Intelligent agent self-learning method and system based on dialogue quality inspection

Abstract

The invention discloses an agent self-learning method based on dialogue quality inspection. The method comprises the steps that a system generates a dialogue; the generated dialogue is stored in a database of the system; the dialogue quality inspection device calls and extracts dialogue samples from the database for quality inspection; performing quality inspection on the extracted dialogue sample through the dialogue quality inspection device, and sending quality inspection result data to the intelligent agent; and the intelligent agent performs self-learning by using quality inspection result data. In addition, the invention also discloses an agent self-learning system based on dialogue quality inspection. According to the method, the self-learning capability of the intelligent agent on the quality inspection marking data is combined, and the business personnel directly connect the intelligent agent through the quality inspection marking interface for debugging, so that the time and energy consumed by the business personnel in optimizing the man-machine conversation quality are greatly saved, and the self-learning performance of the intelligent agent is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of human-computer dialogue, and in particular to an intelligent agent self-learning method and system based on dialogue quality inspection. Background Art

[0002] Intelligent agents in the prior art, such as voice robots, usually need to be optimized based on accumulated historical data. However, this optimization process usually requires a high degree of intervention from the backend personnel of the intelligent agent and takes a lot of time, because the backend personnel need to directly adjust the model parameters of the intelligent agent, and the optimization may not achieve the desired effect because the backend personnel cannot directly understand the business logic. Even if users have a way to perform preliminary optimization, such as quality inspection of conversation data, it will still remain at the optimization of the text information level due to format settings, limited intelligent agent capabilities, etc., and it is difficult to directly convert it into an improvement in the quality of subsequent conversations. Summary of the invention

[0003] Based on this, in order to solve the technical problems in the existing technology, a self-learning method of intelligent agent based on dialogue quality inspection is proposed, including:

[0004] The system generates a dialogue; after the dialogue is generated, the dialogue is stored in the system's database;

[0005] The dialogue quality inspection device calls and extracts dialogue samples from the database for quality inspection; the dialogue quality inspection device performs quality inspection on the extracted dialogue samples, including checking, marking, and correcting problems in the dialogue, and sends the quality inspection result data to the intelligent agent for self-learning;

[0006] The intelligent agent uses the quality inspection result data for self-learning.

[0007] In one embodiment, the conversation quality inspection device includes a learning center and a statistical learning platform; the learning center includes an outbound learning platform and an inbound learning platform; the statistical learning platform includes a statistical learning page;

[0008] Among them, the outbound learning platform and the inbound learning platform are quality inspection working environments, and the statistical learning page is used to check the self-learning data and results;

[0009] The extracted dialogue samples are quality checked by the dialogue quality checking device, including:

[0010] Select the outbound learning platform or inbound learning platform of the learning center for quality inspection according to the business type; the outbound learning platform is for connected conversations and unconnected conversations, and the inbound learning platform is for connected conversations;

[0011] Create a quality inspection batch through the Learning Center; create a quality inspection batch page for the quality inspection batch, and set and adjust the screening conditions and display format of the quality inspection content on the quality inspection batch page;

[0012] Mark for quality inspection through the Learning Center; enter the quality inspection batch page, where the dialogue content of the dialogue to be inspected is displayed on one side of the quality inspection batch page, and the output status bar is displayed on the other side, and key content is marked for quality inspection;

[0013] After the quality inspection of the entire batch is completed, mark the quality inspection batch as completed and export the quality inspection records.

[0014] In one embodiment, the screening conditions and display forms of the quality inspection content include the quality inspection content category, data screening conditions, data push method, sampling quantity, batch name, and business completion standard text;

[0015] Among them, the quality inspection content categories of the outbound learning platform include connected conversations and unconnected conversations. The quality inspection content of unconnected conversations includes the reasons for unconnected conversations. The quality inspection content category of the inbound learning platform is connected conversations.

[0016] Among them, setting data screening conditions and data push methods includes filling in screening conditions to select the data that needs to be inspected in the quality inspection batch. The screening conditions include the dialogue source agent, agent version, and call occurrence time; when the amount of data is large, set the advanced screening mode and refine the screening conditions. The refined screening conditions include call duration, dialogue rounds, keywords, and the number of times that the data is not understood;

[0017] Among them, setting the sampling quantity includes the system user setting the random sampling quantity of the subject or setting the sampling according to a specific time period; the system displays the total number of conversations that meet the filtering criteria and the sampling rate set by the user to ensure that the sampling rate meets the quality inspection requirements.

[0018] In one embodiment, the problems marked by the quality inspection include speech synthesis problems, process and response problems, speech recognition problems, and output status problems;

[0019] Among them, marking speech synthesis problems includes, when the intelligent agent is a voice robot, the system user listens to the voice recording of the conversation made by the voice robot, marks the wrong speech in the sentence and corrects it to the correct speech;

[0020] Among them, marking the process and answering questions includes marking the reasons for answering errors in the intelligent agent dialogue. The system provides a preset list of reasons for answering errors. The reasons for answering errors in the preset list include abnormal user interaction, no added words, and multiple meanings of a sentence.

[0021] Among them, marking speech recognition problems includes, when the intelligent agent is a voice robot, correcting the text of the robot's recognition errors by comparing the recording, marking the interference factors and whether the robot's errors affect the understanding of the conversation content, and the system provides a preset list of interference factors, including dialect accents, environmental noise, and robot interruption;

[0022] Among them, marking the output status problems includes, for the customer's response sentence, the intelligent agent recognizes the customer's intention after understanding and outputs the intention status, and enters the corresponding process according to the intention status; comparing the robot's state output on the sentence, correcting the incorrect output status and marking the influencing factors. The system provides a preset list of influencing factors. The influencing factors in the preset list include speech recognition problems, understanding problems, and process design problems.

[0023] In one embodiment, a statistical analysis is performed on the completion of the quality inspection task through a statistical learning platform to obtain quality inspection statistical results;

[0024] Among them, statistical analysis is performed based on the two dimensions of connected conversations and unconnected conversations to obtain quality inspection statistical results. The statistical quantities used for statistical analysis include the number of quality inspections, working hours, the number of recognition annotations, the number of response annotations, and the number of output annotations;

[0025] In the statistical learning platform, you can use quality inspection time and quality inspection batches to perform customized queries on quality inspection statistical results.

[0026] In addition, in order to solve the technical problems in the prior art, an intelligent agent self-learning system based on dialogue quality inspection is proposed, including a database and a dialogue quality inspection device;

[0027] The system generates a conversation; the database is used to store the conversation data; after the conversation is generated, the conversation is stored in the system's database;

[0028] The dialogue quality inspection device calls and extracts dialogue samples from the database for quality inspection; the dialogue quality inspection device performs quality inspection on the extracted dialogue samples, including checking, marking, and correcting problems in the dialogue, and sends the quality inspection result data to the intelligent agent for self-learning;

[0029] The intelligent agent uses the quality inspection result data for self-learning.

[0030] In one embodiment, the conversation quality inspection device includes a learning center and a statistical learning platform; the learning center includes an outbound learning platform and an inbound learning platform; the statistical learning platform includes a statistical learning page;

[0031] Among them, the outbound learning platform and the inbound learning platform are quality inspection working environments, and the statistical learning page is used to check the self-learning data and results;

[0032] The extracted dialogue samples are quality checked by the dialogue quality checking device, including:

[0033] Select the outbound learning platform or inbound learning platform of the learning center for quality inspection according to the business type; the outbound learning platform is for connected conversations and unconnected conversations, and the inbound learning platform is for connected conversations;

[0034] Create a quality inspection batch through the Learning Center; create a quality inspection batch page for the quality inspection batch, and set and adjust the screening conditions and display format of the quality inspection content on the quality inspection batch page;

[0035] Mark for quality inspection through the Learning Center; enter the quality inspection batch page, where the dialogue content of the dialogue to be inspected is displayed on one side of the quality inspection batch page, and the output status bar is displayed on the other side, and key content is marked for quality inspection;

[0036] After the quality inspection of the entire batch is completed, mark the quality inspection batch as completed and export the quality inspection records.

[0037] In one embodiment, the screening conditions and display forms of the quality inspection content include the quality inspection content category, data screening conditions, data push method, sampling quantity, batch name, and business completion standard text;

[0038] Among them, the quality inspection content categories of the outbound learning platform include connected conversations and unconnected conversations. The quality inspection content of unconnected conversations includes the reasons for unconnected conversations. The quality inspection content category of the inbound learning platform is connected conversations.

[0039] Among them, setting data screening conditions and data push methods includes filling in screening conditions to select the data that needs to be inspected in the quality inspection batch. The screening conditions include the dialogue source agent, agent version, and call occurrence time; when the amount of data is large, set the advanced screening mode and refine the screening conditions. The refined screening conditions include call duration, dialogue rounds, keywords, and the number of times that the data is not understood;

[0040] Among them, setting the sampling quantity includes the system user setting the random sampling quantity of the subject or setting the sampling according to a specific time period; the system displays the total number of conversations that meet the filtering criteria and the sampling rate set by the user to ensure that the sampling rate meets the quality inspection requirements.

[0041] In one embodiment, the problems marked by the quality inspection include speech synthesis problems, process and response problems, speech recognition problems, and output status problems;

[0042] Among them, marking speech synthesis problems includes, when the intelligent agent is a voice robot, the system user listens to the voice recording of the conversation made by the voice robot, marks the wrong speech in the sentence and corrects it to the correct speech;

[0043] Among them, marking the process and answering questions includes marking the reasons for answering errors in the intelligent agent dialogue. The system provides a preset list of reasons for answering errors. The reasons for answering errors in the preset list include abnormal user interaction, no added words, and multiple meanings of a sentence.

[0044] Among them, marking speech recognition problems includes, when the intelligent agent is a voice robot, correcting the text of the robot's recognition errors by comparing the recording, marking the interference factors and whether the robot's errors affect the understanding of the conversation content, and the system provides a preset list of interference factors, including dialect accents, environmental noise, and robot interruption;

[0045] Among them, marking the output status problems includes, for the customer's response sentence, the intelligent agent recognizes the customer's intention after understanding and outputs the intention status, and enters the corresponding process according to the intention status; comparing the robot's state output on the sentence, correcting the incorrect output status and marking the influencing factors. The system provides a preset list of influencing factors. The influencing factors in the preset list include speech recognition problems, understanding problems, and process design problems.

[0046] In one embodiment, a statistical analysis is performed on the completion of the quality inspection task through a statistical learning platform to obtain quality inspection statistical results;

[0047] Among them, statistical analysis is performed based on the two dimensions of connected conversations and unconnected conversations to obtain quality inspection statistical results. The statistical quantities used for statistical analysis include the number of quality inspections, working hours, the number of recognition annotations, the number of response annotations, and the number of output annotations;

[0048] In the statistical learning platform, you can use quality inspection time and quality inspection batches to perform customized queries on quality inspection statistical results.

[0049] Implementing the embodiments of the present invention will have the following beneficial effects:

[0050] In the present invention, the accumulated historical data is used to optimize the quality of human-computer dialogue in the human-computer collaborative dialogue system; the system combines the self-learning ability of the intelligent agent on the marked data and the business personnel directly connecting to the intelligent agent for debugging through the quality inspection mark interface, which greatly saves the time and energy required by the business personnel in optimizing the quality of human-computer dialogue in the prior art, and greatly improves the overall performance of the intelligent agent's self-learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] in:

[0053] Figure 1 Schematic diagram of the process of the intelligent agent self-learning method in the present invention;

[0054] Figure 2 Schematic diagram of the intelligent agent self-learning system in the present invention; DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] The present invention discloses an intelligent agent self-learning method based on dialogue quality inspection, comprising:

[0057] System generated dialogue;

[0058] The interaction process between the customer and the agent is a dialogue; the dialogue types include outbound calls and inbound calls; outbound calls are when the agent initiates a dialogue with the customer, and inbound calls are when the customer initiates a dialogue with the agent; outbound call types include connected and unconnected;

[0059] Among them, the intelligent agent is a computing entity that can function autonomously and sustainably; in the human-computer collaborative dialogue system, it is a voice robot or text robot that can communicate with customers;

[0060] After the conversation is generated, it is stored in the system database as historical data; system users can call the historical data stored in the database;

[0061] Among them, system users include quality inspectors and agent trainers;

[0062] The dialogue quality inspection device calls and extracts dialogue samples from the database for quality inspection; the dialogue quality inspection device performs quality inspection on the extracted dialogue samples, including checking, marking, and correcting problems in the dialogue, and sends the quality inspection result data to the intelligent agent for self-learning;

[0063] The agent uses the quality inspection result data for self-learning;

[0064] Among them, the problems marked by quality inspection include speech synthesis problems, process and response problems, speech recognition problems, and output status problems;

[0065] In particular, the conversation quality inspection device includes a learning center and a statistical learning platform; the learning center includes an outbound learning platform and an inbound learning platform; the statistical learning platform includes a statistical learning page;

[0066] Among them, the outbound learning platform and the inbound learning platform are quality inspection working environments, and the statistical learning page is used to check the self-learning data and results;

[0067] In particular, the extracted dialogue samples are quality checked by the dialogue quality checking device, including:

[0068] Select the learning center's outbound learning platform or inbound learning platform for quality inspection based on the business type;

[0069] The quality inspection and marking process of the outbound learning platform or the inbound learning platform is the same, and the only difference between the two is the marking data they face. The outbound learning platform is for connected conversations and unconnected conversations, and the inbound learning platform is for connected conversations.

[0070] Create a quality inspection batch through the Learning Center; create a quality inspection batch page for the quality inspection batch, and set and adjust the screening conditions and display format of the quality inspection content on the quality inspection batch page;

[0071] Among them, the screening conditions and display forms of quality inspection content include quality inspection content category, data screening conditions, data push method, sampling quantity, batch name, and business completion standard text;

[0072] Among them, the quality inspection content categories of the outbound learning platform include connected conversations and unconnected conversations. The quality inspection content of unconnected conversations includes the reasons for unconnected conversations. The quality inspection content category of the inbound learning platform is connected conversations.

[0073] Among them, setting data screening conditions and data push methods includes filling in screening conditions to select the data that needs to be inspected in the quality inspection batch. The screening conditions include the dialogue source agent, agent version, and call occurrence time; when the amount of data is large, set the advanced screening mode and refine the screening conditions. The refined screening conditions include call duration, dialogue rounds, keywords, and the number of times that the data is not understood;

[0074] Among them, setting the sampling quantity includes the system user setting the random sampling quantity of the subject or setting sampling according to a specific time period; the system displays the total number of conversations that meet the screening conditions and the sampling rate set by the user to ensure that the sampling rate meets the quality inspection requirements;

[0075] Mark for quality inspection through the Learning Center; enter the quality inspection batch page, where the dialogue content of the dialogue to be inspected is displayed on one side of the quality inspection batch page, and the output status bar is displayed on the other side, and key content is marked for quality inspection;

[0076] Among them, the problems marked by quality inspection include speech synthesis problems, process and response problems, speech recognition problems, and output status problems;

[0077] Among them, marking speech synthesis problems includes, when the intelligent agent is a voice robot, the system user listens to the voice recording of the conversation made by the voice robot, marks the wrong speech in the sentence and corrects it to the correct speech;

[0078] Among them, marking the process and answering questions includes marking the reasons for answering errors in the intelligent agent dialogue. The system provides a preset list of reasons for answering errors. The reasons for answering errors in the preset list include abnormal user interaction, no added words, and multiple meanings of a sentence.

[0079] Among them, marking speech recognition problems includes, when the intelligent agent is a voice robot, correcting the text of the robot's recognition errors by comparing the recording, marking the interference factors and whether the robot's errors affect the understanding of the conversation content, and the system provides a preset list of interference factors, including dialect accents, environmental noise, and robot interruption;

[0080] Among them, marking the output status problem includes: for the customer's response sentence, the intelligent agent recognizes the customer's intention after understanding and outputs the intention state, and enters the corresponding process according to the intention state; comparing the robot's state output on the sentence, correcting the incorrect output state and marking the influencing factors. The system provides a preset list of influencing factors, and the influencing factors in the preset list include speech recognition problems, understanding problems, and process design problems;

[0081] After the quality inspection of the entire batch is completed, mark the quality inspection batch as completed and export the quality inspection records;

[0082] Conduct statistical analysis on the completion of quality inspection tasks through the statistical learning platform to obtain quality inspection statistical results;

[0083] Among them, statistical analysis is performed based on the two dimensions of connected conversations and unconnected conversations to obtain quality inspection statistical results. The statistical quantities used for statistical analysis include the number of quality inspections, working hours, the number of recognition annotations, the number of response annotations, and the number of output annotations;

[0084] In the statistical learning platform, use the quality inspection time and quality inspection batch to perform custom queries on the quality inspection statistical results;

[0085] Provide the quality inspection result data of the completed quality inspection to the intelligent agent for self-learning to improve the recognition and synthesis capabilities of the intelligent agent;

[0086] Specifically, for the problem of unconnected recognition, for data that the system cannot recognize, and for recognition error data found in quality inspection, the unrecognizable situations include new categories, new expressions of existing categories such as foreign languages, customization, and noise pollution;

[0087] In the case of new categories that cannot be recognized, the agent adds new categories and supplements relevant data in self-learning; in the case of new expressions of existing categories, the agent supplements relevant data in self-learning; in the case of noise pollution that cannot be recognized, the data is indexed by audio fingerprint, thereby reducing the recognition error of this type of data and having generalization ability;

[0088] Specifically, for speech recognition problems, speech recognition errors that affect semantic understanding are found in dialogue quality inspection. The intelligent agent processes them through full-text representation in self-learning to improve the performance of its recognition engine;

[0089] To address acoustic issues in speech recognition, including dialect accents and pronunciation changes, the intelligent agent performs text and dictionary annotations on sampled data during self-learning, and specifically adjusts the model parameters of the recognition engine to improve its recognition engine performance.

[0090] To address language issues in speech recognition, including Out of Vocabulary (OOV) and new domain words, the agent expands the dictionary during self-learning or adds learning data from related domain texts to adapt to the language rules of the new domain.

[0091] Specifically, for speech synthesis problems, speech synthesis error data and production quality inspection synthesis error data were found during debugging and testing. Speech synthesis errors included incorrect pronunciation of polyphonetic characters and incomplete pronunciation of modal particles.

[0092] For speech synthesis, the agent improves the robustness of the synthesis model by expanding the polyphonetic dictionary, adjusting the polyphonetic model, and expanding the training data of low-frequency characters during self-learning.

[0093] Specifically, with regard to process and response issues, for process issues that arise, the intelligent agent adjusts the overall logic of the conversation during self-learning to avoid redundancy and repetition; for response issues that arise, the intelligent agent adjusts the wording that is prone to ambiguity and misunderstanding by customers during self-learning, so as to convey information to customers in a concise manner and improve the efficiency of human-computer interaction.

[0094] The present invention discloses an intelligent agent self-learning system based on dialogue quality inspection, including a database and a dialogue quality inspection device;

[0095] The system generates a dialogue; the interaction process between the customer and the agent is a dialogue; the dialogue types include outbound calls and inbound calls; outbound calls are when the agent initiates a dialogue with the customer, and inbound calls are when the customer initiates a dialogue with the agent; outbound call types include connected and unconnected;

[0096] Among them, the intelligent agent is a computing entity that can function autonomously and sustainably; in the human-computer collaborative dialogue system, it is a voice robot or text robot that can communicate with customers;

[0097] The database is used to store conversation data. After a conversation is generated, it is stored in the system database as historical data. System users can call the historical data stored in the database.

[0098] Among them, system users include quality inspectors and agent trainers;

[0099] The intelligent agent uses the dialogue quality inspection device to self-learn speech recognition and speech synthesis;

[0100] Among them, in the self-learning for speech recognition and speech synthesis, the dialogue quality inspection device calls and extracts dialogue samples from the database for quality inspection; the dialogue quality inspection device performs quality inspection on the extracted dialogue samples, including checking, marking, and correcting problems in the dialogue, and sends the quality inspection result data to the intelligent agent for self-learning;

[0101] Among them, the problems marked by quality inspection include speech synthesis problems, process and response problems, speech recognition problems, and output status problems;

[0102] In particular, the conversation quality inspection device includes a learning center and a statistical learning platform; the learning center includes an outbound learning platform and an inbound learning platform; the statistical learning platform includes a statistical learning page;

[0103] Among them, the outbound learning platform and the inbound learning platform are quality inspection working environments, and the statistical learning page is used to check the self-learning data and results;

[0104] In particular, the extracted dialogue samples are quality checked by the dialogue quality checking device, including:

[0105] Select the learning center's outbound learning platform or inbound learning platform for quality inspection based on the business type;

[0106] The quality inspection and marking process of the outbound learning platform or the inbound learning platform is the same, and the only difference between the two is the marking data they face. The outbound learning platform is for connected conversations and unconnected conversations, and the inbound learning platform is for connected conversations.

[0107] Create a quality inspection batch through the Learning Center; create a quality inspection batch page for the quality inspection batch, and set and adjust the screening conditions and display format of the quality inspection content on the quality inspection batch page;

[0108] Among them, the screening conditions and display forms of quality inspection content include quality inspection content category, data screening conditions, data push method, sampling quantity, batch name, and business completion standard text;

[0109] Among them, the quality inspection content categories of the outbound learning platform include connected conversations and unconnected conversations. The quality inspection content of unconnected conversations includes the reasons for unconnected conversations. The quality inspection content category of the inbound learning platform is connected conversations.

[0110] Among them, setting data screening conditions and data push methods includes filling in screening conditions to select the data that needs to be inspected in the quality inspection batch. The screening conditions include the dialogue source agent, agent version, and call occurrence time; when the amount of data is large, set the advanced screening mode and refine the screening conditions. The refined screening conditions include call duration, dialogue rounds, keywords, and the number of times that the data is not understood;

[0111] Among them, setting the sampling quantity includes the system user setting the random sampling quantity of the subject or setting sampling according to a specific time period; the system displays the total number of conversations that meet the screening conditions and the sampling rate set by the user to ensure that the sampling rate meets the quality inspection requirements;

[0112] Mark for quality inspection through the Learning Center; enter the quality inspection batch page, where the dialogue content of the dialogue to be inspected is displayed on one side of the quality inspection batch page, and the output status bar is displayed on the other side, and key content is marked for quality inspection;

[0113] Among them, the problems marked by quality inspection include speech synthesis problems, process and response problems, speech recognition problems, and output status problems;

[0114] Among them, marking speech synthesis problems includes, when the intelligent agent is a voice robot, the system user listens to the voice recording of the conversation made by the voice robot, marks the wrong speech in the sentence and corrects it to the correct speech;

[0115] Among them, marking the process and answering questions includes marking the reasons for answering errors in the intelligent agent dialogue. The system provides a preset list of reasons for answering errors. The reasons for answering errors in the preset list include abnormal user interaction, no added words, and multiple meanings of a sentence.

[0116] Among them, marking speech recognition problems includes, when the intelligent agent is a voice robot, correcting the text of the robot's recognition errors by comparing the recording, marking the interference factors and whether the robot's errors affect the understanding of the conversation content, and the system provides a preset list of interference factors, including dialect accents, environmental noise, and robot interruption;

[0117] Among them, marking the output status problem includes: for the customer's response sentence, the intelligent agent recognizes the customer's intention after understanding and outputs the intention state, and enters the corresponding process according to the intention state; comparing the robot's state output on the sentence, correcting the incorrect output state and marking the influencing factors. The system provides a preset list of influencing factors, and the influencing factors in the preset list include speech recognition problems, understanding problems, and process design problems;

[0118] After the quality inspection of the entire batch is completed, mark the quality inspection batch as completed and export the quality inspection records;

[0119] Conduct statistical analysis on the completion of quality inspection tasks through the statistical learning platform to obtain quality inspection statistical results;

[0120] Among them, statistical analysis is performed based on the two dimensions of connected conversations and unconnected conversations to obtain quality inspection statistical results. The statistical quantities include the number of quality inspections, working hours, number of recognition annotations, number of response annotations, and number of output annotations.

[0121] Use quality inspection time and quality inspection batches to perform custom queries on quality inspection statistical results in the statistical learning platform;

[0122] Provide the quality inspection result data of the completed quality inspection to the intelligent agent for self-learning to improve the recognition and synthesis capabilities of the intelligent agent;

[0123] Specifically, for the problem of unconnected recognition, for data that the system cannot recognize, and for recognition error data found in quality inspection, the unrecognizable situations include new categories, new expressions of existing categories such as foreign languages, customization, and noise pollution;

[0124] In the case of new categories that cannot be recognized, the agent adds new categories and supplements relevant data in self-learning; in the case of new expressions of existing categories, the agent supplements relevant data in self-learning; in the case of noise pollution that cannot be recognized, the data is indexed by audio fingerprint, thereby reducing the recognition error of this type of data and having generalization ability;

[0125] Specifically, for speech recognition problems, speech recognition errors that affect semantic understanding are found in dialogue quality inspection. The intelligent agent processes them through full-text representation in self-learning to improve the performance of its recognition engine;

[0126] To address acoustic issues in speech recognition, including dialect accents and pronunciation changes, the intelligent agent performs text and dictionary annotations on sampled data during self-learning, and specifically adjusts the model parameters of the recognition engine to improve its recognition engine performance.

[0127] To address language issues in speech recognition, including Out of Vocabulary (OOV) and new domain words, the agent expands the dictionary during self-learning or adds learning data from related domain texts to adapt to the language rules of the new domain.

[0128] Specifically, for speech synthesis problems, speech synthesis error data and production quality inspection synthesis error data were found during debugging and testing. Speech synthesis errors included incorrect pronunciation of polyphonetic characters and incomplete pronunciation of modal particles.

[0129] To deal with the speech synthesis problem, the intelligent agent improves the robustness of the synthesis model by expanding the polyphonetic dictionary, adjusting the polyphonetic model, and expanding the training data of low-frequency characters during self-learning;

[0130] Specifically, with regard to process and response issues, for process issues that arise, the intelligent agent adjusts the overall logic of the conversation during self-learning to avoid redundancy and repetition; for response issues that arise, the intelligent agent adjusts the wording that is prone to ambiguity and misunderstanding by customers during self-learning, so as to convey information to customers in a concise manner and improve the efficiency of human-computer interaction.

[0131] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An agent self-learning method based on dialogue quality inspection, characterized in that: include: System generated dialogue; After the dialogue is generated, it is stored in the system's database; The dialogue quality inspection device calls and extracts dialogue samples from the database for quality inspection; the dialogue quality inspection device performs quality inspection on the extracted dialogue samples, including checking, marking, and correcting problems in the dialogue, and sends the quality inspection result data to the intelligent agent for self-learning; The intelligent agent uses the quality inspection result data for self-learning.

2. According to the intelligent agent self-learning method based on dialogue quality inspection according to claim 1, It is characterized in that in, The dialogue quality inspection device includes a learning center and a statistical learning platform; the learning center includes an outbound learning platform and an inbound learning platform; The statistical learning platform includes a statistical learning page; Among them, the outbound learning platform and the inbound learning platform are quality inspection working environments, and the statistical learning page is used to check the self-learning data and results; The extracted dialogue samples are quality checked by the dialogue quality checking device, including: Select the outbound learning platform or inbound learning platform of the learning center for quality inspection according to the business type; the outbound learning platform is for connected conversations and unconnected conversations, and the inbound learning platform is for connected conversations; Create a quality inspection batch through the Learning Center; create a quality inspection batch page for the quality inspection batch, and set and adjust the screening conditions and display format of the quality inspection content on the quality inspection batch page; Mark for quality inspection through the Learning Center; enter the quality inspection batch page, where the dialogue content of the dialogue to be inspected is displayed on one side of the quality inspection batch page, and the output status bar is displayed on the other side, and key content is marked for quality inspection; After the quality inspection of the entire batch is completed, mark the quality inspection batch as completed and export the quality inspection records.

3. According to the intelligent agent self-learning method based on dialogue quality inspection according to claim 2, It is characterized in that in, The screening conditions and display forms of quality inspection content include quality inspection content category, data screening conditions, data push method, sampling quantity, batch name, and business completion standard text; Among them, the quality inspection content categories of the outbound learning platform include connected conversations and unconnected conversations. The quality inspection content of unconnected conversations includes the reasons for unconnected conversations. The quality inspection content category of the inbound learning platform is connected conversations. Among them, setting data screening conditions and data push methods includes filling in screening conditions to select the data that needs to be inspected in the quality inspection batch. The screening conditions include the dialogue source agent, agent version, and call occurrence time; when the amount of data is large, set the advanced screening mode and refine the screening conditions. The refined screening conditions include call duration, dialogue rounds, keywords, and the number of times that the data is not understood; Among them, setting the sampling quantity includes the system user setting the random sampling quantity of the subject or setting the sampling according to a specific time period; the system displays the total number of conversations that meet the filtering criteria and the sampling rate set by the user to ensure that the sampling rate meets the quality inspection requirements.

4. According to the intelligent agent self-learning method based on dialogue quality inspection according to claim 2, It is characterized in that in, The issues marked by the quality inspection include speech synthesis issues, process and response issues, speech recognition issues, and output status issues; Among them, marking speech synthesis problems includes, when the intelligent agent is a voice robot, the system user listens to the voice recording of the conversation made by the voice robot, marks the wrong speech in the sentence and corrects it to the correct speech; Among them, marking the process and answering questions includes marking the reasons for answering errors in the intelligent agent dialogue. The system provides a preset list of reasons for answering errors. The reasons for answering errors in the preset list include abnormal user interaction, no added words, and multiple meanings of a sentence. Among them, marking speech recognition problems includes, when the intelligent agent is a voice robot, correcting the text of the robot's recognition errors by comparing the recording, marking the interference factors and whether the robot's errors affect the understanding of the conversation content, and the system provides a preset list of interference factors, including dialect accents, environmental noise, and robot interruption; Among them, marking the output status problems includes, for the customer's response sentence, the intelligent agent recognizes the customer's intention after understanding and outputs the intention status, and enters the corresponding process according to the intention status; comparing the robot's state output on the sentence, correcting the incorrect output status and marking the influencing factors. The system provides a preset list of influencing factors. The influencing factors in the preset list include speech recognition problems, understanding problems, and process design problems.

5. The agent self-learning method based on dialogue quality inspection according to claim 2 is characterized in that: Conduct statistical analysis on the completion of quality inspection tasks through the statistical learning platform to obtain quality inspection statistical results; Among them, statistical analysis is performed based on the two dimensions of connected conversations and unconnected conversations to obtain quality inspection statistical results. The statistical quantities used for statistical analysis include the number of quality inspections, working hours, the number of recognition annotations, the number of response annotations, and the number of output annotations; In the statistical learning platform, you can use quality inspection time and quality inspection batches to perform customized queries on quality inspection statistical results.

6. An intelligent agent self-learning system based on dialogue quality inspection, characterized in that: Including database, dialogue quality inspection device; System generated dialogue; The database is used to store conversation data; after a conversation is generated, it is stored in the system's database; The dialogue quality inspection device calls and extracts dialogue samples from the database for quality inspection; the dialogue quality inspection device performs quality inspection on the extracted dialogue samples, including checking, marking, and correcting problems in the dialogue, and sends the quality inspection result data to the intelligent agent for self-learning; The intelligent agent uses the quality inspection result data for self-learning.

7. According to claim 6, the intelligent agent self-learning system based on dialogue quality inspection, It is characterized in that in, The dialogue quality inspection device includes a learning center and a statistical learning platform; the learning center includes an outbound learning platform and an inbound learning platform; The statistical learning platform includes a statistical learning page; Among them, the outbound learning platform and the inbound learning platform are quality inspection working environments, and the statistical learning page is used to check the self-learning data and results; The extracted dialogue samples are quality checked by the dialogue quality checking device, including: Select the outbound learning platform or inbound learning platform of the learning center for quality inspection according to the business type; the outbound learning platform is for connected conversations and unconnected conversations, and the inbound learning platform is for connected conversations; Create a quality inspection batch through the Learning Center; create a quality inspection batch page for the quality inspection batch, and set and adjust the screening conditions and display format of the quality inspection content on the quality inspection batch page; Mark for quality inspection through the Learning Center; enter the quality inspection batch page, where the dialogue content of the dialogue to be inspected is displayed on one side of the quality inspection batch page, and the output status bar is displayed on the other side, and key content is marked for quality inspection; After the quality inspection of the entire batch is completed, mark the quality inspection batch as completed and export the quality inspection records.

8. According to the intelligent agent self-learning system based on dialogue quality inspection according to claim 7, It is characterized in that in, The screening conditions and display forms of quality inspection content include quality inspection content category, data screening conditions, data push method, sampling quantity, batch name, and business completion standard text; Among them, the quality inspection content categories of the outbound learning platform include connected conversations and unconnected conversations. The quality inspection content of unconnected conversations includes the reasons for unconnected conversations. The quality inspection content category of the inbound learning platform is connected conversations. Among them, setting data screening conditions and data push methods includes filling in screening conditions to select the data that needs to be inspected in the quality inspection batch. The screening conditions include the dialogue source agent, agent version, and call occurrence time; when the amount of data is large, set the advanced screening mode and refine the screening conditions. The refined screening conditions include call duration, dialogue rounds, keywords, and the number of times that the data is not understood; Among them, setting the sampling quantity includes the system user setting the random sampling quantity of the subject or setting the sampling according to a specific time period; the system displays the total number of conversations that meet the filtering criteria and the sampling rate set by the user to ensure that the sampling rate meets the quality inspection requirements.

9. According to claim 7, the intelligent agent self-learning system based on dialogue quality inspection, It is characterized in that in, The issues marked by the quality inspection include speech synthesis issues, process and response issues, speech recognition issues, and output status issues; Among them, marking speech synthesis problems includes, when the intelligent agent is a voice robot, the system user listens to the voice recording of the conversation made by the voice robot, marks the wrong speech in the sentence and corrects it to the correct speech; Among them, marking the process and answering questions includes marking the reasons for answering errors in the intelligent agent dialogue. The system provides a preset list of reasons for answering errors. The reasons for answering errors in the preset list include abnormal user interaction, no added words, and multiple meanings of a sentence. Among them, marking speech recognition problems includes, when the intelligent agent is a voice robot, correcting the text of the robot's recognition errors by comparing the recording, marking the interference factors and whether the robot's errors affect the understanding of the conversation content, and the system provides a preset list of interference factors, including dialect accents, environmental noise, and robot interruption; Among them, marking the output status problems includes, for the customer's response sentence, the intelligent agent recognizes the customer's intention after understanding and outputs the intention status, and enters the corresponding process according to the intention status; comparing the robot's state output on the sentence, correcting the incorrect output status and marking the influencing factors. The system provides a preset list of influencing factors. The influencing factors in the preset list include speech recognition problems, understanding problems, and process design problems.

10. The intelligent agent self-learning system based on dialogue quality inspection according to claim 7, characterized in that: Conduct statistical analysis on the completion of quality inspection tasks through the statistical learning platform to obtain quality inspection statistical results; Among them, statistical analysis is performed based on the two dimensions of connected conversations and unconnected conversations to obtain quality inspection statistical results. The statistical quantities used for statistical analysis include the number of quality inspections, working hours, the number of recognition annotations, the number of response annotations, and the number of output annotations; In the statistical learning platform, you can use quality inspection time and quality inspection batches to perform customized queries on quality inspection statistical results.

Images