Multi-round dialogue engine design method based on intention recognition and state memory
Through a multi-round dialogue engine based on intention recognition and state memory, the problem of insufficient scene recognition accuracy and speech flexibility of the intelligent dialogue engine in the 5G missed conversation secretary business is solved, and the optimization of dialogue management and operation monitoring is achieved, which improves the improvement of dialogue management and operation monitoring. User experience and model performance.
Patent Information
- Application Number
- CN202411092950.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-29
AI Technical Summary
The existing intelligent dialogue engine has low accuracy in the 5G missed conversation secretarial business, poor speech flexibility, complex interface parameters, single functions, and lack of unified monitoring boards to monitor and count business operations and model performance in real time. metrics issues.
A multi-round dialogue engine based on intention recognition and state memory was designed. By acquiring and preprocessing corpus data, using BERT pre-trained model for transfer learning, building a unified visualization of multi-round dialogue engine workflow and real-time log management and monitoring indicators. Monitoring the board to realize the closed loop of dialogue status information memory and operation optimization.
It improves the accuracy of scene recognition, simplifies interface parameters, enhances speech flexibility, realizes dialogue management and operation monitoring, forms a closed loop from data production to engine iteration, and improves user experience and model performance.
Smart Images

Figure CN119047470A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of natural language processing intelligent response technology, and in particular to a multi-round dialogue engine design method based on intent recognition and state memory. Background Art
[0002] Under the guidance of the telecom cloud transformation and digital transformation strategy, the 5G missed call secretary service continues to develop and user coverage is gradually expanding. Intelligent answering is an important part of the intelligent upgrade of the missed call secretary product. Our goal is to enable the AI robot to accurately understand user semantics in the conversation and give more humane answers.
[0003] There are three main types of existing intelligent dialogue engine implementation methods: the first is question and answer based on the knowledge base, which stores question and answer pairs in advance and retrieves similar questions to return the answer with the highest matching value, such as the intelligent question and answer method and system disclosed in patent CN108153876B. This method lacks matching flexibility in scenarios with rich user expressions due to the limited coverage of similar questions; the second is question and answer based on the knowledge graph, which fills slots by building a graph database and extracting entities from user input, such as patent CN111274371B, which discloses an intelligent human-computer dialogue method and device based on a knowledge graph. This method is more suitable for scenarios with more complex knowledge systems; the third is question and answer based on deep learning neural network, which predicts user intentions and returns corresponding words by training intent recognition models such as CNN, RNN, and Bert. This method is more flexible in semantic understanding and response design.
[0004] Combined with the characteristics of the 5G Missed Call Secretary service, however, the existing conversation methods based on intent recognition mostly focus on the response process itself or the model network structure design, lacking a complete system covering engine operation optimization, conversation content summary, and real-time monitoring of business results. Looking at the current situation, the intelligent conversation engine currently used by the 5G Missed Call Secretary has problems such as low scene recognition accuracy, poor flexibility of speech, complex interface parameters, and single functions. In addition, the existing engine lacks a unified monitoring dashboard to monitor and count business operations and model performance indicators in real time. Summary of the invention
[0005] The purpose of the present invention is to provide a multi-round dialogue engine design method based on intent recognition and state memory that can achieve a closed-loop optimization of dialogue engine operations.
[0006] The purpose of the present invention can be achieved by the following technical solutions:
[0007] A multi-round dialogue engine design method based on intention recognition and state memory includes the following steps:
[0008] Obtain relevant corpus data and preprocess it to obtain a training data set;
[0009] The training data set is input into the BERT pre-training model for transfer learning, and the training iteration, parameter tuning and model distillation of the BERT pre-training model are completed to obtain an intent recognition model;
[0010] Based on the intention recognition model, a multi-round dialogue engine workflow is built and a summary of the user-side call content is extracted, wherein the multi-round dialogue engine workflow is used to implement multi-round dialogues, and the summary is sent to the user. The multi-round dialogue engine workflow implements dialogue state information memory by storing the state of active calls, and supports scene jumps, round jumps, and random speech.
[0011] Build a unified visual monitoring dashboard for real-time log management and monitoring indicators;
[0012] The design is processed and analyzed based on the operational results to form an operational optimization closed loop of the multi-round dialogue engine to complete the design process.
[0013] Furthermore, the pre-processing step comprises:
[0014] The relevant corpus data are entity labeled and uniformly cleaned, and the data of low-frequency scenes in the relevant corpus data are oversampled to obtain a training data set, wherein the oversampling process includes data enhancement by random interaction, random insertion, random deletion and synonym replacement.
[0015] Furthermore, the BERT pre-training model is constructed by layering using a Transformer encoder, wherein the Transformer encoder includes a multi-head self-attention layer, a residual connection and normalization layer, and a feedforward network layer;
[0016] The self-attention calculation expression of the multi-head self-attention layer is:
[0017]
[0018] Where attention(Q,K,V) is self-attention, Q, K, V are query, key, and value respectively, and d k is the hidden layer dimension;
[0019] The operational expression of the feedforward network layer is:
[0020] max(0,xw1+b1)*w2+b2
[0021] Where x is the input tensor, w1, b1, w2, b2 are the learnable weight matrices and bias vectors.
[0022] Furthermore, in the model distillation process, the TinyBERT pre-trained model is used as a student model and the BERT pre-trained model is used as a teacher model, wherein knowledge distillation is performed on the embedding layer, prediction layer, multi-head self-attention layer and hidden layer of the TinyBERT pre-trained model.
[0023] Furthermore, the loss function of the model distillation process is the sum of the mean square error of the embedding layer, attention layer and hidden layer of the teacher model and the student model and the cross entropy of the prediction probability. The expression of the loss function is:
[0024]
[0025] Where, L model is the total loss, m is the number of layers of the student model, and a m is the weight of the mth layer, L layer is the loss function of the mth layer, L embedding is the mean square error of the embedding layer, L attention is the mean square error of the attention layer, L hidden is the mean square error of the hidden layer, L prediction is the prediction probability error.
[0026] Furthermore, the specific process of the multi-round dialogue engine workflow includes:
[0027] Obtain the call content on the user side as user input, query the conversation state corresponding to the call ID from the Redis database, if the call ID does not exist, initialize a conversation instance, if it exists, execute the next step, where the conversation state includes the global conversation turn, the current scene, and the current scene conversation turn;
[0028] Use the intent recognition model to perform scene recognition on the incoming user input, and perform entity recognition based on the entity hit rules under the scene recognition results to obtain entity information;
[0029] If the global conversation round is not greater than the preset round, determine whether the current scene of the call ID is consistent with the identification scene input by the user this time. If so, execute the next step. If not, reset the current scene conversation round to 0 to achieve scene jump;
[0030] If the global conversation round is greater than the preset round or the current scene of the call ID is consistent with the identification scene input by the user this time, the conversation round of the current scene is increased by one;
[0031] Determine whether the target entity of the current scene dialogue round has been hit. If so, add one to the current dialogue round. If not, do nothing.
[0032] Returns the corresponding dialogue script for the current scene and dialogue round, where the dialogue script is designed in various ways for each scene and each round.
[0033] Furthermore, the step of obtaining the entity hit rule includes:
[0034] Use regular matching to hit entities from entity annotation data;
[0035] The entity hit rules are summarized based on the occurrence frequency of the hit entities in each business scenario and the slot values corresponding to the hit entities.
[0036] Furthermore, the step of extracting the summary includes:
[0037] Inputting the call content on the user side into the intent recognition model to identify the global intent of the call content as a context constraint for summary extraction;
[0038] Divide the call content into clauses according to punctuation marks and pauses, and input each clause into the intent recognition model to determine the contribution of each clause to the global intent.
[0039] Sort the contributions, select clauses whose contributions are higher than the set threshold, and generate a summary.
[0040] Furthermore, monitoring indicators include the call volume of each interface, response time, status code, distribution of dialogue scenarios and dialogue completion rate of each scenario.
[0041] Furthermore, the step of forming an operation optimization closed loop of the multi-round dialogue engine includes:
[0042] According to the indicator monitoring results, optimize the speech and entity matching rules for scenarios where the dialogue completion rate is lower than the preset threshold, and update the label system, entity library and speech design in stages;
[0043] For scenarios where the accuracy of intent recognition is lower than the preset threshold, error-prone data is summarized and input into the intent recognition model for iterative training, so as to drive multi-round iterative optimization of the dialogue engine based on operational results and form a closed loop.
[0044] Compared with the prior art, the present invention has the following beneficial effects:
[0045] (1) The present invention provides a scalable intelligent dialogue engine design method based on the 5G missed call secretary service. The dialogue engine is based on the intent recognition model, builds a multi-round dialogue engine workflow and summary extraction, and designs a monitoring dashboard to monitor operation indicators, covering dialogue management, intent recognition, operation monitoring, content summary and other parts. It can optimize and iterate each part through the operation results to achieve a closed loop of operation optimization.
[0046] (2) The present invention implements the conversation state memory, context management and round-to-round jumping of the multi-round conversation engine by adding a call instance class in the engine, and transfers the work of historical conversation information management from the client to the server, thereby solving the problems of complex interface parameters and rigid reply process existing in the existing conversation engine, avoiding users from repeating answers and improving user experience.
[0047] (3) The present invention improves the accuracy of scene recognition results by training and fine-tuning the pre-trained BERT model. At the same time, it reduces the size of the model by using knowledge distillation technology, thereby increasing the model reasoning speed and reducing resource usage.
[0048] (4) The dialogue engine of the present invention also covers parts such as speech configuration and entity management, and can further dynamically adjust and improve the underlying models or calling methods of each part, and further support the flexible combination and optimization upgrade of models such as data enhancement and cleaning, intent recognition, entity extraction, and speech generation.
[0049] (5) In the optimization operation of the present invention, the engine optimization is driven by the result of the previous operation, and the business return data and statistical indicators are fully utilized to form a closed loop from data production to engine iteration. By combining automated real-time monitoring with manual spot checks, the effect analysis and optimization point positioning of each scenario of the intelligent dialogue engine are realized; through rolling training of error-prone data, targeted iterative learning of the model is realized. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 This is a flow chart of the intelligent dialogue engine of the present invention;
[0051] Figure 2 It is a structural diagram of the Transformer encoder of the present invention;
[0052] Figure 3 This is a workflow diagram of the intelligent dialogue engine of the present invention. DETAILED DESCRIPTION
[0053] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.
[0054] This embodiment provides a multi-round dialogue engine design method based on intention recognition and state memory. Figure 1As shown, the method mainly includes data preprocessing, model training, model distillation, speech design, entity rule extraction, engine workflow construction, summary extraction, indicator monitoring and operation optimization. The corpus data generated in the actual business of the Missed Talk Secretary is used as the data source. After pre-processing such as annotation, cleaning, and enhancement, the corpus data is input into the BERT (Bidirectional Encoder Representations from Transformers) pre-trained model for transfer learning, and the training iteration, parameter tuning and model distillation of the intent recognition model are completed. The speech process jump is controlled through the engine workflow, and summary extraction is realized by combining active learning and context constraints. The LGTM technology stack is used to build a real-time indicator monitoring dashboard, and then the engine operation optimization closed loop is formed by reprocessing and re-analyzing the reflux data. The specific steps are as follows:
[0055] S1. Original data set acquisition and preprocessing
[0056] The training data used in the present invention mainly include four categories. The first is based on the 5G missed call secretary intelligent answering full-scenario business return data, and the call data after voice recognition is labeled according to the scene and the entities related to each scene, and then the labeled data is spot-checked and manually screened; the second is obtained by writing regular expressions for each business scenario and performing regular matching from the business return data; the third is the error-prone data of the existing model or the old version of the model; the fourth is the data for low-frequency scenarios with the help of large models and manual supplementation.
[0057] The above four types of data are integrated to form the original data set of the intelligent response business. The data set is uniformly cleaned and the data of low-frequency scenes are oversampled. That is, data enhancement is performed through random interaction, random insertion, random deletion and synonym replacement to obtain the model training data set.
[0058] At the same time, by continuously collecting statistics on the proportion of various labels and unclassifiable data in the statistical data set, the existing label system is adjusted to form a data model that is more in line with actual business.
[0059] S2. Model training iteration
[0060] The model training data set is extracted layer by scene and divided into training set and validation set in a ratio of 9:1. The training set is input into the BERT-base-Chinese pre-trained model, the model weights are initialized, and hyperparameters such as learning rate, batch size, and number of iterations are configured. Multi-classification cross entropy is used as the loss function and accuracy is used as the measurement indicator. The training is iterated until the model converges.
[0061] BERT uses the Transformer encoder for layered stacking, and the input is the sum of the token encoding, the part encoding, and the position encoding. The Transformer encoder network structure is as follows: Figure 2 As shown in Figure 1, it mainly includes multi-head attention layer, residual connection and layer normalization and feedforward layer. Among them, the multi-head self-attention mechanism is the core, and the output formula of self-attention is d k is the hidden layer dimension; residual connection can prevent network degradation, and layer normalization can speed up network convergence; the feedforward network layer is a two-layer fully connected layer, the formula is max(0,xw1+b1)*w2+b2, x is the input tensor, w1, b1, w2, b2 are learnable weight matrices and bias vectors.
[0062] S3. Model distillation
[0063] In order to improve the model reasoning speed and reduce resource consumption, the TinyBERT model is used as the student model and the BERT model is used as the teacher model for knowledge distillation, which retains the information learned by the original complex network while shrinking the model.
[0064] In addition to distilling the embedding layer and prediction layer, TinyBERT adds attention-based distillation and hidden-layer-based distillation. Its loss function is the sum of the mean square error of the embedding layer, self-attention layer, and hidden layer output of the teacher BERT model and the student model and the cross entropy of the prediction probability, that is, in
[0065] S4. Speech Design
[0066] By analyzing, summarizing and organizing business return data, we design corresponding response scripts for each business scenario, and control the total number of conversation rounds to 3-7. We design multiple response templates for each scenario and each round for random selection during the conversation, thus improving the diversity of intelligent response scripts.
[0067] S5. Entity rule extraction
[0068] After analyzing the entity annotation data, it is found that its highly dispersed and uneven data distribution is not suitable for training entity recognition models. Therefore, the present invention uses regular matching to hit entities. The entities that appear frequently in actual business and their corresponding slot values are summarized into hit rules.
[0069] S6. Engine workflow construction
[0070] The workflow of the intelligent dialogue engine of the present invention is as follows: Figure 3 As shown, the workflow implements conversation status information memory by storing the status of active calls, supports scene jumps, turn jumps, and random conversations. The specific process includes the following steps.
[0071] Step 1: Use the incoming call ID to query the conversation state corresponding to the call ID from the Redis database. If the call ID does not exist, initialize a conversation instance. The conversation state of a call mainly includes information such as global conversation turn, current scene, and current scene conversation turn;
[0072] Step 2: Use the intent recognition model to classify the incoming user input, and perform entity recognition based on the entity hit rule in the scenario to obtain the scenario and entity information corresponding to the user input;
[0073] Step 3: If the global call round is not greater than 3, determine whether the current scene of the call ID is consistent with the recognition scene input this time. If not, reset the current scene conversation round to 0 to achieve scene jump;
[0074] Step 4: When the global call turn is greater than 3 or the current scene of the call ID is consistent with the recognition scene input this time, increase the current conversation turn by one;
[0075] Step 5: Determine whether the target entity of the current round has been hit. If it has been hit, increase the current dialogue round by one;
[0076] Step 6: Return the corresponding words for the current round of the current scene.
[0077] S7. Summary Extraction
[0078] The summary extraction function is implemented based on the trained intent recognition model, which can extract key information in the call in a targeted manner. The specific implementation process includes the following steps.
[0079] Step 1: Input the user's call content into the intent recognition model to determine the global intent of the call content as the context constraint of the summary model;
[0080] Step 2: Divide the call content into clauses according to punctuation marks and pauses, and input each clause into the intent recognition model to determine the contribution of each clause to the global intent.
[0081] Step 3: Sort by importance, retain the clauses that have a greater impact on the global intent and whose importance is higher than the set threshold, and generate a summary.
[0082] S8. Index monitoring
[0083] By embedding points in the engine code, real-time statistics of relevant operating indicators of the intelligent dialogue service are realized. In addition to common indicators such as the number of interface calls, response time, and status code, business indicators such as call scene distribution and conversation completion rate in each scene are added to meet the business process and product operation needs of 5G missed call secrets. In addition, based on the LGTM technology stack, unified visualization of log management and monitoring indicators of the intelligent dialogue service is realized.
[0084] S9. Optimize operations
[0085] Collect actual business return data after the intelligent dialogue engine goes online, and conduct manual spot checks and audits on the scene and entity recognition results. Combine indicator monitoring and operational audit results to locate scenes with poor results and reasons that affect engine performance. On the one hand, optimize scene scripts and entity extraction rules for scenes with low dialogue completion rates, and periodically update the label system, entity library, and script configuration; on the other hand, for scenes with low intent recognition accuracy, summarize error-prone data and re-enter the model for iterative training to improve the accuracy and robustness of model predictions. Use operational results to drive engine optimization, that is, iteration of models, rules, and script systems, thereby forming a closed loop.
[0086] The technical solution of this embodiment can be deployed and applied in intelligent response scenarios such as missed call secretary and intelligent customer service, providing comprehensive and scalable engine services. At the same time, its sub-functional modules such as intent recognition, entity recognition, and summary extraction can also be decoupled and combined, and can be used as atomic capabilities to access third-party platforms such as AI voice platforms and intelligent outbound call systems.
[0087] The application of the technical solution of this embodiment in the missed call secretary service scenario is as follows:
[0088] Step 1: User A applies for the 5G missed call secretary service;
[0089] Step 2: When calling user B calls user A and user A is busy, missed, or hangs up, the missed call secretary service is triggered;
[0090] Step 3: The call center calls the AI gateway, which then calls the ASR, intelligent dialogue NLP and TTS engines in sequence, and returns the corresponding response audio after speech recognition, speech text confirmation and speech synthesis;
[0091] Step 4: Push the call content, summary text, etc. to user A via SMS or WeChat mini-program messages.
[0092] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0093] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A multi-round dialogue engine design method based on intention recognition and state memory, characterized in that: The following steps are involved: Obtain relevant corpus data and preprocess it to obtain a training data set; Input the training data set into the BERT pre-training model for transfer learning, complete the training iteration, parameter tuning and model distillation of the BERT pre-training model, and obtain the intent recognition model; Based on the intention recognition model, a multi-round dialogue engine workflow is built and a summary of the user-side call content is extracted, wherein the multi-round dialogue engine workflow is used to implement multi-round dialogues, and the summary is sent to the user. The multi-round dialogue engine workflow implements dialogue state information memory by storing the state of active calls, and supports scene jumps, round jumps, and random speech. Build a unified visual monitoring dashboard for real-time log management and monitoring indicators, and obtain operational results based on multiple rounds of conversation data; Processing and analysis are performed based on the operational results to form an operational optimization closed loop of the multi-round dialogue engine to complete the design process.
2. According to claim 1, a multi-round dialogue engine design method based on intention recognition and state memory is characterized in that: The pre-processing step comprises: The relevant corpus data are entity labeled and uniformly cleaned, and the data of low-frequency scenes in the relevant corpus data are oversampled to obtain a training data set, wherein the oversampling process includes data enhancement by random interaction, random insertion, random deletion and synonym replacement.
3. According to the method for designing a multi-round dialogue engine based on intention recognition and state memory according to claim 1, it is characterized in that: The BERT pre-training model is constructed by layering using a Transformer encoder, which includes a multi-head self-attention layer, a residual connection and normalization layer, and a feedforward network layer; The self-attention calculation expression of the multi-head self-attention layer is: Where attention(Q,K,V) is self-attention, Q, K, V are query, key, and value respectively, and d k is the hidden layer dimension; The operational expression of the feedforward network layer is: max(0,xw1+b1)*w2+b2 Where x is the input tensor, w1, b1, w2, b2 are the learnable weight matrices and bias vectors.
4. The method for designing a multi-round dialogue engine based on intention recognition and state memory according to claim 1, characterized in that: In the model distillation process, the TinyBERT pre-trained model is used as the student model, and the BERT pre-trained model is used as the teacher model, wherein knowledge distillation is performed on the embedding layer, prediction layer, multi-head self-attention layer, and hidden layer of the TinyBERT pre-trained model.
5. The method for designing a multi-round dialogue engine based on intention recognition and state memory according to claim 4, characterized in that: The loss function of the model distillation process is the sum of the mean square error of the embedding layer, attention layer and hidden layer of the teacher model and the student model and the cross entropy of the predicted probability. The expression of the loss function is: Where, L model is the total loss, m is the number of layers of the student model, and a m is the weight of the mth layer, L layer is the loss function of the mth layer, L embedding is the mean square error of the embedding layer, L attention is the mean square error of the attention layer, L hidden is the mean square error of the hidden layer, L prediction is the prediction probability error.
6. The method for designing a multi-round dialogue engine based on intention recognition and state memory according to claim 1, characterized in that: The specific process of the multi-round dialogue engine workflow includes: Obtain the call content on the user side as user input, query the conversation state corresponding to the call ID from the Redis database, if the call ID does not exist, initialize a conversation instance, if it exists, execute the next step, where the conversation state includes the global conversation turn, the current scene, and the current scene conversation turn; The intent recognition model is used to perform scene recognition on the incoming user input, and entity recognition is performed according to the entity hit rule under the scene recognition result to obtain entity information; If the global conversation round is not greater than the preset round, determine whether the current scene of the call ID is consistent with the identification scene input by the user this time. If so, execute the next step. If not, reset the current scene conversation round to 0 to achieve scene jump; If the global conversation round is greater than the preset round or the current scene of the call ID is consistent with the identification scene input by the user this time, the conversation round of the current scene is increased by one; Determine whether the target entity of the current scene dialogue round has been hit. If so, add one to the current dialogue round. If not, do nothing. Returns the corresponding dialogue script for the current scene and dialogue round, where the dialogue script is designed in various ways for each scene and each round.
7. The method for designing a multi-round dialogue engine based on intention recognition and state memory according to claim 6, characterized in that: The step of obtaining the entity hit rule includes: Use regular matching to hit entities from entity annotation data; The entity hit rules are summarized based on the occurrence frequency of the hit entities in each business scenario and the slot values corresponding to the hit entities.
8. The method for designing a multi-round dialogue engine based on intention recognition and state memory according to claim 6, characterized in that: The steps of extracting the abstract include: Inputting the call content on the user side into the intent recognition model to identify the global intent of the call content as a context constraint for summary extraction; Divide the call content into clauses according to punctuation marks and pauses, and input each clause into the intent recognition model to determine the contribution of each clause to the global intent. Sort the contributions, select clauses whose contributions are higher than the set threshold, and generate a summary.
9. The method for designing a multi-round dialogue engine based on intention recognition and state memory according to claim 1, characterized in that: Monitoring indicators include the number of interface calls, response time, status code, dialogue scenario distribution, and dialogue completion rate of each scenario.
10. The method for designing a multi-round dialogue engine based on intention recognition and state memory according to claim 1, characterized in that: The steps of forming an operation optimization closed loop of the multi-round dialogue engine include: According to the indicator monitoring results, optimize the speech and entity matching rules for scenarios where the dialogue completion rate is lower than the preset threshold, and update the label system, entity library and speech design in stages; For scenarios where the accuracy of intent recognition is lower than the preset threshold, error-prone data is summarized and input into the intent recognition model for iterative training, so as to drive multi-round iterative optimization of the dialogue engine based on operational results and form a closed loop.
