Method and system for building deep learning dialogue strategy model based on dialogue flow bootstrapping

Through the combination of dialogue flow bootstrapping and deep learning, a dialogue strategy model is built, which solves the problems of insufficient data and lack of flexibility in the dialogue system, and effectively handles complex dialogues and reduces costs.

CN114528387BActive Publication Date: 2024-11-19BEIJING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210074402.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-21
Publication Date
2024-11-19
Estimated Expiration
2042-01-21

AI Technical Summary

Technical Problem

The existing dialogue strategy model construction method of dialogue system has cold start problems caused by insufficient data and insufficient flexibility for complex dialogue tasks. The dialogue strategy model is expensive to maintain, making it difficult to deal with unexpected user replies and dialogue task jumps. .

Method used

Using the deep learning dialogue strategy model construction method based on dialogue flow bootstrapping, the dialogue flow state machine is obtained from the user's multi-dialogue round data through width-first search, and a dialogue sequence labeling data covering the entire dialogue path is constructed, combined with the pre-trained Transformer The language model predicts dialogue action and determines the target dialogue action through confidence thresholds and probability distributions, and interactively corrects and expands the model to deal with complex dialogues.

Benefits of technology

It significantly improves the flexibility of the dialogue strategy model and its ability to deal with complex dialogues, solves the problem of insufficient model training data, reduces the maintenance cost of the dialogue system, and improves the robustness and scalability of the dialogue system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114528387B_ABST
    Figure CN114528387B_ABST
Patent Text Reader

Abstract

The present invention proposes a method and system for building a deep learning dialogue strategy model based on dialogue flow bootstrapping, wherein the method includes: building a corpus training set by dialogue flow bootstrapping, wherein the dialogue flow bootstrapping method includes: obtaining a dialogue flow state machine based on the user's multi-dialogue turn dialogue data, using breadth-first search to obtain all paths from the starting state to any state in the dialogue flow state machine, and then obtaining dialogue sequence annotation data of multiple dialogue turns, wherein the dialogue sequence annotation data of multiple dialogue turns constitutes the corpus training set; building a deep learning dialogue strategy model, inputting the corpus training set into the deep learning dialogue strategy model, and outputting the target dialogue action, wherein the deep learning dialogue strategy model includes a dialogue action prediction model and a confidence comparison model. The method according to the present invention solves the problem of model cold start caused by insufficient model training data in the prior art.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data processing of dialogue systems, and in particular to a method and system for constructing a deep learning dialogue strategy model based on dialogue flow bootstrapping. Background Art

[0002] The rapid development of big data and artificial intelligence has enabled natural language processing technology. With the popularization of smart devices, human-computer interaction technology has a wealth of application scenarios and urgent industrial needs. Dialogue systems have attracted more and more attention in academia and industry. The earliest dialogue systems could only handle simple communication tasks, such as booking airline tickets and providing travel information. Today, dialogue task scenarios are becoming increasingly complex, such as intelligent customer service, voice assistants, smart homes, etc., which have higher requirements for the flexibility and interactivity of dialogue systems. Therefore, research on how to build more flexible and scalable dialogue systems at a lower cost has broad application prospects.

[0003] Currently, more mature dialogue systems often use pipeline-based solutions, with a basic structure such as Figure 1 As shown. The dialogue strategy module is responsible for making decisions on dialogue actions to determine the next action of the dialogue system after receiving the user's reply. The current mainstream dialogue strategy model construction methods for the dialogue strategy module include dialogue strategy models based on dialogue flow and dialogue strategy models based on deep learning. Among them, (1) dialogue strategy model based on dialogue flow: This method models the dialogue process as a finite state machine. Each dialogue state contains a series of different dialogue actions. Entering a dialogue state will trigger the corresponding dialogue action. The state flow between dialogue states is determined by predefined conditions, such as identifying the user's specific intention or the specific entity contained in the user's reply. This method relies on a predefined series of rules, and the output dialogue action is deterministic. (2) dialogue strategy model based on deep learning: This method is based on the dialogue state of the current dialogue turn, that is, the current user intention and the entity contained in the user's reply, as well as the historical state. The deep sequence model is used to train on the labeled dialogue sequence data to obtain the ability to predict the next most likely dialogue action. This method relies on the available labeled dialogue sequence data. By modeling the statistical relationship between the dialogue state and the dialogue action, the predicted dialogue action is uncertain.

[0004] However, the above two dialogue strategy model construction methods still have the following problems: method (1) requires the definition of a strict state machine, which has high manpower maintenance costs. Dialogue strategy model developers need to consider complex boundary conditions and extreme cases, and take them into comprehensive consideration when designing the state machine. After a long period of iteration, the number of state branches will lead to an increase in model complexity. Without affecting the correctness of the state machine, the modification of the old dialogue branch requires clarifying the complex state flow conditions in the state machine, which is not conducive to the addition of new dialogue branches. Moreover, since the model relies heavily on established rules, it has no corresponding response capabilities for unexpected user replies or sudden dialogue task jumps. Given the long-tail characteristics of such dialogues, it is not realistic to explicitly encode all similar situations. Method (2) requires a sufficient amount of dialogue data. In specific fields, it is usually impossible to obtain sufficient labeled data in the early stage of building a dialogue system. Under the setting of few samples, the deep learning model cannot converge to an acceptable range. Therefore, data-driven deep learning methods often face the problem of cold start. Summary of the invention

[0005] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0006] To this end, the first purpose of the present invention is to propose a deep learning dialogue strategy model construction method based on dialogue flow bootstrapping to solve the problem of model cold start caused by insufficient model training data in the prior art.

[0007] The second objective of the present invention is to propose a deep learning dialogue strategy model construction system based on dialogue flow bootstrapping.

[0008] A third objective of the present invention is to provide an electronic device.

[0009] A fourth object of the present invention is to provide a non-transitory computer-readable storage medium.

[0010] To achieve the above-mentioned purpose, the first embodiment of the present invention proposes a method for constructing a deep learning dialogue strategy model based on dialogue flow bootstrapping, comprising:

[0011] Constructing a corpus training set by a dialogue flow bootstrapping method, wherein the dialogue flow bootstrapping method includes: obtaining a dialogue flow state machine based on the user's multi-dialogue turn dialogue data, using breadth-first search to obtain all paths from a start state to any state in the dialogue flow state machine, and then obtaining dialogue sequence annotation data of the multi-dialogue turn, wherein the dialogue sequence annotation data of the multi-dialogue turn constitutes the corpus training set;

[0012] A deep learning dialogue strategy model is constructed, the corpus training set is input into the deep learning dialogue strategy model, and a target dialogue action is output, wherein the deep learning dialogue strategy model includes a dialogue action prediction model and a confidence comparison model, the dialogue action prediction model is used to predict dialogue actions for the corpus training set, the confidence comparison model is used to obtain the confidence of the predicted action, and the target dialogue action is determined based on the confidence threshold and the confidence of the predicted action.

[0013] The method of the embodiment of the present invention utilizes breadth-first search and a dialog flow state machine to convert dialog sequence annotation data covering the entire dialog path from the user's multi-dialogue turn dialog data in a dialog flow bootstrapping manner, so as to fully model the statistical relationship between dialog states and dialog actions, significantly improve the flexibility of the dialog strategy model, and solve the cold start problem of the deep learning dialog strategy model caused by insufficient model training data.

[0014] In a method for constructing a deep learning dialogue strategy model based on dialogue flow bootstrapping according to an embodiment of the first aspect of the present invention, determining a target dialogue action based on a confidence threshold and a confidence of a predicted action includes:

[0015] Compare the confidence threshold and the confidence of the predicted action. If the confidence threshold is less than or equal to the confidence of the predicted action, the predicted action is the target dialogue action. If the confidence threshold is greater than the confidence of the predicted action, recall the dialogue sequence annotation data to obtain the recalled dialogue action, and determine the target dialogue action based on the recalled dialogue action and the probability distribution of the recalled dialogue action.

[0016] In a method for constructing a deep learning dialogue strategy model based on dialogue flow bootstrapping according to an embodiment of the first aspect of the present invention, the step of recalling the dialogue sequence labeled data to obtain recalled dialogue actions, and determining a target dialogue action based on the recalled dialogue actions and a probability distribution of the recalled dialogue actions includes:

[0017] Based on the recall method, for the intent, slot value, action of the previous dialogue round and user reply information in the current dialogue round, fuzzy retrieval is performed on the dialogue sequence annotation data to obtain multiple dialogue actions, and the retrieved dialogue actions are used as the recalled dialogue actions;

[0018] Obtain the probability distribution of the predicted actions in the deep learning dialogue strategy model corresponding to the recalled dialogue action, sort them, and select the recalled dialogue action corresponding to the predicted action with the highest probability as the target dialogue action.

[0019] In a method for constructing a deep learning dialogue strategy model based on dialogue flow bootstrapping in an embodiment of the first aspect of the present invention, it also includes: judging whether the predicted action in the deep learning dialogue strategy model is wrong, and if so, inputting the correct action in the predefined actions, generating a new dialogue data training set based on the multi-dialogue turn dialogue data and the correct action, and retraining the deep learning dialogue strategy model using the new dialogue data training set.

[0020] In a method for constructing a deep learning dialogue strategy model based on dialogue flow bootstrapping according to an embodiment of the first aspect of the present invention, the method for acquiring multi-dialogue turn dialogue data of a user includes:

[0021] Perform natural language understanding on user input sentences during multiple dialogue rounds to obtain user intent and slot values;

[0022] The user intention and slot value are tracked in the dialog state to obtain multi-dialogue turn dialog data, wherein the multi-dialogue turn dialog data includes user intention, slot value, action, state flow condition, current dialog state, historical dialog state and user reply.

[0023] In a method for constructing a deep learning dialogue strategy model based on dialogue flow bootstrapping according to an embodiment of the first aspect of the present invention, the method further includes:

[0024] Determine whether the user intent and slot value are incorrect. If so, input the correct intent and slot value from the predefined intent and slot value, generate new training samples based on the correct intent and slot value, and use the new training samples to retrain the natural language understanding model for natural language understanding.

[0025] In a method for constructing a deep learning dialogue strategy model based on dialogue flow bootstrapping in an embodiment of the first aspect of the present invention, the dialogue action prediction model includes a pre-trained Transformer language model.

[0026] To achieve the above-mentioned purpose, the second embodiment of the present invention proposes a deep learning dialogue strategy model construction system based on dialogue flow bootstrapping, including:

[0027] an acquisition module, configured to construct a corpus training set by a dialogue flow bootstrapping method, wherein the dialogue flow bootstrapping method comprises: obtaining a dialogue flow state machine based on the user's multi-dialogue turn dialogue data, using a breadth-first search to obtain all paths from a start state to any state in the dialogue flow state machine, and then obtaining dialogue sequence annotation data of the multi-dialogue turn, wherein the dialogue sequence annotation data of the multi-dialogue turn constitutes the corpus training set;

[0028] A prediction module is used to build a deep learning dialogue strategy model, input the corpus training set into the deep learning dialogue strategy model, and output a target dialogue action, wherein the deep learning dialogue strategy model includes a dialogue action prediction model and a confidence comparison model, the dialogue action prediction model is used to predict dialogue actions for the corpus training set, the confidence comparison model is used to obtain the confidence of the predicted action, and determine the target dialogue action based on the confidence threshold and the confidence of the predicted action.

[0029] In the system of the embodiment of the present invention, the acquisition module uses breadth-first search and dialogue flow state machine to transform the user's multi-dialogue turn dialogue data into dialogue sequence annotation data covering the entire dialogue path in a dialogue flow bootstrap manner, so as to fully model the statistical relationship between dialogue state and dialogue action, significantly improve the flexibility of the dialogue strategy model, and solve the cold start problem of the deep learning dialogue strategy model caused by insufficient model training data in the prediction module.

[0030] To achieve the above-mentioned purpose, the third aspect of the present invention proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a method for constructing a deep learning dialogue strategy model based on dialogue flow bootstrapping according to the first aspect of the present invention.

[0031] In order to achieve the above-mentioned objectives, the fourth aspect of the present invention proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute a deep learning dialogue strategy model construction method based on dialogue flow bootstrapping according to the first aspect of the present invention.

[0032] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0034] Figure 1 This is the structure diagram of the existing pipeline-based dialogue system;

[0035] Figure 2 A flowchart of a method for building a deep learning dialogue strategy model based on dialogue flow bootstrapping provided by an embodiment of the present invention;

[0036] Figure 3A flowchart of another method for building a deep learning dialogue strategy model based on dialogue flow bootstrapping provided by an embodiment of the present invention;

[0037] Figure 4 A schematic diagram of a dialog flow state machine provided by an embodiment of the present invention;

[0038] Figure 5 A schematic diagram of annotated data of a conversation sequence of multiple conversation turns provided by an embodiment of the present invention;

[0039] Figure 6 An architectural diagram of a pre-trained Transformer language model provided in an embodiment of the present invention;

[0040] Figure 7 A method for predicting dialogue actions in a dialogue system provided by an embodiment of the present invention;

[0041] Figure 8 A model updating method for a dialogue system provided by an embodiment of the present invention;

[0042] Fig. 9 A schematic diagram of the structure of a deep learning dialogue strategy model construction system based on dialogue flow bootstrapping provided by an embodiment of the present invention;

[0043] Fig.10 An architecture diagram of a system for building a deep learning dialogue strategy model based on dialogue flow bootstrapping provided by an embodiment of the present invention;

[0044] Fig.11 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limitations on the present invention. Any process or method description in the flow chart or otherwise described herein may be understood to represent a module, fragment or portion of a code including one or more executable instructions for implementing the steps of a custom logic function or process, and the scope of the preferred embodiments of the present invention includes additional implementations, in which functions may be performed in a substantially simultaneous manner or in reverse order according to the functions involved, which should be understood by a person skilled in the art of the art to which the embodiments of the present invention belong.

[0046] The present invention is described in detail below with reference to specific embodiments.

[0047] Figure 2 A flowchart of a method for building a deep learning dialogue strategy model based on dialogue flow bootstrapping provided by an embodiment of the present invention.

[0048] The embodiment of the present invention provides a method for constructing a deep learning dialogue strategy model based on dialogue flow bootstrapping to solve the problem that the dialogue strategy model based on dialogue flow in the prior art lacks flexibility in dealing with complex dialogues, and the model cold start problem caused by insufficient model training data in the early stage of system construction. Figure 2 As shown, the method for building a deep learning dialogue strategy model based on dialogue flow bootstrapping includes the following steps:

[0049] Step S101, constructing a corpus training set by dialogue flow bootstrapping.

[0050] In step S101, the dialogue flow bootstrapping method may include: obtaining a dialogue flow state machine based on the user's multi-dialogue turn dialogue data, using breadth-first search to obtain all paths from the starting state to any state in the dialogue flow state machine, and then obtaining dialogue sequence annotation data of multiple dialogue turns, and the dialogue sequence annotation data of multiple dialogue turns constitutes a corpus training set.

[0051] In step S101, the method for acquiring user's multi-dialogue turn dialogue data may include: performing natural language understanding on user input sentences in the multi-dialogue turn process to obtain user intention and slot value; performing dialogue state tracking on user intention and slot value to obtain multi-dialogue turn dialogue data, wherein the multi-dialogue turn dialogue data includes user intention, slot value, action, state flow condition, current dialogue state, historical dialogue state and user's reply. The user input sentence may be, for example, Figure 1 Specifically, Figure 1 As shown, the natural language is input into the natural language understanding module for natural language understanding to obtain the user intent and slot value, and then the user intent and slot value are transmitted to the dialogue management module for dialogue state tracking (DST) to obtain the user intent (intent), slot value (slot), action (action), state flow condition (condition), current dialogue state, historical dialogue state and user's reply (utterance).

[0052] It is easy to understand that a dialog flow state machine usually consists of a start state, an intermediate state, an end state, dialog actions, and state transition conditions. Figure 4 Schematic diagram of a dialog flow state machine provided by an embodiment of the present invention. Figure 4 As shown, S0 is the starting state, S i , S j For different intermediate states, S e is the terminal state, conditioni 、condition j 、condition k 、condition l For the state transition conditions between different states, action0, action o 、action n 、action m is the corresponding dialogue action in different states, where the number of intermediate states can be limited to Figure 4 As shown in the figure, the number of state transition conditions is related to the number of states, and the number of dialogue actions in each state may be the same or different. Figure 4 As shown, starting from the initial state S0, the dialogue actions therein are executed. When the user responds, the state transition condition is followed to transfer to the next state and the dialogue actions therein are executed until the termination state S e Since the dialog flow state machine is usually a directed acyclic graph, the dialog flow bootstrapping method in step S101 can use the path traversal of breadth-first search to obtain all possible paths from the starting state to any state in the dialog flow state machine. The time complexity of the dialog flow bootstrapping method in this embodiment is O(n*2 n ). Considering that the number of dialog flow states (i.e., multi-dialog turn dialog data) created during cold start is relatively small, the time complexity of this embodiment is an acceptable time cost.

[0053] In step S101, the algorithm for obtaining the dialog flow bootstrapping method of all paths (i.e., the dialog flow conversion algorithm) is described as follows:

[0054]

[0055] The input of the algorithm is the starting state S0 and the state list States, and the output is the directed acyclic graph traversal walk (i.e., traversal path) exhaustive_walks. The state list States in the algorithm includes all states in the dialog flow state machine, and any state in the state list States is represented by S i The algorithm first initializes the exhaustive walks to an empty list. Then for any state S in the state list States i , the following process is performed: initialize an empty queue Q, add the initial state S0 to the queue, and initialize the current walk current_walk; continuously pop the head element of the queue to update the current walk. If the current head element υ is not equal to the current element S i , then add the current walk to the traversal walk, and perform the queue operation for all neighbors of the current head element υ, and continue to perform the above operations until the queue is empty, indicating that the traversal from Si Wandering to all other states.

[0056] In step S101, the user's multi-dialogue round (can be referred to as multi-round) dialogue data can obtain the dialogue sequence annotation data of the multi-dialogue round through the above-mentioned dialogue flow bootstrapping algorithm, wherein each time step (i.e., each dialogue round) in the dialogue sequence annotation data contains the user's intent (intent), slot value (slot), and user's reply (utterance) of the current dialogue round. In addition, in the dialogue sequence annotation data of each round (can be referred to as each round sequence), the front end of the sequence also includes a vector MEM character, which is used to represent the vector of the current dialogue context (i.e., the previous and next rounds of the current round). Figure 5 A schematic diagram of annotated data of a conversation sequence of multiple conversation turns provided by an embodiment of the present invention. Figure 5 Turn i is the current round, then Turn i-1 The previous round of the current round, Turn i-2 For Turn i-1 The previous round, Turn i+1 For the next round of the current round, Figure 5 Turn i The turn dialogue sequence annotation data includes MEM characters, user's reply (utterance), user's intention (intent) and slot value (slot), but the information included in the turn dialogue sequence annotation data in this embodiment is not limited to this. In addition, Turn i MEM character representation of turn dialogue sequence annotation data i-1 Round and Turn i+1 In this case, bootstrapping the corpus required for training the deep learning dialogue strategy model from the dialogue flow can solve the cold start problem of the deep learning dialogue strategy model.

[0057] Step S102: construct a deep learning dialogue strategy model, input the corpus training set into the deep learning dialogue strategy model, and output the target dialogue action.

[0058] In step S102, the deep learning dialogue strategy model includes a dialogue action prediction model and a confidence comparison model. The dialogue action prediction model is used to predict dialogue actions for a corpus training set, and the confidence comparison model is used to obtain the confidence of the predicted action, and determine the target dialogue action based on the confidence threshold and the confidence of the predicted action.

[0059] In step S102, the dialogue action prediction model includes a pre-trained Transformer language model. The pre-trained Transformer language model is obtained by training the encoding module of a conventional Transformer language model. Specifically, the encoding module of the conventional Transformer language model can be trained using a corpus training set (i.e., dialogue sequence annotation data of multiple dialogue turns) to obtain the pre-trained Transformer language model.

[0060] Figure 6 This is an architecture diagram of the pre-trained Transformer language model provided in an embodiment of the present invention. Figure 6 As shown in the figure, the encoding process of the input sequence in a single dialogue turn starts with the calculation of embedding. The embedding input to the model is the sum of word embedding, position embedding, etc. The calculation formula of position embedding is as follows:

[0061]

[0062]

[0063] Here, pos and i represent the position and the dimension of embedding respectively.

[0064] Then it goes through multiple layers of pre-transformation modules (Pretrained transformer). Each layer contains the following calculations. First, the weighted sum of the input is obtained through multi-head self-attention. The calculation formula is as follows:

[0065] MultiHead(QK,V)=Concat(head1,...,head h )W o

[0066]

[0067] The weight matrix mapped to each subspace is as well as The multi-head self-attention mechanism maps the input Q, K, and V matrices to different subspaces for self-attention calculations, and then rejoins the calculation results of several subspaces. The calculation formula for self-attention is as follows:

[0068]

[0069] The input matrices Q, K, and V of the self-attention calculation are all input sequences. By calculating the similarity of the characters at each position in the sequence, a weighted sum operation is performed on the original sequence to perform dynamic semantic encoding. To normalize the similarity scores and reduce the impact of the input dimension on the results.

[0070] After completing the multi-head self-attention calculation, the residual connection is added and the layer normalization is calculated. Then, it passes through the feed forward network bit by bit, and ReLU is used as the activation function. The expression is as follows:

[0071] FFN(x)=max(0,xW1+b1)W2+b2.

[0072] After calculation of the above-mentioned pre-trained Transformer language model (i.e., the encoding process), a vector representation of the characters at each position in the input sequence can be obtained.

[0073] In step S102, the dialogue action prediction model also includes a softmax layer set after the pre-trained Transformer language model, such as Figure 5 As shown in the figure, the dialogue sequence annotation data of multiple dialogue rounds is input into the pre-trained Transformer language model as the context of the text, and the softmax layer is used to predict the dialogue action of the next round at the current time step. When passing through the softmax layer, the output of the pre-trained Transformer language model can be converted into the predicted probability of the next action through the softmax function.

[0074] In step S102, after obtaining the predicted next action probability, the confidence comparison model can obtain the confidence of the predicted action, and determine the target dialogue action based on the confidence threshold and the confidence of the predicted action.

[0075] In step S102, determining the target dialogue action based on the confidence threshold and the confidence of the predicted action includes: comparing the confidence threshold and the confidence of the predicted action, if the confidence threshold is less than or equal to the confidence of the predicted action, the predicted action is the target dialogue action; if the confidence threshold is greater than the confidence of the predicted action, the dialogue sequence annotation data is recalled to obtain the recalled dialogue action, and the target dialogue action is determined based on the recalled dialogue action and the probability distribution of the recalled dialogue action. The target dialogue action is the final predicted action. In this case, the target dialogue action is determined by combining the model and the dialogue flow state machine, which increases the stability of the dialogue strategy.

[0076] In step S102, the dialogue sequence annotated data is recalled to obtain the recalled dialogue action, and the target dialogue action is determined based on the recalled dialogue action and the probability distribution of the recalled dialogue action, including: based on the recall method, for the intent, slot value, action of the previous dialogue turn and user reply information in the current dialogue turn, fuzzy retrieval is performed on the dialogue sequence annotated data to obtain multiple dialogue actions, and the retrieved dialogue actions are used as the recalled dialogue actions; the probability distribution of the predicted actions in the deep learning dialogue strategy model corresponding to the recalled dialogue actions is obtained, and sorted (for example, fine sorting) to select the recalled dialogue action corresponding to the predicted action with the highest probability as the target dialogue action. In this case, the robustness and flexibility of the dialogue action prediction are improved by combining the stability of the dialogue flow and the ability of the model to cope with complex dialogues.

[0077] Based on the above-mentioned method for constructing a deep learning dialogue strategy model based on dialogue flow bootstrapping, this embodiment also provides a method for predicting dialogue actions of a dialogue system. Figure 7 A method for predicting dialogue actions in a dialogue system is provided in an embodiment of the present invention. Figure 7 As shown in the figure, the dialogue system includes a natural language understanding module, a dialogue state tracker, and a dialogue strategy module. The specific process of the dialogue action prediction method includes the following four steps:

[0078] (1) The user inputs text and sends a request to the multi-round dialogue system (i.e., obtains the user input sentence). If the user input sentence is incorrect, it is re-entered. The natural language understanding module in the dialogue system understands the dialogue based on the user text and parses out information such as the user intent and slot value in the user input sentence.

[0079] (2) The dialogue state is tracked based on the parsed user intent and slot value information. The dialogue state tracking is divided into current dialogue state tracking and historical dialogue state tracking, including different dialogue state information such as intent, slot value, entity, domain information, action, etc., and the dialogue state is formed by combining current and historical data. The historical dialogue state is continuously updated through the current dialogue state and response action, and the current dialogue state is updated in response to the dialogue understanding data.

[0080] (3) The dialogue state obtained from the dialogue state tracker is used to predict dialogue actions in the dialogue strategy module. A neural network model (i.e., dialogue action prediction model) is used to predict dialogue actions, and the confidence in the action prediction is obtained to provide a comparison basis for the next action decision.

[0081] (4) The dialog flow sequence data obtained based on the dialog flow state machine is prepared, and the dialog state obtained in the dialog state tracker is queried and recalled in the dialog flow sequence data, returning multiple (top-k) actions. Based on the probability distribution of each action calculated when predicting the action in the deep learning dialog strategy model, the corresponding confidence of each action in the recalled top-k actions is obtained, and the recalled dialog action with the highest probability is selected from the top-k actions as the machine response action.

[0082] In this embodiment, considering that for complex dialogues and scenarios, directly maintaining and writing dialogue flows requires a large manpower cost in a production environment; for complex dialogue flows, modifying and adding dialogues while ensuring their correctness puts higher requirements on domain knowledge and familiarity with existing systems, which to a certain extent makes the rule-based dialogue system in complex scenarios lack flexibility and scalability. Therefore, this embodiment proposes another deep learning dialogue strategy model construction method based on dialogue flow bootstrapping to interactively correct and expand the deep learning dialogue strategy model.

[0083] Figure 3 A flowchart of another method for building a deep learning dialogue strategy model based on dialogue flow bootstrapping provided by an embodiment of the present invention. Figure 3 As shown, steps S101 and S102 in the method for building a deep learning dialogue strategy model based on dialogue flow bootstrapping can refer to the above Figure 2 The related description of the method shown in the figure, the method for constructing a deep learning dialogue strategy model based on dialogue flow bootstrapping also includes: step S103, judging whether the predicted action in the deep learning dialogue strategy model is wrong, if it is wrong, inputting the correct action in the predefined action, generating a new dialogue data training set based on the dialogue data of multiple dialogue turns and the correct action, and retraining the deep learning dialogue strategy model using the new dialogue data training set. In this way, the ability of the dialogue strategy model to cope with complex dialogues can be increased, while reducing the human maintenance cost of the dialogue system.

[0084] In some embodiments, whether the predicted action is wrong can be determined by human judgment or by the machine teaching module of the dialogue system.

[0085] In this embodiment, the method for constructing a deep learning dialogue strategy model based on dialogue flow bootstrapping also includes: judging whether the user's intention and slot value are incorrect, if so, inputting the correct intention and slot value in the predefined intention and slot value, generating new training samples based on the correct intention and slot value, and retraining the natural language understanding model for natural language understanding using the new training samples. In this way, the ability of the dialogue strategy model to cope with complex dialogues can be further enhanced.

[0086] In some embodiments, whether the user intention and the slot value are incorrect can be determined manually or by a machine teaching module of the dialogue system.

[0087] Based on the above-mentioned method for building a deep learning dialogue strategy model based on dialogue flow bootstrapping, this embodiment also provides a model updating method for a dialogue system. Figure 8 A model updating method for a dialogue system provided by an embodiment of the present invention; Figure 8 As shown in FIG. 1 , the dialogue system includes a natural language understanding module, a dialogue state tracker, a dialogue strategy module, and a machine teaching module. The specific process of the model updating method of the dialogue system (i.e., the interactive machine teaching process) may include the following four steps:

[0088] (1) The user first initiates an interactive dialogue process through the machine teaching module interface, in which the corresponding dialogue content is entered. The dialogue content will confirm with the user whether the input is incorrect. If it is incorrect, the user can re-enter the content, and then call the natural language understanding module to predict the intent and fill in the slot value. The results and the confidence of each prediction will be directly returned to the user for confirmation.

[0089] (2) If the user confirms that there is no problem with the intent prediction and slot value filling, the next step will be entered. Otherwise, the user can submit the correct intent and slot value through the machine teaching module. The intent must be a predefined intent, and the slot value must be a slot value predefined in the field. The user enters it in a specific format, and the slot value is required to be a continuous subsequence of the user input. If it is verified that the intent and slot value entered by the user do not meet the rules, they need to be re-entered. After the user enters the correct intent and slot value, the machine teaching module will annotate the user input, generate new samples, and initiate the natural language understanding model training task after the machine teaching process is completed, and retrain the model for the new samples.

[0090] (3) If the user confirms that the prediction result of the natural language understanding module is correct, or the natural language understanding module has been corrected, the dialogue state tracking module will use the corrected intent and slot value historical dialogue state provided by the user, where the acquisition of the current dialogue state and the historical dialogue state will be retried in case of errors. The updated dialogue state will be sent to the deep learning dialogue strategy model for the neural network to predict dialogue actions. This step will show the user the confidence distribution of the action prediction result, and the user will decide whether the action prediction is incorrect. If the user confirms that it is correct, the current dialogue action will be executed and the next dialogue round will be entered.

[0091] (4) If the user believes that the action prediction result is wrong, the correct next action can be provided. The action must be one of the predefined actions in the library. If the machine teaching module does not find the correct action entered by the user, an exception will be thrown and the user will be asked to re-enter. If the machine teaching module finds the correct action entered by the user, the dialogue system will start to correct the dialogue record by the machine teaching module to generate a new sample including dialogue annotation sequence data. The user can continue the machine teaching process at this time. After exiting, the deep learning dialogue strategy model training task will be initiated to retrain the model on the new sample.

[0092] The method for building a deep learning dialogue strategy model based on dialogue flow bootstrapping proposed in the embodiment of the present invention aims at the fact that the existing dialogue strategy model building method based on dialogue flow cannot cope with complex scene dialogues, unexpected user inputs, and dialogue task conversion. The deep learning dialogue strategy model method based on dialogue flow bootstrapping adopts the method of deep learning dialogue strategy model based on dialogue flow bootstrapping, and uses breadth-first search and dialogue flow state machine to transform dialogue sequence annotation data covering the entire dialogue path from the user's multi-dialogue turn dialogue data in a dialogue flow bootstrapping manner, so as to fully model the statistical relationship between dialogue state and dialogue action, significantly improve the flexibility of the dialogue strategy model, and solve the cold start problem of the deep learning dialogue strategy model caused by insufficient model training data. In addition, in view of the fact that the existing method is difficult to add new dialogue branches and modify existing branches, and the developer needs to consider the complicated correlation relationship between dialogue states and ensure that the changes will not affect the correctness of the existing state machine, the model building method of the present invention interactively modifies and expands the dialogue system (especially the deep learning dialogue strategy model), increases the ability of the dialogue strategy model to cope with complex dialogues, and can significantly reduce the manpower cost of maintaining the dialogue system. The model building method of the present invention is a dialogue strategy model building method for low-resource scenarios, combining dialogue flow and deep learning models, and has low maintenance costs.

[0093] In order to implement the above embodiments, the present invention also proposes a deep learning dialogue strategy model construction system based on dialogue flow bootstrapping. Fig. 9 A schematic diagram of the structure of a deep learning dialogue strategy model construction system based on dialogue flow bootstrapping provided by an embodiment of the present invention, Fig.10 An architectural diagram of a system for building a deep learning dialogue strategy model based on dialogue flow bootstrapping provided in an embodiment of the present invention.

[0094] like Fig. 9 As shown, the deep learning dialogue strategy model construction system 10 based on dialogue flow bootstrapping (referred to as system 10) includes an acquisition module 11 and a prediction module 12.

[0095] In this embodiment, the acquisition module 11 can be used to construct a corpus training set by a dialogue flow bootstrap method, which includes: obtaining a dialogue flow state machine based on the user's multi-dialogue turn dialogue data, using breadth-first search to obtain all paths from the start state to any state in the dialogue flow state machine, and then obtaining dialogue sequence annotation data of multiple dialogue turns, and the dialogue sequence annotation data of multiple dialogue turns constitutes the corpus training set. In this way, the cold start problem of the deep learning dialogue strategy model can be solved. Fig. 9 The acquisition module 11 may include Fig.10 The dialog flow conversion module in .

[0096] In some other embodiments, the acquisition module 11 may also include a dialog state tracking module.

[0097] In this embodiment, the prediction module 12 can be used to construct a deep learning dialogue strategy model, input the corpus training set into the deep learning dialogue strategy model, and output the target dialogue action, wherein the deep learning dialogue strategy model includes a dialogue action prediction model and a confidence comparison model, the dialogue action prediction model is used to predict dialogue actions for the corpus training set, and the confidence comparison model is used to obtain the confidence of the predicted action, and determine the target dialogue action based on the confidence threshold and the confidence of the predicted action. Fig. 9 The prediction module 12 in may correspond to Fig.10 The dialogue policy model module in .

[0098] In some other embodiments, the deep learning dialogue strategy model construction system 10 based on dialogue flow bootstrapping may further include a machine teaching module. The machine teaching module can interactively modify and expand the deep learning dialogue strategy model to reduce manual maintenance costs.

[0099] The machine teaching module can be used to determine whether the predicted action in the deep learning dialogue strategy model is wrong. If it is wrong, the correct action in the predefined actions is input, and a new dialogue data training set is generated based on the dialogue data of multiple dialogue rounds and the correct action. The deep learning dialogue strategy model is retrained using the new dialogue data training set. In this way, the ability of the dialogue strategy model to cope with complex dialogues can be increased, while reducing the human maintenance cost of the dialogue system.

[0100] In addition, the machine teaching module can also determine whether the user's intention and slot value are incorrect. If so, the correct intention and slot value in the predefined intention and slot value are input, and new training samples are generated based on the correct intention and slot value. The natural language understanding model for natural language understanding is retrained using the new training samples. In this way, the ability of the dialogue strategy model to cope with complex dialogues can be further enhanced.

[0101] The deep learning dialogue strategy model construction system based on dialogue flow bootstrapping proposed in the embodiment of the present invention adopts a deep learning dialogue strategy model method based on dialogue flow bootstrapping to address the situation that the existing dialogue strategy model construction method based on dialogue flow cannot cope with complex scene dialogues, unexpected user input, and dialogue task conversion. The acquisition module uses breadth-first search and dialogue flow state machine to transform dialogue sequence annotation data covering the entire dialogue path from the user's multi-dialogue turn dialogue data in a dialogue flow bootstrapping manner to fully model the statistical relationship between dialogue state and dialogue action, significantly improve the flexibility of the dialogue strategy model, and solve the cold start problem of the deep learning dialogue strategy model caused by insufficient model training data. In addition, in view of the fact that the existing method is difficult to add new dialogue branches and modify existing branches, and the developer needs to consider the complicated correlation between dialogue states and ensure that the changes will not affect the correctness of the existing state machine, the model construction method of the present invention uses a machine teaching module to interactively modify and expand the dialogue system (especially the deep learning dialogue strategy model), thereby increasing the ability of the dialogue strategy model to cope with complex dialogues and significantly reducing the labor cost of maintaining the dialogue system.

[0102] It should be noted that the aforementioned explanation of the embodiment of the method for building a deep learning dialogue strategy model based on dialogue flow bootstrapping is also applicable to the system for building a deep learning dialogue strategy model based on dialogue flow bootstrapping of this embodiment, and will not be repeated here.

[0103] According to an embodiment of the present invention, the present invention also provides an electronic device, a readable storage medium and a computer program product.

[0104] Fig.11 A schematic diagram of the structure of an electronic device provided for an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable electronic devices, and other similar computing devices. The components, connections and relationships of the components, and functions of the components shown in the present invention are merely examples and are not intended to limit the implementation of the present invention described and / or required in the present invention.

[0105] like Fig.11As shown, the electronic device 20 includes a computing unit 21, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 22 or a computer program loaded from a storage unit 28 into a random access memory (RAM) 23. In the RAM 23, various programs and data required for the operation of the electronic device 20 can also be stored. The computing unit 21, the ROM 22, and the RAM 23 are connected to each other via a bus 24. An input / output (I / O) interface 25 is also connected to the bus 24.

[0106] A number of components in the electronic device 20 are connected to the I / O interface 25, including: an input unit 26, such as a keyboard, a mouse, etc.; an output unit 27, such as various types of displays, speakers, etc.; a storage unit 28, such as a disk, an optical disk, etc., which is communicatively connected to the computing unit 21; and a communication unit 29, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 29 allows the electronic device 20 to exchange information / data with other electronic devices through a computer network such as the Internet and / or various telecommunication networks.

[0107] The computing unit 21 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 21 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 21 performs the various methods and processes described above, such as executing a deep learning dialogue strategy model construction method based on dialogue flow bootstrapping. For example, in some embodiments, the execution of a deep learning dialogue strategy model construction method based on dialogue flow bootstrapping may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 28. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 20 via the ROM 22 and / or the communication unit 29. When the computer program is loaded into the RAM 23 and executed by the computing unit 21, one or more steps of the execution of the deep learning dialogue strategy model construction method based on dialogue flow bootstrapping described above may be performed. Alternatively, in other embodiments, the computing unit 21 may be configured in any other appropriate manner (e.g., by means of firmware) to execute the deep learning dialog strategy model construction method based on dialog flow bootstrapping.

[0108] Various embodiments of the systems and techniques described above in the present invention can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0109] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0110] In the present invention, machine-readable medium can be a tangible medium that can contain or store a program for use by or in combination with an instruction execution system, device or electronic device. Machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or electronic devices, or any suitable combination of the above. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage electronic devices, magnetic storage electronic devices, or any suitable combination of the above.

[0111] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0112] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0113] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0114] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in the present disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present disclosure can be achieved, and the present invention is not limited here.

[0115] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for building a deep learning dialogue strategy model based on dialogue flow bootstrapping, It is characterized in that include: Constructing a corpus training set by a dialogue flow bootstrapping method, wherein the dialogue flow bootstrapping method includes: obtaining a dialogue flow state machine based on the user's multi-dialogue turn dialogue data, using breadth-first search to obtain all paths from a start state to any state in the dialogue flow state machine, and then obtaining dialogue sequence annotation data of the multi-dialogue turn, wherein the dialogue sequence annotation data of the multi-dialogue turn constitutes the corpus training set; Constructing a deep learning dialogue strategy model, inputting the corpus training set into the deep learning dialogue strategy model, and outputting a target dialogue action, wherein the deep learning dialogue strategy model includes a dialogue action prediction model and a confidence comparison model, the dialogue action prediction model is used to predict dialogue actions for the corpus training set, the confidence comparison model is used to obtain the confidence of the predicted action, and the target dialogue action is determined based on the confidence threshold and the confidence of the predicted action; Determine whether the predicted action in the deep learning dialogue strategy model is wrong. If so, input the correct action in the predefined action, generate a new dialogue data training set based on the multi-dialogue turn dialogue data and the correct action, and retrain the deep learning dialogue strategy model using the new dialogue data training set; wherein the method for acquiring multi-dialogue turn dialogue data of the user includes: performing natural language understanding on the user input sentence in the multi-dialogue turn process to obtain the user intention and slot value; performing dialogue state tracking on the user intention and slot value to obtain multi-dialogue turn dialogue data, the multi-dialogue turn dialogue data includes user intention, slot value, action, state flow condition, current dialogue state, historical dialogue state and user reply; and also includes determining whether the user intention and slot value are wrong. If so, input the correct intention and slot value in the predefined intention and slot value, generate a new training sample based on the correct intention and slot value, and retrain the natural language understanding model for natural language understanding using the new training sample.

2. The method according to claim 1, It is characterized in that The step of determining the target dialogue action based on the confidence threshold and the confidence of the predicted action includes: Compare the confidence threshold and the confidence of the predicted action. If the confidence threshold is less than or equal to the confidence of the predicted action, the predicted action is the target dialogue action. If the confidence threshold is greater than the confidence of the predicted action, recall the dialogue sequence annotation data to obtain the recalled dialogue action, and determine the target dialogue action based on the recalled dialogue action and the probability distribution of the recalled dialogue action.

3. The method according to claim 2, It is characterized in that The step of recalling the dialogue sequence labeled data to obtain recalled dialogue actions, and determining target dialogue actions based on the recalled dialogue actions and probability distribution of the recalled dialogue actions includes: Based on the recall method, for the intent, slot value, action of the previous dialogue round and user reply information in the current dialogue round, fuzzy retrieval is performed on the dialogue sequence annotation data to obtain multiple dialogue actions, and the retrieved dialogue actions are used as the recalled dialogue actions; Obtain the probability distribution of the predicted actions in the deep learning dialogue strategy model corresponding to the recalled dialogue action, sort them, and select the recalled dialogue action corresponding to the predicted action with the highest probability as the target dialogue action.

4. The method according to claim 1, It is characterized in that The dialogue action prediction model includes a pre-trained Transformer language model.

5. A deep learning dialogue strategy model construction system based on dialogue flow bootstrapping, It is characterized in that include: an acquisition module, configured to construct a corpus training set by a dialogue flow bootstrapping method, wherein the dialogue flow bootstrapping method comprises: obtaining a dialogue flow state machine based on the user's multi-dialogue turn dialogue data, using a breadth-first search to obtain all paths from a start state to any state in the dialogue flow state machine, and then obtaining dialogue sequence annotation data of the multi-dialogue turn, wherein the dialogue sequence annotation data of the multi-dialogue turn constitutes the corpus training set; A prediction module, used to construct a deep learning dialogue strategy model, input the corpus training set into the deep learning dialogue strategy model, and output a target dialogue action, wherein the deep learning dialogue strategy model includes a dialogue action prediction model and a confidence comparison model, the dialogue action prediction model is used to predict dialogue actions for the corpus training set, the confidence comparison model is used to obtain the confidence of the predicted action, and determine the target dialogue action based on the confidence threshold and the confidence of the predicted action; The machine teaching module is used to determine whether the predicted action in the deep learning dialogue strategy model is wrong. If so, input the correct action in the predefined action, generate a new dialogue data training set based on the multi-dialogue turn dialogue data and the correct action, and retrain the deep learning dialogue strategy model using the new dialogue data training set, wherein the method for obtaining the multi-dialogue turn dialogue data of the user includes: performing natural language understanding on the user input sentence in the multi-dialogue turn process to obtain the user intention and slot value; performing dialogue state tracking on the user intention and slot value to obtain the multi-dialogue turn dialogue data, and the multi-dialogue turn dialogue data includes user intention, slot value, action, state flow condition, current dialogue state, historical dialogue state and user reply; the machine teaching module is also used to determine whether the user intention and slot value are wrong. If so, input the correct intention and slot value in the predefined intention and slot value, generate a new training sample based on the correct intention and slot value, and retrain the natural language understanding model for natural language understanding using the new training sample.

6. An electronic device, It is characterized in that include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium storing computer instructions, in, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-4.