A task-based dialogue management method and system capable of automatically generating dialogue strategies

Abstract

The present invention relates to a task-based dialogue management method capable of automatically generating dialogue strategies, comprising: obtaining user voice input, obtaining voice input recognition text through a voice recognition module, inputting it into a natural language understanding module, and identifying user intention; a dialogue state tracking module obtains the current dialogue state according to the user intention, and generates a dialogue process tree through a dialogue strategy generation model; the dialogue strategy corresponding to each node of the dialogue process tree is sent to a natural language generation module to generate a corresponding speech text; the speech text is transmitted to a speech synthesis module, and the synthesized speech is transmitted to the user, wherein the above-mentioned dialogue strategy generation model realizes the automatic configuration of the dialogue process tree through a self-learning mechanism based on reinforcement learning.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a dialogue management system. Background Art

[0002] As the basic technology for information exchange between humans and computers in the information age, human-computer interaction has received widespread attention from academia and industry. In recent years, the field of human-computer interaction has developed rapidly, and many products have emerged, including voice assistants, smart speakers, and voice robots. There are currently two main types of dialogue systems: task-based dialogue management systems and chat-based dialogue systems. Task-based dialogue management systems focus on completing tasks and use as few dialogue rounds as possible to achieve their goals. They are mainly aimed at vertical fields, such as: air ticket reservations, hotel reservations, movie ticket reservations, etc. There are two common task-based dialogue management system solutions:

[0003] (1) Pipeline method. A complete task-based dialogue management system built by the pipeline method mainly consists of four parts: speech recognition module (ASR), semantic understanding module (NLU), dialogue management module (DM), natural language generation module (NLG), and speech synthesis module (TTS). The dialogue management module can be further divided into dialogue state representation and dialogue strategy learning. The pipeline method has strong domain relevance. Each domain requires the design of semantic slots, action spaces, and decision rules, which makes it difficult to expand to new domains. At the same time, the modules are independent of each other, and a large amount of labeled data needs to be provided for each module for model training. The models are dependent on each other, which causes the errors of the upstream modules to cascade to the downstream modules, resulting in large errors and unclear error sources.

[0004] (2) End-to-end method. This method directly integrates semantic understanding, dialogue management, and text generation. The text after speech recognition is used as the input of the end-to-end model. The model outputs the user response text, which is sent to the user after speech synthesis. The end-to-end method has the advantages of simple structure and easy portability. It also solves the problem that multiple modules require a large amount of labeled data and greatly optimizes the complex internal design of voice interaction. However, the text synthesized by end-to-end technology is poor in semantics and logic, and the effect depends on the knowledge base. It has the disadvantages of low training efficiency, poor user experience, and uncontrollability.

[0005] At present, there are many problems with task-based dialogue management systems: 1. Poor scalability. Models need to be redeveloped and strategies need to be formulated for different scenarios, especially for building task-based dialogue systems with pipeline methods. 2. Poor user experience. Since user questions are uncontrollable, system definitions and rules cannot be fully covered, resulting in irrelevant answers or ambiguous replies. 3. Difficult maintenance. Task-based dialogue management has different speech specifications for different scenarios and tasks, requiring cross-departmental collaborative maintenance by R&D, products, and operations. The maintenance process is cumbersome and difficult. Summary of the invention

[0006] The problems to be solved by the present invention are:

[0007] 1. The task-based dialogue management system has poor scalability and difficult maintenance. The current development and maintenance process of the enterprise dialogue management system is as follows: (1) The operation department summarizes the dialogue process tree of the scenario based on business data; the dialogue process tree is the dialogue state trend for completing the scenario task, and the nodes in the tree represent the actions of the system or user when the dialogue reaches that state. (2) The operation and product departments design the scenario specification speech template based on the dialogue process tree, and design the corresponding speech and response strategy for the action of each node in the dialogue process tree. (3) The R&D department designs the corresponding regular expression based on the scenario specification speech template. The entire dialogue management process requires multiple effective communications between multiple departments to complete, which consumes a huge amount of time and energy. Communication errors between departments will further lead to poor results of the dialogue system and make maintenance difficult. The dialogue process trees and scenario specification speech templates corresponding to different scenarios are also different. When a new scenario is added, it is necessary to redesign the dialogue process, speech template, response strategy, and regular expression. This affects the entire system and has poor scalability.

[0008] 2. Poor user experience. The users mentioned here include two types: enterprise users who use the dialogue management system and end users who interact with the dialogue system. When it comes to toB business, the poor scalability of the current dialogue management system makes it extremely difficult for enterprise customers to use and maintain the dialogue management system, and the user experience is extremely poor. For end users, since the user's replies are uncontrollable, the system's rules cannot be fully covered, and the manually summarized speech templates are single in form, making the system replies mechanical, regular expressions cannot accurately understand the semantics, and the system replies are sometimes irrelevant or ambiguous, which greatly reduces the user experience.

[0009] In view of the above problems, the present invention proposes a task-based dialogue management method that can automatically generate strategies, including: obtaining user voice input, obtaining voice input recognition text through a voice recognition module, inputting it into a natural language understanding module, and identifying user intention; a dialogue state tracking module obtains the current dialogue state according to the user intention, and generates a dialogue process tree through a dialogue strategy generation model; the dialogue strategy corresponding to each node of the dialogue process tree is sent to the natural language generation module to generate the corresponding speech text; the speech text is transmitted to the speech synthesis module, and the synthesized speech is transmitted to the user, wherein the above-mentioned dialogue strategy generation model realizes the automatic configuration of the dialogue process tree through a self-learning mechanism based on reinforcement learning.

[0010] The present invention also proposes a task-based dialogue management system, including a speech recognition module, a natural language understanding module, a dialogue state tracking module, a dialogue strategy generation model, a natural language generation module and a speech synthesis module; by obtaining user speech input, the above-mentioned speech recognition module obtains speech input recognition text and inputs it into the natural language understanding module to identify the user's intention; the dialogue state tracking module obtains the current dialogue state according to the user's intention, and generates a dialogue process tree through the dialogue strategy generation model; the dialogue strategy corresponding to each node of the dialogue process tree is sent to the natural language generation module to generate the corresponding speech text; the speech text is transmitted to the speech synthesis module, and the synthesized speech is transmitted to the user, wherein the above-mentioned dialogue strategy generation model realizes the automatic configuration of the dialogue process tree through the self-learning mechanism based on reinforcement learning. The dialogue process tree is automatically generated through the dialogue strategy generation tree, which solves the problem of cumbersome process and high maintenance difficulty of manually configuring the dialogue process tree.

[0011] Furthermore, the dialogue strategy generation model self-learning mechanism uses the idea of ​​reinforcement learning to continuously learn and optimize the dialogue strategy generation model trained by scene call data, so that the dialogue process tree generated by the model continues to approach the manually configured dialogue process tree, including:

[0012] (1) First, annotate the existing scene call data, build a data set, and train the dialogue strategy based on the deep learning model to generate the initial model. The X label of the model is the dialogue state extracted from the scene dialogue, and the Y label is the system or user action extracted from the dialogue data. If there is no scene-related call data at the beginning, the dialogue strategy generation model of the approximate scene can be used as the initial model.

[0013] (2) When new data scenario call data is generated, the model is retrained and the model effect is judged based on the model output generated by the current input dialogue state. If the effect is better, positive incentives are generated; if the effect is worse, negative incentives are generated and the corresponding model parameters are updated.

[0014] (3) The dialogue management system continuously cycles through (2) during use, continuously optimizing the dialogue strategy generation model, and ultimately making the dialogue process tree generated by the dialogue strategy generation model close to the manual configuration in terms of content and specifications. This means that the dialogue process tree generated at the beginning requires more manual corrections, and finally a dialogue process tree completely automatically generated by the model is used to achieve fully automated configuration of the dialogue process tree.

[0015] (4) After each model iteration and update, the updated model is automatically deployed, and the corpus is automatically collected and annotated, indicating whether it is the corpus that generates positive incentives or the corpus that generates negative incentives. The automatic iteration, deployment, and collection of corpus data of the model are all automatically completed by the automated program, and the program is run to complete a series of operations.

[0016] Furthermore, the present invention proposes a method for generating a dialogue strategy, which can automatically generate a dialogue process tree and perform self-learning optimization on the model, and has the following beneficial effects:

[0017] 1. Automatically generated dialogue process trees are more automated than manual configuration methods, solving the pain points of complex manual configuration of dialogue process trees and repeated communication between departments. At the same time, the generated dialogue process tree is editable, avoiding the uninterruptibility of model generation and making it more scalable.

[0018] 2. The model automatically generates a dialogue process tree, which avoids errors caused by improper communication between departments and is more maintainable. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic diagram of the dialogue management method and system of the present invention;

[0020] Figure 2 It is a flowchart of the working mechanism of reinforcement learning;

[0021] Figure 3 is a schematic diagram of the parameter update process in the reinforcement learning mechanism;

[0022] Figure 4 It is a graph of the loss function value when the model reaches convergence;

[0023] Figure 5 This is a partial dialogue process tree diagram of the postal savings notification process. DETAILED DESCRIPTION

[0024] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] The technical features, abbreviations, symbols, etc. involved in the present invention are interpreted, defined, and explained based on the common knowledge and common understanding of those skilled in the art.

[0026] The present invention relates to a task-based dialogue management method and system. Figure 1As shown, it includes a speech recognition module, a natural language understanding module, a dialogue state tracking module, a dialogue strategy generation model, a natural language generation module and a speech synthesis module. Specifically, the user's voice input is obtained through the electronic device, and the speech recognition text is obtained through the speech recognition module and input into the natural language understanding module to identify the user's intention; the dialogue state tracking module obtains the current dialogue state according to the user's intention and the historical dialogue record, and generates a dialogue process tree through the dialogue strategy generation model; the dialogue strategy corresponding to each node of the dialogue process tree is sent to the natural language generation module to generate the corresponding speech text; the speech text is passed to the speech synthesis module, and the synthesized speech is passed to the user to complete the interaction. Among them, compared with the traditional manually configured dialogue process tree, the dialogue strategy generation model of the present invention is continuously optimized through a self-learning mechanism based on reinforcement learning, and finally makes the dialogue process tree generated by the dialogue strategy generation model close to the manually configured dialogue process tree in terms of content and specifications, realizing the automatic configuration of the dialogue process tree.

[0027] The dialogue strategy generation model based on reinforcement learning has a dialogue state as input and a corresponding dialogue process tree as output. Each node in the dialogue process tree represents a dialogue strategy, that is, the corresponding action of the system. The automatic configuration of the dialogue process tree is realized through the self-learning mechanism based on reinforcement learning, which mainly includes:

[0028] Step S1, construct the initial model Y=f(WX) of the dialogue strategy generation model using the scene call data. The X label in the model is the dialogue state in the scene dialogue data, including the recognized text, the word slot filling status (information to be collected, such as user name, age, contact information, etc.), the number of dialogue turns, etc. The Y label is the system action extracted from the dialogue data, mainly referring to the reply action made to the user input. Specifically including:

[0029] Step S11: Perform unified preprocessing on the call data text to achieve text normalization, including: converting full-width to half-width, processing special characters, converting letters to lowercase, unified character encoding, text denoising, etc.

[0030] Step S12: Desensitize sensitive data. In order to protect customer information, it is necessary to desensitize the customer privacy information in the call data text, for example, Zhang San is processed into NAME, numbers are replaced by NUM, and organization names are replaced by PRD.

[0031] Step S13: The historical dialogue states and corresponding actions between the user and the system (robot) are extracted through manual annotation, where X = {X1, X2, X3, ... Xt} represents the dialogue state of the system (robot) / user from time 1 to time t, and Y = {Y1, Y2, Y3, ... Yt} represents the actions of the system (robot) / user from time 1 to time t.

[0032] In a specific embodiment, the system (robot): Excuse me, is this Mr. Ma Zilong himself? User: Yes. The corresponding system (robot) dialogue state is {text: Excuse me, is this Mr. Ma Zilong himself, word slot: empty, number of dialogue turns: 1, ...}, the system (robot) action is {action: ask if it is the person himself}; the user dialogue state is {text: yes, word slot: empty, number of dialogue turns: 1, ...}.

[0033] Step S14: Combining the historical conversation states and actions together constitutes a user data set Dataset = {X, Y}.

[0034] Step S15: Use a word vector model to convert the historical state data into a word vector. Common word vector models include word2vect, topic-model, and bert-based word vector models, etc. The present invention uses, but is not limited to, the above-mentioned word vector models.

[0035] Step S16: Construct an initial model of the dialogue strategy generation model. The initial model is represented by Y=f(WX). Where X represents the dialogue state of the system (robot) / user, W is the model parameter, and Y is the action in the dialogue process tree. In a preferred embodiment, the initial model can adopt a neural network model, and f is a transfer function. The present invention does not specify a specific algorithm, and all network structures in the field of deep learning or machine learning are within the scope of patent protection.

[0036] Step S2: Through a self-learning mechanism based on reinforcement learning, the initial model of the dialogue strategy generation model is trained so that the dialogue process tree generated by the dialogue strategy generation model is close to the manually configured dialogue process tree in terms of content and specifications, thereby realizing the automatic configuration of the dialogue process tree.

[0037] Reinforcement learning is a branch of machine learning that emphasizes how to maximize expected benefits based on environmental actions. In reinforcement learning, there are two main bodies: the agent and the environment. The agent determines an action based on the current state and executes the action. After that, it and the environment enter the next state. At the same time, the system gives it a feedback value to reward or punish the action to force the agent to perform the desired action.

[0038] In the present invention, the dialogue strategy generation model is equivalent to the intelligent agent, and the quality of the generated dialogue process tree is equivalent to the environment. When a new scene dialogue material is generated, the dialogue state is extracted from it and sent to the dialogue strategy generation model. Feedback is formed based on the quality of the generated dialogue process tree to update the model parameters. Figure 2 It is a schematic diagram of the reinforcement learning process of the present invention.

[0039] The main process of training the initial model of the dialogue strategy generation model through the self-learning mechanism based on reinforcement learning is as follows Figure 3As shown, specifically including:

[0040] Step S21: Parameter initialization. In the initial model Y=f(WX), W is the model parameter, which constitutes the parameter space as a whole. The initial value of each parameter in the parameter space is randomly determined by the initialization function, and the initial value is preferably in the interval (-0.1, 0.1). After the dialogue strategy model training is completed, the values ​​of the above parameters are uniquely determined and no longer change. The output Y can be uniquely determined by the parameters and input values. In the following, the initial value of the reward function R is calculated by the initial value of the parameter W and the parameter update action A, and the initial value of the value function Q can be zero.

[0041] Step S22: Obtain the parameter update action corresponding to the parameter, and obtain the updated parameter. Select the parameter update action for the parameter in the parameter space during the tth round of training, obtain the parameter update action space subset and update the parameter, and then obtain the value of the parameter space during the t+1th round of training. There are many strategies for selecting the parameter update action, including but not limited to the UCB strategy and the ∈-greedy strategy, etc. The present invention takes the ∈-greedy strategy as an example for explanation. According to the ∈-greedy strategy, during the tth round of training, a parameter Wt is arbitrarily selected from the parameter space W, an update action is randomly selected with a probability of ∈, and the action At corresponding to the maximum value of the value function Q is selected with a probability of 1-∈ as the update action of the parameter Wt, and the range of ∈ is {0.01, 1}. ∈ takes 1 at t=0, 0.99∈ at t=1, and then decays at a rate of ∈=0.99∈. When ∈=0.01, the decay stops and remains at 0.01.

[0042] Step S23: Update and execute the reward function R(W,A) and obtain the reward R.

[0043] Reward function R(W,A)={R1,R2,R3,…,Rt} represents the reward function corresponding to the 1st to tth round of training.

[0044]

[0045] The action space A = {A1, A2, A3..., An} corresponds to the parameter update action, An represents the parameter update action of the nth parameter,

[0046]

[0047] When An is 1, it means that the nth parameter is updated in this round of training; when An is 0, it means that the nth parameter remains unchanged in this round of training; when An is -1, it means that the nth parameter is deleted in this round of training.

[0048] P(t) is the accuracy of the dialogue flow tree generated by the model in the tth round of training. max P(t-1) is the highest accuracy of the dialogue flow tree generated by the model in the previous t-1 round of training. When the accuracy of the dialogue flow tree generated by the model in the tth round of training is higher than the highest record in the previous t-1 round of training, 2 times P(t) is used as the reward value; otherwise, the highest accuracy in the previous t-1 round is used as the penalty. In a specific embodiment, the dialogue flow tree output by the model is compared with the standard dialogue flow tree (manually annotated dialogue flow tree). When the dialogue flow tree output by the model has an accuracy of 85% compared with the standard dialogue flow tree text, P(t) = 85%. When the accuracy of the dialogue flow tree generated by the model in the tth round of training (P(t) = 85%) is higher than the maximum accuracy in the previous t-1 round of training, Rt = 2*85%; otherwise, Rt = -max P(t-1).

[0049] Step S24: Update the value function:

[0050] Q(Wt+1,At+1)=Q(Wt,At)+α[Rt+γmax(R(Wt+1,At)-R(Wt,At))];

[0051] Among them, α is a parameter related to the strategy generation model, which can be between 0.5 and 0.9; γ represents the model error, which can be between 0.0005 and 0.005. According to the value function of the current training and the parameters of the next round of training, the candidate update actions corresponding to the parameters of the next round of training are obtained.

[0052] Step S25: Repeat steps S22 to S24, and continuously update the values ​​in the parameter space until the model reaches a convergence state, at which point the loss function L fluctuates slightly around a certain value.

[0053] The loss function is an important indicator for measuring model performance. It describes the difference between the dialogue process tree generated by the model and the standard dialogue process tree, and its value is between (0,1).

[0054]

[0055] Where N is the number of nodes in the dialog flow tree, y is the true value of the node, and y_pred is the node value generated by the model.

[0056] The effect after the model converges is as follows Figure 4 As shown in the figure, the horizontal axis is the number of training times, 1k means 1000 times, and the vertical axis is the loss, that is, the value of the loss function, which measures the gap between the generated dialogue flow tree and the standard dialogue flow tree. The smaller the loss value, the more similar the two are. In this example figure, the loss value fluctuates around 0.6 after the model converges.

[0057] Finally, an automated deployment program is developed to automatically collect and organize the generated corpus by clicking a button or running a script, and the updated model is automatically deployed online through the script program.

[0058] Through the self-learning mechanism based on reinforcement learning to train the dialogue strategy generation model, the dialogue process tree generated by the dialogue strategy generation model is finally close to the manually configured dialogue process tree in terms of content and specifications. From the initial generation that requires a lot of manual corrections to the final automatic generation by the model, the automatic configuration of the dialogue process tree is realized. Taking the postal savings notification process as an example, Figure 5 The figure shows a partial generated dialogue process tree, where R represents the system / robot node, P represents the user node, and the numbers represent the number of dialogue rounds and nodes. R1.1 means that the action of the system / robot at the first node in the first round of dialogue is to confirm the user dialogue, and P1.1 means that the action of the user at the first node in the first round of dialogue is to confirm that it is himself. The lines between the nodes represent the context and association of the dialogue.

[0059] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for generating a dialogue strategy, comprising: Step S1, constructing an initial model of a dialogue strategy generation model using scene call data; Step S2: Train the above initial model through a self-learning mechanism based on reinforcement learning to achieve automatic configuration of the dialogue process tree; Step S2 includes: Step S21: parameter initialization; Step S22: Obtain the parameter update action corresponding to the parameter, and obtain the updated parameter; Step S23: Update and execute the reward function to obtain a reward; Step S24: Update the value function: Step S25: Repeat steps S22 to S24 until the model reaches a convergence state; The above initial model is represented by Y=f(WX), where X represents the robot / user dialogue state, W is the model parameter, and Y is the action in the dialogue process tree. The above reward function R = {R1, R2, R3, ..., Rt} represents the reward function corresponding to the 1st to tth round of training. The action space A = {A1, A2, A3..., An} corresponds to the parameter update action, An represents the parameter update action of the nth parameter, When An is 1, it means that the nth parameter is updated in this round of training; when An is 0, it means that the nth parameter remains unchanged in this round of training; when An is -1, it means that the nth parameter is deleted in this round of training; P(t) is the accuracy of the dialogue flow tree generated by the model in the tth round of training, and max P(t-1) is the highest accuracy of the dialogue flow tree generated by the model in the first t-1 rounds of training; The above value function Q(Wt+1,At+1)=Q(Wt,At)+α[Rt+γmax(R(Wt+1,At)-R(Wt,At))]; Where α is a model-related parameter, with a value between 0.5 and 0.9; γ represents the model error, with a value between 0.0005 and 0.

005.

2. The method according to claim 1, characterized in that Step S1 includes: Perform unified preprocessing on call data text; Desensitize sensitive data; Manually annotate and extract the historical conversation states and corresponding actions of users and robots; The above historical dialogue states and actions are combined to form a data set; Use the word vector model to convert historical state data into word vectors; Build an initial model for the dialogue policy generation model.

3. The method according to claim 1, characterized in that: The convergence of the above model means that the loss function L fluctuates slightly around a certain value. Where N is the number of nodes in the dialog flow tree, y is the true value of the node, and y_pred is the node value generated by the model.

4. A task-based dialogue management method, comprising: Obtain user voice input, obtain voice input recognition text input through the voice recognition module and input it into the natural language understanding module to identify user intention; The dialogue state tracking module obtains the current dialogue state according to the user's intention and generates a dialogue process tree through the dialogue strategy generation model; The dialogue strategy corresponding to each node of the dialogue process tree is sent to the natural language generation module to generate the corresponding speech text; The speech text is transmitted to the speech synthesis module, and the synthesized speech is transmitted to the user. The above-mentioned dialogue strategy generation model realizes the automatic configuration of the dialogue process tree through the self-learning mechanism based on reinforcement learning; The above-mentioned dialogue strategy generation model realizes the automatic configuration of the dialogue process tree through the self-learning mechanism based on reinforcement learning, and the steps also include: Step S1, constructing an initial model of a dialogue strategy generation model using scene call data; Step S2: Train the above initial model through a self-learning mechanism based on reinforcement learning to achieve automatic configuration of the dialogue process tree; Step S2 includes: Step S21: parameter initialization; Step S22: Obtain the parameter update action corresponding to the parameter, and obtain the updated parameter; Step S23: Update and execute the reward function to obtain a reward; Step S24: Update the value function: Step S25: Repeat steps S22 to S24 until the model reaches a convergence state; The above initial model is represented by Y=f(WX), where X represents the robot / user dialogue state, W is the model parameter, and Y is the action in the dialogue process tree. The above reward function R = {R1, R2, R3, ..., Rt} represents the reward function corresponding to the 1st to tth round of training. The action space A = {A1, A2, A3..., An} corresponds to the parameter update action, An represents the parameter update action of the nth parameter, When An is 1, it means that the nth parameter is updated in this round of training; when An is 0, it means that the nth parameter remains unchanged in this round of training; when An is -1, it means that the nth parameter is deleted in this round of training; P(t) is the accuracy of the dialogue flow tree generated by the model in the tth round of training, and max P(t-1) is the highest accuracy of the dialogue flow tree generated by the model in the first t-1 rounds of training; The above value function Q(Wt+1,At+1)=Q(Wt,At)+α[Rt+γmax(R(Wt+1,At)-R(Wt,At))]; Where α is a model-related parameter, with a value between 0.5 and 0.9; γ represents the model error, with a value between 0.0005 and 0.

005.

5. The method according to claim 4, characterized in that Step S1 includes: Perform unified preprocessing on call data text; Desensitize sensitive data; Manually annotate and extract the historical conversation states and corresponding actions of users and robots; The above historical dialogue states and actions are combined to form a data set; Use the word vector model to convert historical state data into word vectors; Build an initial model for the dialogue policy generation model.

6. The method according to claim 4, characterized in that: The convergence of the above model means that the loss function L fluctuates slightly around a certain value. Where N is the number of nodes in the dialog flow tree, y is the true value of the node, and y_pred is the node value generated by the model.

7. A task-based dialogue management system, comprising a speech recognition module, a natural language understanding module, a dialogue state tracking module, a dialogue strategy generation model, a natural language generation module and a speech synthesis module; by obtaining user speech input, the speech recognition module obtains speech input recognition text input into the natural language understanding module to identify the user's intention; The dialogue state tracking module obtains the current dialogue state according to the user's intention and generates a dialogue process tree through the dialogue strategy generation model; The dialogue strategy corresponding to each node of the dialogue process tree is sent to the natural language generation module to generate the corresponding speech text; The speech text is transmitted to the speech synthesis module, and the synthesized speech is transmitted to the user. The above-mentioned dialogue strategy generation model realizes the automatic configuration of the dialogue process tree through the self-learning mechanism based on reinforcement learning; The above-mentioned dialogue strategy generation model realizes the automatic configuration of the dialogue process tree through the self-learning mechanism based on reinforcement learning, and the steps also include: Step S1, constructing an initial model of a dialogue strategy generation model using scene call data; Step S2: Train the above initial model through a self-learning mechanism based on reinforcement learning to achieve automatic configuration of the dialogue process tree; Step S2 includes: Step S21: parameter initialization; Step S22: Obtain the parameter update action corresponding to the parameter, and obtain the updated parameter; Step S23: Update and execute the reward function and obtain the reward; Step S24: Update the value function: Step S25: Repeat steps S22 to S24 until the model reaches a convergence state; The above initial model is represented by Y=f(WX), where X represents the robot / user dialogue state, W is the model parameter, and Y is the action in the dialogue process tree. The above reward function R = {R1, R2, R3, ..., Rt} represents the reward function corresponding to the 1st to tth round of training. The action space A = {A1, A2, A3..., An} corresponds to the parameter update action, An represents the parameter update action of the nth parameter, When An is 1, it means that the nth parameter is updated in this round of training; when An is 0, it means that the nth parameter remains unchanged in this round of training; when An is -1, it means that the nth parameter is deleted in this round of training; P(t) is the accuracy of the dialogue flow tree generated by the model in the tth round of training, and max P(t-1) is the highest accuracy of the dialogue flow tree generated by the model in the first t-1 rounds of training; The above value function Q(Wt+1,At+1)=Q(Wt,At)+α[Rt+γmax(R(Wt+1,At)-R(Wt,At))]; Where α is a model-related parameter, with a value between 0.5 and 0.9; γ represents the model error, with a value between 0.0005 and 0.

005.

8. The system according to claim 7, characterized in that Step S1 includes: Perform unified preprocessing on call data text; Desensitize sensitive data; Manually annotate and extract the historical conversation states and corresponding actions of users and robots; The above historical dialogue states and actions are combined to form a data set; Use the word vector model to convert historical state data into word vectors; Build an initial model for the dialogue policy generation model.

9. The system according to claim 7, characterized in that: The convergence of the above model means that the loss function L fluctuates slightly around a certain value. Where N is the number of nodes in the dialog flow tree, y is the true value of the node, and y_pred is the node value generated by the model.

Images