Dialogue generation model training method and apparatus and dialogue generation method and apparatus

Abstract

Provided in the embodiments of the present disclosure are a dialogue generation model training method and apparatus, and a dialogue generation method and apparatus. The dialogue generation model training method comprises: acquiring a network structure unit for constructing an adversarial dialogue system; and determining an adversarial training dialogue model on the basis of the network structure unit, and performing adversarial training on the adversarial training dialogue model to obtain a trained adversarial training dialogue model, the adversarial training dialogue model comprising a generator and a discriminator.

Description

Dialogue generation model training method, dialogue generation method and device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application No. 202410070983.X filed in China on January 17, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to the field of artificial intelligence, and more particularly to a method for training a dialogue generation model, a method for generating dialogue, and a device for generating dialogue. Background Art

[0004] Dialogue generation is a challenging problem because it requires not only modeling the context of the conversation, but also leveraging the context to generate coherent and fluent sentences.

[0005] Existing solutions are generally categorized as rule-based or neural network-based. End-to-end dialogue generation still faces a number of challenges. First, language is complex and polysemous; the same sentence can have multiple interpretations, complicating the determination of the correct generation approach. Furthermore, human conversations are often contextual, and dialogue systems need to be able to understand previous conversation history to generate meaningful responses. This requires models to capture long-term dependencies and contextual information, which typically requires strong memory and reasoning capabilities. Another important challenge is emotion and sentiment perception. Conversation is not just about conveying information; it also involves emotional expression and perception. Therefore, dialogue systems need to be able to recognize and generate emotionally charged language to better simulate interpersonal interactions. This increases the complexity of generative models, as they must not only process grammar and semantics but also emotional information. Furthermore, generative models face the problem of data scarcity. Collecting and annotating large-scale dialogue data is an expensive and time-consuming process, leading to data insufficiency. A lack of data of sufficient diversity and quality can limit model performance and generalization. Summary of the Invention

[0006] The embodiments of the present disclosure provide a training method, a dialogue generation method, and a device for a dialogue generation model to solve the problem of how to improve the accuracy of the dialogue model output.

[0007] In a first aspect, a method for training a dialogue generation model is provided, comprising:

[0008] Obtain network structural units for building adversarial dialogue systems;

[0009] According to the network structure unit, an adversarial training dialogue model is determined, and adversarial training is performed on the adversarial training dialogue model to obtain a trained adversarial training dialogue model, wherein the adversarial training dialogue model includes a generator and a discriminator.

[0010] Optionally, obtain the network structure units used to build the adversarial dialogue system, including:

[0011] By applying neural architecture search to a specific dataset, we can obtain the optimal network structure units for building adversarial dialogue systems.

[0012] Optionally, performing adversarial training on the adversarial training dialogue model includes:

[0013] Performing adversarial training on the adversarial training dialogue model to obtain indication information output by the discriminator, where the indication information is used to indicate the quality of the sentence generated by the generator;

[0014] The network parameters of the generator are adjusted according to the instruction information.

[0015] Optionally, adjusting the network parameters of the generator according to the instruction information includes:

[0016] The indication information is sent to the generator through a policy gradient algorithm to adjust the network parameters of the generator.

[0017] Optionally, in the adversarial training of the adversarial training dialogue model, the input statement of the discriminator contains current state related information.

[0018] In a second aspect, a method for generating a dialog is provided, which is applied to a server running an adversarially trained dialog model, wherein the dialog generation model is a model trained by the method of the first aspect, and the method comprises:

[0019] Get the context of the conversation;

[0020] Obtaining one or more candidate utterances corresponding to the context of the conversation through a generator of the adversarial training conversation model;

[0021] Obtaining the quality of each candidate utterance through the discriminator of the adversarial training dialogue model;

[0022] The candidate utterance that matches the context of the conversation and has the highest quality is selected as the target utterance.

[0023] In a fourth aspect, a training device for a dialogue generation model is provided, comprising:

[0024] A first acquisition module is used to acquire a network structure unit for building an adversarial dialogue system;

[0025] A training module is used to determine an adversarial training dialogue model based on the network structure unit, and perform adversarial training on the adversarial training dialogue model to obtain a trained adversarial training dialogue model, wherein the adversarial training dialogue model includes a generator and a discriminator.

[0026] Optionally, the first acquisition module is further used to: obtain the optimal network structure unit for constructing an adversarial dialogue system by applying neural architecture search on a specific data set.

[0027] Optionally, the training module is further used to: perform adversarial training on the adversarial training dialogue model, obtain indication information output by the discriminator, and the indication information is used to indicate the quality of the sentences generated by the generator; and adjust the network parameters of the generator according to the indication information.

[0028] Optionally, the training module is further used to: send the indication information to the generator through a policy gradient algorithm to adjust the network parameters of the generator.

[0029] In a fourth aspect, a dialog generation device is provided, which is applied to a server running an adversarial training dialog model, wherein the dialog generation model is a model trained by the method described in the first aspect, and the device includes:

[0030] The second acquisition module is used to obtain the context of the conversation;

[0031] A first processing module is configured to obtain one or more candidate utterances corresponding to the context of the conversation through a generator of the adversarial training conversation model;

[0032] A third acquisition module is used to obtain the quality of each candidate utterance through the discriminator of the adversarial training dialogue model;

[0033] The second processing module is configured to select a candidate utterance that matches the context of the conversation and has the highest quality as a target utterance.

[0034] In a fifth aspect, an electronic device is provided, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect or the second aspect.

[0035] In a sixth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0036] In an embodiment of the present disclosure, a network structure unit for constructing an adversarial dialogue system is obtained; based on the network structure unit, an adversarial training dialogue model is determined, and adversarial training is performed on the adversarial training dialogue model to obtain a trained adversarial training dialogue model. The adversarial training dialogue model includes a generator and a discriminator, and combines neural architecture search and adversarial training to achieve an automatic closed loop from end-to-end dialogue generation to dialogue evaluation, thereby improving the accuracy of the dialogue model output. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present disclosure. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0038] FIG1 is a flow chart of a method for training a dialogue generation model according to an embodiment of the present disclosure;

[0039] FIG2 is a flow chart of a method for generating a dialogue according to an embodiment of the present disclosure;

[0040] FIG3 is a schematic diagram of adversarial learning enhanced by neural architecture search according to an embodiment of the present disclosure;

[0041] FIG4 is a flowchart of the method implementation provided by an embodiment of the present disclosure;

[0042] FIG5 is a diagram of a state-aware adversarial training framework provided by an embodiment of the present disclosure;

[0043] FIG6 is a schematic diagram of an RNN unit structure searched in a MultiWOZ dataset using DARTS according to an embodiment of the present disclosure;

[0044] FIG7 is a structural block diagram of a training apparatus for a dialogue generation model according to an embodiment of the present disclosure;

[0045] FIG8 is a structural block diagram of a dialogue generating device provided by an embodiment of the present disclosure;

[0046] FIG9 is a schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present disclosure in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0048] The term "comprise" and any variations thereof in the specification and claims of the present disclosure are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to the process, method, product, or apparatus. In addition, the use of "and / or" in the specification and claims to indicate at least one of the connected objects, such as A and / or B, means that A alone, B alone, and both A and B are included.

[0049] In the embodiments of the present disclosure, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present disclosure should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0050] In order to better understand the embodiments of the present disclosure, the following technical points are first introduced:

[0051] 1. About state-aware dialogue.

[0052] State-aware conversations refer to conversational systems (such as chatbots or virtual assistants) that can understand and remember the current state of a conversation while interacting with a user, and respond accordingly. These systems typically track and manage conversational context, including identifying the user's intent, needs, mentioned facts, and emotional state.

[0053] State awareness allows the dialogue system to maintain coherence across multiple rounds of conversation, ensuring that each response takes into account the context of the previous exchange. For example, in a restaurant reservation scenario, the system needs to track information such as the user's selected date, time, and number of diners.

[0054] Conversational systems that include state awareness often record the following:

[0055] (1) User Intents: The goals that users want to achieve, such as booking a hotel, purchasing goods, etc.

[0056] (2) Entities: Specific information related to the intent, such as date, location, quantity, etc.

[0057] (3) Dialogue History: previous question-answer sequence;

[0058] (4) User Preferences: User’s past behavior and expressed preferences;

[0059] (5) Emotional States: Analyze the user's tone and words to infer their emotional attitude.

[0060] With this information, the state-aware dialogue system can generate more personalized, relevant, and effective responses, greatly enhancing the user experience.

[0061] The following, in conjunction with the accompanying drawings, describes in detail the training method, dialogue generation method, and device for the dialogue generation model provided by the embodiments of the present disclosure through some embodiments and their application scenarios.

[0062] Referring to FIG1 , an embodiment of the present disclosure provides a method for training a dialogue generation model, the specific steps of which include:

[0063] Step 101: Obtain a network structure unit for constructing an adversarial dialogue system;

[0064] Optionally, the network structure unit includes but is not limited to a convolutional neural network.

[0065] Step 102: Determine an adversarial training dialogue model based on the network structure unit, and perform adversarial training on the adversarial training dialogue model to obtain a trained adversarial training dialogue model, where the adversarial training dialogue model includes a generator and a discriminator.

[0066] In one embodiment of the present disclosure, obtaining a network structure unit for constructing an adversarial dialogue system includes:

[0067] By applying neural architecture search to a specific dataset, we can obtain the optimal network structure units for building adversarial dialogue systems.

[0068] In this implementation, a two-stage learning framework is used, combining neural architecture search and adversarial training, to achieve an automatic closed loop from end-to-end dialogue generation to dialogue evaluation.

[0069] In one embodiment of the present disclosure, performing adversarial training on the adversarial training dialogue model includes:

[0070] Performing adversarial training on the adversarial training dialogue model to obtain indication information output by the discriminator, where the indication information is used to indicate the quality of the sentence generated by the generator;

[0071] The network parameters of the generator are adjusted according to the instruction information.

[0072] Optionally, during adversarial training of the adversarial training dialogue model, the input sentence of the discriminator contains information related to the current state, that is, the input of the discriminator includes state-aware sentences. This state-related information can help the discriminator more accurately evaluate whether the generated content is not only authentic and credible in form, but also whether the content conforms to a specific context or dialogue state. In such a system, the discriminator considers the sentence itself and the state information associated with it to make an evaluation. This makes the entire adversarial network more complex and powerful, and can generate output that is more consistent with a given state.

[0073] In one embodiment of the present disclosure, adjusting the network parameters of the generator using the instruction information includes:

[0074] The indication information is sent to the generator through a policy gradient algorithm to adjust the network parameters of the generator.

[0075] Policy gradient algorithms belong to the field of reinforcement learning and are a method for directly optimizing policies. In reinforcement learning, "policy" refers to the rules or model that an agent uses to decide what actions to take in a given state. Policy gradient algorithms optimize policies by adjusting policy parameters to maximize the expected cumulative reward.

[0076] This approach differs from value-based reinforcement learning methods, which evaluate the goodness of actions and then choose actions based on these evaluations. Policy gradient algorithms directly adjust the policy function itself, giving better actions higher probabilities.

[0077] The policy gradient algorithm includes at least one of the following:

[0078] 1) REINFORCE: Also known as Monte Carlo policy gradient, it updates the policy by sampling the complete sequence and using its returns.

[0079] 2) Actor-Critic: This method combines the policy gradient and value function methods, where the "actor" generates actions and the "critic" evaluates the quality of the actions to guide the "actor" to learn.

[0080] 3) Proximal Policy Optimization (PPO) and Trusted Region Policy Optimization (TRPO): They are both improved policy gradient algorithms designed to improve the stability and efficiency of training.

[0081] In this implementation, the quality calculated by the discriminator is backpropagated to the generator via a policy gradient algorithm. The dialogue generation process can be viewed as a series of actions taken according to the policy defined by the generator. The generator generates controllable sentences based on the input conditions. The discriminator determines the quality of the sentences generated by the generator and feeds rewards back to the generator via policy gradients. This closed-loop process solves the non-differentiable concatenation problem of discrete natural language data and enables automatic optimization and updating of the model.

[0082] This approach of using network structural units to construct and adversarially train a dialogue model consisting of a generator and a discriminator can achieve the following technical effects:

[0083] 1) Improved generation performance: Through adversarial training, the generator can learn to produce more natural, fluent, and context-relevant utterances, which improves the quality of responses generated by the dialogue system.

[0084] 2) Conversational fidelity: The discriminator continuously evaluates the output of the generator, prompting it to produce content that is difficult to distinguish from real conversations. This training process can significantly improve the fidelity of the conversation content.

[0085] 3) Generalization ability: Adversarial training helps enhance the generalization ability of the model and avoid overfitting, because the generator learns a wider range of dialogue patterns in the process of trying to deceive the discriminator.

[0086] 4) Flexibility and robustness: Within the adversarial training framework, by continuously adjusting the parameters of the generator and discriminator, the model can adapt to various data distributions and better handle diverse inputs and scenario changes.

[0087] 5) Automatic Optimization: Use neural architecture search technology to automatically select the optimal network structure unit, reducing manual parameter adjustment work and potentially bringing model performance that exceeds the level of human expert design.

[0088] This embodiment can build a more efficient dialogue system with powerful expression and understanding capabilities, thereby providing a better user experience in various practical application scenarios (such as customer service robots, virtual assistants, interactive entertainment, etc.).

[0089] Referring to FIG2 , an embodiment of the present disclosure provides a method for generating a dialogue, which is applied to a server running an adversarial training dialogue model. The specific steps include:

[0090] Step 201: Obtain the context of the conversation;

[0091] Step 202: Obtain one or more candidate utterances corresponding to the context of the conversation by adversarially training the generator of the conversation model;

[0092] Step 203: Obtain the quality of each candidate utterance through the discriminator of the adversarial training dialogue model;

[0093] Step 204: The candidate utterance that matches the context of the conversation and has the highest quality is selected as the target utterance.

[0094] In one embodiment of the present disclosure, before or after step 201, the method further includes: obtaining a network structure unit for constructing an adversarial dialogue system; determining an adversarial training dialogue model based on the network structure unit, and performing adversarial training on the adversarial training dialogue model to obtain a trained adversarial training dialogue model, wherein the adversarial training dialogue model includes a generator and a discriminator.

[0095] In one embodiment of the present disclosure, obtaining a network structure unit for constructing an adversarial dialogue system includes:

[0096] By applying neural architecture search to a specific dataset, we can obtain the optimal network structure units for building adversarial dialogue systems.

[0097] In one embodiment of the present disclosure, performing adversarial training on the adversarial training dialogue model includes:

[0098] Performing adversarial training on the adversarial training dialogue model to obtain indication information output by the discriminator, where the indication information is used to indicate the quality of the sentence generated by the generator;

[0099] The network parameters of the generator are adjusted according to the instruction information.

[0100] In one embodiment of the present disclosure, adjusting the network parameters of the generator using the instruction information includes:

[0101] The indication information is sent to the generator through a policy gradient algorithm to adjust the network parameters of the generator.

[0102] In this embodiment, neural architecture search and adversarial training are combined to achieve an automatic closed loop from end-to-end dialogue generation to dialogue evaluation.

[0103] The embodiment of the present disclosure includes a two-stage learning process, as shown in Figure 3. Step 1: neural network search to obtain the target network architecture. Step 2: using the target network architecture as the input of the generator to perform adversarial training to achieve an automatic closed loop from end-to-end dialogue generation to dialogue evaluation.

[0104] In step 1, in Neural Architecture Search (NAS), the interaction between the controller, child network, and dataset (MultiWOZ) completes the task of searching for the best network architecture. The following are the steps in their interaction:

[0105] Step 11: Initialization: The controller is initialized, which may be a probability-based model or a recurrent neural network (RNN) that outputs architecture hyperparameters.

[0106] Step 12: Architecture Generation: The controller proposes a new neural network architecture, which may be achieved by sampling a search space. This architecture describes the structure of the network, including information such as layers and connections, thereby defining a child network.

[0107] Step 13: Subnetwork Construction and Training: Based on the architecture provided by the controller, a subnetwork model is constructed and trained using a dataset such as MultiWOZ. The training process involves optimizing the model weights to achieve specific tasks, such as language understanding or dialogue state tracking.

[0108] Step 14: Performance Evaluation: After training is complete, the performance of the subnetwork is evaluated on the validation set (which is also a subset of the MultiWOZ dataset that was not used for training). Evaluation criteria typically include accuracy, loss function value, and other metrics.

[0109] Step 15: Feedback Signal: The performance evaluation results of the sub-network are used as feedback signals back to the controller. In the reinforcement learning setting, this may affect the reward signal of the controller; in the gradient-based method, this affects the gradient calculation used by the controller to update its parameters.

[0110] Step 16: Update the controller: Based on the feedback from the previous step, update the controller's strategy or parameters to generate a better network architecture in subsequent iterations.

[0111] Step 17: Iteration: The above process is repeated several times, each time generating different sub-network architectures, training them, evaluating them, feeding back to the controller and updating the controller.

[0112] Step 18: Select the optimal architecture: After multiple iterations, select the sub-network architecture with the best performance as the "target network architecture".

[0113] In this way, through this iterative process, neural network architecture search can finally determine the neural network architecture that best meets the requirements through continuous searching, evaluation and improvement.

[0114] In step 2 of Figure 3, system typically refers to a part of the dialogue system, such as the responses or actions it generates. State describes the information accumulated during the dialogue, including the user's intent, desired information, and the content of the conversation. When these are provided as input to the generator: Input: The system's current response / action and the current state of the dialogue. Output: The generator outputs the user's expected response, which is the simulated user response (User'). In a dialogue system, the "User" output simulates the response a real user might give given the system response and dialogue state. The model's task is to generate natural and reasonable user statements based on these inputs to continue the conversation and achieve the user's goal. Such models can help improve data augmentation in machine learning training, test the robustness of dialogue systems, or be integrated into dialogue systems to predict likely user reactions.

[0115] In step 2 of Figure 3, after obtaining the simulated user response (User'), the simulated user response (User') and User are used as inputs to the discriminator to obtain a true / false response. The process described here involves using a generator (e.g., a generative adversarial network) to simulate user responses, and having the discriminator evaluate the authenticity of the simulated responses. In this setting: Simulated user responses (User'): generated by the generator, intended to mimic how real users would respond to specific conversation scenarios. User: may refer to real user data, i.e., actual user responses in the training set. Discriminator: A model whose function is to distinguish whether the input data comes from a real data distribution (i.e., real user responses) or a data distribution simulated by the generator (i.e., simulated user responses (User')). Real / Fake: The output of the discriminator. If it believes the input data is similar to the responses of real users in the training set, it outputs "Real"; if it looks like it was fabricated by the generator, it outputs "Fake." Throughout this process, the "simulated user responses" and real user responses are evaluated by the discriminator, thereby training the generator to produce more realistic user responses. The generator's goal is to deceive the discriminator as much as possible, making it unable to distinguish between true and false, while the discriminator's task is to improve its judgment ability. Through this adversarial training process, the system aims to improve the naturalness and quality of generated conversational responses.

[0116] The discriminator returns a reward to the generator, typically in dialogue systems using generative adversarial networks (GANs) or reinforcement learning frameworks. The reward here is a signal or feedback that evaluates the generator's output and guides the generator to adjust its parameters to improve future output. The detailed steps are as follows:

[0117] The discriminator receives these generated outputs and compares them with real data to determine whether these outputs are more similar to real user corpus (marked as Real) or obviously appear to be machine-generated (marked as Fake). Based on the discriminator's evaluation results, a reward value (Reward) is calculated. If the generated output can deceive the discriminator into thinking it is real, the generator will receive a higher reward; conversely, if the output is obviously fake, the reward will be lower. The generator uses this reward value (or the gradient calculated based on the reward value) to update its network parameters. If reinforcement learning technology is used, the reward usually directly affects its learning process. This process implements the training of the generative adversarial network through a reward mechanism, so that the generator gradually learns to produce increasingly difficult to distinguish and higher quality simulated user responses.

[0118] In this embodiment, when the target network architecture is used as the input of the generator and adversarial training is performed to achieve an automatic closed loop from end-to-end dialogue generation to dialogue evaluation, the following steps are included:

[0119] Step 21: Determine the target network architecture: Select the best performing target network architecture from the neural network architecture search. This architecture is considered as the generator for generating conversational text.

[0120] Step 22: Initialize the Discriminator: Create a discriminator network whose task is to distinguish the difference between the text output by the generator and the real conversation data.

[0121] Step 23: Generator training: Use the target network architecture to train the generator. This may involve using historical dialogue information from a real dialogue dataset (such as MultiWOZ) to "start" the dialogue.

[0122] Step 24: Discriminator training: Synchronously train the discriminator to enable it to better distinguish between the dialogues generated by the generator and the real user-system dialogues.

[0123] Step 25: Adversarial training loop: Enter the adversarial training loop, where the generator tries to generate increasingly realistic dialogues, while the discriminator tries to more accurately identify which dialogues are generated. The specific steps include:

[0124] Step 251: The generator generates a dialogue.

[0125] Step 252: The discriminator receives the real dialogue data and the generated data and gives a true / false evaluation.

[0126] Step 253: The evaluation result of the discriminator is fed back to the generator, and the generator adjusts the parameters based on this feedback to generate a more natural dialogue.

[0127] Step 26: Performance Evaluation: Regularly evaluate the quality of the generated dialogues, using metrics such as perplexity, Bilingual Evaluation Understudy (BLEU) scores, and dialogue success rate to assess the fluency and effectiveness of the generated dialogues.

[0128] Step 27: Adjust the parameters of the generator and discriminator: Based on the evaluation indicators and the performance of the generator and discriminator, further fine-tune the network parameters of the two.

[0129] Step 28: Stop condition achieved: When the preset performance standard or the upper limit of the number of iterations is achieved, the training loop is stopped.

[0130] Step 29: Closing the Loop: The final step is to implement and test a fully automated closed-loop system—that is, an end-to-end system where the generator produces dialogue, the discriminator evaluates the quality of the dialogue, and the system summarizes the feedback to improve the generator output, all without human intervention.

[0131] This approach allows us to create a dialogue system that can automatically generate and evaluate the dialogue it generates. Continuous adversarial iterations allow the system to continuously learn and improve, with the ultimate goal of generating high-quality dialogue that is indistinguishable from real human communication.

[0132] As shown in Figure 4, the data-adapted network structure unit obtained in step 1 is used as the component of the generator in step 2. After step 2, high-quality dialogue text is obtained after adversarial training.

[0133] Step 1: Structure search.

[0134] 1) Initialize the controller: Start the controller in the neural network architecture search process, which can be a probability-based model or a recurrent neural network (RNN) that can output network architecture description information.

[0135] 2) Generate candidate network structures: The controller produces a series of network architecture parameters that describe details such as the connection method between different layers and layer types.

[0136] 3) Training and evaluating sub-networks: A sub-network is constructed based on each generated network architecture parameter, and trained and validated on a dataset (e.g., MultiWOZ) to evaluate its performance.

[0137] 4) Collect feedback and update the controller: Use the performance results of the sub-network on the validation set as feedback to update the controller's strategy or parameters.

[0138] 5) Iterative improvement: Repeat the above steps until a stopping condition is met, such as reaching a preset number of iterations or performance threshold.

[0139] 6) Final selection of network structure: The network structure with the best performance is selected as the target network structure for the subsequent adversarial training step.

[0140] Step 2: Use the network structure as the generator input for adversarial training.

[0141] 1) Build the generator: Use the optimal network structure obtained in step 1 to build the generator network.

[0142] 2) Initialize the discriminator: Create a discriminator network to distinguish generated dialogues from real dialogue samples.

[0143] 3) Adversarial training begins:

[0144] 31) Generator Output: The generator attempts to generate data (i.e., dialogue) that mimics samples from the real dataset.

[0145] 32) Discriminator Evaluation: The discriminator receives generated dialogues and real dialogue samples and classifies them as “real” or “fake”.

[0146] 33) Optimize the generator: Using the feedback from the discriminator (usually gradient information), update the generator parameters to produce more realistic outputs.

[0147] 34) Optimize the discriminator: At the same time, the discriminator also updates its parameters based on whether it correctly identifies the generated and real samples.

[0148] 4) Iteration: The generator and discriminator are trained for multiple rounds until their performance no longer improves significantly or a predetermined number of iterations is reached.

[0149] 5) Dialogue Generation: After adversarial training, the optimized generator is used to generate dialogue text. These dialogues should be difficult for the discriminator to distinguish, thus behaving more like human-generated text.

[0150] This process achieves end-to-end automated dialogue generation, including structural search to determine the optimal network architecture and adversarial training to improve the quality of generated dialogue. The final generator can produce realistic and relevant dialogue content, while the discriminator can distinguish the quality and realism of the dialogue.

[0151] As shown in Figure 5, the process of solid arrows represents the training process of the generator, and the process of hollow arrows represents the training process of the discriminator, with a total of two loops. The loop process of solid arrows is used for generator learning and updating the model parameters of the generator. The loop process of hollow arrows is used for discriminator learning and periodically updating the discriminator model. The learning of the generator and the discriminator is carried out in an alternating manner. Unlike the traditional generative dialogue model trained by maximum likelihood estimation (MLE), in this embodiment, the sentence generation process is regarded as a series of actions taken according to the strategy defined by the generator. The generator generates controllable sentences based on the input conditions, and the discriminator judges the quality of the sentences generated by the generator and feeds back rewards to the generator through policy gradients.

[0152] The generator described above is a predictive model whose goal is to generate sentences that meet certain input conditions. For example, in text generation tasks, these input conditions can be keywords, categories, topics, or sentiment. The generator is typically a sequence model, such as a recurrent neural network (RNN) or a transformer, which learns to generate data with a distribution similar to the real data based on the input conditions.

[0153] The discriminator described above is a classification model that attempts to distinguish whether the input sentence is generated by the generator ("fake" data) or a real sentence from the training set ("real" data). The discriminator provides feedback on the quality of the generated sentence, usually by evaluating how realistic the sentence is or how it conforms to the given input conditions.

[0154] Policy Gradient (PG) is an algorithm in reinforcement learning that adjusts the parameters of a generator to produce higher-quality output. In the reinforcement learning framework, the generator is considered an agent, the generated sentences are considered actions in a certain environment, and the discriminator provides a reward signal.

[0155] Here is the specific process:

[0156] 1) Generate a statement: The generator produces a statement in response to an input condition.

[0157] 2) Quality evaluation: The discriminator evaluates the quality of the generated sentence and provides it with a score, i.e. reward (positive or negative).

[0158] 3) Updating the Generator: The generator uses a policy gradient algorithm to update its parameters. Policy gradients use the rewards given by the discriminator to adjust the generation probability distribution in the direction of increasing the expected reward, making the generator more likely to produce sentences that the discriminator considers to be of higher quality.

[0159] This process can be iterated continuously, with the ultimate goal of training a model that can generate high-quality text.

[0160] Adversarial training can output matrix X T , matrix X T =[R1U1,...,R T U T ] to represent the dialogue turn sequence, where U is the user utterance, R represents the system response, and T represents the number of turns. T represents the user's words in the T-th round of dialogue, R T Indicates that the system responds to U in the Tth round of dialogue T Response.

[0161] Therefore, if T is 3, then X T This is a six-column data string arranged in chronological order: R1U1, R2U2, R3U3, reflecting the complete conversation flow from round 1 to round 3. This method is effective in representing conversation structure and flow in machine learning and natural language processing, especially when building conversational systems.

[0162] In the Dialogue State Tracking (DST) task in a dialogue system, the system needs to track and understand the user's goals in each round of dialogue, and these goals are updated or changed as the dialogue progresses.

[0163] In each round of dialogue, the user's goal can be viewed as a certain number of domain-slot-value triplets.

[0164] Among them, domain-slot-value pairs: In order to express the user's goals, "domain-slot-value" triples are usually used. Among them:

[0165] 1) Domain refers to the specific area that the conversation focuses on, such as restaurant reservations, flight inquiries, etc.

[0166] 2) Slot refers to the category of information that users need to provide or inquire about in this area, such as time, date, number of people, etc.

[0167] 3) Value refers to the specific value or content of a slot. For example, in the field of "restaurant reservation", a possible slot is "date" and the corresponding value may be "September 1st".

[0168] The task of dialogue state tracking is to track X t The value of each slot on S (1≤t≤T). t represents the information introduced in round t, B t represents the information accumulated from the first round to the tth round. Dialogue state tracking means that in each round of dialogue (t), the system will try to maintain and update a record containing all relevant slots and their current values, which is the "dialogue state". Therefore, the task of DST is to determine X t Which slots have been filled or modified explicitly or indirectly by the user.

[0169] S t : represents the newly added dialogue state information in the (t)th round of dialogue. It allows the system to understand what new information the user mentioned or what old information was changed in this round.

[0170] B t : Represents all the accumulated dialogue state information from the first round to the present (i.e., round (t)). It is an aggregated view of the entire dialogue history during the dialogue state tracking process, representing a complete picture of the current dialogue state.

[0171] For example, in a travel booking conversation, the user might initially simply express their destination (revealing the value of "destination"), and then provide information such as the departure and return dates. The conversation state tracking task is responsible for continuously tracking and updating this information, enabling the conversation system to accurately understand and meet the user's needs.

[0172] In this embodiment, according to the turn-level dialogue state S t and the corresponding system response R t Generate user utterance U t .

[0173] The goal is to be based on the turn-level dialogue state S t and the system response R t , automatically generates the next sentence the user may say (U t This task usually appears in the Natural Language Generation (NLG) process, and its core steps can be summarized as follows:

[0174] Turn-level dialogue state S t: Refers to all user intent and contextual information identified and tracked by the system during the current conversation. This information, in the form of domain-slot-value pairs (triplets), reflects the specific goals the user hopes to achieve through the conversation.

[0175] System response R t :According to S t , the system provides answers or information designed to keep the conversation going, answer user questions, or satisfy user requests.

[0176] Generate user utterance U t After processing the user's request, we need to generate the user's next possible sentence. This is like simulating the user's role, inferring what the user might say next based on the previous conversation content and the system's response.

[0177] Specifically, the implementation process of the method shown in FIG4 is described in detail below.

[0178] Step 1: Data-driven structure search, outputting network structure units.

[0179] Data-driven architecture search generally refers to the use of large amounts of data to guide and optimize network structures in neural network architecture search. In this process, "outputting network structural units" means that the neural network architecture search system automatically discovers or selects the basic units (such as convolutional layers, recurrent layers, and attention mechanisms) that constitute the computational model based on a given dataset (such as data related to tasks such as image recognition and natural language processing). These units typically involve various types of operations and activation functions, which together form a complete model architecture.

[0180] The optional, data-driven structure search process is briefly described as follows:

[0181] 1) Define the search space: fix some basic components of the network architecture, such as possible layer types, activation functions, connection methods, etc.

[0182] 2) Design a controller: Set up a controller (such as a model designed using RNN) that will explore the previously defined search space through learning.

[0183] 3) Collect data: Obtain training data from relevant task domains. This data will be used to train and verify the performance of the sub-network and train the controller.

[0184] 4) Generate and evaluate sub-networks: The controller generates sub-network architectures based on the learned strategies and evaluates their performance on the training data.

[0185] 5) Update the controller: Based on the performance feedback of the sub-network, adjust the parameters of the controller so that it is more inclined to generate a more promising network architecture in the next round of iteration.

[0186] 6) Iteration process: Repeat the above steps until a sufficiently strong network architecture is generated or other termination conditions are met.

[0187] 7) Select the best network structure: Select the network structure unit with the best performance so far as the final product output.

[0188] Data-driven means that the entire neural network architecture search process is influenced and driven by data—from network architecture proposal to selection, everything is based on experimental performance on the dataset. This approach can reduce the time and effort required for manual neural network design, thereby improving efficiency and performance.

[0189] Optionally, choose Differentiable Architecture Search (DARTS) to implement neural architecture search on the Multi-Domain Wizard-of-Oz (MultiWOZ) dataset.

[0190] MultiWOZ is a multi-domain, task-based dialogue dataset containing dialogue samples from various scenarios (such as hotel reservations and restaurant searches). It is widely used for training and evaluating dialogue systems. We use DARTS, an automatic architecture search technique, to find the optimal recurrent neural network (RNN) unit structure for building dialogue systems trained on the MultiWOZ dataset.

[0191] As shown in Figure 6, DARTS represents the network as a directed acyclic graph (DAG) and searches for a suitable architecture on a given dataset. For a DAG, each node can be represented as:

[0192] Among them, x represents the feature map of the node, o represents the operation (such as convolution, maximum pooling, zeroing), i and j represent the sequence number of the node, x (j) Represents the feature map of the jth node, o (i,j) Represents the operation of applying the output of the i-th node and affecting the j-th node. The equation indicates that the intermediate node is the sum of all operations of all previous nodes. The entire equation clarifies the feature map x of each intermediate node. (j) It is achieved by taking the feature graph x of the predecessor node (the sequence number is less than the jth node) (i) Apply Operation (i,j) And add up their results.

[0193] In order to obtain all possible operations o between two nodes (i,j) , which can be expressed as:

[0194] Among them, O is the candidate operation set. ω represents the operation weight from the i-th node to the j-th node, which is also the parameter of the network structure to be searched. o (i,j) Represents the weight parameter of the specific operation O from the i-th node to the j-th node. The softmax function converts a set of unnormalized weights into normalized weights that are positive and sum to 1, so that each weight represents the probability of performing the corresponding operation. Softmax ensures that the sum of all operation weights is 1, making it possible to obtain a probability distribution. The equation indicates that each possible operation between the i-th node and the j-th node receives a normalized weight, and the results of these weighted operations are accumulated to form a mixed version of the output.

[0195] The goal of the search is to find the edge with the highest probability from all edges: (i,j) =argmax o∈O ω o (i,j)

[0196] argmax: It is the abbreviation of "maximum argument" in mathematical terms. It returns the argument when the function reaches its maximum value, rather than the maximum value of the function itself.

[0197] DARTS is used to search for a Recurrent Neural Network (RNN) unit suitable for the MultiWOZ dataset as a sequence to sequence (Seq2Seq) encoder. Based on the optimization goal of DARTS, a language model is constructed to complete the search for RNN units. In this embodiment, the MultiWOZ system response and user response [R T ,U T ] can search for an RNN unit structure with good performance, as shown in Figure 3. The search structure can be named SearchRNN.

[0198] Step 2: Combine the output of step 1 to form a generator, perform adversarial training, and output high-quality dialogue.

[0199] After obtaining the optimal network structure unit in step 1, proceed to step 2, which uses this structure unit to build a generator and further improve the quality of the dialogue text through adversarial training:

[0200] 1) Build the generator: Use the optimal network structure unit output in Step 1 to build a generator model that can generate dialogue texts.

[0201] 2) Initialize the discriminator: Create a discriminator model that is specifically designed to distinguish between real dialogue texts and those generated by the generator.

[0202] 3) Adversarial training loop:

[0203] 31) The generator attempts to generate dialogue texts that conform to the data distribution.

[0204] 32) The discriminator evaluates the difference between the generated text and the real text and sends this evaluation result back as feedback to the generator.

[0205] 33) The generator focuses on improving specific aspects that fail the discriminator test based on the feedback.

[0206] This process loops continuously, and as the generator and discriminator interact, the output quality of the generator gradually improves.

[0207] 4) Continuously optimize until the goal is met: When the generated dialogue texts reach a quality similar to real texts, or when the evaluation metrics reach a preset threshold such that the discriminator cannot easily distinguish between real and generated texts, the adversarial training is completed.

[0208] 5) Output high-quality dialogues: The generator is optimized to be sufficient to produce high-quality and credible dialogue texts, and this generator can be used in dialogue systems or other applications that require such natural language generation.

[0209] Through this process, the machine learning model can not only generate more natural and user-friendly dialogues, but also continuously improve itself until the quality of its output dialogues meets the requirements of the application scenario.

[0210] Generator: The generator defines the strategy for generating the user utterance U t from the given dialogue history R t and the turn-level user goal S t . Its form is similar to the Seq2Seq model and consists of an encoder and a decoder. In this paper, a generator based on the Gated Recurrent Unit (GRU) and SearchRNN is used to approximate P(U t 丨R t , S t ), where the concatenation of R t and S t is used as the input to the encoder, and U t is set as the target sequence generated by the decoder.

[0211] Dialogue history R t : Represents the previous dialogue content between both parties (user and system) up to time (t), which may include several rounds of dialogue exchanges.

[0212] Turn-level user goal S t : Represents the specific goal that the user wants to achieve in the dialogue. It can change (i.e., turn) over time and be influenced by the dialogue history.

[0213] User utterance U t : Represents the user's speech or response at time (t).

[0214] P(U t 丨R t ,S t ): Represents the conditional probability of the occurrence of the next user utterance U given the known dialogue history R t and the turn-level user goal S t . The task of the generator is to approximate this probability distribution, that is, to generate a reasonable and natural user response based on the current dialogue context. t The discriminator is a binary classifier used to determine whether the user utterance is generated or from the original dataset. To understand the belief state condition, the concatenation of the turn-level belief state and the user utterance is used as the input to the discriminator. For example, following the settings of SeqGAN, a Convolutional Neural Network (CNN) is used as the backbone model of the discriminator. SeqGAN refers to the Sequence Generative Adversarial Networks. First, the input sequence is represented as

[0215] where each token is represented as a k-dimensional token embedding, and is the concatenation operator for building the input matrix. Second, the kernel applies a convolutional operation to the word window size to generate a new feature map and performs a max-over-time pooling operation. Finally, the output vector of a fully connected layer is fed into a two-class sigmoid activation layer, which returns the probability that the input discourse is generated by the generator or from the original dialogue. The specific steps are as follows:

[0216] 1) Input sequence representation: Concatenate the two sequences U

[0217] and S t ( t represents the concatenation operator), which represent two different input sequences respectively (for example, in a dialogue system, U t ​Can be user statement, S t is the system response), and each token (word or character) is represented as a (k)-dimensional embedding vector.

[0218] 2) Embedding and concatenation to form a matrix: Convert words into dense vectors through word embedding, and concatenate these embedded vectors in sequence to form the input matrix.

[0219] 3) Convolution operation: A series of convolution kernels (or filters) are applied to the word windows of the embedding matrix, with each convolution kernel covering a specific number of words to extract local features from the data. The result is a series of new feature maps, each corresponding to a convolution kernel.

[0220] 4) Max-overtime pooling: Each feature map passes through a max-pooling layer, usually to reduce the dimensionality of the features and extract the most prominent signal, that is, the maximum value in each feature map.

[0221] 5) Fully connected layer: The output after convolution and pooling is sent to one or more fully connected layers, which can integrate all the previous local information for further processing.

[0222] 6) Sigmoid Activation Layer: The output of the fully connected layer is finally fed into the sigmoid activation function for binary classification. Here, the sigmoid function will generate a probability value between 0 and 1, indicating the probability that the input sequence was generated by the generator (relative to the probability that it originated from the original human-created conversation data).

[0223] Adversarial training (solid and hollow arrow flows in Figure 5): State-aware dialogue generation is formulated as a reinforcement learning problem. The error calculated by the discriminator is backpropagated to the generator using a policy gradient algorithm. The generator can be viewed as an agent whose parameters θ define a policy. At each time step, it takes an action by generating a token and receives a reward from the discriminator through Monte Carlo search.

[0224] The above process treats state-aware dialogue generation as a reinforcement learning problem. The specific workflow of backpropagating the error calculated by the discriminator to the generator through the policy gradient algorithm is as follows:

[0225] 1) State-aware dialogue generation: This means that the dialogue system must not only generate grammatically correct and logically coherent sentences, but also understand the context of the dialogue and maintain the dialogue state, such as tracking user intent and considering previous interactions.

[0226] 2) Reinforcement Learning Problem: In this setting, the task of generating dialogue is viewed as a sequential decision-making problem. At each step, the generator (i.e., the agent) must make decisions (output tokens or words) based on the current state (dialogue context) and aim to maximize long-term rewards (e.g., generating natural, fluent, and purposeful dialogue).

[0227] 3) Generator as an agent: The generator contains parameters θ, which define a "policy", that is, the probability distribution of choosing the next word given the dialogue state.

[0228] The parameter θ can represent the set of parameters of the generator. In machine learning and especially neural networks, parameters usually refer to model weights and biases, which determine the specific behavior and output of the model.

[0229] For a generator, these parameters control how to generate an output sequence (in this case, tokens or words) based on the input (such as the state or dialogue context mentioned above). At the beginning of training, these parameters are usually randomly initialized and then gradually adjusted by the learning algorithm so that the generator can optimize its performance according to the desired objective function (for example, the reward signal in the reinforcement learning framework).

[0230] When using policy gradient methods for reinforcement learning, the generator's policy (i.e., the probability distribution of the next token) can be improved by updating the parameters θ based on the reward signal generated by the policy. These parameters are adjusted by calculating the gradient of the loss function with respect to the parameters and applying the backpropagation algorithm. Each parameter update aims to improve the quality of the generated output, that is, to generate more realistic text that is more consistent with the task objectives.

[0231] 4) Policy Gradient Algorithms: These algorithms optimize the policy by adjusting the generator's parameters θ to maximize the expected total reward for the generated dialogue. Optimization is typically performed by calculating the gradient of the expected reward and then adjusting the parameters to increase the reward.

[0232] 5) Backpropagation of the error calculated by the discriminator: The error here usually refers to the signal obtained from the discriminator, indicating the difference between the generated dialogue and the real dialogue. This error can be converted into a score or a paired reward value.

[0233] 6) Monte Carlo Search: To estimate the reward value for each action (word), the Monte Carlo method is used to simulate the entire dialogue process and calculate the reward. This usually involves taking multiple actions in a leading manner from the current step until a termination condition is reached or the dialogue ends, and recording the cumulative reward.

[0234] 7) Obtaining the reward value: After a series of actions are completed using the Monte Carlo method, the reward value for each step of the action is calculated retrospectively based on the final result.

[0235] 8) Time step and generated token: At each time point during the process of generating a conversation, the generator outputs a token (such as a word or a character) based on its policy, which is equivalent to taking an action in the terms of reinforcement learning.

[0236] Converting the dialogue generation into a reinforcement learning problem can bring more flexible dialogue management and has the potential to generate more natural and personalized text responses. This method combining reinforcement learning attempts to create an intelligent dialogue agent that can operate effectively in various dialogue scenarios.

[0237] The true probability estimated by the discriminator is used to calculate the reward:

[0238] where U represents the sequence generated by the generator, u l is the l-th word (token) in the sequence U generated by the generator, S represents some conditional information, such as the previous dialogue content or other relevant context, and in the design of some dialogue systems, it can be used as a clue to guide the generator to generate responses. D φ (U 1:l 丨S) represents the discriminator D, parameterized with parameters φ, calculating the authenticity probability of all the generated words U1, U2,..., U l given the context S. That is, the discriminator evaluates how likely this partial sequence is to be a real dialogue. represents the reward for U l , and φ is the parameter of the discriminator. Since the role of the discriminator in GAN is usually to distinguish between generated data and real data, so here D φ (U 1:l 丨S) output value (usually a probability value between 0 and 1) is directly used as the reward A high probability value means a high reward, indicating that the discriminator believes that the currently generated partial sequence is very "real".

[0239] The goal of the generator is to minimize the negative expected reward of the generated statement using a reinforcement learning algorithm (e.g., the REINFORCE algorithm): J G (θ) = -E U～G ​​​​​​​(U|S) represents the probability of the discriminator estimating the authenticity of the statement (S) under the given context (S), which is controlled by the parameter (θ).

[0241] J G (θ) represents that the generator generates the statement (U) under the parameter (θ), and then the discriminator (D) gives the average "true" score (i.e., reward) for these statements. The expected value of all these scores can be taken and negated to form the loss function to be minimized. It can be understood that the generator hopes to maximize the authenticity score of the statements it generates. Therefore, it tries to generate more and more realistic statements by adjusting the parameter (θ).

[0242] Using the likelihood ratio method, the gradient of θ can be derived as follows:

[0243] where J G (θ) represents the objective function of the generator, and in this embodiment, this function needs to be minimized, that is, to increase the probability that the statement U generated by the generator is recognized as true by the discriminator D φ (U|S).

[0244] represents the gradient of the objective function J G (θ), that is, how a certain change in the parameter θ will affect the value of the objective function.

[0245] represents the gradient of the expected objective function.

[0246] represents the approximation of the Monte Carlo estimate for the sampled data. The negative sign indicates that we want to update the parameter (θ) by gradient ascent (instead of descent), that is, we hope to increase the probability of the generated statements that can be considered "true" by the discriminator.

[0247] represents the gradient of the log-likelihood (logG(U|S)) with respect to the parameter (θ), and in machine learning, this is obtained through the backpropagation algorithm. [[ID=3​​​​

[0250] Among them, D φ (U|S) is the probability that U comes from the original conversation,

[0251] U ∼ ground - truth means that the statement U comes from the label of the target.

[0252] E U～ground-truth logD φ (U|S) represents the expected logarithm of the probability that the discriminator correctly identifies a statement as a real statement when the statement is true (from ground - truth), that is, when the statement (U) is generated by a real conversation.

[0253] E U～G log(1 - D φ (U|S)) represents the expected logarithm that the discriminator correctly identifies a statement as a generated statement (assigning it a low probability score, so (1 - D φ (U|S)) should be high) when the statement is generated by the generator, that is, when the statement (U) is a generated utterance.

[0254] The discriminator tries to minimize this joint loss function by adjusting its parameters (φ), that is, to improve its ability to identify real and generated statements. That is, the discriminator attempts to enable itself to accurately give a high probability to identify real conversations and a low probability to generated conversations, so as to distinguish between real data and data generated by the generator.

[0255] Automatic and manual evaluation of the output dialogue data: The quality of the generated statements is measured by the Bilingual Evaluation Understudy (BLEU) score and the Bidirectional Encoder Representations from Transformers (BERT) score.

[0256] Table 1 shows the experimental results on the MultiWOZ 2.0 and MultiWOZ 2.1 datasets in the full - scale dataset and low - resource scenarios respectively. The first and third rows in Table 1 are experimental comparisons without adversarial training. It can be seen that replacing the encoder with the searched RNN cell structure generally improves the MLE training results by about 1%. In addition, adding adversarial training on this basis can effectively improve the upper limit of MLE training (the last row in the table). Especially in the experiment on the MultiWOZ 2.1 dataset, since the data is cleaner and has fewer annotation errors than MultiWOZ 2.0, the BLEU - 2 score even increases by 3%. <00OO615>

[0257] Table 1: Automatic evaluation of MLE and adversarial training in full dataset and low-resource scenarios.

[0258] Here, Model refers to the model, Full Datasets refers to the full dataset, and Low-resource scenario refers to the low-resource scenario.

[0259] To further explore the performance of the disclosed framework in low-resource scenarios, 100 complete conversation examples were randomly sampled from the training dataset, and 50 complete conversation examples were randomly sampled from the validation dataset. The last four rows in the table show the performance of the two models in low-resource environments on MultiWOZ 2.0 and MultiWOZ 2.1. As expected, performance will degrade to varying degrees. On the other hand, the performance improvement under the same settings demonstrates the effectiveness of the framework of this embodiment in low-resource environments.

[0260] 100 instances generated by the model were randomly selected, and three experts were invited to manually evaluate the data. The generated data was evaluated from two perspectives: sentence fluency and slot accuracy. Sentence fluency indicates whether the generated sentences conform to natural human expression and are suitable as answers to questions. Slot accuracy indicates whether the generated sentences contain the input dialogue state. Table 2 shows the manual evaluation results of the naturalness and controllability of the GRU-based and SearchRNN-based generators on the MultiWOZ 2.0 dataset. As can be seen from Table 2, compared with the GRU-based model, the SearchRNN-based model generator can generate smoother responses and the generated responses contain more comprehensive belief states. This indicates that the model has a stronger perception of belief states.

[0261] Table 2: Results of manual evaluation.

[0262] This embodiment proposes a business-oriented end-to-end artificial intelligence (AI) closed-loop global optimization technology that uses adversarial training and neural architecture search to achieve an automatic closed loop from end-to-end dialogue generation to dialogue evaluation. This technology is of great significance in the rapidly developing field of AI. It can improve the performance of dialogue systems and provide users with a higher quality, more accurate and more intelligent dialogue experience. In addition, this technology has broad application potential. In a business environment, improving the performance of dialogue systems can increase customer service efficiency, reduce customer waiting time, and improve user satisfaction. For example, the 10086 customer service robot can solve user problems more quickly and provide more accurate information, thereby enhancing the company's image and customer loyalty. Improving the performance of dialogue systems will also help provide better services and user experience.

[0263] 7 , an embodiment of the present disclosure provides a training device for a dialogue generation model. The device 700 includes:

[0264] A first acquisition module 701 is used to acquire a network structure unit for building an adversarial dialogue system;

[0265] The training module 702 is used to determine an adversarial training dialogue model based on the network structure unit, and perform adversarial training on the adversarial training dialogue model to obtain a trained adversarial training dialogue model, where the adversarial training dialogue model includes a generator and a discriminator.

[0266] In one embodiment of the present disclosure, the first acquisition module is further used to: obtain the optimal network structure unit for constructing an adversarial dialogue system by applying neural architecture search on a specific data set.

[0267] In one embodiment of the present disclosure, the training module is further used to: perform adversarial training on the adversarial training dialogue model, obtain indication information output by the discriminator, and the indication information is used to indicate the quality of the sentences generated by the generator; and adjust the network parameters of the generator according to the indication information.

[0268] In one embodiment of the present disclosure, the training module is further used to: send the indication information to the generator through a policy gradient algorithm to adjust the network parameters of the generator.

[0269] Optionally, in the adversarial training of the adversarial training dialogue model, the input statement of the discriminator contains current state related information.

[0270] The device provided in the embodiment of the present disclosure can implement each process implemented by the method embodiment shown in Figure 1 and achieve the same technical effect. To avoid repetition, it will not be described here.

[0271] 8 , an embodiment of the present disclosure provides a dialog generation apparatus, which is applied to a server running an adversarial training dialog model. The dialog generation model is a model trained using the method described in FIG. 1 . The apparatus 800 includes:

[0272] The second acquisition module 801 is used to acquire the context of the conversation;

[0273] A first processing module 802 is configured to obtain one or more candidate utterances corresponding to the context of the conversation through a generator of the adversarial training conversation model;

[0274] A third acquisition module 803 is configured to obtain the quality of each candidate utterance through the discriminator of the adversarial training dialogue model;

[0275] The second processing module 804 is configured to select the candidate utterance that matches the context of the conversation and has the highest quality as the target utterance.

[0276] The device provided in the embodiment of the present disclosure can implement each process implemented by the method embodiment shown in Figure 2 and achieve the same technical effect. To avoid repetition, it will not be described here.

[0277] As shown in FIG9 , an embodiment of the present disclosure further provides an electronic device 900, comprising a processor 901, a memory 902, and a program or instruction stored in the memory 902 and executable on the processor 901. When executed by the processor 901, the program or instruction implements the various processes of the method embodiments of FIG1 or FIG2 above, and can achieve the same technical effects. To avoid repetition, detailed description is omitted here.

[0278] The embodiments of the present disclosure also provide a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the method embodiment shown in Figure 1 or Figure 2 above are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0279] The processor is the processor in the terminal described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk.

[0280] The steps of the method or algorithm described in conjunction with the present disclosure can be implemented in hardware or by executing software instructions on a processor. The software instructions can be composed of corresponding software modules, which can be stored in RAM, flash memory, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, read-only optical disks, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be carried in an application-specific integrated circuit (ASIC). In addition, the ASIC can be carried in a core network interface device. Of course, the processor and the storage medium can also exist as discrete components in the core network interface device.

[0281] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in this disclosure can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0282] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present disclosure in detail. It should be understood that the above description is only a specific implementation method of the present disclosure and is not intended to limit the scope of protection of the present disclosure. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the present disclosure should be included in the scope of protection of the present disclosure.

[0283] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the embodiments of the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the embodiments of the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, Compact Disc Read-Only Memory (CD-ROM), optical storage, etc.) containing computer-usable program code.

[0284] The present disclosure embodiments are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present disclosure embodiments. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0285] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0286] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0287] Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the embodiments of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include such modifications and variations.

Claims

1. A training method for a dialogue generation model, comprising: Obtaining network structure units for constructing an adversarial dialogue system; Determining an adversarial training dialogue model according to the network structure units, and performing adversarial training on the adversarial training dialogue model to obtain a trained adversarial training dialogue model, where the adversarial training dialogue model includes a generator and a discriminator.

2. The method according to claim 1, wherein Obtaining network structure units for constructing an adversarial dialogue system includes: Obtaining the optimal network structure units for constructing an adversarial dialogue system by applying neural architecture search to a specific data set.

3. The method according to claim 1, wherein, Performing adversarial training on the adversarial training dialogue model includes: Performing adversarial training on the adversarial training dialogue model to obtain indication information output by the discriminator, where the indication information is used to indicate the quality of the statements generated by the generator; Adjusting the network parameters of the generator through the indication information.

4. The method according to claim 3, wherein Adjusting the network parameters of the generator through the indication information includes: Sending the indication information to the generator through a policy gradient algorithm to adjust the network parameters of the generator.

5. The method according to claim 3, wherein During the adversarial training of the adversarial training dialogue model, the statements input to the discriminator include information related to the current state.

6. A dialogue generation method, applied to a server running an adversarial training dialogue model, where the dialogue generation model is a model trained by the method according to any one of claims 1 to 5, and the method includes: Obtaining the context of the dialogue; Obtaining one or more candidate utterances corresponding to the context of the dialogue through the generator of the adversarial training dialogue model; Obtaining the quality of each candidate utterance through the discriminator of the adversarial training dialogue model; Taking the candidate utterance that matches the context of the dialogue and has the highest quality as the target utterance.

7. A training device for a dialogue generation model, comprising: A first obtaining module, configured to obtain network structure units for constructing an adversarial dialogue system; A training module, configured to determine an adversarial training dialogue model according to the network structure units, and perform adversarial training on the adversarial training dialogue model to obtain a trained adversarial training dialogue model, where the adversarial training dialogue model includes a generator and a discriminator.

8. The apparatus according to claim 7, wherein The first obtaining module is further configured to: obtain the optimal network structure units for constructing an adversarial dialogue system by applying neural architecture search to a specific data set.

9. The apparatus according to claim 7, wherein, The training module is further configured to: perform adversarial training on the adversarial training dialogue model to obtain indication information output by the discriminator, where the indication information is used to indicate the quality of the statements generated by the generator; adjust the network parameters of the generator through the indication information.

10. The device according to claim 9, wherein, The training module is further configured to: send the indication information to the generator through a policy gradient algorithm to adjust the network parameters of the generator.

11. A dialogue generation device, applied to a server running an adversarial training dialogue model, where the dialogue generation model is a model trained by the method according to any one of claims 1 to 5, and the device includes: A second obtaining module, configured to obtain the context of the dialogue; The first processing module is configured to obtain one or more candidate utterances corresponding to the context of the dialogue through the generator of the adversarial training dialogue model; The third acquisition module is configured to obtain the quality of each candidate utterance through the discriminator of the adversarial training dialogue model; The second processing module is configured to use the candidate utterance that matches the context of the dialogue and has the highest quality as the target utterance.

12. An electronic device, comprising a processor, a memory, and a program or instruction stored on the memory and executable on the processor, wherein when the program or instruction is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

13. A readable storage medium, having a program or instruction stored thereon, wherein when the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images