Dialogue state generation method and model training method, device, medium and equipment

Abstract

The application discloses a dialogue state generation method, a model training method, a device, a medium and equipment, wherein the dialogue state generation method comprises the following steps: obtaining a last round dialogue state, historical dialogue information and current round dialogue information; performing prediction processing on the last round dialogue state, the historical dialogue information and the current round dialogue information based on a slot detector in a dialogue state generation model to obtain a last round to-be-corrected slot and a current round updated slot; performing prediction processing on the last round dialogue state, the historical dialogue information, the current round dialogue information, the last round to-be-corrected slot and the current round updated slot based on a dialogue information generator in the dialogue state generation model to obtain a current round dialogue state. The last round dialogue state is corrected in the prediction process of the current round dialogue state, the interference of the last round error information on the current round prediction is avoided, and the prediction accuracy of the current round dialogue state is improved.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to methods for generating dialogue states, methods for training models, apparatus, media and devices. Background Technology

[0002] Dialogue State Tracking (DST) is an important component of task-oriented dialogue systems. DST extracts the user's target slots from each turn of the dialogue and then executes the user's requests through subsequent processes. It is widely used in fields such as intelligent customer service, intelligent transportation, and intelligent office.

[0003] In the process of realizing this invention, it was found that at least the following technical problems exist in the prior art: erroneous information in historical dialogues during the current dialogue state tracking process will affect the accuracy of subsequent dialogue state prediction. Summary of the Invention

[0004] This invention provides a method for generating dialogue states, a model training method, an apparatus, a medium, and a device to address the interference of erroneous information in historical dialogues on the prediction of subsequent dialogue states and improve the accuracy of dialogue state prediction.

[0005] According to one aspect of the present invention, a method for generating a dialogue state is provided, comprising:

[0006] Get the dialogue status of the previous round, historical dialogue information, and current round dialogue information;

[0007] Based on the slot detector in the dialogue state generation model, the dialogue state of the previous round, historical dialogue information and the dialogue information of the current round are predicted to obtain the slot to be corrected in the previous round and the updated slot in the current round.

[0008] Based on the dialogue information generator in the dialogue state generation model, the dialogue state of the previous round, the historical dialogue information, the dialogue information of the current round, the slot to be corrected in the previous round, and the updated slot of the current round are predicted to obtain the dialogue state of the current round.

[0009] Optionally, the slot detector includes an encoding module, an attention module, and a slot detection module;

[0010] The slot detector in the dialogue state generation model performs prediction processing on the dialogue state of the previous round, historical dialogue information, and dialogue information of the current round to obtain the slots to be corrected in the previous round and the updated slots in the current round. This includes: encoding the dialogue state of the previous round, the historical dialogue information, and the dialogue information of the current round based on the encoding module to obtain an encoding vector; performing attention mechanism processing on the encoding vector based on the attention module to obtain an attention vector; and performing detection processing on the attention vector based on the slot detection module to obtain the slots to be corrected in the previous round and the updated slots in the current round.

[0011] Optionally, the slot detection module includes a first probability prediction unit, a second probability prediction unit, and a detection unit;

[0012] The process of detecting and processing the attention vector based on the slot detection module to obtain the slots to be corrected in the previous round and the updated slots in the current round includes: transforming the slot features in the attention vector based on the first probability prediction unit to obtain the activation probability of each slot in the previous round; transforming the slot features in the attention vector based on the second probability prediction unit to obtain the update probability of each slot in the current round; determining the slots to be corrected in the previous round based on the dialogue state in the previous round and the activation probability of each slot in the previous round based on the detection unit, and determining the updated slots in the current round based on the update probability of each slot in the current round.

[0013] Optionally, the slots to be corrected in the previous round include over-predicted slots and / or under-predicted slots;

[0014] The detection unit determines the slots to be corrected in the previous round based on the dialogue state of the previous round and the activation probability of each slot in the previous round, and determines the updated slots in the current round based on the update probability of each slot in the current round, including: determining the theoretically activated slots in the previous round based on the activation probability and activation threshold of the previous round; determining the actual activated slots in the previous round based on the dialogue state of the previous round; determining over-predicted slots and / or under-predicted slots based on the theoretically activated slots and the actual activated slots in the previous round; and determining the updated slots in the current round based on the update probability and update threshold of each slot in the current round.

[0015] Optionally, the encoding module includes multiple slot encoders and a context encoder;

[0016] The encoding module performs encoding processing on the previous round's dialogue state, the historical dialogue information, and the current round's dialogue information to obtain an encoding vector, including: encoding slot information based on the slot encoder to obtain a slot vector; and encoding concatenated information formed based on the previous round's dialogue state, the historical dialogue information, and the current round's dialogue information based on the context encoder to obtain a context vector.

[0017] According to another aspect of the present invention, a method for training a dialogue state generation model is provided, comprising:

[0018] Multiple historical dialogue information is acquired, and error simulation is performed on the multiple historical dialogue information to obtain a simulation sample set. The simulation sample set includes first sample data with error information and second sample data without error information.

[0019] The dialogue state generation model to be trained is trained based on the simulated sample set to obtain a trained dialogue state generation model. The dialogue state generation model includes a slot detector and a dialogue information generator. During the training process, a first loss function of the slot detector and a second loss function of the dialogue information generator are obtained respectively. The model parameters of the slot detector are adjusted based on the first loss function, and the model parameters of the dialogue information generator are adjusted based on the second loss function.

[0020] Optionally, the step of performing error simulation on the plurality of historical dialogue information to obtain a simulation sample set includes: for any one of the historical dialogue information, setting the non-empty slot value in the historical dialogue information to empty based on a first probability; and / or, for any one of the historical dialogue information, setting the non-empty slot value in the historical dialogue information to other slot values ​​based on a second probability.

[0021] Optionally, the method further includes:

[0022] For each simulated sample in the simulated sample set, a slot error correction label is generated based on the simulated sample and the dialogue history data before the error simulation; a slot update label is generated based on the next round dialogue state corresponding to the simulated sample.

[0023] Accordingly, obtaining the first loss function of the slot detector includes: processing the simulated sample based on the slot detector to obtain the slot activation probability and slot update probability, and generating the first loss function based on the slot activation probability, the slot update probability, the slot error correction label and the slot update label.

[0024] Optionally, the method further includes: generating slot labels based on the simulated sample corresponding to the next round of dialogue state;

[0025] Accordingly, obtaining the second loss function of the dialogue information generator includes: generating a second loss function based on the slot prediction probability output by the dialogue information generator and the slot label.

[0026] According to another aspect of the present invention, a dialogue state generation apparatus is provided, comprising:

[0027] The information acquisition module is used to acquire the dialogue status of the previous round, historical dialogue information, and current round dialogue information;

[0028] The information generation module is used to perform predictive processing on the dialogue state of the previous round, historical dialogue information, and current round dialogue information based on the slot detector in the dialogue state generation model to obtain the slot to be corrected in the previous round and the updated slot in the current round; and to perform predictive processing on the dialogue state of the previous round, historical dialogue information, current round dialogue information, slot to be corrected in the previous round, and updated slot in the current round based on the dialogue information generator in the dialogue state generation model to obtain the dialogue state of the current round.

[0029] According to another aspect of the present invention, a training apparatus for a dialogue state generation model is provided, comprising:

[0030] The sample processing module is used to acquire multiple historical dialogue information, perform error simulation on the multiple historical dialogue information, and obtain a simulation sample set. The simulation sample set includes first sample data with error information and second sample data without error information.

[0031] The model training module is used to train the dialogue state generation model to be trained based on the simulated sample set, so as to obtain a trained dialogue state generation model. The dialogue state generation model includes a slot detector and a dialogue information generator. During the training process, a first loss function of the slot detector and a second loss function of the dialogue information generator are obtained respectively. The model parameters of the slot detector are adjusted based on the first loss function, and the model parameters of the dialogue information generator are adjusted based on the second loss function.

[0032] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0033] At least one processor; and

[0034] A memory communicatively connected to the at least one processor; wherein,

[0035] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to execute the dialogue state generation method and / or the training method of the dialogue state generation model according to any embodiment of the present invention.

[0036] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the dialogue state generation method according to any embodiment of the present invention, and / or, the training method for the dialogue state generation model.

[0037] The technical solution of this invention, by setting a slot detector, detects the dialogue state of the previous round, historical dialogue information, and the dialogue information of the current round to obtain the slots to be corrected in the previous round and the updated slots in the current round. The slots to be corrected are used to prompt the dialogue information generator to correct errors that occurred in the previous round, and the updated slots are used to prompt the dialogue information generator to determine the slot value of the updated slots. This achieves the correction of the previous round's dialogue state during the prediction process of the current round's dialogue state, avoiding interference from previous round's erroneous information on the current round's prediction, and improving the prediction accuracy of the current round's dialogue state.

[0038] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart of a dialogue state generation method provided in an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram of the structure of a dialogue state generation model provided in an embodiment of the present invention;

[0042] Figure 3 This is a flowchart of a training method for a dialogue state generation model provided in an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of the structure of a dialogue state generation device provided in an embodiment of the present invention;

[0044] Figure 5 This is a schematic diagram of the structure of a training device for a dialogue state generation model provided in an embodiment of the present invention;

[0045] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0046] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0047] It should be noted that the terms "first probability prediction unit," "second probability prediction unit," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0048] Figure 1 This is a flowchart of a dialogue state generation method provided by an embodiment of the present invention. This embodiment is applicable to correcting historical dialogue information during the process of dialogue state prediction and predicting the current round of dialogue state based on the error correction information. This method can be executed by a dialogue state generation device, which can be implemented in hardware and / or software and can be configured in electronic devices such as mobile terminals such as mobile phones, computers, and servers. Figure 1 As shown, the method includes:

[0049] S110: Obtain the dialogue status of the previous round, historical dialogue information, and current round dialogue information.

[0050] S120. Based on the slot detector in the dialogue state generation model, predict the dialogue state of the previous round, historical dialogue information and the dialogue information of the current round to obtain the slot to be corrected in the previous round and the updated slot in the current round.

[0051] S130. Based on the dialogue information generator in the dialogue state generation model, perform prediction processing on the dialogue state of the previous round, the historical dialogue information, the dialogue information of the current round, the slot to be corrected in the previous round, and the updated slot of the current round to obtain the dialogue state of the current round.

[0052] In this embodiment, there are multiple rounds of dialogue state prediction during the dialogue state tracking process. In each round of dialogue state prediction, the dialogue state of the previous round and the historical dialogue information of the previous multiple rounds of dialogue are used as reference information. At the same time, the dialogue information of the current round is obtained. The above reference information and the dialogue information of the current round are used as the input information for the current round prediction. The prediction is performed by a pre-set dialogue state generation model to obtain the dialogue state of the current round.

[0053] The previous dialogue state can be the prediction result output by the dialogue state generation model in the previous prediction process, and can be represented as a concatenated sequence of slots and slot values. Historical dialogue information can include dialogue information from previous multiple rounds of dialogue. For the first prediction round, both the previous dialogue state and historical dialogue information are empty.

[0054] The dialogue information for the current round includes user dialogue information and system dialogue information. Optionally, it can involve obtaining the dialogue interaction information between the user and the system in the current round, extracting key information from the dialogue interaction information, and determining the extracted key information as the dialogue information for the current round.

[0055] In this embodiment, a dialogue state generation model is pre-trained, and the dialogue state is predicted for each round using this model. The dialogue state generation model includes a slot detector and a dialogue information generator. The slot detector identifies erroneous slots (slots to be corrected) among the predicted slots. Simultaneously, the slot detector also determines updated slots relative to the predicted slots. Updated slots can be newly added active slots in the current round, or slots whose values ​​have changed relative to the predicted slots.

[0056] The dialogue information generator connects to the slot detector, taking the detector's output as input to obtain the dialogue state for the current round. This state is a concatenated sequence of the slots and their values ​​output for the current round. Through the cooperation of the slot detector and the dialogue information generator, slots to be corrected and updated are identified from the predicted slots. These serve as references and prompts for the generator to produce the current round's dialogue state, improving its accuracy.

[0057] Based on the above embodiments, the slot detector includes an encoding module, an attention module, and a slot detection module. Optionally, the slot detector in the dialogue state generation model performs prediction processing on the previous round's dialogue state, historical dialogue information, and the current round's dialogue information to obtain the slots to be corrected in the previous round and the updated slots in the current round, including: encoding the previous round's dialogue state, historical dialogue information, and current round's dialogue information based on the encoding module to obtain an encoding vector; performing attention mechanism processing on the encoding vector based on the attention module to obtain an attention vector; and performing detection processing on the attention vector based on the slot detection module to obtain the slots to be corrected in the previous round and the updated slots in the current round.

[0058] The encoding module is used to encode the previous round's dialogue state, the historical dialogue information, and the current round's dialogue information to obtain an encoding vector. Specifically, the encoding module includes multiple slot encoders and a context encoder. The slot encoders encode slot information to obtain slot vectors, and the context encoders encode concatenated information formed based on the previous round's dialogue state, the historical dialogue information, and the current round's dialogue information to obtain a context vector. Correspondingly, the encoding module's process of encoding the previous round's dialogue state, the historical dialogue information, and the current round's dialogue information to obtain an encoding vector includes: encoding slot information using the slot encoders to obtain slot vectors; and encoding concatenated information formed based on the previous round's dialogue state, the historical dialogue information, and the current round's dialogue information using the context encoder to obtain a context vector.

[0059] In some embodiments, the slot encoder can be a BERT model; the specific form of the slot encoder is not limited here, as long as the slot information is encoded. Optionally, the slot information can be a slot name. Before inputting the slot information into the slot encoder, the method further includes setting an identifier for the slot information, which includes a start identifier and an end identifier. The start identifier can be CLS, and the end identifier can be SEP. The start identifier, slot information, and end identifier are concatenated to form the input information for the slot encoder.

[0060] Taking the slot encoder as an example in the BERT model, the process of the slot encoder processing slot information can be represented as follows: ,in, This refers to the information for the j-th slot. For the pre-trained model BERT with fixed parameters, The output is the special character [CLS] for slot information, where d is the size of the hidden vector dimension. This indicates a sequence cascading operation.

[0061] The slot encoder in the slot detector can be multiple, and the number of slot encoders can be determined based on the number of slots in the business scenario. In some embodiments, the dialogue state generation model predicts the dialogue state for a fixed business scenario, and correspondingly, the number of slot encoders in the slot detector of the dialogue state generation model is determined based on the number of slots in that fixed business scenario. In some embodiments, the dialogue state generation model can predict the dialogue state for multiple business scenarios, and correspondingly, the number of slot encoders in the slot detector of the dialogue state generation model can be determined based on the number of slots corresponding to each of the multiple business scenarios. For example, the maximum number of slots in each business scenario can be determined as the number of slot encoders.

[0062] When the dialogue state generation model is designed for a fixed business scenario, the fixed business scenario corresponds to a fixed number of slot information. In any round of prediction, the above multiple slot information is called and input into each slot encoder to obtain multiple slot vectors.

[0063] When the dialogue state generation model can predict multiple business scenarios, the current business scenario can be determined, and multiple slot information of the current business scenario can be called and input into each slot encoder to obtain multiple slot vectors.

[0064] The context encoder encodes the dialogue state of the previous round, historical dialogue information, and current round dialogue information. Before being input into the context encoder, the dialogue state of the previous round, historical dialogue information, and current round dialogue information are concatenated. Specifically, the dialogue state of the previous round, historical dialogue information, and current round dialogue information are concatenated based on a start identifier, a connection identifier, and an end identifier. There can be multiple connection identifiers, and each connection identifier can be different, used to represent the type of information being connected.

[0065] For example, by Representing historical dialogue information, through Indicates the state of the previous dialogue, via This represents the dialogue information for the current round; correspondingly, the input information of the context encoder can be represented as... ,in, and These are connection identifiers. The current round's dialogue information is also included. ,in, For system dialogue information, This refers to user dialogue information. Historical dialogue information is a collection of multi-turn dialogue information, which can be represented as... The previous dialogue state could be... ,in, For slot information, The slot value for wheel t-1.

[0066] The input information is encoded using a context encoder to obtain a context vector, which can be represented by the following formula: ,in, Characterization context encoder. For the hidden state of the context encoder, It is the length of the input sequence.

[0067] The attention module is connected to the encoding module and processes the obtained encoding vector using an attention mechanism to obtain an attention vector. The attention module includes a slot-word attention unit and a slot self-attention unit. The slot-word attention unit is used to determine the slot attention vector between the slot vector and the context vector. Optionally, the slot-word attention unit can be implemented using a multi-head attention mechanism. For example, the slot attention vector can be generated as follows: . It is a representation specific to the j-th slot.

[0068] The slot self-attention unit is used to learn the dependencies between slots. The slot self-attention unit can sequentially include: a multi-head attention mechanism layer, an FFN (Feedforward neural network) layer, a ReLU layer, and another FFN layer. Correspondingly, the processing within the slot self-attention unit can include: and ,in, The input information is formed by integrating the slot attention vectors output by multiple slot-word attention units, where, Correspondingly, The output of the last layer, i.e., the attention vector. . Let be the slot feature of the j-th slot in the attention vector of the t-th round.

[0069] The slot detection module is connected to the attention module and performs detection processing based on the attention vector output by the attention module to obtain the slots to be corrected in the previous round and the updated slots in the current round. The slot detection module includes a first probability prediction unit, a second probability prediction unit, and a detection unit.

[0070] The first probability prediction unit transforms the slot features in the attention vector to obtain the activation probability of each slot in the previous round, which is the theoretical activation probability of the slot in the previous round. The second probability prediction unit transforms the slot features in the attention vector to obtain the update probability of each slot in the current round. The detection unit determines the slots to be corrected in the previous round based on the dialogue state in the previous round and the activation probabilities of each slot in the previous round, and determines the updated slots in the current round based on the update probabilities of each slot in the current round. Accordingly, the attention vector is processed by the slot detection module to obtain the slots to be corrected in the previous round and the updated slots in the current round, including: transforming the slot features in the attention vector based on the first probability prediction unit to obtain the activation probability of each slot in the previous round; transforming the slot features in the attention vector based on the second probability prediction unit to obtain the update probability of each slot in the current round; determining the slots to be corrected in the previous round based on the dialogue state in the previous round and the activation probability of each slot in the previous round based on the detection unit, and determining the updated slots in the current round based on the update probability of each slot in the current round.

[0071] Specifically, the detection unit determines the theoretical activation slots of the previous round by setting an activation threshold. For example, slots with an activation probability greater than the activation threshold can be identified as theoretical activation slots. The actual activation slots of the previous round are determined based on the dialogue state of the previous round. For example, slots with non-empty values ​​in the dialogue state of the previous round can be identified as actual activation slots. Based on the theoretical and actual activation slots of the previous round, slots to be corrected in the previous round are determined. These slots include over-predicted slots and / or under-predicted slots. Optionally, the detection unit determines the slots to be corrected in the previous round based on the dialogue state of the previous round and the activation probability of each slot in the previous round, and determines the updated slots in the current round based on the update probability of each slot in the current round, including: determining the theoretically activated slots in the previous round based on the activation probability and activation threshold of the previous round; determining the actual activated slots in the previous round based on the dialogue state of the previous round; determining over-predicted slots and / or under-predicted slots based on the theoretically activated slots and the actual activated slots in the previous round; and determining the updated slots in the current round based on the update probability and update threshold of each slot in the current round.

[0072] Specifically, over-predicted slots can be determined by finding the intersection of the theoretically activated slots and the actual activated slots from the previous round, and then identifying the slots from the previous round that are not included in the intersection as over-predicted slots. Under-predicted slots can be determined by identifying the slots from the previous round that are not included in the intersection as under-predicted slots.

[0073] For example, the method for determining the slot to be corrected can be:

[0074] Previous round of actual activated slots ;

[0075] Over-predicting slots ;

[0076] Under-predicted slots ;

[0077] in, To activate the slots in the previous theoretical round, Let be the activation probability of slot j in the previous round. This is the activation threshold. The activation probability from the previous round is obtained through processing by the first probability prediction unit, i.e. , It is a linear layer.

[0078] The detection unit sets an update threshold and determines the update probability of the current round output by the second probability prediction unit to identify the update slot for the current round. For example, slots with update probabilities greater than the update threshold can be identified as update slots for the current round. The update probability of the j-th slot in the current round is determined as follows: ,in, This is a linear layer. Accordingly, the update slot for the current round is... , To update the threshold.

[0079] The over-predicted slots, under-predicted slots, and the updated slots of the current round obtained from the slot detector are used as prompts and input into the dialogue information generator to obtain the dialogue state of the current round. The over-predicted and under-predicted slots obtained from the slot detector prompt the dialogue information generator to correct the errors that occurred in the previous round, and the updated slots of the current round obtained from the slot detector prompt the dialogue information generator to determine the slot value of the updated slots. The dialogue information generator includes an encoder and a decoder. The encoder's input information is generated based on the dialogue state and historical dialogue information of the previous round, the dialogue information of the current round, the slots to be corrected in the previous round, and the updated slots of the current round. It is obtained by cascading a start flag, an end flag, and a connection flag. For example, the encoder's input information can be represented as:

[0080] in, , , , and These are connection identifiers, which, when connecting to the next piece of information, can also characterize the information flow type of the connected information.

[0081] The encoder's processing of input information can be as follows: , Enhance the context vector for slot information. Indicates the length of the input sequence.

[0082] The decoder enhances the context vector based on slot information to generate the dialogue state. The processing procedure can be as follows:

[0083] ;

[0084] ;

[0085] , , is the length of the state sequence. Where, The current hidden state feature is output by the decoder. Let l-1 tokens be decoded in the t-th round of prediction, where the decoder generates words gradually over time, and l represents the current decoding time. To project the hidden state feature space to A linear layer of a 3D vocabulary space. Predict probabilities for words. The decoder gradually generates words over time until the generation of special words ends [EOS]. This refers to the current dialogue state, including updating the slot value corresponding to the slot. .

[0086] For example, see Figure 2 , Figure 2 This is a schematic diagram of the structure of a dialogue state generation model provided in an embodiment of the present invention.

[0087] The technical solution of this embodiment, by setting a slot detector, detects the dialogue state of the previous round, historical dialogue information, and the dialogue information of the current round to obtain the slots to be corrected in the previous round and the updated slots in the current round. The slots to be corrected are used to generate dialogue information to correct errors that occurred in the previous round, and the updated slots are generated to determine the slot value of the updated slots. This achieves the correction of the previous round's dialogue state during the prediction process of the current round's dialogue state, avoiding interference from previous round's erroneous information on the current round's prediction and improving the prediction accuracy of the current round's dialogue state.

[0088] Figure 3This is a flowchart of a training method for a dialogue state generation model provided by an embodiment of the present invention. This embodiment is applicable to situations where the dialogue state generation model is pre-trained. The method can be executed by a training device for the dialogue state generation model, which can be implemented in hardware and / or software. This training device for the dialogue state generation model can be configured in electronic devices such as computers and servers. Figure 3 As shown, the method includes:

[0089] S210. Obtain multiple historical dialogue information, perform error simulation on the multiple historical dialogue information to obtain a simulation sample set, wherein the simulation sample set includes first sample data with error information and second sample data without error information.

[0090] S220. The dialogue state generation model to be trained is trained based on the simulated sample set to obtain a trained dialogue state generation model. The dialogue state generation model includes a slot detector and a dialogue information generator. During the training process, the first loss function of the slot detector and the second loss function of the dialogue information generator are obtained respectively. The model parameters of the slot detector are adjusted based on the first loss function, and the model parameters of the dialogue information generator are adjusted based on the second loss function.

[0091] In this embodiment, an initial dialogue state generation model is created in advance, which includes a slot detector and a dialogue information generator with undetermined parameters. The initial dialogue state generation model is iteratively trained using a simulated sample set to adjust the model parameters and obtain a trained dialogue state generation model.

[0092] To avoid the mismatch between historical dialogue information during training and inference, the system simulates the possibility of errors in the historical dialogue information during inference. The acquired historical dialogue information is processed to include both first sample data with errors and second sample data without errors, thereby improving the matching degree between historical dialogue information during training and historical dialogue information during inference.

[0093] Optionally, multiple historical dialogue information can be determined based on the applicable business scenario of the dialogue state generation model. The business scenario can be one or more, for example, including but not limited to hotel booking, movie ticket booking, travel booking, and restaurant consultation. Multiple historical dialogue information can be generated separately for different business scenarios to generate sample data for different business scenarios. The dialogue state generation model can then be trained using this sample data to improve its scenario adaptability and generalization.

[0094] Historical dialogue information may include at least one slot and slot value. Error simulation is performed on the historical dialogue information to obtain simulated samples. Simulated samples from multiple historical dialogue information sets form a simulated sample set. Optionally, a simulator is pre-created, and the historical dialogue information is input into the simulator to obtain simulated samples corresponding to the historical dialogue information. This simulator has the function of performing error simulation on historical dialogue information based on a certain probability. Specifically, the simulator adjusts one or more slots and / or one or more slot values ​​in the historical dialogue information based on a certain probability to obtain simulated samples.

[0095] Optionally, the simulated samples can also be obtained by processing according to a simulation strategy. Specifically, error simulation is performed on the multiple historical dialogue information to obtain a simulated sample set, including: for any one of the historical dialogue information, setting the non-empty slot value in the historical dialogue information to empty based on a first probability; and / or, for any one of the historical dialogue information, setting the non-empty slot value in the historical dialogue information to other slot values ​​based on a second probability.

[0096] By setting non-empty slot values ​​to empty, underfitting of slots is simulated; by setting non-empty slot values ​​to other slot values, overfitting of slots is simulated. For any historical dialogue information, error simulation is performed based on a first probability or a second probability, and the adjusted non-empty slot values ​​can be randomly selected, which improves the generalization and randomness of the simulation samples and is beneficial to the realism of error simulation of the reasoning process.

[0097] The historical sample information containing erroneous information after the above simulation is determined as the first sample data, and the historical sample information without erroneous information after the above simulation is determined as the second sample data. Multiple first sample data sets and multiple second sample data sets constitute a simulation sample set, which is used to iteratively train the dialogue state generation model.

[0098] During any iteration of training, any sample data is input into the dialogue state generation model to be trained. The slot detector outputs the activation probability of the slot in the previous round and the update probability in the current round, further obtaining the slot to be corrected and the updated slot, which are then input into the dialogue information generator to obtain the current round of dialogue state. A loss function is generated based on the pre-set labels of the sample data, the output of the slot detector, and the output of the dialogue information generator to adjust the model parameters of the slot detector and the dialogue information generator. Optionally, a first loss function is generated for the slot detector to adjust the model parameters of the slot generator, and a second loss function is generated for the dialogue information generator to adjust the model parameters of the dialogue information generator.

[0099] Optionally, for the simulated samples in the simulated sample set, a slot correction label is generated based on the simulated sample and the dialogue history data before the error simulation. For example, the slot values ​​in the simulated sample and the dialogue history data before the error simulation are compared, and the slot correction label for the changed slot is set to 1, and the slot correction label for the unchanged slot is set to 0.

[0100] Slot update tags are generated based on the simulated sample corresponding to the next round of dialogue state. For example, the simulated sample is compared with the slot values ​​of each slot in the next round of dialogue state, and the slot update tags of the changed slots are set to 1, while the slot update tags of the unchanged slots are set to 0.

[0101] Accordingly, obtaining the first loss function of the slot detector includes: processing simulated samples based on the slot detector to obtain slot activation probabilities and slot update probabilities; and generating a first loss function based on the slot activation probabilities, the slot update probabilities, the slot correction labels, and the slot update labels. The first loss function includes a correction loss term and an update loss term, and can be determined by the sum (or weighted sum) of the update loss term and the correction loss term.

[0102] The correction loss term is determined based on the slot activation probability and the slot error correction label. For example, the correction loss term may be: ;in, Add error correction tags for slots. Let be the activation probability of the j-th slot in the previous round.

[0103] The update loss term is generated based on the slot update probability and the slot update label. For example, the update loss term could be: , Update the label for the j-th slot in the current round. Update the probability of the j-th slot in the current round.

[0104] Optionally, a slot label is generated based on the simulated sample corresponding to the next round of dialogue state. The slot label can be a slot value label in the next round of dialogue state, and the slot value of the updated slot in the next round of dialogue state is set to 1.

[0105] Accordingly, obtaining the second loss function of the dialogue information generator includes: generating the second loss function based on the slot prediction probability output by the dialogue information generator and the slot label. For example, the second loss function may be: ,in, For slot labels, The word prediction probability for the slot value output by the dialogue information generator. is the length of the state sequence.

[0106] Optionally, the encoder in the dialogue information generator and the context encoder in the slot detector share parameters during training.

[0107] By iteratively executing the above training process, the model parameters in the dialogue state generation model are adjusted. When the training termination condition is met, a trained dialogue state generation model is obtained, which can be used to predict the dialogue state of any round in a business scenario.

[0108] The technical solution of this embodiment simulates errors in the acquired historical dialogue information to mimic errors in the historical dialogue information during training, matching the derivation process and avoiding interference from errors in the historical dialogue information on dialogue state prediction. By iteratively training the dialogue state generation model using a simulated sample set, a dialogue state generation model capable of correcting errors in the historical dialogue information and obtaining the current round of dialogue state is obtained, thus improving the quality of dialogue state generation.

[0109] Figure 4 This is a schematic diagram of a dialogue state generation device provided in an embodiment of the present invention. Figure 4 As shown, the device includes:

[0110] The information acquisition module 310 is used to acquire the dialogue status of the previous round, historical dialogue information, and current round dialogue information;

[0111] The information generation module 320 is used to perform predictive processing on the previous round's dialogue state, historical dialogue information, and the current round's dialogue information based on the slot detector in the dialogue state generation model, to obtain the slot to be corrected in the previous round and the updated slot in the current round; and to perform predictive processing on the previous round's dialogue state, historical dialogue information, the current round's dialogue information, the slot to be corrected in the previous round, and the updated slot in the current round based on the dialogue information generator in the dialogue state generation model, to obtain the current round's dialogue state.

[0112] Optionally, the slot detector includes an encoding module, an attention module, and a slot detection module. The information generation module 320 is used to: encode the previous round's dialogue state, the historical dialogue information, and the current round's dialogue information based on the encoding module to obtain an encoding vector; perform attention mechanism processing on the encoding vector based on the attention module to obtain an attention vector; and perform detection processing on the attention vector based on the slot detection module to obtain the slots to be corrected in the previous round and the updated slots in the current round.

[0113] Optionally, the slot detection module includes a first probability prediction unit, a second probability prediction unit, and a detection unit, wherein the information generation module 320 is further used for:

[0114] The first probability prediction unit transforms the slot features in the attention vector to obtain the activation probability of each slot in the previous round; the second probability prediction unit transforms the slot features in the attention vector to obtain the update probability of each slot in the current round; the detection unit determines the slots to be corrected in the previous round based on the dialogue state in the previous round and the activation probability of each slot in the previous round, and determines the updated slots in the current round based on the update probability of each slot in the current round.

[0115] Optionally, the slots to be corrected in the previous round include over-predicted slots and / or under-predicted slots; the information generation module 320 is also used for:

[0116] The detection unit determines the theoretical activation slots of the previous round based on the activation probability and activation threshold of the previous round, determines the actual activation slots of the previous round based on the dialogue state of the previous round, and determines over-predicted slots and / or under-predicted slots based on the theoretical activation slots and the actual activation slots of the previous round; and determines the updated slots of the current round based on the update probability and update threshold of each slot in the current round.

[0117] Optionally, the encoding module includes multiple slot encoders and a context encoder, and the information generation module 320 is further used for:

[0118] The slot information is encoded using the slot encoder to obtain a slot vector; the concatenated information formed by the previous round's dialogue state, the historical dialogue information, and the current round's dialogue information is encoded using the context encoder to obtain a context vector.

[0119] The dialogue state generation apparatus provided in this embodiment of the invention can execute the dialogue state generation method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0120] Figure 5 This is a schematic diagram of the structure of a training device for a dialogue state generation model provided in an embodiment of the present invention. Figure 5 As shown, the device includes:

[0121] The sample processing module 410 is used to acquire multiple historical dialogue information, perform error simulation on the multiple historical dialogue information, and obtain a simulation sample set. The simulation sample set includes first sample data with error information and second sample data without error information.

[0122] The model training module 420 is used to train the dialogue state generation model to be trained based on the simulated sample set to obtain a trained dialogue state generation model. The dialogue state generation model includes a slot detector and a dialogue information generator. During the training process, a first loss function of the slot detector and a second loss function of the dialogue information generator are obtained respectively. The model parameters of the slot detector are adjusted based on the first loss function, and the model parameters of the dialogue information generator are adjusted based on the second loss function.

[0123] Optionally, the sample processing module 410 is used for:

[0124] For any of the aforementioned historical dialogue information, based on a first probability, the non-empty slot value in the historical dialogue information is set to empty; and / or,

[0125] For any of the historical dialogue information, the non-empty slot values ​​in the historical dialogue information are set to other slot values ​​based on the second probability.

[0126] Optionally, the device may also include:

[0127] The tag setting module is used to generate slot error correction tags for simulated samples in the simulation sample set based on the simulated samples and the dialogue history data before the error simulation; and to generate slot update tags based on the next round dialogue state corresponding to the simulated samples.

[0128] The model training module 420 is used to: process simulated samples based on the slot detector to obtain slot activation probability and slot update probability, and generate a first loss function based on the slot activation probability, the slot update probability, the slot error correction label and the slot update label.

[0129] Optionally, the tag setting module is also used to generate slot tags based on the simulated sample corresponding to the next round of dialogue state;

[0130] The model training module 420 is also used to generate a second loss function based on the slot prediction probability output by the dialogue information generator and the slot label.

[0131] The training device for the dialogue state generation model provided in the embodiments of the present invention can execute the training method for the dialogue state generation model provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0132] Figure 6This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0133] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0134] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0135] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as dialogue state generation methods or methods for training dialogue state generation models.

[0136] In some embodiments, the dialogue state generation method or the dialogue state generation model training method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the dialogue state generation method or the dialogue state generation model training method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured by any other suitable means (e.g., by means of firmware) to execute the dialogue state generation method and / or the dialogue state generation model training method.

[0137] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0138] Computer programs used to implement the dialogue state generation method and / or the training method for the dialogue state generation model of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0139] This invention also provides a computer-readable storage medium storing computer instructions for causing a processor to execute a dialogue state generation method, the method comprising:

[0140] Get the dialogue status of the previous round, historical dialogue information, and current round dialogue information;

[0141] Based on the slot detector in the dialogue state generation model, the dialogue state of the previous round, historical dialogue information and the dialogue information of the current round are predicted to obtain the slot to be corrected in the previous round and the updated slot in the current round.

[0142] Based on the dialogue information generator in the dialogue state generation model, the dialogue state of the previous round, the historical dialogue information, the dialogue information of the current round, the slot to be corrected in the previous round, and the updated slot of the current round are predicted to obtain the dialogue state of the current round.

[0143] And / or, computer instructions are used to cause the processor to execute a training method for a dialogue state generation model, the method comprising:

[0144] Multiple historical dialogue information is acquired, and error simulation is performed on the multiple historical dialogue information to obtain a simulation sample set. The simulation sample set includes first sample data with error information and second sample data without error information.

[0145] The dialogue state generation model to be trained is trained based on the simulated sample set. The trained dialogue state generation model includes a slot detector and a dialogue information generator. During the training process, a first loss function of the slot detector and a second loss function of the dialogue information generator are obtained respectively. The model parameters of the slot detector are adjusted based on the first loss function, and the model parameters of the dialogue information generator are adjusted based on the second loss function.

[0146] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0147] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0148] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0149] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0150] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0151] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for generating dialogue states, characterized in that, include: Get the dialogue status of the previous round, historical dialogue information, and current round dialogue information; Based on the slot detector in the dialogue state generation model, the dialogue state of the previous round, historical dialogue information and the dialogue information of the current round are predicted to obtain the slot to be corrected in the previous round and the updated slot in the current round. Based on the dialogue information generator in the dialogue state generation model, the dialogue state of the previous round, the historical dialogue information, the dialogue information of the current round, the slot to be corrected in the previous round, and the updated slot of the current round are predicted to obtain the dialogue state of the current round. The slot detector includes an encoding module, an attention module, and a slot detection module. The slot detector in the dialogue state generation model performs prediction processing on the dialogue state of the previous round, historical dialogue information, and the dialogue information of the current round to obtain the slots to be corrected in the previous round and the updated slots in the current round, including: The encoding module encodes the previous round's dialogue state, the historical dialogue information, and the current round's dialogue information to obtain an encoding vector. The attention module is used to process the encoded vector using an attention mechanism to obtain an attention vector. The attention vector is processed by the slot detection module to obtain the slots to be corrected in the previous round and the updated slots in the current round.

2. The method according to claim 1, characterized in that, The slot detection module includes a first probability prediction unit, a second probability prediction unit, and a detection unit; Based on the slot detection module, the attention vector is processed to obtain the slots to be corrected in the previous round and the updated slots in the current round, including: Based on the first probability prediction unit, the slot features in the attention vector are transformed to obtain the activation probability of each slot in the previous round. Based on the second probability prediction unit, the slot features in the attention vector are transformed to obtain the update probability of each slot in the current round; The detection unit determines the slots to be corrected in the previous round based on the dialogue state of the previous round and the activation probability of each slot in the previous round, and determines the updated slots in the current round based on the update probability of each slot in the current round.

3. The method according to claim 2, characterized in that, The slots to be corrected in the previous round include over-predicted slots and / or under-predicted slots; The detection unit determines the slots to be corrected in the previous round based on the dialogue state of the previous round and the activation probability of each slot in the previous round, and determines the updated slots in the current round based on the update probability of each slot in the current round, including: The detection unit determines the theoretical activation slots of the previous round based on the activation probability and activation threshold of the previous round, determines the actual activation slots of the previous round based on the dialogue state of the previous round, and determines over-predicted slots and / or under-predicted slots based on the theoretical activation slots and the actual activation slots of the previous round; and determines the updated slots of the current round based on the update probability and update threshold of each slot in the current round.

4. The method according to claim 1, characterized in that, The encoding module includes multiple slot encoders and a context encoder; The encoding module encodes the previous round's dialogue state, the historical dialogue information, and the current round's dialogue information to obtain an encoding vector, including: The slot information is encoded based on the slot encoder to obtain the slot vector; The context encoder encodes the concatenated information formed by the previous round's dialogue state, the historical dialogue information, and the current round's dialogue information to obtain a context vector.

5. A training method for a dialogue state generation model, characterized in that, include: Multiple historical dialogue information is acquired, and error simulation is performed on the multiple historical dialogue information to obtain a simulation sample set. The simulation sample set includes first sample data with error information and second sample data without error information. The dialogue state generation model to be trained is trained based on the simulated sample set to obtain a trained dialogue state generation model, wherein the dialogue state generation model includes a slot detector and a dialogue information generator: during the training process, the first loss function of the slot detector and the second loss function of the dialogue information generator are obtained respectively, the model parameters of the slot detector are adjusted based on the first loss function, and the model parameters of the dialogue information generator are adjusted based on the second loss function. The method further includes: For the simulated samples in the simulated sample set, generate slot error correction tags based on the simulated samples and the dialogue history data before the error simulation. Based on the simulated sample, generate slot update tags corresponding to the next round of dialogue state; Accordingly, obtaining the first loss function of the slot detector includes: Based on the slot detector, the simulated samples are processed to obtain the slot activation probability and slot update probability. A first loss function is generated based on the slot activation probability, the slot update probability, the slot error correction label, and the slot update label.

6. The method according to claim 5, characterized in that, The process of simulating errors in the multiple historical dialogue information to obtain a simulation sample set includes: For any of the aforementioned historical dialogue information, based on a first probability, the non-empty slot value in the historical dialogue information is set to empty; and / or, For any of the historical dialogue information, the non-empty slot values ​​in the historical dialogue information are set to other slot values ​​based on the second probability.

7. The method according to claim 5, characterized in that, The method further includes: Slot labels are generated based on the simulated sample corresponding to the next round of dialogue state; Accordingly, obtaining the second loss function of the dialogue information generator includes: A second loss function is generated based on the slot prediction probability output by the dialogue information generator and the slot label.

8. A training method for a dialogue state generation model, characterized in that, include: Multiple historical dialogue information is acquired, and error simulation is performed on the multiple historical dialogue information to obtain a simulation sample set. The simulation sample set includes first sample data with error information and second sample data without error information. The dialogue state generation model to be trained is trained based on the simulated sample set to obtain a trained dialogue state generation model, wherein the dialogue state generation model includes a slot detector and a dialogue information generator: during the training process, the first loss function of the slot detector and the second loss function of the dialogue information generator are obtained respectively, the model parameters of the slot detector are adjusted based on the first loss function, and the model parameters of the dialogue information generator are adjusted based on the second loss function. The method further includes: For the simulated samples in the simulated sample set, slot labels are generated based on the next round of dialogue state corresponding to the simulated samples; Accordingly, obtaining the second loss function of the dialogue information generator includes: A second loss function is generated based on the slot prediction probability output by the dialogue information generator and the slot label.

9. The method according to claim 8, characterized in that, The process of simulating errors in the multiple historical dialogue information to obtain a simulation sample set includes: For any of the aforementioned historical dialogue information, based on a first probability, the non-empty slot value in the historical dialogue information is set to empty; and / or, For any of the historical dialogue information, the non-empty slot values ​​in the historical dialogue information are set to other slot values ​​based on the second probability.

10. A dialogue state generation device, characterized in that, include: The information acquisition module is used to acquire the dialogue status of the previous round, historical dialogue information, and current round dialogue information; The information generation module is used to predict the dialogue state of the previous round, historical dialogue information and the dialogue information of the current round based on the slot detector in the dialogue state generation model, so as to obtain the slot to be corrected in the previous round and the updated slot in the current round. Based on the dialogue information generator in the dialogue state generation model, the dialogue state of the previous round, the historical dialogue information, the dialogue information of the current round, the slot to be corrected in the previous round, and the updated slot of the current round are predicted to obtain the dialogue state of the current round. The slot detector includes an encoding module, an attention module, and a slot detection module. The information generation module is specifically used to: encode the previous round's dialogue state, the historical dialogue information, and the current round's dialogue information based on the encoding module to obtain an encoding vector; and to process the encoding vector using an attention mechanism based on the attention module to obtain an attention vector. The attention vector is processed by the slot detection module to obtain the slots to be corrected in the previous round and the updated slots in the current round.

11. A training device for a dialogue state generation model, characterized in that, include: The sample processing module is used to acquire multiple historical dialogue information, perform error simulation on the multiple historical dialogue information, and obtain a simulation sample set. The simulation sample set includes first sample data with error information and second sample data without error information. The model training module is used to train the dialogue state generation model to be trained based on the simulated sample set to obtain a trained dialogue state generation model. The dialogue state generation model includes a slot detector and a dialogue information generator. During the training process, the first loss function of the slot detector and the second loss function of the dialogue information generator are obtained respectively. The model parameters of the slot detector are adjusted based on the first loss function, and the model parameters of the dialogue information generator are adjusted based on the second loss function. The device further includes: a tag setting module, used to generate slot error correction tags for simulated samples in the simulated sample set based on the simulated samples and the dialogue history data before the error simulation; and to generate slot update tags based on the next round of dialogue state corresponding to the simulated samples. The model training module is used to: process simulated samples based on the slot detector to obtain slot activation probability and slot update probability, and generate a first loss function based on the slot activation probability, the slot update probability, the slot error correction label and the slot update label.

12. A training device for a dialogue state generation model, characterized in that, include: The sample processing module is used to acquire multiple historical dialogue information, perform error simulation on the multiple historical dialogue information, and obtain a simulation sample set. The simulation sample set includes first sample data with error information and second sample data without error information. The model training module is used to train the dialogue state generation model to be trained based on the simulated sample set to obtain a trained dialogue state generation model. The dialogue state generation model includes a slot detector and a dialogue information generator. During the training process, the first loss function of the slot detector and the second loss function of the dialogue information generator are obtained respectively. The model parameters of the slot detector are adjusted based on the first loss function, and the model parameters of the dialogue information generator are adjusted based on the second loss function. The device further includes: a tag setting module, used to generate slot tags for simulated samples in the simulated sample set based on the next round of dialogue state corresponding to the simulated sample; The model training module is also used to generate a second loss function based on the slot prediction probability output by the dialogue information generator and the slot label.

13. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the dialogue state generation method of any one of claims 1-4, and / or the training method of the dialogue state generation model of any one of claims 5-9.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the dialogue state generation method of any one of claims 1-4, and / or the training method of the dialogue state generation model of any one of claims 5-9.

Images