Method and system for generating multi-round dialogues with diverse responses based on multi-task learning

Abstract

The present invention provides a multi-round dialogue generation method, system, storage medium and electronic device with diverse responses based on multi-task learning, and relates to the technical field of multi-round dialogue generation. The present invention first constructs a multi-task learning model, the multi-task learning model includes a pre-trained multi-round dialogue model and a VAE model, the multi-round dialogue model includes a discourse-level encoder, an inter-discourse encoder and a first decoder; each sentence sequence vector in the pre-processed multi-round dialogue history information is sequentially input into the discourse-level encoder and the inter-discourse encoder to obtain a hidden vector containing the entire multi-round dialogue history information; the hidden vector is input into the first decoder, the parameters of the first decoder are loaded from the second decoder parameters in the VAE model, and a response sentence sequence is obtained. The VAE model decoder parameters are shared with the multi-round dialogue generation model, which enhances the decoding ability of the decoder in the multi-round dialogue generation model and improves the text quality of the generated response.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-round dialogue generation, and in particular to a method, system, storage medium and electronic device for generating multi-round dialogues with diverse responses based on multi-task learning. Background Art

[0002] The application of deep learning in multi-round dialogue generation mainly includes neural language models, sequence-to-sequence models, attention mechanisms, and hierarchical sequence-to-sequence models.

[0003] Most early models were based on non-hierarchical structures (including neural language models and sequence-to-sequence models). However, in a non-hierarchical framework, whether directly concatenating historical utterances and queries or receiving utterances as input in sequence, the dependency between queries and responses will be weakened and noise will be introduced. Most recent multi-round dialogue generation models are implemented based on a hierarchical sequence-to-sequence model framework, and the vector representation of the dialogue context is optimized by improving the framework. Comparing the model based on the hierarchical framework with the model based on the non-hierarchical framework, the improved hierarchical framework model directly models the hierarchical relationship of utterances, which has better results and can generate more diverse and meaningful responses.

[0004] However, in multi-round dialogue generation, simply using the attention mechanism to extract dialogue history information is prone to a large amount of information redundancy. Usually, the quality of the generated reply text cannot be guaranteed, and context inconsistency is prone to occur. Summary of the invention

[0005] 1. Technical issues to be solved

[0006] In view of the deficiencies in the prior art, the present invention provides a method, system, storage medium and electronic device for generating multi-round dialogues with diverse responses based on multi-task learning, which solves the technical problem of low quality of response texts generated in the prior art multi-round dialogue generation.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] A method for generating multi-round dialogues with diverse responses based on multi-task learning. The method first constructs a multi-task learning model, wherein the multi-task learning model includes a pre-trained multi-round dialogue model and a VAE model, wherein the multi-round dialogue model includes an utterance-level encoder, an inter-utterance encoder, and a first decoder; and comprises:

[0010] Obtain and preprocess multi-round dialogue history information;

[0011] Inputting each sentence sequence vector in the preprocessed multi-round dialogue history information into the utterance-level encoder to obtain the utterance-level encoding vector corresponding to the multi-round dialogue history information;

[0012] Inputting the utterance-level encoding vector into the inter-utterance encoder to obtain a hidden vector containing the entire multi-round dialogue history information;

[0013] The hidden vector is input into a first decoder, and the parameters of the first decoder are loaded from the parameters of the second decoder in the VAE model to obtain a reply sentence sequence.

[0014] Preferably, the VAE model includes an encoder and a second decoder, and the model construction process specifically includes:

[0015] Acquire and preprocess multiple sets of complete multi-round dialogue history information, obtain the real response sentence sequence in each set of complete multi-round dialogue history information, and construct a training vector set;

[0016] Input each of the true response sentence sequences into the encoder to obtain an encoded true response sentence sequence vector;

[0017] According to the true reply sentence sequence vector, linear layer sampling is used to obtain an intermediate vector with added noise;

[0018] The intermediate vector with added noise is input into the second decoder, unsupervised layer-by-layer training is performed, the parameters of the second decoder are obtained, and the pre-trained VAE model is constructed.

[0019] Preferably, the VAE model building process selects the first loss function:

[0020] L VAE =L reconstruction +λ vae L kl (1)

[0021] Among them, L VAE represents the first loss function; λ vae is a hyperparameter; L reconstruction Represents the reconstruction loss function, which adopts the first cross entropy loss function:

[0022] L reconstruction =CrossEntropy1(y,y′) (2)

[0023] y represents the characters in the real reply sentence sequence; y′ represents the characters in the reply sentence sequence obtained by the VAE model;

[0024] L kl Represents the kl loss function:

[0025] L KL =μ 2 +σ 2 -log(σ 2 ) (3)

[0026] μ represents the first neural network fitting parameter in the linear layer, and σ represents the second neural network fitting parameter in the linear layer.

[0027] Preferably, the loss function is selected during the construction of the multi-task learning model:

[0028] L=λ1L Dialog +λ2L VAE +λ3L Laten (4)

[0029] Among them, L represents the loss function of multi-task learning joint training; λ1, λ2, λ3 are all hyperparameters; L Dialog represents the second loss function selected in the process of building the multi-round dialogue model, and the second loss function adopts the second cross entropy loss function:

[0030]

[0031] Represents characters in a reply sentence sequence obtained through the multi-round dialogue model;

[0032] L Laten =D(Z,C s ) (6)

[0033] L Laten It represents the intermediate vector Z with added noise and the hidden vector C corresponding to each set of complete multi-round dialogue history information. s , the third loss function that constrains the approximation of the two in the semantic space; D represents the distance function.

[0034] Preferably, the speech-level encoder adopts an RNN model;

[0035] Preferably, the inter-utterance encoder adopts the RNN model

[0036] Preferably, the first decoder adopts an RNN model with an attention mechanism.

[0037] Preferably, the preprocessing includes data cleaning and word embedding of the multi-round dialogue history information.

[0038] A multi-round dialogue generation system with diverse responses based on multi-task learning. The system first constructs a multi-task learning model, wherein the multi-task learning model includes a pre-trained multi-round dialogue model and a VAE model, wherein the multi-round dialogue model includes an utterance-level encoder, an inter-utterance encoder, and a first decoder; and includes:

[0039] The original information preprocessing module is used to obtain and preprocess the multi-round dialogue history information;

[0040] A first vector acquisition module, configured to input each sentence sequence vector in the preprocessed multi-round dialogue history information into the utterance-level encoder to obtain an utterance-level encoding vector corresponding to the multi-round dialogue history information;

[0041] A second vector acquisition module, configured to input the utterance-level encoding vector into the inter-utterance encoder to obtain a hidden vector containing the entire multi-round dialogue history information;

[0042] The reply sentence sequence acquisition module is used to input the hidden vector into the first decoder, and the parameters of the first decoder are loaded from the second decoder parameters in the VAE model to obtain the reply sentence sequence.

[0043] A storage medium stores a computer program for generating multi-round dialogues with diverse responses based on multi-task learning, wherein the computer program enables a computer to execute the multi-round dialogue generation method with diverse responses as described above.

[0044] An electronic device, comprising:

[0045] one or more processors;

[0046] Memory; and

[0047] One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, the programs including a program for executing the method for generating multi-round dialogues with diverse responses as described above.

[0048] (III) Beneficial effects

[0049] The present invention provides a method, system, storage medium and electronic device for generating multi-round dialogues with diverse responses based on multi-task learning. Compared with the prior art, the present invention has the following beneficial effects:

[0050] The present invention first constructs a multi-task learning model, which includes a pre-trained multi-round dialogue model and a VAE model, wherein the multi-round dialogue model includes an utterance-level encoder, an inter-utterance encoder, and a first decoder; each sentence sequence vector in the pre-processed multi-round dialogue history information is sequentially input into the utterance-level encoder and the inter-utterance encoder to obtain a hidden vector containing the entire multi-round dialogue history information; the hidden vector is input into the first decoder, and the parameters of the first decoder are loaded from the second decoder parameters in the VAE model to obtain a reply sentence sequence. The VAE model decoder parameters are shared with the multi-round dialogue generation model, thereby enhancing the decoding capability of the decoder in the multi-round dialogue generation model and improving the text quality of the generated reply. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 A flowchart of a method for generating multi-round dialogues with diverse responses based on multi-task learning provided by an embodiment of the present invention;

[0053] Figure 2 A structural framework diagram of a VAE model provided by an embodiment of the present invention;

[0054] Figure 3 A structural block diagram of a multi-round dialogue generation system with diverse responses based on multi-task learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] The embodiments of the present application solve the technical problem of low quality of reply texts generated in the existing multi-round dialogue generation by providing a method, system, storage medium and electronic device for generating multi-round dialogues with diverse replies based on multi-task learning, thereby ensuring the accuracy and diversity of the reply texts.

[0057] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:

[0058] The embodiment of the present invention first constructs a multi-task learning model, which includes a pre-trained multi-round dialogue model and a VAE model, and the multi-round dialogue model includes a discourse-level encoder, an inter-discourse encoder, and a first decoder; each sentence sequence vector in the pre-processed multi-round dialogue history information is sequentially input into the discourse-level encoder and the inter-discourse encoder to obtain a hidden vector containing the entire multi-round dialogue history information; the hidden vector is input into the first decoder, and the parameters of the first decoder are loaded from the second decoder parameters in the VAE model to obtain a reply sentence sequence. The VAE model decoder parameters are shared with the multi-round dialogue generation model, which enhances the decoding ability of the decoder in the multi-round dialogue generation model and improves the text quality of the generated reply.

[0059] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0060] Embodiment 1:

[0061] First, as Figure 1 As shown, an embodiment of the present invention provides a method for generating multi-round dialogues with diverse responses based on multi-task learning. The method first constructs a multi-task learning model, wherein the multi-task learning model includes a pre-trained multi-round dialogue model and a VAE model, wherein the multi-round dialogue model includes an utterance-level encoder, an inter-utterance encoder, and a first decoder; and includes:

[0062] Obtain and preprocess multi-round dialogue history information;

[0063] Inputting each sentence sequence vector in the preprocessed multi-round dialogue history information into the utterance-level encoder to obtain the utterance-level encoding vector corresponding to the multi-round dialogue history information;

[0064] Inputting the utterance-level encoding vector into the inter-utterance encoder to obtain a hidden vector containing the entire multi-round dialogue history information;

[0065] The hidden vector is input into a first decoder, and the parameters of the first decoder are loaded from the parameters of the second decoder in the VAE model to obtain a reply sentence sequence.

[0066] The embodiment of the present invention shares the VAE model decoder parameters with the multi-round dialogue generation model, thereby enhancing the decoding capability of the decoder in the multi-round dialogue generation model and improving the text quality of the generated responses.

[0067] The following will introduce each step of the plan in combination with specific content:

[0068] It is important to understand that the multi-turn dialogue generation scheme in the open domain is first formally described as: in the context of historical dialogue information, the complete dialogue history is used as the model input, and the reply sentence is output, which can also be called a response or answer.

[0069] S1. Obtain and preprocess multi-round dialogue history information.

[0070] The multi-round dialogue history information can be expressed as U = (u1, ..., u i ,...,u s ),u i is the i-th sentence in the conversation, u s is the last sentence in the conversation, that is, the current question sentence; each sentence sequence vector is represented as: where w j,k is the kth character in the jth sentence.

[0071] The preprocessing includes data cleaning:

[0072] 1) Build a stop word dictionary and delete the stop words in the corpus. Change all English letters in the multi-round dialogue history information corpus to lowercase.

[0073] 2) Delete the dialogue data with less than 2 dialogue rounds in the multi-round dialogue history information

[0074] The multi-round dialogue history information is word-embedded. The glove model is used to obtain the word vector of each word in the multi-round dialogue history information. Specifically, e(x) is used to represent the word embedding of character x.

[0075] like Figure 2 As shown, the VAE model, i.e., the variational autoencoder model, includes an encoder and a second decoder. The model construction process specifically includes:

[0076] S100, obtaining and preprocessing multiple sets of complete multi-round dialogue history information, obtaining a real response sentence sequence in each set of complete multi-round dialogue history information, and constructing a training vector set.

[0077] The multiple sets of complete multi-round dialogue history information data sets, compared with the above multi-round dialogue history information, include the content of the current reply and can be expressed as: For each round of dialogue, (U, Y) contains a reply sentence, that is, the real reply sentence sequence Y = (y1, ..., y T ) and the conversation history U = (u1, ..., u s ), where y j is the jth character in the current reply sentence, u i is the i-th sentence in the conversation, u sis the last sentence in the conversation, i.e., the current question sentence, U = (u1, ..., u s-1 ) is the previous conversation sentence in the conversation. For multi-round conversations, set s≥2, that is, the conversation contains at least two sentences (i.e., it meets the multi-round requirement); where w j，k is the kth character in the jth sentence in the conversation history.

[0078] For example, the conversation history U = ((Tomorrow the weather is fine, let's organize a spring outing), (Great, we can relax), (I'm going to the supermarket to buy some things for the spring outing)), the current reply Y = (Then I'll make some sandwiches at home). Each word is separated by a space.

[0079] The true reply sentence sequence Y = (y1, ..., y T )After word embedding, we get e(Y)=(e(y1),...,e(y t ), ..., e(y T )).

[0080] S200, input each of the true response sentence sequences e(Y) into the encoder to obtain the encoded true response sentence sequence vector h T .

[0081] The encoder of the VAE model adopts an RNN model, e(Y)=(e(y1),...,e(y T )) Input the RNN model and get the hidden state of the RNN model:

[0082] h t =f1(h t-1 , e(y t )) (1)

[0083] Wherein, f1 represents the RNN model. Specifically, in order to avoid the problem of gradient explosion, the LSTM model in the RNN model can be used. t represents the hidden state at time step t, and the hidden state h at the last time step T T is regarded as the feature vector representation of the sentence.

[0084] S300, based on the true reply sentence sequence vector h T , linear layer sampling is used to obtain the intermediate vector Z with noise added.

[0085] Construct the first neural network fitting parameters: in represents the first linear function.

[0086] Construct the second neural network fitting parameters: in Represents the second linear function. Note that this and the first linear function do not share parameters.

[0087] According to the fitting parameters u and σ 2 Sampling to obtain the intermediate vector containing the reply sentence information and adding noise

[0088] Z=μ+h T ×σ (2)

[0089] S400, input the intermediate vector Z with added noise into the second decoder, perform unsupervised layer-by-layer training, obtain the parameters of the second decoder, and complete the construction of the pre-trained VAE model.

[0090] The VAE model building process selects the first loss function:

[0091] L VAE =L reconstruction +λ vae L kl (3)

[0092] Among them, L VAE represents the first loss function; λ vae is a hyperparameter; L reconstruction Represents the reconstruction loss function, which adopts the first cross entropy loss function:

[0093] L reconstruction =CrossEntropy1(y,y′) (4)

[0094] y represents the characters in the real reply sentence sequence; y′ represents the characters in the reply sentence sequence obtained by the VAE model;

[0095] L kl Represents the kl loss function:

[0096] L KL =μ 2 +σ 2 -log(σ 2 ) (5)

[0097] μ represents the first neural network fitting parameter in the linear layer; σ represents the second neural network fitting parameter in the linear layer.

[0098] S2, each sentence sequence vector u in the preprocessed multi-round dialogue history information i The utterance-level encoder is input to obtain an utterance-level encoding vector H corresponding to the multi-round dialogue history information.

[0099] Each sentence sequence vector Each character in is embedded in the word to obtain e(w i，t The utterance-level encoder adopts an RNN model to convert e(w i，t ) Input the RNN model and get the hidden state of the RNN model:

[0100] h′ t =f2(h t-1 ,e(w i，t )) (6)

[0101] f2 represents the RNN model. Specifically, in order to avoid the problem of gradient explosion, the LSTM model in the RNN model can be used; h′ t represents the hidden state at time step t, and the hidden state at the last time step T is regarded as the feature vector of the sentence h i ; Finally, the speech-level encoding vector H corresponding to the multi-round dialogue history information is obtained = (h1, ..., h i , ..., h s ).

[0102] S3: Input the utterance-level encoding vector H into the inter-utterance encoder to obtain a hidden vector C containing the entire multi-round dialogue history information. s .

[0103] The inter-utterance encoder adopts an RNN model, and the utterance-level encoding vector H is input into the RNN model. At each time step t, the model formula is expressed as follows: t =f(C t-1 ,h t ), get C at the last time step s s .

[0104] Specifically, in order to avoid the problem of gradient explosion when using the RNN model, the LSTM model in the RNN model can be used.

[0105] S4, the hidden vector C s Input the first decoder, the parameters of the first decoder are loaded from the parameters of the second decoder in the VAE model (the remaining parameters of the dialogue model are randomly initialized), and obtain the reply sentence sequence.

[0106] The first decoder adopts an RNN model with an attention mechanism, and its model parameters are loaded from the second decoder parameters in the VAE model. Based on the pre-training operation of the VAE model in the aforementioned steps, the decoding capability of the decoder in the multi-round dialogue generation model is enhanced.

[0107] This step first calculates each sentence sequence vector u iThe attention weights are:

[0108] e i，t =V T tanh(Wh i +Us t-1 ) (7)

[0109] Among them, V, W and U are the parameters of the attention mechanism, the superscript T represents the transpose of V; tanh represents the activation function; s t-1 is the hidden state of the decoder at time step t-1.

[0110] The calculated weights are normalized to obtain the normalized weights:

[0111]

[0112] According to the calculated normalized weight α i，t Perform a weighted sum calculation on the hidden state vector hi of each sentence to obtain the vector representation of the conversation history after the attention mechanism is adopted:

[0113] c t =∑ i α i，t h i (9)

[0114] In the decoding stage, the decoder selects the output character that maximizes the probability p according to the probability formula (10). As the decoding result, the reply sentence sequence is finally obtained

[0115]

[0116] in, Represents the characters in the reply sentence sequence obtained by the decoder at time step t.

[0117] In particular, the loss function is selected during the construction of the multi-task learning model:

[0118] L=λ1L Dialog +λ2L VAE +λ3L Laten (11)

[0119] Among them, L represents the loss function of multi-task learning joint training; λ1, λ2, λ3 are all hyperparameters; L Dialog represents the second loss function selected in the process of building the multi-round dialogue model, and the second loss function adopts the second cross entropy loss function:

[0120]

[0121] Represents characters in a reply sentence sequence obtained through the multi-round dialogue model;

[0122] L Laten =D(Z,C s ) (13)

[0123] L Laten It represents the intermediate vector Z with added noise and the hidden vector C corresponding to each set of complete multi-round dialogue history information. s , the third loss function that constrains the approximation of the two in the semantic space; D represents the distance function.

[0124] From the above, it can be seen that the embodiment of the present invention proposes to add the noise-added intermediate vector Z and the hidden vector C corresponding to each set of complete multi-round dialogue history information s , constraining the two to be close in semantic space. During the multi-task training process, the multi-round dialogue model is controlled so that the vector containing the entire dialogue history information after being encoded by the utterance-level encoder and the inter-utterance encoder is close to the hidden vector obtained by the VAE model through self-encoding of the reply in the semantic space, thereby ensuring that the dialogue history information vector contains both the information contained in the reply (improving the encoding ability of the encoder) and additional sampling information (improving the diversity of reply generation).

[0125] Second, as Figure 3 As shown, an embodiment of the present invention provides a multi-round dialogue generation system with diverse responses based on multi-task learning. The system first constructs a multi-task learning model, wherein the multi-task learning model includes a pre-trained multi-round dialogue model and a VAE model, wherein the multi-round dialogue model includes an utterance-level encoder, an inter-utterance encoder, and a first decoder; and includes:

[0126] The original information preprocessing module is used to obtain and preprocess the multi-round dialogue history information;

[0127] A first vector acquisition module, configured to input each sentence sequence vector in the preprocessed multi-round dialogue history information into the utterance-level encoder to obtain an utterance-level encoding vector corresponding to the multi-round dialogue history information;

[0128] A second vector acquisition module, configured to input the utterance-level encoding vector into the inter-utterance encoder to obtain a hidden vector containing the entire multi-round dialogue history information;

[0129] The reply sentence sequence acquisition module is used to input the hidden vector into the first decoder, and the parameters of the first decoder are loaded from the second decoder parameters in the VAE model to obtain the reply sentence sequence.

[0130] It can be understood that the response diversity multi-round dialogue generation system based on multi-task learning provided by the present invention corresponds to the response diversity multi-round dialogue generation method based on multi-task learning provided by the present invention. The explanations, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the response diversity multi-round dialogue generation method, which will not be repeated here.

[0131] In a third aspect, an embodiment of the present invention provides a storage medium storing a computer program for generating multi-round dialogues with diverse responses based on multi-task learning, wherein the computer program enables a computer to execute the method for generating multi-round dialogues with diverse responses as described above.

[0132] In a fourth aspect, an embodiment of the present invention provides an electronic device, including:

[0133] one or more processors;

[0134] Memory; and

[0135] One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, the programs including a program for executing the method for generating multi-round dialogues with diverse responses as described above.

[0136] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0137] 1. The embodiment of the present invention first constructs a multi-task learning model, which includes a pre-trained multi-round dialogue model and a VAE model, and the multi-round dialogue model includes an utterance-level encoder, an inter-utterance encoder, and a first decoder; each sentence sequence vector in the pre-processed multi-round dialogue history information is sequentially input into the utterance-level encoder and the inter-utterance encoder to obtain a hidden vector containing the entire multi-round dialogue history information; the hidden vector is input into the first decoder, and the parameters of the first decoder are loaded from the second decoder parameters in the VAE model to obtain a reply sentence sequence. The VAE model decoder parameters are shared with the multi-round dialogue generation model, which enhances the decoding ability of the decoder in the multi-round dialogue generation model and improves the text quality of the generated reply.

[0138] 2. The embodiment of the present invention proposes to add the noise-added intermediate vector Z to the hidden vector C corresponding to each set of complete multi-round dialogue history information s, constraining the two to be close in semantic space. During the multi-task training process, the multi-round dialogue model is controlled so that the vector containing the entire dialogue history information after being encoded by the utterance-level encoder and the inter-utterance encoder is close to the hidden vector obtained by the VAE model through self-encoding of the reply in the semantic space, thereby ensuring that the dialogue history information vector contains both the information contained in the reply (improving the encoding ability of the encoder) and additional sampling information (improving the diversity of reply generation).

[0139] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0140] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-round dialogue generation method with diverse responses based on multi-task learning, characterized in that: The method first constructs a multi-task learning model, wherein the multi-task learning model includes a pre-trained multi-round dialogue model and a VAE model, wherein the multi-round dialogue model includes an utterance-level encoder, an inter-utterance encoder, and a first decoder; and includes: Obtain and preprocess multi-round dialogue history information; Inputting each sentence sequence vector in the preprocessed multi-round dialogue history information into the utterance-level encoder to obtain the utterance-level encoding vector corresponding to the multi-round dialogue history information; Inputting the utterance-level encoding vector into the inter-utterance encoder to obtain a hidden vector containing the entire multi-round dialogue history information; The hidden vector is input into a first decoder, and the parameters of the first decoder are loaded from the parameters of the second decoder in the VAE model to obtain a reply sentence sequence; The VAE model includes an encoder and a second decoder, and the model construction process specifically includes: Acquire and preprocess multiple sets of complete multi-round dialogue history information, obtain the real response sentence sequence in each set of complete multi-round dialogue history information, and construct a training vector set; Input each of the true response sentence sequences into the encoder to obtain an encoded true response sentence sequence vector; According to the true reply sentence sequence vector, linear layer sampling is used to obtain an intermediate vector with added noise; Inputting the intermediate vector with added noise into the second decoder, performing unsupervised layer-by-layer training, obtaining the parameters of the second decoder, and completing the construction of the pre-trained VAE model; The VAE model building process selects the first loss function: L VAE =L reconstruction +λ vae L kl (1) Among them, L VAE represents the first loss function; λ vae is a hyperparameter; L reconstruction Represents the reconstruction loss function, which adopts the first cross entropy loss function: L reconstruction =CrossEntropy1(y,y′) (2) y represents the characters in the real reply sentence sequence; y′ represents the characters in the reply sentence sequence obtained by the VAE model; L kl Represents the kl loss function: L KL =μ 2 +s 2 -log(σ 2 ) (3) μ represents the fitting parameter of the first neural network in the linear layer; σ represents the fitting parameter of the second neural network in the linear layer; The loss function is selected during the construction of the multi-task learning model: L=λ1L Dialog +λ2L VAE +λ3L Laten (4) Among them, L represents the loss function of multi-task learning joint training; λ1, λ2, λ3 are all hyperparameters; L Dialog represents the second loss function selected in the process of building the multi-round dialogue model, and the second loss function adopts the second cross entropy loss function: Represents characters in a reply sentence sequence obtained through the multi-round dialogue model; L Laten =D(Z,C s ) (6) L Laten It represents the intermediate vector Z with added noise and the hidden vector C corresponding to each set of complete multi-round dialogue history information. s , the third loss function that constrains the approximation of the two in the semantic space; D represents the distance function.

2. The method for generating multi-round dialogues with diverse responses as claimed in claim 1, characterized in that: The speech-level encoder adopts an RNN model; And / or the inter-utterance encoder adopts an RNN model; And / or the first decoder adopts an RNN model with an attention mechanism.

3. The method for generating multi-round dialogues with diverse responses as claimed in claim 1, wherein: The preprocessing includes data cleaning and word embedding of the multi-round dialogue history information.

4. A multi-turn dialogue generation system with diverse responses based on multi-task learning, characterized in that: For executing the method for generating multi-round dialogues with diverse responses as claimed in claim 1, the system first constructs a multi-task learning model, wherein the multi-task learning model includes a pre-trained multi-round dialogue model and a VAE model, wherein the multi-round dialogue model includes an utterance-level encoder, an inter-utterance encoder, and a first decoder; and includes: The original information preprocessing module is used to obtain and preprocess the multi-round dialogue history information; A first vector acquisition module, configured to input each sentence sequence vector in the preprocessed multi-round dialogue history information into the utterance-level encoder to obtain an utterance-level encoding vector corresponding to the multi-round dialogue history information; A second vector acquisition module, configured to input the utterance-level encoding vector into the inter-utterance encoder to obtain a hidden vector containing the entire multi-round dialogue history information; The reply sentence sequence acquisition module is used to input the hidden vector into the first decoder, and the parameters of the first decoder are loaded from the second decoder parameters in the VAE model to obtain the reply sentence sequence.

5. A storage medium, characterized in that: It stores a computer program for generating multi-round dialogues with diverse responses based on multi-task learning, wherein the computer program enables a computer to execute the method for generating multi-round dialogues with diverse responses as described in any one of claims 1 to 3.

6. An electronic device, characterized in that: include: one or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include commands for executing the method for generating multi-round dialogues with diverse responses according to any one of claims 1 to 3.

Images