An emotion dialogue generation method based on an improved generative adversarial network

Abstract

The emotion dialogue generation method based on improved generative adversarial network belongs to the dialogue system field under natural language processing, realizes the interaction between memory matrices through a self-attention mechanism, enhances the long-distance transmission energy between information, thereby improves the expression ability and feature extraction ability of the model, and solves the problem that the existing model has weak expression ability and generates short sentences. Meanwhile, a multi-class discriminator is set, which respectively discriminates the true and false of the text and the emotion category, calculates the category relative loss and the emotion information loss two parts to feed back and update the generator, so as to improve the consistency of the emotion information, and make the emotion expression of the reply generation sentence more obvious and clear. Finally, the iterative evolution algorithm is used in the generator update, the temperature parameter and the quality parameter are controlled respectively, and the child generator in the optimal direction of the evolution temperature is selected to complete the generation task. The dialogue generation method of the present application realizes the emotion embedding which considers the quality and diversity of the reply.

Description

Technical Field

[0001] This invention belongs to the field of dialogue systems under natural language processing, and particularly relates to an improved method for generating emotional dialogues using generative adversarial networks. Background Technology

[0002] With the innovation and development of deep learning technology, the field of artificial intelligence has entered a new stage. Dialogue systems, as a major task in natural language processing, have wide applications and have received high attention from various industries. Early dialogue systems were designed for specific tasks within limited domains. These tasks required ensuring the logical accuracy of generated responses without considering the user's emotional feedback, thus lacking empathy. In recent years, with the maturity of human-computer interaction technology, people have begun to shift from improving the semantic quality of dialogue to enhancing the emotional interactivity of the dialogue process. Initial research on emotional dialogue generation was based on templates and rules, where emotional statements were generated in a pattern-based manner. Designers set different templates according to different scenarios, but this approach was difficult to build and not easily scalable. Subsequently, neural network models were widely applied to generative dialogue model research. Emotional dialogue generation mainly completed the following tasks: emotion embedding encoding, response emotion control, and emotion response decoding, thereby solving problems such as emotional consistency and semantic diversity in dialogue generation. Emotion is generally transformed into two dimensions of emotional information: discrete emotion models use labels to represent emotion categories; dimensional emotion models map emotions to a multi-dimensional space, using continuous numbers to describe emotions. Most existing dialogue generation models fall into three categories: Seq2Seq-based models, GAN-based models, and VAE-based models.

[0003] GAN models have achieved remarkable success in image generation, but they cannot be directly applied to text generation because characters are discrete. The discrete output of the generator model makes it difficult to pass gradient updates from the discriminator model to the generator model to guide the generator's sequence generation. Applying GANs to text generation is a major research hotspot. Compared to the quality ambiguity problem of VAE models in generation tasks, GANs can improve the quality of results in generation tasks. Research attempts to introduce reinforcement learning into GANs, modeling the generator as a stochastic policy, bypassing the generator's differentiation problem by directly executing gradient policy updates, and introducing Monte Carlo search to ensure that the current score and the score of the entire generation sequence are considered when determining the next word to be generated. The closest research to this invention specifies the sentiment type to be generated, sets up multiple generators to generate text of different sentiment categories, and then sets up a multi-class discriminator to distinguish between the authenticity of the text and the consistency of sentiment information. A teacher-coercion mechanism is added to improve the situation where the discriminator's reward cannot directly and effectively guide the generator to generate better sequences, thus indirectly leading to the training collapse problem of adversarial networks, while also correcting the direction of sentiment expression.

[0004] Current sentiment-based dialogue generation models suffer from instability and a lack of diversity in generated results. Specifically, most existing models rely on LSTM networks for their encoders and decoders, embedding sentiment word vectors during encoding or decoding. While LSTM networks enhance long-distance information transmission capabilities, they also suffer significant loss of sentiment information during transmission, leading to inconsistencies in sentiment and shorter generated dialogues with a higher probability of safe responses. Secondly, existing GAN-based generation models also exhibit poor dialogue diversity. During adversarial training, the vanishing gradient problem frequently occurs, resulting in identical dialogue responses being output repeatedly. Although experiments have shown that using conditional variational encoding in dialogue generators can improve response diversity, this often comes at the cost of sentence quality, making a balance difficult to achieve.

[0005] This invention improves the generator part of the GAN model by replacing the commonly used LSTM or memory augmentation network with a relational memory network. The relational memory network designs a relational memory kernel based on a multi-head self-attention mechanism. Its basic idea is to achieve interaction between memory matrices through self-attention, enhancing the long-distance energy transfer of information, thereby improving the model's expressive power and feature extraction capabilities, addressing the problems of weak expressive power and short generated sentences in existing models. Simultaneously, a multi-class discriminator is set up to distinguish between true / false text and sentiment categories. The relative loss of categories and the loss of sentiment information are calculated to feed back and update the generator, improving the consistency of sentiment information and making the emotional expression of the generated responses more obvious and clear. Finally, this invention uses an iterative evolutionary algorithm in the generator update. By controlling temperature and quality parameters separately, the offspring generator with the optimal evolutionary temperature direction completes the generation task, balancing the relationship between the diversity and quality of dialogue texts, and reducing the problem of existing models losing the ability to generate different texts due to gradient vanishing during adversarial training. Through these improvements, this invention achieves a dialogue generation method that balances response quality and diversity based on an improved generative adversarial network with sentiment embedding. Summary of the Invention

[0006] To address some of the problems existing in current technologies for generating emotional dialogues, this invention proposes an emotional dialogue generation method based on an improved generative adversarial network. This method includes: acquiring dialogue data with sentiment annotations, classifying the data into six sentiment categories: Angry, Disgust, Happy, Like, Sad, and Other (hereinafter, the number of sentiment categories is defined as k); semantically specializing the sentiment word vectors in the dialogue sentences to obtain specialized dialogue data with different sentiment labels; employing a relational memory network as the generator and setting up multiple discriminators to provide feedback on semantic and sentiment information; and using an iterative evolutionary learning algorithm to balance the quality and diversity of sentence generation, generating a dialogue response sentence Y with a specified sentiment category. c =(y1,…, t ,…, T ), y t ∈V represents the word vector of the response statement, V represents the vocabulary of candidate tags, T represents the length of the generated statement, and c represents the specified sentiment category of the generated dialogue.

[0007] This invention improves the training process of generative adversarial networks for emotion-based dialogue generation models, as follows: Figure 1 As shown.

[0008] 1. A sentiment-based dialogue generation method based on an improved generative adversarial network, characterized by: acquiring sentiment-annotated dialogue data and classifying the data into six sentiment categories: Angry, Disgust, Happy, Like, Sad, and Other (hereinafter, the number of sentiment categories is defined as k); semantically specializing the sentiment word vectors in the dialogue sentences to obtain specialized dialogue data with different sentiment labels; employing a relational memory network as the generator and setting multiple discriminators to provide feedback on semantic and sentiment information respectively; and adjusting the balance between sentence generation quality and diversity through an iterative evolutionary learning algorithm to generate dialogue response sentences Y with specified sentiment categories. c =(y1,…,y t ,…,y T ), y t ∈V represents the word vector of the response statement, V represents the vocabulary of candidate tags, T represents the length of the generated statement, and c represents the specified sentiment category of the generated dialogue.

[0009] S1: First, process the dataset by performing sentence-level sentiment classification; select multiple standard dialogue data with sentiment annotations to process the existing BERT pre-trained model to obtain a multi-class sentiment classifier; then, preprocess the obtained dialogue data for training through the sentiment classifier to obtain the semantic vector X = {x1, x2, ..., x} corresponding to each sentence. n}, where x nThis represents the nth word vector in the sentence; the probability of the sentence belonging to each sentiment category is obtained through the softmax function, and the sentiment label with the highest probability among k sentiment labels is selected as the sentiment label; constructing a sentiment classifier effectively reduces training costs and improves the expression of emotions in dialogue;

[0010] S2: Perform sentiment specialization on the word vectors of the dialogue data and perform sentiment classification at the phrase level;

[0011] A convolutional neural network is used, including a word embedding layer, a convolutional pooling layer, and a classification layer. The input parameter of the word embedding layer is a word embedding matrix of size V∈n×d, where n is the sentence length and d is the dimension of the word vector. The height of the convolutional kernel of the convolutional layer is set to be the same as the dimension of the word vector, and the width is a hyperparameter, initially set to 2. The pooling layer uses max pooling and then concatenates the resulting values. The classification layer uses a linear softmax function to obtain the final classification, thus completing the sentiment specialization of the word vectors.

[0012] S3: Construct a category-aware model, which includes a generator G. θ With discriminator D c c∈{1,…,k}, where D c Let D represent the probability that the generated dialogue belongs to the c-th sentiment category among k sentiment categories. k+1 The probability that the represented text is real text; for a given time t, the new semantic vector y t Using embedded tag E yt This indicates that, simultaneously, for a given emotion c, an embedded category label E is constructed. c To control the sentiment category information; then to E yt and E c Linear transformation of W in series x The input vector of the generator is obtained. [;] indicates line concatenation;

[0013] The category-aware model sets a relative sentiment category target and updates the generator based on the loss feedback of the discriminator on the sentiment category; it adopts a multi-class discriminator; it uses the sigmoid function to judge the relative category target of sentiment category and text authenticity, and then uses the softmax function to judge the similarity of k sentiment categories separately, thereby enhancing the expression of emotional dialogue;

[0014] The discriminator is based on the actual data distribution of the specified category c. Random sampling Data distribution from this category Random sampling The total category loss was calculated using information from all actual categories. Random sampling And in all generated category information Random sampling

[0015] The following is the judgment formula for the relative category target discriminator:

[0016]

[0017] The first term on the right-hand side of the above equation measures the relative distance between the actual data and the generated data for each category, while the second term measures the relative distance between the actual data and the generated data for all categories; where...

[0018]

[0019] This formula represents the game process of a generative adversarial network, where... Indicates the distribution of real data P r Sample Y r The expectation of Y r Depending on the data input into the relative category discriminator's judgment formula, it represents the sampling of a specified category c or all categories all in the real data. and Indicates the distribution of generated data P θ Sample Y θ Y's expectations θ Similarly, depending on the different data input into the relative category target discriminator judgment formula, it indicates the sampling of a specified category c or all categories all in the generated data. and The relative discriminative metric of the generative adversarial network discriminator when given real or generated input data is defined as follows:

[0020]

[0021] Where Y is based on The corresponding input in the formula represents the real data sample Y. r Or generate data sample Y θ D k+1 (Y) represents the probability that the input text Y to the discriminator is real text. and The definition is the same as the formula above;

[0022] The sentiment enhancement discriminator uses a multi-class cross-entropy loss function. The complete discriminator objective function is as follows:

[0023]

[0024] The above formula The formula for the relative category target discriminator is as follows: the second term represents the sentiment enhancement discrimination target. For the true text distribution of a given sentiment label c, Y s Indicates distribution Random sampling in D c ( s ) indicates that the discriminator is sensitive to the input Y. s The probability of determining whether the emotion is of type c is further determined by the softmax function to determine whether the generated dialogue emotion conforms to the specified emotion category, providing feedback information to the generator;

[0025] S4: A relational memory network is used as the generator. During the generation process, the memory matrix is ​​updated step by step, and combined with the output of the previous steps, specifically, the memory matrix M from the previous time step is updated. t With the processed word vector x t As input, the memory matrix M t Each row represents a memory slot, and self-attention is used to achieve interaction between memory slots, constructing query, key, and value vectors from word vectors; Multi-head attention (MHA) means that H heads are used to obtain information from different levels, extracting features from multiple dimensions, and the query vector of each group of heads h. key vector value vector Where [;] indicates line concatenation, h represents a group of heads in multi-head attention, and The weight matrices representing the linear transformations applied to the query, key, and value are parameters obtained through model training. The attention weights of vector V are obtained by multiplying vectors Q and K, and the memory matrix of this layer is updated accordingly. The calculation formula is as follows:

[0026]

[0027]

[0028] Where σ(*) is the softmax function, T represents the transpose of the matrix, [:] represents concatenating different headers column by column, and d k Let H represent the column number of the key vector K, and h∈H represent a set of heads. The memory matrices of all heads are concatenated column-wise, so the memory matrix and the output of the relational memory network at the next time step are shown in the following formula:

[0029]

[0030]

[0031] In the parameterized function f θ The paper introduces a combination of residual connections, multilayer perceptrons (MLPs), and gated unit operations, represented as follows: Where fmlp (*) indicates a multilayer perceptron (MLP);

[0032] M t+1 The memory matrix output representing the next time step will be compared with the word vector input x at the next time step. t+1 Generate new queries, keys and values, o t This represents the output of the relational memory network;

[0033] S5: Apply the Gumbel-softmax distribution to the output sample o t To approximate the solution, we can directly solve the problem from the multinomial distribution σ(o) t Sample the next output word y in ) t+1 This will lead to non-differentiability issues. The process is divided into two parts: first, sampling discrete sentences using Gumbel-Max, and then approximating the argmax function using the softmax function; first, Gumbel-Max samples the discrete vector y... t+1 The sampling formula is:

[0034]

[0035] in o t The d-th dimension value, For g t The d-th dimension value, g t This is called Gumbel noise, which is obtained by random sampling from the Gumbel distribution, and the formula is:

[0036]

[0037] in This term represents an independent sample taken from a uniform distribution U(0,1), and its purpose is to make y t+1 The return result is not fixed, increasing the diversity of generated responses; since the argmax function is not differentiable and cannot be learned through backpropagation, a differentiable approximation is obtained. It can be obtained from the following formula:

[0038]

[0039] Where σ(*) represents the softmax function, and τ represents the temperature parameter, controlling the smoothness of the softmax function. It is differentiable, so we use it to replace the non-differentiable y. t+1 As input to the discriminator;

[0040] S6: Adding a temperature parameter τ to softmax controls the approximation level. Adjusting the temperature parameter τ is equivalent to generating the vector y in the next time step.t+1 Adjusting the bias and variance in time allows the generator to gain diversity. The closer τ is to 0, the closer the sampled vector will be to one-hot form. The higher the temperature, the smoother the generated distribution and the closer the output is to a uniform distribution.

[0041] Two mutation mechanisms are set up: temperature mutation and quality mutation. The parent generator G... θ Multiple different temperature parameter changes are set, and the temperature set is... Where τ tar Indicates the target temperature. Let τ represent a linear monotonic function at that temperature, where n∈[1,N] represents the current iteration with a maximum iteration count of N, and N is set to 2000. tar The initial value is set to 100; after each training iteration, the temperature parameter will have a small increment. With a fixed quality parameter, when the training reaches the maximum number of iterations N, the temperature is evaluated according to the following formula. The optimal temperature parameter τ from the temperature set is selected as the fixed value for the next training stage; the offspring generator at this temperature parameter is retained as the new parent generator, and the evaluation formula for this temperature stage is... for:

[0042]

[0043] in The above relative discriminant metric formula measures the difference between generated samples and real samples. The larger the value, the higher the quality of the generated dialogue. θ Y represents the distribution of generated data. θ This represents random sampling in the generated data distribution, and the generator offspring at the optimal temperature is selected according to this formula;

[0044] Then comes the quality mutation phase, where the newly selected generator is trained at a fixed temperature, and the best-performing generator is used as the parent generator for the next iteration; the evaluation formula for the quality phase is as follows:

[0045]

[0046] The first item on the right is The temperature assessment formula, μ is set to increase from 0 to 1 during training, and the value corresponding to the highest score of the BLUE evaluation index is selected as the current μ value, which is mainly used to balance quality and diversity. Indicates the distribution of generated data P θ Random sampling Y θ Expectations; NLL qua The diversity of dialogue generation is evaluated by calculating the negative log-likelihood of the samples generated by the generator, using the following formula:

[0047]

[0048] Where P θ Y represents the sample distribution of the generated data. θ Indicates in P θ Random sampling in y T The word vectors generated by the generator are represented by T, and the length of the generated sentence is represented by T. The best-quality offspring generator at a fixed temperature is selected as the optimal choice in this iteration based on the quality judgment formula to perform the task of generating emotional dialogues.

[0049] S7: Each time the generator is trained, the discriminator parameters are updated based on the sampled data using the relative sentiment category target formula.

[0050] 1. This invention modifies the generator part of the generative adversarial network used in traditional text generation tasks, and introduces the relational memory network into the emotional dialogue generation model. Based on its multi-head self-attention mechanism, it enhances the long-distance transmission energy between information, and uses the emotional and semantic information of the preceding text as part of the generator reference, thereby improving the model's expressive ability and feature extraction ability, and making emotional information the focus of input data processing.

[0051] 2. This invention enhances the emotional component of generated emotional dialogues, improving the expression of emotional information in responses. It sets relative target loss and emotional information loss for the discriminator and optimizes the gradient vanishing problem that occurs during training. All emotional categories are sampled and included in the relevant target loss, making classification more precise. The relative distance of a specified emotion among all emotional categories is calculated to control the clarity of the currently generated emotion.

[0052] 3. This invention improves the iterative evolution algorithm. Unlike previous generation tasks performed at a fixed temperature, this invention proposes to adjust the temperature parameter in multiple dynamic learning processes to obtain the optimal temperature, and adds a target parameter as a reference coefficient. This ensures the diversity of emotional dialogue generation without compromising the quality and length of response statements. Attached Figure Description

[0053] Figure 1 Model training flowchart

[0054] Figure 2 Generator G based on relational memory network θ Model diagram

[0055] Figure 3 Overall model diagram Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0057] This invention proposes an emotion-based dialogue generation model based on an improved generative adversarial network. The training process of this model is as follows: Figure 1 As shown. The overall process is as follows: First, acquire the dialogue dataset, perform sentence-level sentiment classification using a pre-trained BERT model, and then perform word-level sentiment specialization to obtain the input vector for the generator. Initialize the generator and discriminator, and pre-train the generator on K types of samples. Sample a batch of sentiment category dialogues, and train the generator with different temperature parameters to find the best descendant generator. Then, while fixing the optimal effect temperature parameter, control the quality parameter to find a generator that balances diversity and dialogue quality. After one generator training cycle, update the discriminator parameters based on the generated data and the objective function relative to the real data, and one round of training is complete. The following is the training process in a specific embodiment.

[0058] S1: First, process the dataset and perform sentence-level sentiment classification. Select 3000 standard dialogues with sentiment annotations to process the existing BERT pre-trained model, obtaining a multi-class sentiment classifier suitable for this task. Then, preprocess the obtained dialogue data used for training through the sentiment classifier to obtain the semantic vector X = {x1, x2, ..., x...} for each sentence. n}, where x n This represents the nth word vector in the sentence. The probability of the sentence belonging to each sentiment category is obtained using the softmax function, and the sentiment label with the highest probability among k sentiment labels is selected as the sentiment marker. Building a sentiment classifier can effectively reduce training costs and improve the expression of sentiment in dialogue.

[0059] S2: Sentiment specialization is applied to the word vectors in the dialogue data, performing phrase-level sentiment classification. Traditional word vectors often fail to differentiate between words with similar contexts and grammatical levels, frequently grouping antonyms like "like" and "dislike" together, which significantly impacts sentiment generation tasks. Therefore, sentiment specialization of word vectors is necessary, primarily measured by calculating cosine distance to determine the distance between different categories. A convolutional neural network is used, consisting of a word embedding layer, convolutional pooling layers, and a classification layer. The word embedding layer's input parameter is a word embedding matrix of size V∈n×d, where n is the sentence length and d is the dimension of the word vectors. The convolutional layer's kernel height is set to the same as the word vector dimension, and its width is a hyperparameter initially set to 2. The pooling layer uses max pooling, concatenating the resulting values. The classification layer uses a linear softmax function to derive the final classification, completing the sentiment specialization of the word vectors.

[0060] S3: Construct a category-aware model, such as Figure 3 As shown, the model includes a generator G. θ With discriminator D c c∈{1,…,k}, where D c Let D represent the probability that the generated dialogue belongs to the c-th sentiment category among k sentiment categories. k+1 This represents the probability that the text is real text. For a new semantic vector y at a given time t... t Using embedded tag E yt This indicates that, for a given emotion c, an embedded category label E is constructed. c To control the sentiment category information. Then for E yt and E c Linear transformation of W in series x The input vector of the generator is obtained. The brackets [;] indicate line concatenation.

[0061] The category-aware model sets a relative sentiment category target and updates the generator based on the discriminator's loss feedback on the sentiment category. Because the discriminator needs to determine both the authenticity of the text and the sentiment of the dialogue, this invention employs a multi-class discriminator. Specifically, a sigmoid function is used to determine the relative category target between sentiment category and text authenticity, and a softmax function is used to separately determine the similarity of k sentiment categories, thus enhancing the expression of emotional dialogue.

[0062] The discriminator is based on the actual data distribution of the specified category c. Random sampling Data distribution from this category Random sampling The total category loss was calculated using information from all actual categories. Random sampling And in all generated category information Random sampling This item can calculate the offset loss of a specified category among all categories.

[0063] The following is the judgment formula for the relative category target discriminator:

[0064]

[0065] The first term on the right side of the above equation measures the relative distance between the actual data and the generated data for each category, while the second term measures the relative distance between the actual data and the generated data for all categories.

[0066]

[0067] This formula represents the game process of a generative adversarial network, where... Indicates the distribution of real data P r Sample Y r The expectation of Y r Depending on the data input into the relative category discriminator's judgment formula, it can represent the sampling of a specified category c or all categories all in the real data. and Indicates the distribution of generated data P θ Sample Y θ Y's expectations θ Similarly, depending on the data input into the relative category discriminator's judgment formula, it can represent the sampling of a specified category c or all categories all in the generated data. and The relative discriminative metric of the generative adversarial network discriminator when given real or generated input data is defined as follows:

[0068]

[0069] Where Y is based on The corresponding input in the formula represents the real data sample Y. r Or generate data sample Y θ D k+1 (Y) represents the probability that the input text Y to the discriminator is real text. and The definitions are the same as in the formula above. The above part calculates the difference between the true probability on real samples and the true probability on generated samples. For each category, the generator aims to reduce this difference, while the discriminator aims to increase the probability that real text is more realistic than generated text. Therefore, the relative evolutionary objective of the generator is opposite to that of the discriminator. Compared to the standard GAN objective, the relative classification objective effectively trains our sentiment classification dialogue generation model.

[0070] The sentiment enhancement discriminator uses a multi-class cross-entropy loss function. The complete discriminator objective function is as follows:

[0071]

[0072] The above formula The formula for the relative category target discriminator is as follows: the second term represents the sentiment enhancement discrimination target. For the true text distribution of a given sentiment label c, Y s Indicates distribution Random sampling in D c ( s ) indicates that the discriminator is sensitive to the input Y. s The probability of determining whether the emotion is of type c is further determined by the softmax function to determine whether the generated dialogue emotion conforms to the specified emotion category, providing feedback information to the generator.

[0073] S4: Current GAN-based dialogue generation models commonly use LSTM or memory-enhanced networks as generators, which lack expressive power in generating text and neglect the interaction between memory access. This invention uses a relational memory network as the generator, such as... Figure 2 As shown.

[0074] During the generation process, the memory matrix will be updated step by step, and combined with the output of the previous steps. Specifically, this means that the memory matrix M from the previous time step will be updated. t With the processed word vector x t As input, the memory matrix M t Each row represents a memory slot, and self-attention is used to enable interaction between memory slots, constructing query, key, and value vectors from word vectors. Multi-head attention (MHA) means that H heads are used to acquire information from different levels, extracting features from multiple dimensions, with each group of h heads producing a query vector. key vector value vector Where [;] indicates line concatenation, and h represents a group of heads in a multi-head attention mechanism. These represent the weight matrices for linear transformations of the query, key, and value, respectively, and are parameters obtained through model training. The attention weights of vector V are obtained by multiplying vectors Q and K, and the memory matrix of this layer is updated accordingly. The calculation formula is as follows:

[0075]

[0076]

[0077] Where σ(*) is the softmax function, T represents the transpose of the matrix, [:] represents concatenating different headers column by column, and d k Let H represent the column number of the key vector K, and h∈H represent a set of heads. The memory matrices of all heads are concatenated column-wise, so the memory matrix and the output of the relational memory network at the next time step are shown in the following formula:

[0078]

[0079]

[0080] Considering that relational memory is executed in a cyclical manner during decoding, it may be affected by vanishing and bursting phenomena. Therefore, in the parameterized function f... θ The paper introduces a combination of residual connections, multilayer perceptrons (MLPs), and gated unit operations, which can be represented as follows: Where f mlp (*) indicates a multilayer perceptron (MLP).

[0081] M t+1 The memory matrix output representing the next time step will be compared with the word vector input x at the next time step. t+1 Generate new queries, keys and values, o t This represents the output of the relational memory network.

[0082] S5: Apply the Gumbel-softmax distribution to the output sample o t To approximate the solution, we can directly solve the problem from the multinomial distribution σ(o) t Sample the next output word y in ) t+1 This will lead to non-differentiability issues. The process is divided into two parts: first, sampling discrete sentences using Gumbel-Max, and then approximating the argmax function using the softmax function. First, Gumbel-Max samples the discrete vector y... t+1 The sampling formula is:

[0083]

[0084] in o t The d-th dimension value, For g t The d-th dimension value, g tThis is called Gumbel noise, which is obtained by random sampling from the Gumbel distribution, and the formula is:

[0085]

[0086] in This term represents an independent sample taken from a uniform distribution U(0,1), and its purpose is to make y t+1 The return result is not fixed, increasing the diversity of generated responses. Since the argmax function is not differentiable and cannot be learned through backpropagation, a differentiable approximation is obtained. It can be obtained from the following formula:

[0087]

[0088] Where σ(*) represents the softmax function, and τ represents the temperature parameter, controlling the smoothness of the softmax function. It is differentiable, so we use it to replace the non-differentiable y. t+1 As input to the discriminator.

[0089] S6: Since improving diversity by adjusting the temperature parameter τ can lead to a decrease in generated dialogue quality, this invention proposes an iterative optimization learning algorithm. In each training iteration, the algorithm finds the direction of temperature change that maximizes training effectiveness. Based on this, it further filters the generated dialogue quality, selecting the generator with the best performance and using it to achieve the task of generating emotional dialogues. The temperature parameter τ is added to Gumbel-softmax to control the approximation level; adjusting the temperature parameter τ is equivalent to approximating the generated vector y in the next time step. t+1 Adjusting the bias and variance in time allows the generator to gain diversity. When τ is closer to 0, the sampled vector will be closer to the one-hot form, solving the difference between forward and backward propagation. The higher the temperature, the smoother the generated distribution and the closer the output is to a uniform distribution.

[0090] The implementation mainly involves setting up two mutation mechanisms: a temperature mutation mechanism and a quality mutation mechanism, which affect the parent generator G. θ Multiple different temperature parameter changes are set, and the temperature set is... Where τ tar Indicates the target temperature. Let τ represent a linear monotonic function at that temperature, where n∈[1,N] represents the current iteration with a maximum iteration count of N, and N is set to 2000. tar The initial value is set to 100. After each training iteration, the temperature parameter will have a small increment. With a fixed quality parameter, when the training reaches the maximum number of iterations N, the temperature is evaluated according to the following formula: The optimal temperature parameter τ from the temperature set is selected as the fixed value for the next training stage. The offspring generator at this temperature parameter is retained as the new parent generator, and the evaluation formula for this temperature stage is as follows: for:

[0091]

[0092] in The above relative discriminant metric formula can measure the discrepancy between generated samples and real samples. A larger value for P indicates higher quality dialogue generation. θ Y represents the distribution of generated data. θ This represents random sampling in the generated data distribution, and the generator offspring at the optimal temperature is selected according to this formula.

[0093] The process then proceeds to the quality mutation phase, where the newly selected generator is trained at a fixed temperature. The generator with the best performance is selected as the parent generator for the next iteration. The evaluation formula for the quality phase is as follows:

[0094]

[0095] The first item on the right is The temperature assessment formula, μ is set to increase from 0 to 1 during training, and the value corresponding to the highest score of the BLUE evaluation index is selected as the current μ value, which is mainly used to balance quality and diversity. Indicates the distribution of generated data P θ Random sampling Y θ The expectation. NLL qua The diversity of dialogue generation is evaluated by calculating the negative log-likelihood of the samples generated by the generator, using the following formula:

[0096]

[0097] Where P θ Y represents the sample distribution of the generated data. θ Indicates in P θ Random sampling in y T Let T represent the word vectors generated by the generator, and T represent the length of the generated sentence. Based on a quality judgment formula, the best-quality offspring generator at a fixed temperature is selected as the optimal choice in this iteration for the task of generating emotional dialogues.

[0098] S7: After each generator training iteration, the discriminator parameters are updated based on the sampled data using the relative sentiment category target formula. The overall model structure is as follows: Figure 3 .

Claims

1. An emotion dialogue generation method based on an improved generative adversarial network, characterized in that: The dialogue data with emotional labels is obtained, and the data is divided into six categories of Angry, Disgust, Happy, Like, Sad and Other according to the emotional type, and the number of emotional categories is defined as k in the following; the emotional word vector in the dialogue sentence is semantically specialized to obtain specialized dialogue data with different emotional labels; A relational memory network is used as the generator, and multiple discriminators are set up to provide feedback on semantic and sentiment information respectively. An iterative evolutionary learning algorithm is used to adjust the balance between the quality and diversity of the generated sentences, generating dialogue response sentences Y with a specified sentiment category. c =(y1,…,y t ,…,y T ), y t ∈V represents the word vector of the response statement, V represents the vocabulary of candidate tags, T represents the length of the generated statement, and c represents the specified sentiment category of the generated dialogue. S1: First, process the data set and perform sentence-level sentiment classification; Select multiple standard dialogue data with emotional annotation to process the existing BERT pre-training model to obtain a multi-classification emotion classifier; then the obtained dialogue data for training is preprocessed by the emotion classifier to obtain the semantic vector X={x1, x2, …, x n} corresponding to each sentence, wherein x n n represents the n th word vector in the sentence; the probability of the sentence belonging to each emotion category is obtained by a softmax function, and the one with the maximum probability in the k emotion labels selected for the sentence is selected as the emotion label; the construction of the emotion classifier effectively reduces the training cost and improves the dialogue emotion expression; S2: Perform sentiment specialization on the dialogue data word vector and perform phrase-level sentiment classification; The convolutional neural network includes a word embedding layer, a convolutional pooling layer and a classification layer. The input parameter of the word embedding layer is a word embedding matrix with a size of V∈n×d, where n is the sentence length and d is the dimension of the word vector. The convolution kernel height of the convolution layer is set to be the same as the word vector dimension, and the width is a hyperparameter, initially set to 2. The maximum pooling is used in the pooling layer, then the obtained values are spliced, and the final classification is obtained by the linear softmax function, completing the sentiment specialization of the word vector; S3: Construct a category-aware model, which includes a generator G. θ With discriminator D c c∈{1,…,k}, where D c Let D represent the probability that the generated dialogue belongs to the c-th sentiment category among k sentiment categories. k+1 The probability that the represented text is real text; for a given time t, the new semantic vector y t , using embedded tags This indicates that, simultaneously, for a given emotion c, an embedded category label E is constructed. c To control the sentiment category information; then to and E c Linear transformation of W in series x The input vector of the generator is obtained. [;] indicates line concatenation; The category-aware model updates the generator according to the loss feedback of the discriminator on the emotional category by setting a relative emotional category target; A multi-class discriminator is used; the sigmoid function is used to judge the relative category target of the emotional category and the true or false of the text, and the softmax function is used to separately judge the similarity of the k emotional categories, so as to strengthen the expression of emotional dialogue; The discriminator randomly samples from the actual data distribution of a specified class c The generator randomly samples from the generated data distribution of this class and computes the total class loss by randomly sampling over all actual class information and randomly sampling over all generated class information ​​​​ The following is the judgment formula of the relative category target discriminator: The first term on the right side of the above formula measures the relative distance between the actual data and the generated data of each category, and the second term measures the relative distance between the actual data and the generated data of all categories. Wherein This formula represents the game process of a generative adversarial network, where... Indicates the distribution of real data P r Sample Y r The expectation of Y r Depending on the data input into the relative category discriminator's judgment formula, it represents the sampling of a specified category c or all categories all in the real data. and Indicates the distribution of generated data P θ Sample Y θ Y's expectations θ Similarly, depending on the different data input into the relative category target discriminator judgment formula, it indicates the sampling of a specified category c or all categories all in the generated data. and The relative discriminative metric of the generative adversarial network discriminator when given real or generated input data is defined as follows: where Y is according to where the respective inputs represent real data samples Y r or generated data samples Y θ , D k+1 (Y) represents the probability that the discriminator input text Y is real text, and are defined the same as the above equations; The emotional enhancement discriminator part uses a multi-class cross-entropy loss function, and the complete discriminator target function formula is: The above formula is the judgment formula of the relative category target discriminator, and the second term is the emotion enhancement judgment target, is the real text distribution of the specified emotion label c, Y s represents random sampling in the distribution D c (Y s ) represents the probability that the discriminator judges that the input Y s is the cth emotion, and this term further judges whether the generated dialogue emotion conforms to the specified emotion category through the softmax function, thereby providing feedback information for the generator; S4: Adopting relational memory network as the generator, the memory matrix will be updated step by step during the generation process, and the output of the previous step will be combined, which is specifically represented as the memory matrix M t and the processed word vector x t as input, the memory matrix M t Each row of the memory matrix represents a memory slot, and the interaction between the memory slots is realized by using self-attention. The word vector is constructed as a query, key, and value vector. The multi-head attention mechanism (MHA) represents H heads to obtain information at different levels and extract features from multiple dimensions. The query vector of each head h key vector value vector where [;] represents row connection, h∈H, and represents a group of heads in multi-head attention, respectively represent the weight matrix for linear transformation of the query, key, and value. The parameters are obtained through model training. The attention weight of the vector V is obtained by multiplying the Q and K vectors, and the output of the updated memory matrix is The calculation formula is as follows: where σ (*) is a softmax function, T denotes the transpose of a matrix, [:] denotes the concatenation of different heads by column, d k denotes the number of columns of the key vector K, h e H denotes one of the set of heads, denotes the concatenation of all heads' memory matrices by column, thus the memory matrix at the next time step and the output of the relational memory network are given by the following equations: In the parametric function f θ a combination of residual connections, multi-layer perceptrons (MLP), gated unit operations is introduced, denoted as where f mlp (*) denotes a multi-layer perceptron (MLP); M t+1 memory matrix output representing the next time instant, to be inputted with the word vector of the next time instant x t+1 generate new queries, keys and values, o t output representing the relational memory network; S5: Gumbel-softmax distribution is used to sample the output sample o t Approximation, solve the problem of non-differentiable sampling the next output word y t from the polynomial distribution σ(o t+1 ), the process is divided into two parts, respectively using Gumbel-Max sampling discrete sentence, and then through the softmax function to approximate argmax function; first Gumbel-Max sampling formula for discrete vector y t+1 : wherein denotes o t the d-th dimension value of is g t the d-th dimension value of g t known as Gumbel noise, is randomly sampled from a Gumbel distribution, given by where represents independent samples drawn from the uniform distribution U(0, 1), and the term is present to make y t+1 The return result is not fixed, and the diversity of the generated reply is increased; since the argmax function is not derivable, it cannot be learned by back propagation, so a differentiable approximation value is obtained, and the differentiable approximation value can be obtained by the following formula: where σ (*) denotes the softmax function, and τ denotes a temperature parameter that controls the degree of smoothing of the softmax, since is differentiable, it is used instead of the non-differentiable y t+1 as input to the discriminator; S6: Add temperature parameter τ in Gumbel-softmax to control the degree of approximation, adjusting the temperature parameter τ is equivalent to generating vector y at the next moment t+1 Adjust the bias and variance at this time, so that the generator obtains diversity, when τ is closer to 0, the vector sampled will be closer to one-hot form; the higher the temperature, the smoother the generated distribution, and the output is closer to uniform distribution; Two variation mechanisms are set, temperature variation mechanism and quality variation mechanism, to generate parent generator G θ Set multiple different temperature parameter variation directions, and the temperature set is Where τ tar represents the target temperature, represents a linear monotonic function at the temperature, n∈[1,N] represents the current iteration of the maximum iteration number N, N is set to 2000, τ tar is initially set to 100; after each iteration training, the temperature parameter has a small increment, and under the premise of fixing the quality parameter, when the training reaches the maximum iteration number N, the temperature evaluation formula is The temperature parameter τ with the best effect in the temperature set is selected as the fixed value of the next stage training; the child generator under the temperature parameter is retained as the new parent generator, and the evaluation formula of the temperature stage is : wherein represents the above relative discriminative measure objective formula, measures the gap between the generated sample and the real sample, The larger the value, the higher the quality of the dialogue generation, P θ represents the generated data distribution, Y θ represents a random sampling in the generated data distribution, and a generator offspring at an optimal temperature is selected according to the formula; Then enter the quality variation stage, train the newly selected generator at a fixed temperature to obtain the best generator as the parent generator for the next iteration; The judgment formula of the quality stage is where the first term on the right is The temperature evaluation formula, μ is set from 0 to 1 in training, and the value corresponding to the highest BLUE evaluation index score is selected as the current μ value, mainly used to balance quality and diversity; Indicates that the generated data distribution P θ Randomly sample Y θ The expectation of NLL qua The negative log likelihood of the sample generated by the generator is calculated, and the diversity of the dialogue generation is evaluated, and the formula is where P θ represents the sample distribution of the generated data, Y θ represents random sampling in P θ , y T represents the word vector generated by the generator, and T represents the length of the generated sentence; according to the quality judgment formula, the best child generator with the best quality at a fixed temperature is selected as the optimal choice in the current iteration process, and the task of generating emotional dialogue is performed; S7: After training the generator once, update the discriminator parameters according to the relative emotional category target formula based on the sampling data.

Images