A conversation emotion recognition method based on supervised contrastive learning and response generation assistance

Abstract

The present invention provides a dialogue emotion recognition method based on supervised contrastive learning and response generation assistance. Firstly, due to the constructed pre-training model CoG-BART, a dialogue-level Transformer module is used to perform context modeling on the dialogue to be tested, thereby solving the problem of long-distance dependency between discourses. At the same time, since the overall model architecture is composed of Transformers, fewer structural prior assumptions are introduced; secondly, since supervised contrastive learning is used to train the model, while making full use of label information, not only the stability of the model during training can be increased and the generalization of the model can be enhanced, but also samples of the same emotion can be cohesive and samples of different emotions can be mutually exclusive, thereby improving the problem that similar emotions are difficult to distinguish; finally, since the auxiliary response generation task can obtain more accurate context according to the given context information, the model can consider richer context information when judging the emotion of the discourse, thereby improving the accuracy of the emotion label of the model in recognizing the dialogue.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning and natural language processing, and specifically relates to a conversation emotion recognition method based on supervised contrastive learning and auxiliary response generation. Background Art

[0002] With the development and popularization of personal intelligent terminal technology and social networks, the importance of building a dialogue system that can understand user emotions and intentions and conduct effective dialogue interactions has increased significantly. A particularly critical module in the dialogue system is the natural language understanding module that analyzes user emotions. Analyzing user emotions with contextual relationships is not only an advancement of simple sentiment classification tasks, but also more in line with usage scenarios in the real world and has more research value. The task of dialogue emotion recognition is to classify the emotions of all utterances in a historical dialogue on one end with a contextual relationship, and each historical dialogue contains interactions between multiple different interlocutors.

[0003] There are three main challenges in the task of conversational emotion recognition: (1) The emotion of each utterance in the conversation process will be affected by the context information. For example, a specific emotion will depend on the specific utterance in the context. At the same time, utterances with the same expression may have completely different emotion labels in different contexts. Therefore, the problem of efficiently modeling context dependency and speaker dependency is the main difference between this task and traditional emotion recognition tasks. (2) The emotion of each speaker will be affected by the utterances of other speakers in the conversation process, so there may be sudden changes in the speaker's emotion during the conversation. (3) There are some semantically similar emotions, such as "frustrated" and "sad", "happy" and "excited", etc. There is a certain degree of difficulty in distinguishing such similar emotions.

[0004] Some current work uses various graph neural networks to handle context dependency and speaker dependency. However, as the number of graph network layers increases, oversmoothing begins to appear, making the representation of similar emotions more difficult to distinguish. Summary of the invention

[0005] In order to solve the above problems, a method for dialogue emotion recognition is provided to better model context dependency and speaker dependency. The present invention adopts the following technical solutions:

[0006] The present invention provides a method for dialogue emotion recognition based on supervised contrastive learning and response generation assistance, characterized in that it includes the following steps: step S1, constructing a pre-training model CoG-BART based on an encoder-decoder, the pre-training model CoG-BART having a dialogue-level Transformer module and a shared embedding layer; step S2, using the pre-training model CoG-BART to encode the training dialogue to obtain the hidden state of the discourse contained in the training dialogue; step S3, the pre-training model CoG-BART performs an auxiliary response generation task based on the hidden state of the discourse and the context text and calculates the generation loss, and at the same time inputs the hidden state into the dialogue-level The Transformer module performs context modeling of the discourse to obtain a new discourse hidden state; step S4, calculates the supervised contrastive learning loss and the cross entropy loss based on the new discourse hidden state; step S5, calculates the overall training loss based on the generation loss, the supervised contrastive learning loss and the cross entropy loss, and trains the pre-trained model CoG-BART until the trained pre-trained model CoG-BART is obtained and used as the dialogue emotion recognition model; step S6, inputs the dialogue to be tested containing several sentences into the dialogue emotion recognition model for emotion recognition and outputs the emotion labels corresponding to several sentences in the dialogue to be tested, wherein the context text pairs are generated based on the training dialogue.

[0007] The method for dialogue emotion recognition based on supervised contrastive learning and response generation assistance provided by the present invention may also have such a technical feature, wherein step S2 includes the following sub-steps: step S2-1, splicing an utterance in the training dialogue with the corresponding speaker to obtain a spliced ​​utterance:

[0008]

[0009] In the formula, u t For a certain utterance in the training dialogue, p(u t ) is the discourse u t Speaker; Step S2-2, chunking the concatenated utterances to obtain chunked utterances:

[0010]

[0011] In the formula, w t，i Indicates the chunk after chunking, |n t | represents the length of the sequence after the t-th round of speech is chunked, and <s> and < / s> represent the special chunks at the beginning and end of the sentence respectively; Step S2-3, the chunked speech is sequentially input into the shared embedding layer and the encoder to obtain the sentence representation corresponding to the chunked speech, i.e., the hidden state of the speech:

[0012]

[0013]

[0014] In the formula, H t represents the sentence representation after embedding, is the encoded sentence representation, Among them, s and d represent the sequence length and hidden state dimension respectively.

[0015] The dialogue emotion recognition method based on supervised contrastive learning and response generation assistance provided by the present invention may also have such a technical feature, wherein, in step S3, context modeling includes the following sub-steps: step S3-1, representing the encoded sentence Get the aggregate representation of each utterance through max pooling

[0016]

[0017] In the formula, M represents the mask operation on the filled position; Step S3-2, for all conversations in a context, the multi-head attention between each two different utterances is calculated based on the multi-head attention mechanism, and the different features of each utterance are aggregated to obtain the final hidden state representation. The hidden state of each two different utterances is The calculation formula of multi-head attention is:

[0018]

[0019]

[0020] MultiHead(Q,K,V)=[head1;…;head n ]W O

[0021] In the formula, Q, K, and V represent the hidden states of two different utterances. With three trainable parameter matrices W i Q , W i K , W i V The query vector, key vector, and value vector obtained by multiplication, M represents the mask vector, W O Represents the parameter vector that aggregates the information in each attention head headi; Step S3-3, modeling the context dependency based on the dialogue-level Transformer module to obtain the new utterance hidden state:

[0022]

[0023]

[0024] In the formula, Represents a dialog in a window. Indicates a conversation within the window size bs in a conversation. Represents the new hidden state of the utterance after context modeling.

[0025] The dialogue emotion recognition method based on supervised contrastive learning and response generation assistance provided by the present invention may also have such a technical feature, wherein, in step S3, the auxiliary response generation task is: the pre-trained model CoG-BART is based on the above u t Generate the corresponding context u t+1 , below t+1 The hidden state of each chunk in is output by the decoder sequence:

[0026]

[0027]

[0028]

[0029] Where BART-Encoder and BART-Decoder represent the encoder and decoder of the pre-trained model CoG-BART, respectively. The encoder and the hidden state H t After encoding, the encoded hidden state is generated The decoder generates the hidden state of the next word block And get the word block u through the Softmax function t+1，j .

[0030] The dialogue emotion recognition method based on supervised contrastive learning and response generation assistance provided by the present invention may also have such a technical feature, wherein the supervised contrastive learning is: for a batch of N training examples, for each training example in the batch, 2N enhanced training examples are obtained by a predetermined enhancement method, and the predetermined enhancement method is to transform the latent state of a speech into Make a copy and delete its gradient, and we get

[0031]

[0032] The method for dialogue emotion recognition based on supervised contrastive learning and response generation assistance provided by the present invention may also have such a technical feature, wherein the overall training loss is a weighted sum of a cross entropy loss, a supervised contrastive loss, and a generation loss, and the cross entropy loss is to calculate the distribution of the probability of emotion labels obtained by a multi-layer perceptron after the discourse representation generated by context modeling:

[0033]

[0034]

[0035]

[0036] Where P i For statement It represents the conditional probability in the label space obtained after the Softmax function is calculated, and the predicted label with the highest probability is obtained through the argmax function. y i，c Indicates the true emotional label of a certain utterance, represents the probability distribution of the density layer output on category c, L CE is the cross entropy loss, generating loss for:

[0037]

[0038] In the formula, θ is the parameter of the pre-trained model CoG-BART that needs to be optimized, and the weighted sum is 1, that is:

[0039]

[0040] In the formula, α represents the weight of the supervised contrast loss, β represents the weight of the generated loss, To generate losses.

[0041] Function and Effect of the Invention

[0042] According to the dialogue emotion recognition method based on supervised contrastive learning and response generation assistance of the present invention, firstly, due to the pre-training model CoG-BART built based on the Transformer encoder-decoder, the dialogue-level Transformer module is used to perform context modeling on the dialogue to be tested to obtain the discourse representation, thereby solving the long-distance dependency problem between discourses. At the same time, since the overall model architecture is composed of Transformers, fewer structural prior assumptions are introduced. Secondly, since supervised contrastive learning (Supervised Contrastive Learning, SCL) is used to train the model, while making full use of label information, the samples of the same emotion are cohesive and the samples of different emotions are mutually exclusive, thereby improving the problem of similar emotions being difficult to distinguish. At the same time, when processing noisy labels, compared with the cross entropy loss, the supervised contrast loss can increase the stability of the model during training and enhance the generalization of the model. Finally, when performing context modeling, an auxiliary response generation task is also performed to obtain a more accurate context based on the given context information, so that the model can consider more abundant context information when judging the emotion of the discourse, and improve the accuracy of the emotion label of the model recognition dialogue. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flow chart of a method for dialogue emotion recognition based on supervised contrastive learning and response generation assistance in an embodiment of the present invention;

[0044] Figure 2 Schematic diagram of the structure of the dialogue emotion recognition model in the embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the following is a detailed description of the conversation emotion recognition method based on supervised contrastive learning and response generation assistance of the present invention in combination with embodiments and drawings.

[0046] <Example>

[0047] Figure 1 This is a flowchart of a method for dialogue emotion recognition based on supervised contrastive learning and response generation assistance in an embodiment of the present invention.

[0048] like Figure 1 As shown in FIG. 1 , the dialogue emotion recognition method based on supervised contrastive learning and response generation assistance includes the following sub-steps:

[0049] Step S1, constructing a pre-trained model CoG-BART based on the encoder-decoder, where the pre-trained model CoG-BART has a dialogue-level Transformer module and a shared embedding layer.

[0050] Step S2: Use the pre-trained model CoG-BART to encode the training dialogue to obtain the hidden state of the speech contained in the training dialogue.

[0051] In this embodiment, in order to model the dependency relationship between speakers and utterances, for a certain utterance u in the training dialogue t , first compare it with the corresponding speaker p(u t ) to obtain the concatenated utterance

[0052]

[0053] Then the spliced ​​words Chunking is performed to obtain chunked utterances:

[0054]

[0055] In the formula, w t，i Indicates the chunk after chunking, |n t | represents the length of the sequence after the word block in the tth round of discourse, and <s> and < / s> represent the special word blocks at the beginning and end of the sentence respectively.

[0056] Finally, the chunked utterances are input into the shared embedding layer and the encoder in turn to obtain the sentence representation corresponding to the chunked utterances, i.e., the hidden state of the utterances:

[0057]

[0058]

[0059] In the formula, H t represents the sentence representation after embedding, is the encoded sentence representation, Among them, s and d represent the sequence length and hidden state dimension respectively.

[0060] In step S3, the pre-trained model CoG-BART performs auxiliary response generation tasks based on the hidden state and context text of the utterance and calculates the generation loss. At the same time, the hidden state is input into the dialogue-level Transformer module to model the context of the utterance to obtain a new hidden state of the utterance.

[0061] The context text pair is obtained by combining the nth sentence and the n+1th sentence in the training dialogue in the form of a dialogue. For example, each sentence in the training dialogue is represented as {n1, n2, n3, n4, n5}, and the context text pairs are {n1, n2}, {n2, n3}, {n3, n4} and {n4, n5}.

[0062] Among them, auxiliary response generation enables the model to consider richer contextual information when judging the sentiment of discourse.

[0063] The auxiliary reply generation task is: the pre-trained model CoG-BART is based on the above u t Generate the corresponding context u t+1 , the following text t+1 The hidden state of each chunk in is output by the decoder sequence:

[0064]

[0065]

[0066]

[0067] Where BART-Encoder and BART-Decoder represent the encoder and decoder of the pre-trained model CoG-BART, respectively. The encoder and the hidden state H t After encoding, the encoded hidden state is generated The decoder generates the hidden state of the next word block And get the word block u through the Softmax function t+1，j .

[0068] Among them, the generation loss of the auxiliary reply task is:

[0069]

[0070] Where θ is the parameter of the pre-trained model CoG-BART that needs to be optimized.

[0071] The process of the context modeling task is as follows:

[0072] Step S3-1, the encoded sentence is represented as Get the aggregate representation of each utterance through max pooling

[0073]

[0074] In the formula, M represents the mask operation on the filled position, so that the influence of irrelevant word blocks on the sentence representation is removed during the maximum pooling process.

[0075] In step S3-2, for all conversations in a context, the multi-head attention between every two different utterances is calculated based on the multi-head attention mechanism, and the different features of each utterance are aggregated to obtain the final latent state representation, thereby fully modeling the complex dependencies between different utterances in the context.

[0076] Among them, the hidden state of each two different utterances The calculation formula of multi-head attention is:

[0077]

[0078]

[0079] MultiHead(Q,K,V)=[head1;…;head n ]W O

[0080] In the formula, Q, K, and V represent the hidden states of two different utterances. With three trainable parameter matrices W i Q , W i K , W i V The query vector, key vector, and value vector obtained by multiplication, M represents the mask vector, W O Indicates that each attention head i The parameter vector for aggregating the information in .

[0081] Step S3-3, modeling the context dependency based on the dialogue-level Transformer module to obtain the new utterance hidden state:

[0082]

[0083]

[0084] In the formula, Represents a dialog in a window. Indicates a conversation within the window size bs in a conversation. Represents the new hidden state of the utterance after context modeling.

[0085] Step S4, calculate the supervised contrastive learning loss and cross entropy loss based on the new utterance hidden state.

[0086] Figure 2 Schematic diagram of the structure of the dialogue emotion recognition model in the embodiment of the present invention.

[0087] Compared with ordinary self-supervised contrastive learning, this embodiment uses supervised contrastive learning to make full use of label information, so that it is not limited to the relationship between a single positive and negative example, but also takes into account all similar information in a batch, so that samples with the same emotion are cohesive, and samples with different emotions are mutually exclusive (such as Figure 2 shown).

[0088] For conversation sentiment classification, because the number of samples in each category in some data sets is extremely unbalanced, and the loss of supervised contrastive learning will mask itself during calculation, if there is only one sample of a category in a batch, it cannot be directly applied to calculate the loss.

[0089] Therefore, in this embodiment, in order to prevent its gradient from being back-propagated and ensure that the parameter optimization of the model remains stable, the hidden state of a discourse is Make a copy and delete its gradient, and we get

[0090]

[0091] For a batch with N training examples, the above method is used to obtain enhanced 2N training examples for each training example in the batch.

[0092] Among them, the loss of all training examples in a batch when training the model is:

[0093]

[0094]

[0095]

[0096] where i∈I={1,2,…,2N} represents the index of a sample in a multi-view batch, Φ(·)∈R d represents the hidden state of the encoder output after L2 regularization and before the input density layer, τ∈R + Represents the temperature coefficient used to control the distance, P(i) = I j＝i -{i, N+i} represents the sample with the same category as i except itself, A(i)=I-{i, N+i} represents the other samples in the batch except itself, is the supervised contrastive learning loss.

[0097] The discourse representation (new discourse hidden state) generated by context modeling is passed through a multi-layer perceptron to obtain the distribution of sentiment label probabilities for calculating the cross entropy loss:

[0098]

[0099]

[0100]

[0101] Where P i For statement It represents the conditional probability in the label space obtained after the Softmax function is calculated, and the predicted label with the highest probability is obtained through the argmax function. y i，c Indicates the true emotional label of a certain utterance, represents the probability distribution of the density layer output on category c, L CE is the cross entropy loss.

[0102] Step S5, calculating the overall training loss based on the generation loss, the supervised contrastive learning loss and the cross entropy loss, and training the pre-trained model CoG-BART until the trained pre-trained model CoG-BART is obtained and used as the dialogue emotion recognition model.

[0103] In this embodiment, the overall training loss is a weighted sum of the cross entropy loss, the supervised contrast loss, and the generation loss, and the weighted sum is 1.

[0104]

[0105] Where α represents the weight of the supervised contrast loss, and β represents the weight of the auxiliary reply generation loss.

[0106] Step S6, inputting the dialogue to be tested containing several sentences into the dialogue emotion recognition model for emotion recognition and outputting the emotion labels corresponding to the several sentences in the dialogue to be tested.

[0107] Example Function and Effect

[0108] According to the dialogue emotion recognition method based on supervised contrastive learning and response generation assistance provided in this embodiment, since the pre-trained model CoG-BART is established based on the Transformer encoder-decoder architecture, and the dialogue-level Transformer module is used to perform context modeling on the dialogue to be tested to obtain discourse representation, so as to solve the long-distance dependency problem between discourses. Compared with the previous method of only using the pre-trained model as a feature extractor and using the extracted features for downstream graph network node representation, since the model architecture in this embodiment is composed of Transformers, fewer structural prior assumptions are introduced; at the same time, when performing context modeling, an auxiliary response generation task is also performed to obtain a more accurate context based on the given previous information, so that the model can consider richer context information when judging the emotion of the discourse, thereby improving the accuracy of the model's recognition of the emotion label of the dialogue.

[0109] In the embodiment, since supervised contrastive learning (SCL) is used to train the model, by making full use of label information, samples of the same emotion are cohesive and samples of different emotions are mutually exclusive, thereby improving the problem of similar emotions being difficult to distinguish. At the same time, when processing noisy labels, compared to cross entropy loss, supervised contrast loss can increase the stability of the model during training and enhance the generalization of the model. At the same time, compared to the original SCL, the supervised contrastive learning of this embodiment copies the hidden state of a batch of samples during the model training process and deletes its gradient as a multi-view representation, thereby avoiding calculation errors caused by unbalanced data set samples.

[0110] The above embodiments are only used to illustrate specific implementation modes of the present invention, and the present invention is not limited to the description scope of the above embodiments.

Claims

1. A method for conversation emotion recognition based on supervised contrastive learning and response generation assistance, characterized in that: The following steps are involved: Step S1, constructing a pre-trained model CoG-BART based on an encoder-decoder, wherein the pre-trained model CoG-BART has a dialogue-level Transformer module and a shared embedding layer; Step S2, using the pre-trained model CoG-BART to encode the training dialogue to obtain the latent state of the speech contained in the training dialogue; Step S3, the pre-trained model CoG-BART performs an auxiliary response generation task based on the hidden state and context text of the utterance and calculates the generation loss, and at the same time inputs the hidden state into the dialogue-level Transformer module to perform context modeling of the utterance to obtain a new hidden state of the utterance; Step S4, calculating supervised contrastive learning loss and cross entropy loss based on the new utterance hidden state; Step S5, calculating the overall training loss based on the generation loss, the supervised contrastive learning loss and the cross entropy loss, and training the pre-trained model CoG-BART until a trained pre-trained model CoG-BART is obtained and used as a dialogue emotion recognition model; Step S6, inputting the dialogue to be tested containing several sentences into the dialogue emotion recognition model for emotion recognition and outputting emotion labels corresponding to the several sentences in the dialogue to be tested, wherein the context text pairs are generated based on the training dialogue, The step S2 comprises the following sub-steps: Step S2-1, concatenate an utterance in the training dialogue with the corresponding speaker to obtain a concatenated utterance: In the formula, u t is the utterance in the training dialogue, p(u t ) is the discourse u t The speaker of Step S2-2, chunking the concatenated utterances to obtain chunked utterances: In the formula, w t,i Indicates the chunk after chunking, |n t | represents the length of the sequence after the word block in the tth round of discourse, <s> and< / s> Special chunks that indicate the beginning and end of a sentence; Step S2-3, input the chunked utterances into the shared embedding layer and the encoder in sequence to obtain the sentence representation corresponding to the chunked utterances, i.e., the hidden state of the utterances: In the formula, H t represents the sentence representation after embedding, is the encoded sentence representation, Among them, s and d represent the sequence length and hidden state dimension respectively. In step S3, the context modeling includes the following sub-steps: Step S3-1, the encoded sentence is represented as Get the aggregate representation of each utterance through max pooling In the formula, M represents the mask operation on the filled position; Step S3-2, for all conversations in a context, the multi-head attention between each two different utterances is calculated based on the multi-head attention mechanism, and the different features of each utterance are aggregated to obtain the final hidden state representation. The hidden state of each two different utterances is The calculation formula of multi-head attention is: MultiHead(Q,K,V)=[head1;…;head n ]W O In the formula, Q, K, and V represent the hidden states of two different utterances. With three trainable parameter matrices W i Q , W i K , W i V The query vector, key vector, and value vector obtained by multiplication, M represents the mask vector, W O Indicates that each attention head i The parameter vector for aggregating the information in ; Step S3-3, modeling the context dependency based on the dialogue-level Transformer module to obtain the new utterance hidden state: In the formula, Represents a dialog in a window. Indicates a conversation within the window size bs in a conversation. represents the new utterance hidden state after context modeling, In step S3, the auxiliary reply generation task is: The pre-training model CoG-BART is based on the above t Generate the corresponding context u t+1 , The following t+1 The hidden state of each chunk in is output by the decoder sequence: Where BART-Encoder and BART-Decoder represent the encoder and decoder of the pre-trained model CoG-BART, respectively, and the encoder and utterance hidden state H t After encoding, the encoded hidden state is generated The decoder generates the hidden state of the next word block And get the word block u through the Softmax function t+1,j .

2. According to claim 1, a method for dialogue emotion recognition based on supervised contrastive learning and response generation assistance is characterized in that: in, The supervised contrastive learning is: For a batch of N training examples, for each training example in the batch, 2N enhanced training examples are obtained by a predetermined enhancement method, The predetermined enhancement method is to transform the hidden state of an utterance Make a copy and delete its gradient, and we get 3. The method for dialogue emotion recognition based on supervised contrastive learning and response generation assistance according to claim 2, characterized in that: in, The overall training loss is a weighted sum of the cross entropy loss, the supervised contrastive learning loss, and the generation loss, The cross entropy loss is to calculate the distribution of the probability of the emotion label obtained by the new utterance hidden state generated by the context modeling through the multi-layer perceptron: Where P i For statement It represents the conditional probability in the label space obtained after calculation by the Softmax function, and the predicted label with the highest probability is obtained through the argmax function. y i,c Indicates the true emotional label of a certain utterance, represents the probability distribution of the density layer output on category c, is the cross entropy loss, The generation loss for: Where θ is the parameter of the pre-trained model CoG-BART that needs to be optimized, The weighted sum is 1, that is: Where α represents the weight of the supervised contrastive learning loss, β represents the weight of the generation loss, A loss is generated for the said.

Images