A dialogue emotion recognition method combining personality cues and emotional uncertainty

Abstract

The application provides a dialogue emotion recognition method combining personality clues and emotional uncertainty, and belongs to the technical field of natural language processing; solves the problem of incomplete speech representation in current text dialogue emotion recognition; includes the following steps: constructing a speech-level background representation; constructing a dialogue-level background representation based on the speech-level background representation; obtaining a personality representation vector of each speech in the dialogue, and obtaining a personality feature vector of each speaker by using a statistical model; obtaining a personality perception vector; converting the dialogue-level background representation into a multivariate Gaussian distribution by using a probability distribution encoder; sampling a text timing background vector from the multivariate Gaussian distribution; converting the text timing background vector and the personality perception vector into the same concept space, and then using splicing to combine the text timing background vector and the personality perception vector into a new vector; and using an emotion classifier to predict the emotion category of each speech; and the application is applied to dialogue emotion recognition.

Description

Technical Field

[0001] This application relates to the field of fine-grained text sentiment analysis in natural language processing, and in particular to a dialogue sentiment recognition system and method that combines personality cues and emotional uncertainty. Background Technology

[0002] Emotion recognition aims to identify the emotional state of each utterance by fusing information from multiple sources. With the increasing number of conversations on social media platforms, conversational emotion recognition has garnered growing attention from researchers. As an extension of traditional emotion recognition, conversational emotion recognition aims to identify the emotional state of all constituent utterances in a conversation.

[0003] Emotional dynamics are one of the most challenging problems in dialogue emotion recognition. Emotional dynamics in dialogue refers to the emotional impact of interactions between speakers. Traditional methods use recurrent neural networks or graph neural networks to capture intra-speaker and inter-speaker dependencies. However, these methods still have limitations in uncovering deeper and richer emotional dynamic cues. This may be because they neglect speaker personality traits. Speaker personality traits include the speaker's personality characteristics in the current utterance and their personality characteristics in the global dialogue. Furthermore, subtle emotions and subjective emotion labeling during emotion perception and evaluation lead to uncertainty in the emotion itself. Although current deep learning-based text dialogue emotion recognition methods attempt to detect the potential paradigm of ambiguity caused by subjectivity, they rarely consider the uncertainty arising from the ambiguity of emotional expression. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, this application proposes a dialogue emotion recognition method that combines personality cues and emotional uncertainty.

[0005] The technical solution adopted in this application is: a dialogue emotion recognition method that combines personality cues and emotional uncertainty, including the following steps:

[0006] Step 1: Fine-tune the pre-trained model on the dialogue emotion recognition dataset, and then use the fine-tuned pre-trained model to achieve discourse-level context representation;

[0007] Step 2: Based on the above discourse-level context representation, construct a dialogue-level context representation using a bidirectional gated loop unit;

[0008] Step 3: Train the personality detection model on the ChaLearn First Impression dataset, then use the trained personality detection model to predict on the dialogue emotion recognition dataset to obtain the personality representation vector of each utterance in the dialogue, and finally use a statistical model to obtain the personality feature vector of each speaker.

[0009] Step 4: Construct the interaction relationship between the same speaker and different speakers by using the speaker's personality representation and the background representation of the current discourse. Then, transform the two into weights in the same conceptual space and combine them with the speaker's global dialogue representation to obtain the background representation of personality perception, i.e., the personality perception vector.

[0010] Step 5: Use a probability distribution encoder to transform the dialogue-level background representation into a multivariate Gaussian distribution to achieve emotional uncertainty modeling;

[0011] Step 6: Sample the text temporal background vector from the above multivariate Gaussian distribution;

[0012] Step 7: Transform the text temporal background vector and the personality perception vector into the same concept space, and then use the concatenation operation to combine the text temporal background vector and the personality perception vector into a new vector;

[0013] Step 8: Use an emotion classifier to predict the emotion category of each utterance;

[0014] Step 9: Construct the corresponding cross-loss function to train the model.

[0015] Furthermore, step 1 employs a widely used pre-trained language model to obtain discourse-level context representations, i.e., discourse-level feature vectors, specifically including:

[0016] First, in each word u i A special symbol [CLS] is appended to the beginning position to obtain the sequence;

[0017] Then, the pre-trained RoBERTa model was fine-tuned using the above sequence and utterance-level emotion classification tasks, and emotion classification was achieved by passing the special symbol [CLS] of the last layer to the pooling layer;

[0018] Finally, a finely tuned pre-trained RoBERTa model is used to obtain the discourse-level feature vector c. i .

[0019] Furthermore, in step 3, the Big Five model is used to label personality data samples on the ChaLearn First Impression dataset. Each sample is labeled with a 5-dimensional personality label vector, which includes five dimensions: openness, conscientiousness, extraversion, agreeableness, and neuroticism. The value of each dimension is between [0,1].

[0020] Furthermore, the statistical model used in step 3 is the mean, standard deviation, median, maximum or minimum value.

[0021] Furthermore, step 5 specifically includes:

[0022] Step 5.1: Represent the dialogue-level background as h i As input features, they are fed into the feedforward neural network to obtain feature-level interactions;

[0023]

[0024] Where FFN() is a feedforward neural network;

[0025] Step 5.2: Obtain the feature vector The data is divided into two parts and fed into a multi-head attention network layer. The mean and variance of the estimated multivariate Gaussian distribution are obtained for each head. The final mean and variance vectors of the multivariate Gaussian distribution are then obtained by concatenating the output vectors of all heads.

[0026] Furthermore, in step 6, the Markov chain Monte Carlo method is used to sample the text temporal background vector from the multivariate Gaussian distribution.

[0027] Furthermore, in step 7, a feedforward neural network is used to transform the text temporal background vector and the personality perception vector into the same conceptual space.

[0028] A computer device includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method.

[0029] A computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the method.

[0030] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the method.

[0031] The advantages of this application over the prior art are as follows:

[0032] 1. To address the problem that current dialogue emotion recognition technology ignores the speaker's personality state in the current discourse and the global discourse, we propose an enhanced background representation based on speaker personality cue retrieval.

[0033] 2. To address the issue of incomplete discourse representation in text-based emotion recognition, this paper proposes a method that combines personality cues and emotional uncertainty to construct a comprehensive background representation of discourse, thereby enabling emotion recognition of dialogue text and helping machines to more accurately identify human emotions.

[0034] This application proposes an efficient speaker personality detection method that can identify the speaker's personality characteristics in the current discourse and in the global discourse. Furthermore, by constructing a probability distribution encoder and Markov chain Monte Carlo sampling method, it obtains the emotional uncertainty feature representation of the discourse, thereby constructing a dialogue emotion recognition method that combines personality cues and emotional uncertainty, which can help machines more accurately identify the emotions in the discourse. Attached Figure Description

[0035] The following description, in conjunction with the accompanying drawings, further illustrates this application:

[0036] Figure 1 A flowchart illustrating a dialogue emotion recognition method that combines personality cues and emotional uncertainty, provided as an embodiment of this application;

[0037] Figure 2 This is a structural diagram of the probability distribution encoder provided in an embodiment of this application. Detailed Implementation

[0038] like Figure 1 and 2 As shown, current methods for dialogue emotion recognition neglect the speaker's personality state in the current utterance and the overall utterance, information that can provide crucial clues for dialogue emotion recognition tasks. This application employs speaker personality cue retrieval to enhance the dialogue context representation. The ambiguity of emotion expression in dialogue text introduces uncertainty; therefore, this application uses a probability distribution encoder to construct an emotion distribution in a continuous latent space, and employs a Markov Monte Carlo method to extract the emotion uncertainty representation from this probability distribution. Finally, the two representations are combined to achieve emotion label classification. Figure 1 The corresponding framework diagram is shown.

[0039] The dialogue emotion recognition method combining personality cues and emotional uncertainty provided in this application captures rich cues of emotional dynamics in the dialogue and achieves fuzzy modeling of emotions by employing a personality cue retrieval module and an emotion distribution estimation module, respectively. Specifically, it includes the following steps:

[0040] Step 1: Fine-tune the pre-trained model on the dialogue emotion recognition dataset, and then use the fine-tuned pre-trained model to realize the discourse-level background representation, i.e., discourse-level feature vector.

[0041] Step 2: Based on the above discourse-level context representation, construct the dialogue-level context representation using a bidirectional gated loop unit.

[0042] Step 3: Train the personality detection model on the ChaLearn First Impression dataset, then use the trained personality detection model to make predictions on the dialogue emotion recognition dataset to obtain the personality representation vector of each utterance in the dialogue. Finally, use a statistical model to obtain the personality feature vector of each speaker.

[0043] Step 4: Construct the interaction relationship between the same speaker and different speakers by using the speaker's personality representation and the background representation of the current discourse. Then, transform the two into weights in the same conceptual space and combine them with the speaker's global dialogue representation to obtain the background representation of personality perception, i.e., the personality perception vector.

[0044] Step 5: Use a probability distribution encoder to transform the dialogue-level background representation into a multivariate Gaussian distribution to achieve emotional uncertainty modeling.

[0045] Step 6: Use the Markov chain Monte Carlo method to sample the text temporal background vector from the above multivariate Gaussian distribution.

[0046] Step 7: Use a feedforward neural network to transform the text temporal background vector and personality perception vector into the same concept space, and then use a concatenation operation to combine the two into a new vector.

[0047] Step 8: Use an emotion classifier to predict the emotion category of each utterance.

[0048] Step 9: Construct the corresponding cross-loss function to train the model.

[0049] Thus far, from step 1 to step 9, the dialogue emotion recognition combining personality cues and emotional uncertainty has been completed.

[0050] Step 1 uses a widely used pre-trained language model to obtain utterance-level feature vectors, as follows:

[0051] First, in each word u i =(w1,w2,…,w n By concatenating a special symbol [CLS] at the beginning of the sequence {[CLS], w1, w2, ..., w}, we obtain the sequence {[CLS], w1, w2, ..., w}. n}, where w n Words in a sentence or phrase.

[0052] Then, the pre-trained RoBERTa model was fine-tuned using the sequence and utterance-level emotion classification task, where emotion classification was achieved by passing the symbol [CLS] of the last layer to the pooling layer.

[0053] Finally, a finely tuned pre-trained RoBERTa model is used to obtain the discourse-level feature vector c. ic i =RoBERTa([CLS],w1,w2,…,w n )

[0054] Step 2 specifically employs a bidirectional gated recurrent unit (GRU) to construct the dialogue-level context representation h. i This allows us to capture the dependencies between adjacent utterances, namely:

[0055]

[0056] Step 3 uses a personality detection model and the utterance set of each speaker in the dialogue to construct a speaker personality representation, including the speaker personality feature vector p of the current utterance. i And the personality trait vectors s of each speaker j .

[0057] The specific implementation process of step 3 includes:

[0058] Step 3.1: The Big Five model is used to label personality data samples on the ChaLearn First Impression dataset. Each sample is labeled with a 5-dimensional personality label vector, which includes five dimensions: openness, conscientiousness, extraversion, agreeableness, and neuroticism. The value of each dimension is between [0,1].

[0059] Step 3.2: Train the personality detection model on the ChaLearn First Impression dataset. Based on the performance on the validation set, select the best personality detection model as the pre-trained personality detection model in this application embodiment.

[0060] Step 3.3: Use a pre-trained personality detection model to obtain a 5-dimensional personality feature score vector p for each utterance. i ,

[0061] p i =Person(u i );

[0062] Person() is a pre-trained personality detection model.

[0063] Step 3.4: Using statistical models, such as mean, standard deviation, median, maximum, and minimum values, aggregate all discourse personality trait score vectors for each speaker to obtain a 25-dimensional personality trait vector s. i .

[0064] P i ={p j ·mask j} j∈N ;

[0065]

[0066] v = statModel(P i );

[0067] s i =concat({v});

[0068] Among them, P i For the current discourse u i The corresponding global personality feature vector of the speaker, p j For the words in the dialogue u j The corresponding speaker personality feature vector, where N is the number of utterances in the dialogue, and S... φ() As a speaker symbol, φ() can map the index of a utterance to the corresponding speaker, h i and h j They are respectively the words u i and u j The background representation is that mask is a masking operation, and statModel() can be the mean, standard deviation, median, maximum, and minimum values.

[0069] Step 4 uses a personality cue retrieval module to obtain rich background representations. The specific implementation steps are as follows:

[0070] Step 4.1: Construct the interaction relationship between the utterances of the same speaker by combining personality traits, and assign a corresponding weight to each utterance, i.e.

[0071]

[0072] Where α ij The words of the same speaker i and words u j The weights between them, f q (x), f k (x), f e (x) are all linear transformation functions, d h This represents the vector dimension after the transformation. Ensure the target discourse's dialogue-level contextual representation h i It can obtain changes in the internal personality cues of the same speaker and can guarantee the retrieval of historical personality cues, including:

[0073]

[0074] Where φ() maps the index of a utterance to the corresponding speaker, i is the index of the current utterance, and j is the index of other utterances of the speaker corresponding to the current utterance.

[0075] Step 4.2: Construct the interaction relationship between the utterances of different speakers by combining personality traits, and assign a corresponding weight to each utterance, i.e.

[0076]

[0077] Where β ij For different speakers' discourses i and words u j The weights between them Ensure the target discourse's dialogue-level contextual representation h i It can detect changes in the internal personality cues of different speakers;

[0078]

[0079] Where φ() maps the index of a utterance to the corresponding speaker, i is the index of the current utterance, and j is the index of other utterances of the speaker corresponding to the current utterance.

[0080] Step 4.3: Unify the discourse relationships between the same speaker and between different speakers to the same scale to obtain the weight value γ between discourses. ij for:

[0081] γ ij =softmax(α) ij +β ij ).

[0082] Step 4.4: Integrate all discourse representations related to the target discourse with the corresponding speaker personality representations to obtain a rich background representation of personality perception, i.e., a personality perception vector.

[0083]

[0084] In step 5, to model emotional uncertainty, a probability distribution encoder is used to represent the dialogue-level context h. i This is transformed into a multivariate Gaussian distribution. Therefore, the dialogue-level context representation h is... i As the input feature vector, the output parameters (mean, variance) of the multivariate normal distribution are defined as follows: The specific implementation process of step 5 is as follows:

[0085] Step 5.1: Represent the dialogue-level background as h i As input features, they are fed into the feedforward neural network to obtain feature-level interactions;

[0086]

[0087] FFN() is a feedforward neural network.

[0088] Step 5.2: Obtain the feature vector The data is divided into two parts and fed into a multi-head attention network layer. The mean and variance of the estimated multivariate Gaussian distribution are obtained for each head. The final mean and variance vectors of the multivariate Gaussian distribution are then obtained by concatenating the output vectors of all heads.

[0089]

[0090] μ=concat(MultAttention(h′ i ));

[0091] σ=concat(MultAttention(h″ i ));

[0092] Here, `split()` is the segmentation operation, `MultAttention()` is the multi-head attention operation, `concat()` is the concatenation operation, and `h′` is the concatenation operation. i For the first intermediate hidden variable, h″ i σ is the second intermediate hidden variable, μ is the mean vector, and σ is the variance vector.

[0093] In step 6, the Markov chain Monte Carlo method is used to sample the text temporal background vector from the Gaussian distribution mentioned above. Formalized as follows:

[0094]

[0095] in Let be the dimensional space, and d be the feature dimension. Sampling based on probability distribution can minimize intra-class imbalance and resolve the bias of inter-class imbalanced distributions.

[0096] In step 7, a feedforward neural network is used to combine the text temporal background vector X and the personality perception vector. Transform them into the same conceptual space, then merge the two together. The merged result is:

[0097]

[0098] Where h f For fusion representation, FFN() is a feedforward neural network, and concat() is a concatenation operation.

[0099] In step 8, the above-described fusion representation h is used. f As input, a sentiment classifier is used to predict the sentiment category of each utterance:

[0100]

[0101] in Here, W represents the model's predicted value, W represents the training weights, and b represents the bias.

[0102] The expression for the crossover loss function L in step 9 is as follows:

[0103]

[0104] Where N is the number of utterances in the dialogue, and E is the number of emotion categories. It is the true emotional label of the words.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A dialogue emotion recognition method combining personality cues and emotional uncertainty, characterized in that: Includes the following steps: Step 1: Fine-tune the pre-trained model on the dialogue emotion recognition dataset, and then use the fine-tuned pre-trained model to achieve discourse-level context representation; Step 2: Based on the above discourse-level context representation, construct a dialogue-level context representation using a bidirectional gated loop unit; Step 3: Train the personality detection model on the ChaLearn First Impression dataset, then use the trained personality detection model to predict on the dialogue emotion recognition dataset to obtain the personality representation vector of each utterance in the dialogue, and finally use a statistical model to obtain the personality feature vector of each speaker. Step 4: Construct the interaction relationship between the same speaker and different speakers by using the speaker's personality representation and the background representation of the current discourse. Then, transform the two into weights in the same conceptual space and combine them with the speaker's global dialogue representation to obtain the background representation of personality perception, i.e., the personality perception vector, which specifically includes: Step 4.1: Construct the interaction relationship between the utterances of the same speaker by combining personality traits, and assign a corresponding weight to each utterance, i.e. ； in Discourses of the same speaker and words The weights between them , , They are all linear transformation functions. and Distinguished by words and Background information, For the words in the dialogue The corresponding speaker personality trait vector, The dimension of the transformed vector is represented by the masking operation. Ensure the target discourse is presented in a conversational context. It can obtain changes in the internal personality cues of the same speaker and can guarantee the retrieval of historical personality cues, including: ； in () can map the index of a utterance to the corresponding speaker. For the index of the current discourse, An index of other utterances by the speaker corresponding to the current utterance; Step 4.2: Construct the interaction relationship between the utterances of different speakers by combining personality traits, and assign a corresponding weight to each utterance, i.e. ； in Discourses of different speakers and words The weights between them Speaker symbol, Ensure the target discourse is presented in a conversational context. It can detect changes in the internal personality cues of different speakers; ； in () can map the index of a utterance to the corresponding speaker. For the index of the current discourse, An index of other utterances by the speaker corresponding to the current utterance; Step 4.3: Unify the discourse relationships between the same speaker and between different speakers to the same scale to obtain the weight values ​​between discourses. for: ； Step 4.4: Integrate all discourse representations related to the target discourse with the corresponding speaker personality representations to obtain a rich background representation of personality perception, i.e., a personality perception vector. , ； Step 5: Use a probability distribution encoder to transform the dialogue-level background representation into a multivariate Gaussian distribution to achieve emotional uncertainty modeling; Step 6: Sample the text temporal background vector from the above multivariate Gaussian distribution; Step 7: Transform the text temporal background vector and the personality perception vector into the same concept space, and then use the concatenation operation to combine the text temporal background vector and the personality perception vector into a new vector; Step 8: Use an emotion classifier to predict the emotion category of each utterance; Step 9: Construct the corresponding cross-loss function to train the model.

2. The dialogue emotion recognition method combining personality cues and emotional uncertainty according to claim 1, characterized in that: Step 1 uses a widely used pre-trained language model to obtain discourse-level context representations, i.e., discourse-level feature vectors, which specifically include: First, in every word A special symbol [CLS] is appended to the beginning position to obtain the sequence; Then, the pre-trained RoBERTa model was fine-tuned using the above sequence and utterance-level emotion classification tasks, and emotion classification was achieved by passing the special symbol [CLS] of the last layer to the pooling layer; Finally, a finely tuned pre-trained RoBERTa model is used to obtain discourse-level feature vectors. .

3. The dialogue emotion recognition method combining personality cues and emotional uncertainty according to claim 1, characterized in that: In step 3, the Big Five model is used to label personality data samples on the ChaLearn First Impression dataset. Each sample is labeled with a 5-dimensional personality label vector, which includes five dimensions: openness, conscientiousness, extraversion, agreeableness, and neuroticism. The value of each dimension is between [0,1].

4. The dialogue emotion recognition method combining personality cues and emotional uncertainty according to claim 3, characterized in that: The statistical model used in step 3 is the mean, standard deviation, median, maximum or minimum value.

5. The dialogue emotion recognition method combining personality cues and emotional uncertainty according to claim 1, characterized in that: Step 5 specifically includes: Step 5.1: Represent the dialogue-level background As input features, they are fed into the feedforward neural network to obtain feature-level interactions; ； Where FFN() is a feedforward neural network; Step 5.2: Obtain the feature vector The data is divided into two parts and fed into a multi-head attention network layer. The mean and variance of the estimated multivariate Gaussian distribution are obtained for each head. The final mean and variance vectors of the multivariate Gaussian distribution are then obtained by concatenating the output vectors of all heads.

6. The dialogue emotion recognition method combining personality cues and emotional uncertainty according to claim 1, characterized in that: In step 6, the Markov chain Monte Carlo method is used to sample the text temporal background vector from the multivariate Gaussian distribution.

7. The dialogue emotion recognition method combining personality cues and emotional uncertainty according to claim 1, characterized in that: In step 7, a feedforward neural network is used to transform the text temporal background vector and personality perception vector into the same conceptual space.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that: When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1-7.

10. A computer program product comprising a computer program / instructions, characterized in that: When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1-7.

Images