A conversation emotion recognition method and system based on dual-channel emotion reasoning

Abstract

The present invention discloses a method and system for dialogue emotion recognition based on dual-channel emotion reasoning, which introduces character feature extraction technology to improve the accuracy and robustness of emotion recognition. The method includes: inputting the dialogue into a label prediction module, using a large language model to predict the emotion label of the target sentence, which is called the predicted label; in the label retrieval module, using any data set containing the emotion label of the dialogue sentence as the retrieval library of the sentence label pair; splicing the target sentence and the predicted label into a sentence label pair, using SBERT to calculate the similarity with the retrieval library, and obtaining the retrieved label pair set; inputting the dialogue into a character feature extraction module, extracting the description text of the character feature; inputting the dialogue into the reasoning emotion chain module to analyze the emotion change of the current scene, and obtaining the reasoning reason of the emotion change; inputting the sentence label pair set obtained in step 3 and the reasoning reason obtained in step 5 into the label generation module, and inferring the emotion label of the target sentence.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a dialogue emotion recognition method and system based on dual-channel emotion reasoning. Background Art

[0002] With the rapid advancement of artificial intelligence technology, the importance of emotion recognition in the field of natural language processing has become increasingly prominent. Emotion recognition aims to analyze and judge the emotional state expressed by an individual during communication. In conversation scenarios, traditional emotion recognition models usually deal with emotion classification tasks, and therefore often fail to effectively capture complex emotion transfer and dynamic changes. Existing methods are not accurate enough in terms of multiple speakers, emotion transfer, and context understanding, and it is difficult to fully capture the deep-level information that can be explained behind the emotion transition.

[0003] Current research shows that technologies based on large language models perform well in language context processing and semantic capture, but still have obvious limitations in identifying emotional transfer in dynamic conversations. At the same time, how to improve the interpretability and generalization ability in emotion recognition remains an unresolved problem. Summary of the invention

[0004] In order to make up for the shortcomings of the prior art, the present invention provides a conversation emotion recognition method and system based on dual-channel emotion reasoning, which takes into account both general knowledge and personalized differences by collaboratively analyzing general scenarios and specific situations. At the same time, it uses the advantages of large language models in semantic understanding and contextual reasoning to build an emotion reasoning chain, deeply analyze the multi-dimensional emotion information of the current scene, and supplement the explainability of emotion transfer. In addition, the character feature extraction technology is introduced to improve the ability to cope with complex emotion transfer scenarios, thereby effectively improving the accuracy and robustness of emotion recognition, making conversation emotion analysis more comprehensive and flexible.

[0005] The technical problem solved by the present invention can be achieved through the following technical solutions:

[0006] On the one hand, a conversation emotion recognition method based on dual-channel emotion reasoning is provided, comprising the following steps:

[0007] Step 1: Input the conversation into the label prediction module and use the large language model (LLM) to predict the sentiment label of the target sentence, which is called the predicted label.

[0008] Step 2: In the label retrieval module, use any dataset containing sentiment labels of dialogue sentences, and the processed dataset is used as the retrieval library of sentence label pairs;

[0009] Step 3: Concatenate the target sentence and the predicted label into a sentence label pair, use SBERT to calculate the similarity with the retrieval library, and obtain the retrieved label pair set;

[0010] Step 4: Input the conversation into the character feature extraction module to extract the description text of the character features;

[0011] Step 5: Input the dialogue into the reasoning emotion chain module to analyze the emotion changes in the current scene and obtain the reasoning reasons for the emotion changes;

[0012] Step 6: Input the sentence label pair set obtained in step 3 and the reasoning reason obtained in step 5 into the label generation module to infer the sentiment label of the target sentence.

[0013] Furthermore, in step 1, the dialogue is input into the label prediction module, and by using a large language model (LLM) and designing prompt words, the target speech is analyzed according to the input dialogue to determine which of the emotion labels is in the set, and the large model is allowed to generate the emotion label of the target sentence. The formula can be expressed as:

[0014] ,

[0015] Prompt = ["You are an expert in emotion analysis", U, L],

[0016] ,

[0017] ,

[0018] in, Represents the predicted sentiment label, For the prompt word, The input dialogue text includes each sentence, and L represents the set of emotion label types of the current training dataset.

[0019] Furthermore, in step 2, only the dialogue sentences and the corresponding emotion labels are retained in the processed data set, and the formula can be expressed as:

[0020] ,

[0021] in, Represents the first dialogue sentences, Indicates The emotion label corresponding to each dialogue utterance.

[0022] Furthermore, in step 3, the SBERT dual-head encoder generates an independent cls embedding for each element in each sentence label pair, sorts all sentence label pairs in the retrieval library using co-sine similarity, and selects the three groups of sentence label pairs with the highest scores as the most relevant sentence label pairs. The formula can be expressed as:

[0023] ,

[0024] ,

[0025] in, Indicates that the target sentence and the predicted label are concatenated into a sentence label pair, Indicates the sentence label pair with the highest similarity.

[0026] Furthermore, in step 3, SBERT is used to calculate the similarity with the retrieval library. In order to ensure that the sentence label pair formed by splicing the target sentence and the predicted emotional label can be highly consistent with the most matching label pair in the retrieval library in terms of emotional semantics, a similarity ranking loss is designed, and the formula is expressed as follows:

[0027] ,

[0028] in, represents the similarity ranking loss function, represents the distance between two sentence label pairs, Represents the sentence sentiment label pair predicted by the model, It indicates the sentence label pair that is judged to be closest to the predicted sentiment label, i.e., it is regarded as a positive sample; Indicates sentence label pairs that are judged to have dissimilar predicted sentiment labels; is a hyperparameter that ensures that the distance between positive samples is at least equal to the distance between negative samples. distance.

[0029] Furthermore, in step 4, a large language model is used to analyze all the words of the speakers in the conversation, describe the specific speaking characteristics, word usage habits and personality characteristics of different speakers, and supplement this information as character characteristics. The formula can be expressed as:

[0030] ,

[0031] prompt = "You are an analysis expert.Based on the speech, analyze theman's speaking characteristics, possible personality traits and so on?(answerwithin 200 words),

[0032] ,

[0033] in, Indicates the speaker's personality traits. Indicates prompt words, Represents the set of all sentences of the current speaker in the conversation.

[0034] Furthermore, the reasoning emotion chain module in step 5 includes the following contents:

[0035] (a) Input the dialogue into the large language model. The prompt word is based on the input dialogue to obtain the emotional information conveyed by the speaker. The formula is expressed as:

[0036] ,

[0037] ,

[0038] prompt1=“What's emotional information did the speaker convey based onconversations”,

[0039] in, It represents the first step of the reasoning emotion chain. Indicates the corresponding prompt word in the first step, Indicates the answer to the first step question;

[0040] (b) Put the answer obtained in (a) at the front of the current prompt word, introduce and concatenate the speaker's character feature description text obtained in step 4, and at the same time, add the emotional information that the target speaker may convey next time. The formula is expressed as:

[0041] ,

[0042] ,

[0043] prompt2=“What's emotional information did the speaker convey based on [pi]”,

[0044] in, Indicates the second step of the reasoning emotion chain, Indicates the corresponding prompt word in the second step, Indicates the answer to the second step question;

[0045] (c) All contents in (a) and (b) are combined as the reasoning reason of the inference emotion chain, and the formula is expressed as:

[0046] ,

[0047] in, Indicates the reasoning reason of the inference emotion chain.

[0048] Furthermore, in step 6, a large language model is used to select the most appropriate reasoning label in the sentence label set according to the reasoning reason, and the formula is expressed as:

[0049] =LLM(“Select the most suitable label from”, Top3_pair, “based on ”,Reasoning),

[0050] ,

[0051] in, represents the inferred sentiment label, Represents the cross entropy loss function of the true sentiment label and the inference label, represents the true label of the target sentence;

[0052] Based on the above loss function, we need to minimize the comprehensive total loss function to improve the accuracy of emotion recognition. The formula can be expressed as:

[0053] ,

[0054] in, and They are loss functions And the loss function The weight of .

[0055] On the other hand, a conversation emotion recognition system based on dual-channel emotion reasoning is provided, which consists of a label prediction channel and an inference emotion channel, including the following modules:

[0056] The label prediction module is used to predict the sentiment label of the target sentence;

[0057] The label retrieval module finds the three most similar sentence label pairs in the processed dataset;

[0058] The character feature extraction module uses a large language model to analyze the character feature description text of the speaker in the conversation;

[0059] The inference emotion chain module uses the designed inference emotion chain to gradually infer the dynamic changes of the speaker's emotions in the conversation and use it as the reasoning reason;

[0060] The label generation module combines the reasoning reason and the most similar sentence label pair to generate the most appropriate sentiment label as the reasoning label.

[0061] Compared with the prior art, the present invention has the following advantages:

[0062] (1) This invention combines the semantic reasoning ability of a large language model, and through a dual-channel structure of a label prediction channel and an inference emotion channel, it uses the combination of label prediction and emotion chain reasoning to make emotion recognition more accurate. This design enables the model to effectively analyze emotion changes in complex dialogue scenarios and improve the accuracy of dialogue emotion recognition.

[0063] (2) The present invention uses the sentence label pair retrieval library and SBERT similarity calculation to ensure high consistency between the target sentence and the most relevant sentiment label through the sentence label pair similarity ranking loss. This optimization enables the model to select the label pair that best matches the sentiment semantics based on similarity, further improving the accuracy of sentiment label matching.

[0064] (3) The present invention analyzes the speaker's expression style, personality characteristics and other information through the character feature extraction module, providing a personalized emotional background for emotion recognition, enabling the model to recognize the specific emotion change trajectory and expression habits of each speaker in the conversation, thereby having a higher emotion recognition ability in multi-role scenarios.

[0065] (4) The present invention uses the reasoning emotion chain to gradually analyze the emotion changes and construct an emotion change path based on the dialogue scenario. By combining character characteristics and the emotion reasoning process, the model can accurately capture the dynamic changes of emotions in the dialogue and improve the ability to understand complex emotional scenes.

[0066] (5) The present invention designs a modular emotion recognition system, including modules such as label prediction, retrieval, character feature extraction, emotion chain reasoning, and label generation, which facilitates flexible expansion and optimization in different task scenarios, making the invention widely adaptable.

[0067] (6) The present invention improves the robustness and robustness of the model in emotion recognition by combining a comprehensive loss function of label prediction loss, similarity ranking loss, and inference emotion chain loss. This comprehensive optimization strategy ensures high accuracy of emotion recognition results and maintains consistent performance under different emotional states. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is a flow chart of the conversation emotion recognition method based on dual-channel emotion reasoning of the present invention;

[0069] Figure 2 This is a schematic diagram of the structure of the label prediction module of the present invention processing and predicting labels;

[0070] Figure 3 This is a schematic diagram of the structure of the tag retrieval module processing and retrieval of the present invention;

[0071] Figure 4A schematic diagram of a character feature extraction module of the present invention analyzing and extracting character features;

[0072] Figure 5 It is a schematic diagram of the framework of the conversation emotion recognition system based on dual-channel emotion reasoning of the present invention. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Example 1

[0074] like Figure 1 As shown, this embodiment provides a conversation emotion recognition method based on dual-channel emotion reasoning, comprising the following steps:

[0075] Step 1: Input the conversation into the label prediction module and use the large language model to predict the sentiment label of the target sentence, which is called the predicted label.

[0076] like Figure 2 As shown, the dialogue is input into the label prediction module. By using the large language model (LLM) and designing prompt words, for example, the prompt word is "Now you are an expert in sentiment analysis", the target speech is analyzed according to the input dialogue. Which one of the sentiment labels is it in the set, let the large model generate the sentiment label of the target sentence. The formula can be expressed as:

[0077] ,

[0078] Prompt = ["You are an expert in emotion analysis", U, L],

[0079] ,

[0080] ,

[0081] in, Represents the predicted sentiment label, For the prompt word, The input dialogue text includes each sentence, and L represents the set of emotion label types of the current training dataset.

[0082] Step 2: In the label retrieval module, use any dataset containing sentiment labels of dialogue sentences, and use the processed dataset as the retrieval library for sentence label pairs.

[0083] like Figure 3The figure is a schematic diagram of the structure of the label retrieval module of the present invention. The processed data set is used as a retrieval library for sentence label pairs, in which only the dialogue sentences and the corresponding emotion labels are retained. The formula can be expressed as:

[0084] ,

[0085] in, Represents the first dialogue sentences, Indicates The emotion label corresponding to each dialogue utterance.

[0086] Step 3: Concatenate the target sentence and the predicted label into a sentence label pair, use SBERT to calculate the similarity with the retrieval library, and obtain the retrieved label pair set.

[0087] In this embodiment, the SBERT dual-head encoder generates an independent cls embedding for each element in each sentence label pair. After sorting all sentence label pairs in the retrieval library using co-sine similarity, the three groups of sentence label pairs with the highest scores are selected as the most relevant sentence label pairs. The formula can be expressed as:

[0088] ,

[0089] ,

[0090] in, Indicates that the target sentence and the predicted label are concatenated into a sentence label pair, Indicates the sentence label pair with the highest similarity.

[0091] In order to ensure that the sentence label pair composed of the target sentence and the predicted emotional label can be highly consistent with the most matching label pair in the retrieval library in terms of emotional semantics, a similarity ranking loss is designed, which is expressed as follows:

[0092] ,

[0093] in, represents the similarity ranking loss function, represents the distance between two sentence label pairs, Represents the sentence sentiment label pair predicted by the model, It indicates the sentence label pair that is judged to be closest to the predicted sentiment label, i.e., it is regarded as a positive sample; Indicates sentence label pairs that are judged to have dissimilar predicted sentiment labels; is a hyperparameter that ensures that the distance between positive samples is at least equal to the distance between negative samples. distance.

[0094] Step 4: Input the conversation into the character feature extraction module to extract the description text of the character features.

[0095] like Figure 4 As shown, it is a schematic diagram of the character feature extraction module of the present invention analyzing and extracting character features. In this embodiment, a large language model is used to analyze all the words of the speakers in the conversation, describe the specific speaking characteristics, word usage habits and personality characteristics of different speakers, and supplement this information as character features. The formula can be expressed as:

[0096] ,

[0097] prompt = "You are an analysis expert.Based on the speech, analyze theman's speaking characteristics, possible personality traits and so on?(answerwithin 200 words),

[0098] ,

[0099] in, Indicates the speaker's personality traits. Indicates prompt words, Represents the set of all sentences of the current speaker in the conversation.

[0100] Step 5: Input the dialogue into the reasoning emotion chain module to analyze the emotional changes in the current scene and obtain the reasoning reasons for the emotional changes.

[0101] In this embodiment, the reasoning emotion chain module includes the following contents:

[0102] (a) Input the conversation into the large language model. The prompt word is based on the input conversation to obtain the emotional information conveyed by the speaker. The formula is expressed as:

[0103] ,

[0104] ,

[0105] prompt1=“What's emotional information did the speaker convey based on conversations”,

[0106] in, It represents the first step of the reasoning emotion chain. Indicates the corresponding prompt word in the first step, Indicates the answer to the first step question;

[0107] (b) Put the answer obtained in (a) at the front of the current prompt word, introduce and concatenate the speaker's character feature description text obtained in step 4, and at the same time, add the emotional information that the target speaker may convey next time. The formula is expressed as:

[0108] ,

[0109] ,

[0110] prompt2=“What's emotional information did the speaker convey based on [pi]”,

[0111] in, Indicates the second step of the reasoning emotion chain, Indicates the corresponding prompt word in the second step, Indicates the answer to the second step question;

[0112] (c) All contents in (a) and (b) are combined as the reasoning reason of the inference emotion chain, and the formula is expressed as:

[0113] ,

[0114] in, Indicates the reasoning reason of the inference emotion chain.

[0115] Step 6: Input the sentence label pair set obtained in step 3 and the reasoning reason obtained in step 5 into the label generation module to infer the sentiment label of the target sentence.

[0116] Using a large language model, the most appropriate reasoning label in the sentence label set is selected based on the reasoning reason. The formula is expressed as:

[0117] =LLM(“Select the most suitable label from”, Top3_pair, “based on ”,Reasoning),

[0118] ,

[0119] in, represents the inferred sentiment label, Represents the cross entropy loss function of the true sentiment label and the inference label, represents the true label of the target sentence;

[0120] Based on the above loss function, we need to minimize the comprehensive total loss function to improve the accuracy of emotion recognition. The formula can be expressed as:

[0121] ,

[0122] in, and They are loss functions And the loss function The weight of . Example 2

[0123] like Figure 5 As shown, this embodiment provides a conversation emotion recognition system based on dual-channel emotion reasoning, which consists of a label prediction channel and an inference emotion channel, including the following modules:

[0124] The label prediction module is used to predict the sentiment label of the target sentence;

[0125] The label retrieval module finds the three most similar sentence label pairs in the processed dataset;

[0126] The character feature extraction module uses a large language model to analyze the character feature description text of the speaker in the conversation;

[0127] The inference emotion chain module uses the designed inference emotion chain to gradually infer the dynamic changes of the speaker's emotions in the conversation and use it as the reasoning reason;

[0128] The label generation module combines the reasoning reason and the most similar sentence label pair to generate the most appropriate sentiment label as the reasoning label.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A conversation emotion recognition method based on dual-channel emotion reasoning, characterized in that: The following steps are involved: Step 1: Input the conversation into the label prediction module and use the large language model to predict the sentiment label of the target sentence, which is called the predicted label; Step 2: In the label retrieval module, use any dataset containing sentiment labels of dialogue sentences, and the processed dataset is used as the retrieval library of sentence label pairs; Step 3: Concatenate the target sentence and the predicted label into a sentence label pair, use SBERT to calculate the similarity with the retrieval library, and obtain the retrieved label pair set; Step 4: Input the conversation into the character feature extraction module to extract the description text of the character features; Use a large language model to analyze all the words of the speaker in the conversation, describe the specific speaking characteristics, word usage habits and personality traits of different speakers, and supplement this information as character features. The formula can be expressed as: p i =LLM(prompt,U i ),prompt="You are an analysis expert.Based on thespeech,analyze the man’s speaking characteristics,possible personality traitsand so on?(answerwithin 200words)”, U i ={u i ,i=1,2,...,n}, Among them, p i Indicates the speaker's personality characteristics, prompt indicates the prompt word, U i Represents the set of all sentences in the conversation by the current speaker; Step 5: Input the dialogue into the inference emotion chain module to analyze the emotion changes in the current scene and obtain the inference reasons for the emotion changes; the inference emotion chain module includes the following contents: (a) Input the dialogue into the large language model. The prompt word is based on the input dialogue to obtain the emotional information conveyed by the speaker. The formula is expressed as: Question1=(U,prompt1), Answer1=LLM(Question1),prompt1="What's emotional information did the speaker conveybased on conversations", Among them, Quetion1 represents the first step of the reasoning emotional chain, prompt1 represents the corresponding prompt word in the first step, and Answer1 represents the answer to the first step; (b) Put the answer obtained in (a) at the front of the current prompt word, introduce and concatenate the speaker's character feature description text obtained in step 4, and at the same time, add the emotional information that the target speaker may convey next time. The formula is expressed as: Question2=(Answer1,p i ,prompt2), Answer2=LLM(Question2),prompt2="What's emotional information did the speaker conveybased on[pi]", Among them, Question2 represents the second step question of the reasoning emotion chain, prompt2 represents the corresponding prompt word in the second step, and Answer2 represents the answer to the second step question; (c) All contents in (a) and (b) are combined as the reasoning reason of the inference emotion chain, and the formula is expressed as: Reasoning=(Question1,Answer1,Question2,Answer2), Among them, Reasoning represents the reasoning reason of the reasoning emotion chain; Step 6: Input the sentence label pair set obtained in step 3 and the reasoning reason obtained in step 5 into the label generation module to infer the sentiment label of the target sentence; use the large language model to select the most appropriate reasoning label in the sentence label set according to the reasoning reason, and the formula is expressed as: in, represents the inferred sentiment label, represents the cross entropy loss function of the true sentiment label and the reasoning label, and L represents the true label of the target sentence; Based on the above loss function, we need to minimize the comprehensive total loss function to improve the accuracy of emotion recognition. The formula can be expressed as: Among them, λ Trople and λ CE They are loss functions And the loss function The weight of .

2. According to claim 1, a method for dialogue emotion recognition based on dual-channel emotion reasoning is characterized in that: In step 1, the dialogue is input into the label prediction module. By using the large language model and designing prompt words, the target speech is analyzed according to the input dialogue to determine which of the emotion label sets it belongs to, and the large model is allowed to generate the emotion label of the target sentence. The formula can be expressed as: Prompt=["You are an expert in emotion analysis",U,L], U=[u1,u2,...,u n ], L＝[l1,l2,...,l n ]， in, Represents the predicted emotional label, Prompt is the prompt word, U is the input dialogue text including each sentence, and L represents the emotional label type set of the current training dataset.

3. According to the method of conversation emotion recognition based on dual-channel emotion reasoning according to claim 1, it is characterized in that: In step 2, only the conversation sentences and the corresponding emotion labels are retained in the processed data set, and the formula can be expressed as: Database={[s i ,dl i ],i=1,2,...,n}, Among them, s i represents the i-th dialogue sentence in the dataset, dl i Represents the sentiment label corresponding to the i-th dialogue utterance.

4. According to the method of conversation emotion recognition based on dual-channel emotion reasoning according to claim 1, it is characterized in that: In step 3, the SBERT dual-head encoder generates an independent cls embedding for each element in each sentence label pair. After sorting all sentence label pairs in the retrieval library using co-sine similarity, the three groups of sentence label pairs with the highest scores are selected as the most relevant sentence label pairs. The formula can be expressed as: Among them, sl_pair represents the target sentence and the predicted label concatenated into a sentence label pair, and Top3_pair represents the sentence label pair with the highest similarity.

5. According to claim 4, a method for dialogue emotion recognition based on dual-channel emotion reasoning is characterized in that: In step 3, a similarity ranking loss is designed, and the formula is expressed as: in, represents the similarity ranking loss function, d represents the distance between two sentence label pairs, Represents the sentence sentiment label pair predicted by the model, (s + ,dl + ) indicates the sentence label pair that is judged to be closest to the predicted sentiment label, i.e., the positive sample; (s - ,dl - ) represents the sentence label pairs that are judged to be dissimilar in predicting sentiment labels; α is a hyperparameter that ensures that the distance between positive samples is smaller than the distance between negative samples by a minimum distance of α.

6. A conversation emotion recognition system based on dual-channel emotion reasoning, executing the conversation emotion recognition method based on dual-channel emotion reasoning according to claim 1, characterized in that: It consists of a label prediction channel and an inference sentiment channel, including the following modules: The label prediction module is used to predict the sentiment label of the target sentence; The label retrieval module finds the three most similar sentence label pairs in the processed dataset; The character feature extraction module uses a large language model to analyze the character feature description text of the speaker in the conversation; The inference emotion chain module uses the designed inference emotion chain to gradually infer the dynamic changes of the speaker's emotions in the conversation and use it as the reasoning reason; The label generation module combines the reasoning reason and the most similar sentence label pair to generate the most appropriate sentiment label as the reasoning label.

Images