Multi-source knowledge fused emotion support dialogue reply generation model and method

Abstract

An emotion support dialogue reply generation model and method fusing multi-source knowledge belongs to the field of natural language processing, and comprises the following steps: supplementing user external situation cognitive knowledge in combination with common sense knowledge, and supplementing user internal emotional state cognitive knowledge in combination with psychological health dialogue sample knowledge; carrying out mutual constraint modeling for a multi-source cognitive knowledge and emotion support reply strategy; modeling fine-grained transition information of each dialogue turn level based on the coding context and common knowledge; and by modifying a pre-training language model Blenderbottom decoder, performing cross attention calculation on the context hidden state and multiple factors to generate a final emotion support reply. According to the method, the knowledge selection ability, the strategy prediction ability and the emotion cognitive ability of the model are jointly improved, fine-grained emotion understanding is realized, the dialogue model is assisted to deeply understand the external situation and the internal emotion state of the user, and the deficiency of the emotion support dialogue model in the emotion cognitive ability is filled up.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, specifically relating to an emotion-supported dialogue response generation model and method that integrates multi-source knowledge. Background Technology

[0002] In the fields of artificial intelligence and natural language processing, emotion-supporting dialogue is a key technology for dialogue response generation. In multi-turn human-computer dialogue, the model automatically generates responses based on the user's statements and maintains the dialogue. Emotion-supporting dialogue requires that, as the dialogue progresses, the model effectively reduces the user's negative emotions, alleviates their emotional distress, and provides emotional support. This technology is widely used in daily life, including in intelligent customer service, chatbots, and mobile voice assistants, and has significant application value.

[0003] Emotional support dialogue tasks have received widespread attention from scholars. Professor Huang Minlie and others from the Interactive Artificial Intelligence (CoAI) research group at Tsinghua University proposed that the research direction of emotional intelligent dialogue systems can be divided into the following three tasks: Emotional Chatting, i.e., the ability to generate responses to accurately express (specified or unspecified) emotions; Empathetic Responding, i.e., the ability to understand users' feelings and respond accordingly; and Emotional Support, i.e., understanding users' experiences and feelings, exploring users' problems, reducing users' emotional distress, and helping people overcome difficulties. (Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021. Towards Emotional Support Dialog Systems. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3469–3483, Online. Association for Computational...) Linguistics.

[0004] Unlike emotional dialogue and empathic dialogue tasks, emotional support dialogue is a multi-turn dialogue task that requires consideration of the user's feedback and changes in emotional state during multiple rounds of interaction. It aims to reduce the user's emotional distress through specific emotional support response strategies (such as asking questions, restating the user's situation, comforting, and providing advice).

[0005] Professor Shen Xiangyang and others from Tsinghua University pointed out that empathetic social chatbots need to be able to identify users' emotions from conversations, detect how emotions evolve over time, and understand users' emotional needs. This requires query understanding, user profiling, emotion detection, emotion recognition, and dynamic tracking of users' emotions in conversations. (Li Zhou, Jianfeng Gao, Di Li, and Heung-Yeung Shum. 2020. The Design and Implementation of XiaoIce, an Empathetic Social Chatbot. Computational Linguistics, 46(1):53–93.) The Qin Bing group of the Social Computing and Information Retrieval Laboratory at Harbin Institute of Technology proposed that emotional dialogue technology focuses on the "emotional intelligence" of chatbots, aiming to enable chatbots to observe, understand, and express emotions like humans, taking into account both the "intelligence" and "emotional intelligence" of chatbots, thereby achieving spiritual companionship, emotional comfort, and psychological guidance for users. (Zhao Yanyan, Lu Xin, Zhao Weixiang, Tian Yijian, Qin Bing. A Review of Emotional Dialogue Technology. Journal of Software, ,():1-26.) Professor Wai Lam's group at the Chinese University of Hong Kong points out that emotional support is crucial for addressing widespread emotional distress and the increasing risk of mental illness. Researchers have proposed various Emotional Support Dialogue (ESC) systems to provide timely and convenient emotional support to those seeking help, including mental health support, counseling, or motivational interviews. (Yang Deng, Wenxuan Zhang, Yifei Yuan, and Wai Lam. 2023. Knowledge-enhanced Mixed-initiative Dialogue System for Emotional Support Conversations. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4079–4095, Toronto, Canada. Association for Computational Linguistics.)

[0006] Enhancing the emotional capabilities of dialogue models is crucial for helping them deeply understand the meaning of user conversations and building a bridge between artificial intelligence and human communication. Emotional intelligence is a significant area of ​​artificial intelligence, and emotion-supported dialogue is a vital component of natural language processing in achieving emotional intelligence. The research goal of emotion-supported dialogue is to enable AI algorithms to understand emotions and user situations, helping users alleviate stress and reduce depressive moods. Emotion-supported dialogue is a key technology in many dialogue generation scenarios, including intelligent customer service, mental health support, and companion chatbots, and has broad application value. Beyond specific applications in vertical fields such as mental health support, it is also an essential path to the vast future of artificial intelligence, making it of significant research value.

[0007] Currently, existing research on emotional support dialogue includes:

[0008] (1) Liu et al. (2021) proposed the emotional support dialogue task and released the benchmark dataset ESCNV, which is the open-source dataset on which this technology is based.

[0009] (2) The MISC model employs multiple emotion support strategies to guide response generation and assist in the smooth transition of conversation topics. At the same time, it utilizes the common sense reasoning model COMET to supplement the emotional knowledge in the conversation. (Quan Tu, Yanran Li, Jianwei Cui, Bin Wang, Ji-Rong Wen, and Rui Yan. 2022. MISC: A Mixed Strategy-Aware Model integrating COMET for Emotional Support Conversation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 308–319, Dublin, Ireland. Association for Computational Linguistics.)

[0010] (3) The TransESC model models the state transitions in each round of dialogue from three perspectives: semantic transition, policy transition, and sentiment transition, thereby maintaining a smooth transition between dialogues. (Weixiang Zhao, Yanyan Zhao, Shilong Wang, and Bing Qin. 2023. TransESC: Smoothing Emotional Support Conversation via Turn-Level State Transition. In Findings of the Association for Computational Linguistics: ACL 2023, pages 6725–6739, Toronto, Canada. Association for Computational Linguistics.)

[0011] (4) The KEMI model, combined with the knowledge graph HEAL, studied a hybrid proactive emotional support dialogue model, enabling both users and the system to actively lead the dialogue. (Yang Deng, Wenxuan Zhang, Yifei Yuan, and Wai Lam. 2023. Knowledge-enhanced Mixed-initiative Dialogue System for Emotional Support Conversations. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4079–4095, Toronto, Canada. Association for Computational Linguistics.)

[0012] Existing research on emotional support dialogue still faces the following technical challenges:

[0013] (1) Most existing studies do not use external knowledge, or only use common sense knowledge (ConceptNet or COMET) to improve the cognitive ability of the model: The scope and methods of using external knowledge in existing studies are limited, and they cannot improve the emotional cognitive ability of the model from multiple dimensions and aspects for the emotion support dialogue generation task; in addition, common sense knowledge is too concise and one-sided, often in the form of phrases, lacking the background semantic information of external knowledge, and external knowledge contains noise that is not related to the dialogue context. Existing studies have ignored the filtering of noise in external knowledge.

[0014] (2) Existing research lacks an exploration of the relationship between external knowledge and emotional support strategies: External knowledge is usually used as an independent module or method to improve the performance of the model, which leads to an overly one-sided analysis of knowledge and strategies and an inability to fully capture the influencing factors of the model's emotional cognitive ability.

[0015] (3) Existing research neglects the transitional information of user emotional states during dialogue: Most existing studies treat the dialogue history as a long sequence and input it into the model as a whole. This approach has certain limitations. Because the model can usually only perform sentiment analysis on the last user dialogue, it ignores the transitional information of the user's emotional state during the dialogue. However, dialogue is a dynamic process, and the user's emotional state may continuously evolve as the dialogue progresses. Therefore, this will lead to the model's understanding of user emotions being neither comprehensive nor accurate. Summary of the Invention

[0016] To address the shortcomings of existing research and comprehensively enhance the model's emotional cognition capabilities, this invention provides an emotion-supported dialogue response generation model and method that integrates multi-source knowledge.

[0017] The main objectives of this invention are as follows:

[0018] (1) The purpose of this invention is to integrate multi-source knowledge to supplement rich background semantic information.

[0019] Based on the common knowledge COMET, this invention uses the psychological health dialogue example knowledge HEAL to supplement the background semantic information of external knowledge, and calculates the emotional intensity based on the emotion dictionary NRC_VAD to filter knowledge noise, thereby making more scientific and reasonable use of external knowledge.

[0020] (2) The purpose of this invention is to comprehensively model the relationship between knowledge and strategy.

[0021] This invention explores the mutual constraints between emotional cognitive knowledge and emotional support strategies on the generation of emotional support responses. It achieves comprehensive perception of multiple factors, including dialogue context, external emotional cognitive knowledge, and emotional support strategies, allowing these elements to jointly participate in guiding the generation of emotional support responses, thereby further improving the model's emotional cognitive ability.

[0022] (3) The purpose of this invention is to model the user's emotional state transition information.

[0023] To better capture user emotions, it is necessary to incorporate the emotional changes in the dialogue history into the model's considerations, so as to gain a more comprehensive understanding of the user's emotional state and thus supplement the emotional information related to the dialogue context.

[0024] The technical solution adopted by this invention to solve the technical problem is as follows:

[0025] This invention provides a method for generating emotion-supported dialogue responses that integrates multi-source knowledge, mainly comprising the following steps:

[0026] Step 1: Multi-source cognitive knowledge fusion mechanism;

[0027] By combining common sense knowledge (COMET) to supplement users' knowledge of their external situation, and by combining knowledge of mental health dialogue examples (HEAL) to supplement users' knowledge of their internal emotional state, the combined effect of the two types of knowledge on the generation of emotional support responses is analyzed.

[0028] Step 2: Modeling the mutual constraints between multi-source cognitive knowledge and emotional support response strategies;

[0029] Step 3: Modeling the transition of user emotional state;

[0030] Based on the encoding context and common-sense knowledge COMET, fine-grained transition information at the level of each dialogue round is modeled, including semantic and emotional transitions, to achieve smooth and natural transitions between dialogues that support emotional support.

[0031] Step 4: Multi-factor perception decoder;

[0032] By modifying the Blenderbot decoder of the pre-trained language model, the contextual hidden state is cross-attentioned with multiple factors. The final emotional support response is generated through interaction with multiple factors, so as to achieve comprehensive perception that includes dialogue context, user emotion, emotional support strategy and cognitive knowledge factors.

[0033] Furthermore, in step one, the context sequence {[CLS],u1,…,[CLS],u n The input is a context encoder, which encodes the hidden context state to obtain the corresponding context statement. i The encoded vector, where u i [CLS] represents the i-th round of dialogue text in the dialogue context, where i = 1, ..., n, n represents the total number of rounds of dialogue, and [CLS] represents the dialogue text u. i The starting marker is used, and the context sequence represents all dialogues from the beginning to the last round in chronological order. The COMET knowledge text and HEAL knowledge text are added to the same context sequence and then input into the knowledge encoder. The encoded context hidden state with external knowledge is used to select knowledge based on the calculated sentiment intensity value. The sentiment intensity value of the entire sentence is obtained by summing the sentiment intensity values ​​of each word in the sentence, and the context hidden state H with the highest sentiment intensity value with external knowledge is selected.

[0034] Furthermore, the Knowledge Encoder and the Context Encoder share all parameters.

[0035] Furthermore, the external knowledge NRC-VAD sentiment lexicon is used to calculate the sentiment intensity value of multi-source cognitive knowledge. The external knowledge NRC-VAD sentiment lexicon contains VAD three-dimensional vectors of 20,000 English words.

[0036] The formula for calculating the sentiment intensity value for word x is as follows:

[0037]

[0038] Where min-max() represents minimum-maximum normalization; ||·||2 represents the 2-norm; Va(x) and Ar(x) represent the values ​​of the Valence dimension and Arousal dimension in the VAD 3D vector of word x, respectively; if word x is not in the NRC-VAD sentiment lexicon, then η(x) is set to 0.

[0039] Furthermore, in step two, the emotional support response strategy labels corresponding to the dialogue statements are semantically expanded. In the dataset, each dialogue providing emotional support corresponds to an emotional support response strategy label. Explanatory text is added to different emotional support response strategy labels through the Strategy Dictionary, and the hidden state t of the emotional support response strategy is obtained by inputting it into the Strategy Encoder.

[0040] Furthermore, the specific prediction steps of the emotional support response strategy are as follows:

[0041] The policy classifier is trained based on the contextual hidden state H obtained in step one, and the probability distribution of the policy is calculated using a multilayer perceptron (MLP) to represent the prediction probability p of different policies. g =[p 1 ,…,p 8 Then, combine this with the hidden state T = [t1, ..., t8] of the policy. T The weighted calculation yields the following comprehensive strategy representation:

[0042] p g =MLP(H) H g =p g ·T

[0043] Where t j This refers to the j-th strategy. There are eight strategies in total, therefore j = 1, ..., 8; p i It refers to the probability of choosing the i-th strategy, where i = 1, ..., 8.

[0044] Furthermore, in step three, each time n rounds of dialogue are selected to construct an emotion state transition graph, and the dialogue text u i The hidden state is represented by the semantic node Semi in the sentiment state transition graph. The nodes in the sentiment state transition graph include the semantic node Semi in the dialogue context and the sentiment node Emo i that is only for user speech. Connect all nodes in the sentiment state transition graph, construct sentiment edges and role edges, and assign values ​​to the nodes in the sentiment state transition graph.

[0045] Furthermore, the role edges are two directed edges constructed for the two role models in the ESC task and for the user, namely user-model and model-user; the emotion edges are two directed edges constructed for the semantic and emotion relationships, namely user semantic-emotion and emotion state transition, thereby modeling the transition relationship between dialogue semantics and emotion.

[0046] Furthermore, the semantic nodes of the emotion state transition graph are formed by the dialogue text u i The corresponding hidden state is used for assignment; for sentiment nodes, the dialogue text is used. i The corresponding hidden states and common-sense knowledge COMET are used to supplement the user's emotional state knowledge and capture the user's emotional feelings; then, common-sense knowledge is used to generate the last layer of hidden state representation as emotional knowledge; the final representation of the emotional state is the sum of the dialogue hidden states and emotional knowledge, that is:

[0047]

[0048] Where COMET represents the commonsense knowledge generation model; the input is the i-th round of dialogue text u from the dialogue context. i And the j-th relation r predefined in the common sense knowledge generation model j Output dialogue text u i Corresponding to the j-th relation r j Common sense knowledge text c j ; This represents combining all the common-sense knowledge generated from the correspondences into the same text sequence C. ui ={c1,…,c Nr};N r This represents the number of predefined relations in the common sense knowledge generation model.

[0049] Furthermore, in step four, c-Attn, g-kAttn, scAttn, and mcAttn are added to the multi-factor perception decoder; c-Attn refers to the self-attention calculation of the context hidden state; g-kAttn refers to the cross-attention calculation between the policy hidden state and the context hidden state of fused knowledge; scAttn refers to the cross-attention calculation between a single context hidden state and the context hidden state of fused COMET knowledge; and m-cAttn refers to the cross-attention calculation between a single context hidden state and the context hidden state of fused HEAL knowledge.

[0050] The beneficial effects of this invention are:

[0051] The present invention provides an emotion-supported dialogue response generation model and method that integrates multi-source knowledge for conducting emotion-supported dialogue in multi-turn human-computer dialogue scenarios, which has the following advantages:

[0052] 1. By integrating multi-source cognitive knowledge, and adding examples of mental health dialogues and an emotional lexicon on the basis of common sense knowledge, a comprehensive and multi-dimensional emotional cognitive model is established, which transcends the limitations of existing cognitive knowledge.

[0053] 2. This study innovatively investigates the bidirectional constraint relationship between emotional cognitive knowledge and emotional support strategies, which jointly improves the model's knowledge selection ability, strategy prediction ability, and emotional cognitive ability, breaking the limitations of previous studies that isolated each module.

[0054] 3. By modeling the user's emotional state transition process based on common sense knowledge, fine-grained emotional understanding is achieved, which helps the dialogue model to deeply understand the user's external situation and internal emotional state, thereby filling the gap in the emotional cognitive ability of the emotional support dialogue model. Attached Figure Description

[0055] Figure 1 Design the model framework diagram.

[0056] Figure 2 This is a process for the fusion mechanism of multi-source cognitive knowledge.

[0057] Figure 3 Modeling process for the two-way interaction of knowledge strategies.

[0058] Figure 4 This is a diagram of emotional state transitions.

[0059] Figure 5 This is the process for a multi-factor perception decoder.

[0060] Figure 6 This is the emotional state transition diagram obtained in Example 1. Detailed Implementation

[0061] Firstly, this invention provides an emotion-supported dialogue response generation model that integrates multi-source knowledge. The overall framework of this model is an Encoder-Decoder model based on Blenderbot-small, which mainly includes the following modules:

[0062] The multi-source cognitive knowledge fusion mechanism module aims to supplement users' cognitive knowledge of their external situation by combining common sense knowledge COMET with cognitive knowledge of users' internal emotional state by combining mental health dialogue examples HEAL, and to analyze the combined effect of the two types of knowledge on the generation of emotional support responses.

[0063] The knowledge strategy bidirectional interaction modeling module aims to model the mutual constraints between multi-source cognitive knowledge and emotional support response strategies.

[0064] The User Emotional State Transition Modeling Module aims to model fine-grained transition information at the level of each dialogue round, including semantic and emotional transitions, based on the encoded context and common-sense knowledge COMET, in order to achieve smooth and natural transitions between emotionally supported dialogues.

[0065] The multi-factor perception decoder module modifies the pre-trained language model Blenderbot decoder to perform cross-attention calculations between the contextual hidden state and the aforementioned factors. Through interaction with multiple factors, it generates the final emotional support response, thereby achieving comprehensive perception of multiple factors including dialogue context, user emotions, emotional support strategies, and cognitive knowledge.

[0066] Secondly, the present invention provides a method for generating emotion-supported dialogue responses that integrates multi-source knowledge. This method is implemented using an emotion-supported dialogue response generation model that integrates multi-source knowledge, as provided in the first aspect.

[0067] like Figure 1 As shown, the specific workflow of the emotion-supported dialogue response generation method integrating multi-source knowledge of the present invention is as follows:

[0068] Step 1: Multi-source cognitive knowledge fusion mechanism;

[0069] The multi-source cognitive knowledge fusion mechanism proposed in this invention is as follows: Figure 2 As shown, the multi-source cognitive knowledge fusion mechanism aims to supplement the user's cognitive knowledge of the external situation by combining common sense knowledge (COMET) and cognitive knowledge of the user's internal emotional state by combining mental health dialogue example knowledge (HEAL), and analyze the combined effect of the two types of knowledge on the generation of emotional support responses.

[0070] The specific workflow for step one is as follows:

[0071] The context sequence {[CLS],u1,…,[CLS],u n The input is a context encoder, which encodes the hidden context state to obtain the corresponding context statement. i The encoded vector, where u i [CLS] represents the i-th round of dialogue text in the dialogue context, where i = 1, ..., n, n represents the total number of rounds of dialogue, and [CLS] represents the dialogue text u. i The starting marker. The context sequence represents all dialogues in chronological order from the beginning to the last round.

[0072] The COMET knowledge text and HEAL knowledge text are added to the same context sequence. Specifically, the COMET knowledge text is added to the context sequence to obtain {[CLS],u1,…,[CLS],u n,[COMET]}, input the Knowledge Encoder. Additionally, add the HEAL knowledge text to the context sequence to obtain {[CLS],u1,…,[CLS],u n, [HEAL]}, input the Knowledge Encoder.

[0073] The Knowledge Encoder and the Context Encoder share all parameters, but their functions differ.

[0074] The contextual hidden state (encoded vector) with external knowledge obtained through the above encoding is used for knowledge selection based on calculated emotion intensity values. This invention uses an external knowledge NRC-VAD emotion dictionary to calculate the emotion intensity values ​​of multi-source cognitive knowledge. It contains three-dimensional VAD (Valence-Arousal-Dominance) vectors (Va, Ar, Do) for 20,000 English words and is widely used in emotion computing research. In addition to multi-source cognitive knowledge fusion, this invention achieves a scientifically reliable knowledge selection method based on emotion intensity calculation.

[0075] The specific process for calculating the sentiment intensity for a given word x is as follows:

[0076]

[0077] Where min-max() is the minimum-maximum normalization; ||·||2 represents the 2-norm; Va(x) and Ar(x) represent the values ​​of the Valence dimension and Arousal dimension in the VAD 3D vector of word x, respectively; if word x is not in the NRC-VAD sentiment lexicon, then η(x) is set to 0.

[0078] The sentiment intensity value of the entire sentence is obtained by summing the sentiment intensity values ​​of each word in the sentence, and the contextual hidden state H with the highest sentiment intensity value and external knowledge is selected.

[0079] Step 2: Modeling the two-way interaction of knowledge strategies;

[0080] like Figure 3 As shown, this invention proposes a mutual constraint modeling approach for multi-source cognitive knowledge and emotional support response strategies. The specific operation process is as follows:

[0081] Semantic augmentation of the emotional support response strategy labels corresponding to dialogue statements: In the dataset, each dialogue providing emotional support corresponds to an emotional support response strategy label, such as [Providing Suggestions], but such labels lack semantic information.

[0082] Semantic augmentation refers to adding explanatory text to different sentiment support response strategy labels using a strategy dictionary, such as [Providing Suggestions: provide suggestions about how to get over the tough and change the current situation.]. This is input into the strategy encoder to obtain the hidden state t of the sentiment support response strategy. The strategy encoder, knowledge encoder, and context encoder share all parameters; their names differ because they serve different purposes. g1 and g2 both represent sentiment support response strategy labels, such as [Providing Suggestions]. i The subscript i in the dialogue indicates the i-th round of dialogue in which the party providing emotional support is speaking.

[0083] The eight sentiment support response strategies defined in the dataset and their original definitions are as follows (the semantic augmentation step involves adding sentiment support response strategy labels to the original definitions of each sentiment support response strategy in the dataset paper):

[0084] ·Question:ask for information related to the problem to help thehelp-seeker articulate the issues that they face.

[0085] ·Restatement or Paraphrasing: a simple, more concise rephrasing of thesupportseeker's statements that could help them see their situation more clearly.

[0086] ·Reflection of Feelings: describe the helpseeker's feelings to show the understanding of the situation and empathy.

[0087] ·Self-disclosure:share similar experiences or emotions that the supporter has also experienced to express your empathy.

[0088] ·Affirmation and Reassurance:affirm the help-seeker's ideas, motivations, and strengths to give reassurance and encouragement.

[0089] ·Providing Suggestions:provide suggestions about how to get over thetough and change the current situation.

[0090] ·Information: provide useful information to the help-seeker, forexample with data, facts, opinions, resources, or by answering questions.

[0091] ·Others: other support strategies that do not fall into the above categories.

[0092] The specific prediction steps of the emotional support response strategy are as follows:

[0093] The policy classifier is trained based on the contextual hidden state H of the fused knowledge obtained from the multi-source cognitive knowledge fusion mechanism in step one. The probability distribution of the policy is calculated using a multilayer perceptron (MLP) to represent the predicted probability p of different policies. g =[p 1 ,…,p 8Then, combine this with the hidden state T = [t1, ..., t8] of the policy. T The weighted calculation yields the following comprehensive strategy representation:

[0094] p g =MLP(H)H g =p g ·T

[0095] Where t j This refers to the j-th strategy. There are eight strategies in total, therefore j = 1, ..., 8. i It refers to the probability of choosing the i-th strategy, where i = 1, ..., 8.

[0096] Step 3: Modeling the transition of user emotional state;

[0097] The user emotional state transition modeling module did not... Figure 1 It is not directly reflected in the text, but rather it is executed along with the fusion process of common sense knowledge COMET in the multi-source cognitive knowledge fusion mechanism of step one.

[0098] To achieve smooth and natural transitions between emotion-supported dialogues, this invention models fine-grained transition information at the level of each dialogue turn, including semantic and emotional transitions, based on encoded context and common-sense knowledge COMET. Specifically, this is achieved by constructing an EmotionTransitions Graph to propagate and update transition information.

[0099] like Figure 5 As shown, to avoid the influence of semantic noise in the dialogue history, the sentiment state transition graph is constructed over n rounds of dialogue. Taking n=5 as an example, the sentiment state transition graph is constructed by selecting 5 rounds of dialogue each time, {u i u i+1 u i+2 u i+3 u i+4}, where u i This represents the i-th round of dialogue text within the dialogue context. The dialogue text u... i The hidden state is represented by the semantic node Sem i in the sentiment state transition graph. The dialogue text u i Used only for extracting semantic nodes, not as elements in the sentiment state transition graph. Context Utterance represents the dialogue context, i.e., the text sequence arranged according to the dialogue rounds; SemanticNodes represent semantic nodes; and EmotionNodes represent sentiment nodes.

[0100] The nodes in the emotional state transition graph include semantic nodes Sem i representing the dialogue context and emotional nodes Emo i representing only the user's speech.

[0101] To connect all nodes in the sentiment state transition graph, sentiment edges and role edges are constructed: role edges are two types of directed edges (User-Model and Model-User) constructed for the two roles (Model and User) in the ESC task; sentiment edges are two types of directed edges (User-Emo and Emo-Trans) constructed for semantic and sentiment relationships, thereby modeling the transition relationship between dialogue semantics and sentiment.

[0102] Regarding the assignment of values ​​to nodes in the emotion state transition graph: The semantic nodes of the emotion state transition graph are derived from the dialogue text. i The corresponding hidden state is used for assignment; for sentiment nodes, in addition to using the dialogue text, the corresponding hidden state is used for assignment. i In addition to the corresponding hidden states, common-sense knowledge (COMET) is used to supplement the user's emotional state knowledge and capture the user's emotional feelings. Then, common-sense knowledge is used to generate the final hidden state representation of the model as emotional knowledge. The final representation of the emotional state is the sum of the dialogue's hidden states and the emotional knowledge, i.e.:

[0103]

[0104] Where COMET represents the commonsense knowledge generation model. The input is the i-th round of dialogue text u from the dialogue context. i And the j-th relation r predefined in the common sense knowledge generation model j (e.g., xAttr, xReact, etc.). Output dialogue text u i Corresponding to the j-th relation r j Common sense knowledge text c j ; This represents combining all the common-sense knowledge generated from the correspondences into the same text sequence C. ui ={c1,…,c Nr};N r This represents the number of predefined relations in the common sense knowledge generation model.

[0105] Step 4: Multi-factor perception decoder;

[0106] To achieve comprehensive perception encompassing multiple factors such as dialogue context, user emotion, emotional support strategies, and cognitive knowledge, this invention proposes a multi-factor perception-based emotional support response generation method. The pre-trained language model Blenderbot decoder is modified to perform cross-attention calculations between the contextual hidden states and the aforementioned factors, generating the final emotional support response through interaction with these multiple factors.

[0107] like Figure 5 As shown, the input generates the hidden state sequence {[CLS],y1,y2,y3,…,y}.n}, where [CLS] is the sentence start marker, y i This represents the hidden state of the i-th word in the generated sentence.

[0108] The innovative work of the multi-factor perception decoder lies in the modification of the cross-attention layer, which incorporates c-Attn, g-kAttn, s-cAttn, and m-cAttn, such as... Figure 2 As shown.

[0109] c-Attn refers to self-attention computation of the contextual hidden state; the self-attention mechanism allows the model to focus its attention on information in other positions when computing the representation of each word, which helps to better capture dependencies and long-distance dependencies in the context sequence.

[0110] c-Attn(H c ) = Attention(H c H c H c )

[0111] Where Hc represents the context hidden state matrix, which contains the hidden states of all context positions.

[0112] Attention represents the typical computational process of an attention mechanism, used to compute a weighted contextual representation.

[0113] g-kAttn refers to the cross-attention calculation between the policy hidden state and the context hidden state of fused knowledge:

[0114] g-kAttn(H g H comet H heal ) = Attention(H g H comet +H heal H comet +H heal )

[0115] Where H g H represents the weighted calculation strategy representation of the output in the two-way interaction modeling of knowledge strategies in step two. comet H represents the context hidden state of COMET knowledge fused together. heal This represents the contextual hidden state that incorporates HEAL knowledge.

[0116] s-cAttn refers to the cross-attention computation between individual contextual hidden states and contextual hidden states that incorporate COMET knowledge:

[0117] s-cAttn(H c Hcomet ) = Attention(H c H comet H comet )

[0118] m-cAttn refers to the cross-attention calculation between individual contextual hidden states and contextual hidden states incorporating HEAL knowledge:

[0119] m-cAttn(H c H heal ) = Attention(H c H heal H heal )

[0120] The other layers are standard Transformer decoders, only used for calling, so they will not be explained in detail. Self-Attention refers to the self-attention mechanism, Add & Norm refers to residual connections and normalization, Feed Forward refers to the feedforward network, and Linear & Softmax refers to linear layers and word probability calculation.

[0121] In summary, this invention provides an emotion-supporting dialogue response generation model and method that integrates multi-source knowledge, enhancing the model's emotion cognition capabilities for emotion-supporting dialogue. This invention combines the mental health knowledge graph HEAL and the emotion dictionary NRC_VAD to supplement the background semantic information of external knowledge, filtering knowledge noise by calculating emotion intensity; it jointly analyzes external knowledge and emotion support strategy modules, modeling the bidirectional constraint relationship between emotion cognition knowledge and emotion support strategies, thereby improving the model's emotion cognition capabilities in multiple ways; and it models the user's emotional state transition process based on the common-sense knowledge COMET, achieving in-depth emotional understanding.

[0122] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0123] Example 1

[0124] This invention provides an emotion-supported dialogue response generation model and method that integrates multi-source knowledge. Taking a dialogue between a user and the model as an example, the input dialogue contexts U1, U2, and U3 are:

[0125]

[0126] Apart from the text in Context and Response Generation, all other text is high-dimensional vectors generated during the model calculation process. For readability reasons, it has been replaced with text.

[0127] 1) Context: Input dialogue context, where U1 and U3 are user statements, and U2 is model statements;

[0128] 2) COMET: Input user statement U and reasoning relationship r, output common sense reasoning result;

[0129] 3) COMET NRC-VAD: Based on NRC-VAD sentiment intensity calculation, select the knowledge with the highest sentiment intensity;

[0130] 4) HEAL: The HEAL knowledge text that best fits the context is matched by similarity calculation and used as a reference example for generating responses;

[0131] 5) HEAL NRC-VAD: Based on NRC-VAD sentiment intensity calculation, the response with the highest sentiment intensity is selected to generate a reference example;

[0132] 6) Strategy Dictionary: Adds explanatory semantics to the policy labels [Reflection offerings] predicted by the Multilayer Perceptron (MLP);

[0133] 7) EmotionTransitions Graph: Construct an emotional state transition graph to model user emotional shifts;

[0134] 8) Cross Attention: Cross attention is calculated for context, external knowledge, and emotional support dialogue response strategies respectively;

[0135] 9) Response Generation: The model decodes and generates the final emotional support dialogue response text.

[0136] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for generating emotional support dialogue responses by integrating multi-source knowledge, characterized in that: The following steps are involved: Step 1: Multi-source cognitive knowledge fusion mechanism; Combine common sense knowledge COMET to supplement the user's external situation cognition knowledge, combine psychological health dialogue sample knowledge HEAL to supplement the user's internal emotional state cognition knowledge, and analyze the joint effect of the two kinds of knowledge on the generation of emotional support replies; Step 2: Modeling the mutual constraints between multi-source cognitive knowledge and emotional support response strategies; Step 3: Modeling the user's emotional state transition; Based on COMET, which encodes context and common sense knowledge, it models fine-grained transition information at each dialogue turn level, including semantic and emotional transitions, to achieve smooth and natural transitions between emotional support dialogues. Step 4: Multi-factor perception decoder; By modifying the pre-trained language model Blenderbot decoder, the contextual hidden states are cross-attention calculated with multiple factors respectively, and the final emotional support reply is generated by interacting with multiple factors to achieve comprehensive perception including conversation context, user emotion, emotional support strategy and cognitive knowledge factors.

2. According to the method for generating emotional support dialogue responses by integrating multi-source knowledge according to claim 1, it is characterized in that: In step 1, the context sequence {[CLS],u1,…,[CLS],u n } Input the context encoder Context Encoder, encode the context hidden state, and get the corresponding context sentence u i The encoding vector of i represents the i-th round of dialogue text in the dialogue context, i = 1, ..., n, n represents the total number of dialogue rounds, [CLS] represents the dialogue text u i The context sequence represents all the dialogues from the beginning to the last round in chronological order. The COMET knowledge text and the HEAL knowledge text are added to the same context sequence and then input into the knowledge encoder. The encoded context hidden state with external knowledge is used for knowledge selection based on the calculated sentiment intensity value; the sentiment intensity value of the entire sentence is obtained by summing up the sentiment intensity values ​​of each word in the sentence, and the context hidden state H with external knowledge with the highest sentiment intensity value is screened out.

3. The method for generating emotional support dialogue responses by integrating multi-source knowledge according to claim 2, characterized in that: The knowledge encoder Knowledge Encoder and the context encoder Context Encoder share all parameters.

4. The method for generating emotional support dialogue responses by integrating multi-source knowledge according to claim 2, characterized in that: Using the external knowledge NRC-VAD sentiment dictionary to calculate the sentiment intensity value of multi-source cognitive knowledge, the external knowledge NRC-VAD sentiment dictionary contains VAD three-dimensional vectors of 20,000 English words; For word x, the sentiment intensity value is calculated as follows: Where min-max() represents minimum-maximum normalization; ||·||2 represents the 2-norm; Va(x) and Ar(x) represent the values ​​of the Valence dimension and Arousal dimension in the VAD three-dimensional vector of word x, respectively; if word x is not in the NRC-VAD sentiment dictionary, η(x) is set to 0.

5. The method for generating emotional support dialogue responses by integrating multi-source knowledge according to claim 2, characterized in that: In step 2, the emotional support response strategy labels corresponding to the dialogue sentences are semantically expanded. In the dataset, each dialogue that provides emotional support corresponds to an emotional support response strategy label. Explanation texts are added to different emotional support response strategy labels through the strategy dictionary StrategyDictionary and input into the strategy encoder StrategyEncoder to obtain the hidden state t of the emotional support response strategy.

6. The method for generating emotional support dialogue responses by integrating multi-source knowledge according to claim 5, characterized in that: The specific prediction steps of the emotional support response strategy are as follows: The strategy classifier is trained based on the contextual hidden state H of the fusion knowledge obtained in step 1, and the probability distribution of the strategy is calculated through the multi-layer perceptron MLP to represent the predicted probability p of different strategies. g =[p 1 ,…,p 8 ], and then with the strategy hidden state T = [t1,…,t8] T , the weighted calculation to obtain the comprehensive strategy is expressed as follows: p g =MLP(H) H g =p g ·T where t j refers to the jth strategy, there are eight strategies in total, so j = 1,…,8; p i It refers to the probability of choosing the i-th strategy, i=1,…,8.

7. The method for generating emotional support dialogue responses by integrating multi-source knowledge according to claim 1, characterized in that: In step 3, each time we select n rounds of dialogue to construct the emotional state transition graph, we convert the dialogue text u i The hidden state is represented as a semantic node Semi in the emotional state transition graph, where the nodes in the emotional state transition graph include a semantic node Semi of the dialogue context and an emotional node Emo i that is only for the user's speech; Connect all nodes in the emotional state transition graph, construct emotion edges and role edges, and assign values ​​to the nodes in the emotional state transition graph.

8. The method for generating emotional support dialogue responses by integrating multi-source knowledge according to claim 7, characterized in that: The role edges are two directed edges constructed for the two role models and users in the ESC task, namely user-model and model-user; the emotion edges are two directed edges constructed for the semantic and emotional relationship, namely user semantics-emotion and emotional state conversion, thereby modeling the transition relationship between dialogue semantics and emotion.

9. The method for generating emotional support dialogue responses by integrating multi-source knowledge according to claim 7, characterized in that: The semantic nodes of the emotional state transition graph are composed of the dialogue text u i The corresponding hidden state is assigned; for emotional nodes, the dialogue text u i The corresponding hidden state and common sense knowledge COMET are used to supplement the user's emotional state knowledge and capture the user's emotional feelings. Then the common sense knowledge is used to generate the last layer of hidden state representation of the model as emotional knowledge. The final representation of the emotional state is the sum of the dialogue hidden state and the emotional knowledge, that is: Among them, COMET represents the common sense knowledge generation model; Enter the i-th round of dialogue text u in the dialogue context i And the jth relationship r predefined by the common sense knowledge generation model j ; Output dialogue text u i Corresponding to the jth relationship r j Common sense knowledge text c j ; Represents the common sense knowledge generated by combining all corresponding relations into the same text sequence C ui ={c1,…,c Nr }; N r Represents the number of predefined relations in the commonsense knowledge generation model.

10. The method for generating emotional support dialogue responses by integrating multi-source knowledge according to claim 1, characterized in that: In step 4, c-Attn, g-kAttn, s-cAttn and m-cAttn are added to the multi-factor perception decoder; the c-Attn refers to the self-attention calculation of the context hidden state; the g-kAttn refers to the cross-attention calculation between the strategy hidden state and the context hidden state of the fusion knowledge; the sc Attn refers to the cross-attention calculation between the separate context hidden state and the context hidden state of the fusion COMET knowledge; the mc Attn refers to the cross-attention calculation between the separate context hidden state and the context hidden state of the fusion HEAL knowledge.

Images