Emotion support dialogue generation method and system based on heterogeneous graph network
Through heterogeneous graph networks and bidirectional attention flow networks, combined with common sense knowledge generators, the insufficient capture of dynamic changes in dialogue history and response strategies in the existing technology is solved, and the accuracy and strategy relevance of emotional support dialogue generation are improved. A more suitable system reply is achieved.
Patent Information
- Application Number
- CN202410992592.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-12
AI Technical Summary
The existing emotional support dialogue generation method is difficult to effectively capture the dynamic changes in user status and system response strategies in the conversation history, resulting in the generated responses being lengthy and deviating from human intentions, and the accuracy of response strategy prediction is low. The system response is related to the selected strategy. Not very sexual.
Using a heterogeneous graph network method, the graph structure information between dialogue history, strategy history and common sense knowledge is constructed through the attention mechanism, and the two-way attention flow network interaction dialogue history characteristics and response strategy characteristics are introduced to generate emotional support responses. And supervise the supportive strategies adopted by the MLP model fitting system responses.
The contextual correlation and coherence between dialogue history and response strategies are achieved, the accuracy of emotional support dialogue generation and the prediction accuracy of response strategies are improved, and the diversity and relevance of system responses are enhanced.
Smart Images

Figure CN118939774A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing and dialogue generation, and in particular to a method and system for generating emotion support dialogue based on a heterogeneous graph network. Background Art
[0002] Emotional Support Conversation, as a core branch in the field of empathetic dialogue, is committed to integrating human emotional intelligence into the dialogue system. Its core concept is not only to perceive the user's emotional fluctuations, but also to provide emotional support to the user through empathetic and supportive responses. Compared with conventional empathetic dialogue, emotional support dialogue focuses more on strategically exploring the user's deep emotional needs, first by expressing understanding and empathy to comfort the help-seeker, then by patiently communicating to understand the difficulties faced by the help-seeker, and finally providing reasonable suggestions or solutions to help the help-seeker out of the predicament. In the current social context where mental health problems are frequent, this task is particularly important.
[0003] At present, the emotion support dialogue generation method is mainly based on end-to-end model design, that is, generating responses directly from the dialogue history. Researchers continue to explore efficient encoder and decoder structures, striving to accurately capture the behavioral and emotional characteristics in the dialogue history by integrating external knowledge, so as to generate emotion support responses that conform to specific strategies.
[0004] In recent years, large language models have attracted much attention for their excellent language understanding and generation capabilities. At the same time, some researchers have found that by introducing heterogeneous graph neural networks, the ability of large language models in capturing multimodal information and processing complex data can be significantly improved. However, in the field of emotional support dialogue, heterogeneous graph neural networks have not yet been fully and deeply applied. Existing methods using large language models can be roughly divided into two categories: one is to directly input complex information into the large language model by designing specific prompt templates to generate responses in one go. However, this method may cause the generated responses to be lengthy and deviate from human intentions, making it difficult to achieve the expected emotional support effect; the second is to capture the contextual features in the dialogue history by designing an efficient encoder, and the decoder generates emotional support responses that conform to specific strategies based on this. However, this method often pays too much attention to the content of the dialogue itself, and fails to fully consider the process in which the user status and system response strategy change dynamically as the dialogue progresses.
[0005] Therefore, it is necessary to design an emotion-support dialogue model based on heterogeneous graph networks to integrate the common sense knowledge in user discourse and the supportive strategies adopted by the system's historical responses to solve the above technical problems. Summary of the invention
[0006] In view of the deficiencies in the prior art, the present invention proposes a method, system, storage medium and electronic device for generating emotion support dialogues based on heterogeneous graph networks, aiming to obtain an external common sense knowledge set based on a common sense knowledge generator, and capture rich semantic information and complex relationship features between dialogue history, strategy history and common sense knowledge through a heterogeneous graph network, and formalize the emotion support dialogue into a dialogue process jointly constrained by dialogue history and response strategy. Different from traditional emotion recognition methods based on prompt templates or contextual features, the technical solution of the present invention can not only select the most appropriate response strategy based on strategy history and external common sense knowledge during the response generation process, but also ensure the contextual relevance and coherence of the system response and the support strategy.
[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions:
[0008] A method for generating emotional support dialogue based on a heterogeneous graph network comprises the following steps:
[0009] (1) Collect and preprocess the dialogue dataset; any dialogue data includes multiple rounds of user utterances and system responses, as well as the support strategies adopted by the system in each round of responses.
[0010] (2) Input each user utterance in the dialogue dataset into a common sense knowledge generator to obtain external common sense knowledge of each user utterance.
[0011] (3) Based on each preprocessed dialogue data, the dialogue history, strategy history, common sense knowledge and the system response to be generated are obtained in the form of multi-round label sequences, and the dialogue history features, strategy history features and common sense knowledge features of each round of dialogue are obtained through a multi-source encoder.
[0012] (4) Constructing a heterogeneous graph network based on an attention mechanism to model the graph structure information between the conversation history features, strategy history features, and common sense knowledge features, so as to generate the response strategy features required for emotional support replies; wherein the heterogeneous graph network is composed of four types of nodes and three types of edges, and the attention mechanism is used to calculate the importance weights of adjacent nodes.
[0013] (5) Through the bidirectional attention flow network, the dialogue history features and response strategy features are interacted to establish the dialogue history to response strategy flow and the response strategy to dialogue history flow, so as to use the dialogue history features to select the response strategy related to the dialogue history, and use the response strategy features to select the dialogue history features related to the response strategy.
[0014] (6) Inputting the dialogue history features related to the response strategy into the response generation decoder, generating a system response for the emotional support dialogue to be generated, and calculating the loss value of the system response; at the same time, inputting the response strategy related to the dialogue history into the supervised MLP model to fit the support strategy adopted by the system response, and calculating the loss value of the strategy selection task.
[0015] (7) The loss value of the system response and the loss value of the strategy selection task are integrated to train the emotion support dialogue generation model composed of steps S1 to S6 until the emotion support dialogue generation model converges or reaches a specified number of training times.
[0016] Furthermore, in step (2), the method of obtaining external common sense knowledge of each user utterance is as follows: a common sense knowledge generator is constructed based on a pre-trained knowledge graph, the user utterance and the common sense relationship are input into the common sense knowledge generator, and common sense knowledge such as psychological state and cognitive state are obtained in the form of text blocks to expand the information provided by the user utterance.
[0017] Furthermore, in step (3), for each round of dialogue in each piece of dialogue data: the user utterances of the current round and the previous round and the system responses of the previous round are spliced together as the dialogue history of the round; the support strategies adopted by the system responses of the previous round are spliced together as the strategy history of the round; the set of external common sense knowledge obtained from the user utterances of the current round is taken as the common sense knowledge of the round; and the system responses of the current round are taken as the system responses to be generated in the round.
[0018] Furthermore, in step (3), a training dataset, a test dataset and a verification dataset are obtained based on the dialogue dataset, the dialogue history, strategy history and common sense knowledge in character form are converted into label sequence representations, and a multi-source encoder is used to encode the dialogue history, strategy history and common sense knowledge, and convert them into high-dimensional vector representations to capture the semantic information and contextual information in the dialogue history sequence and common sense knowledge, as well as the position information in the strategy history sequence.
[0019] Furthermore, in step (4), the information of each node and its neighbors is aggregated through a heterogeneous graph network, and the strategy features guiding the generation of supportive responses are obtained based on the graph attention network:
[0020]
[0021] Among them, α t represents the weight of each neighbor of the node, which is calculated by the attention mechanism; s t The response strategy features that guide the system's reply are obtained by concatenating the strategy node states output by the heterogeneous graph.
[0022] The HeteGraph heterogeneous graph network structure is as follows:
[0023] G=(v,ε)
[0024] Among them, v represents four types of nodes in the graph, including the user's historical discourse, the system's historical replies, the system's historical strategies, and external common sense knowledge; ε represents three types of edges in the graph, including dialogue connections, strategy connections, and state connections.
[0025] The heterogeneous graph network uses a graph attention network to extract high-dimensional features of nodes and edges, and obtains response strategy features for emotional support dialogue by fusing the top-level historical strategy node features.
[0026] Furthermore, in step (5), a bidirectional attention flow network is introduced to establish a flow from dialogue history to response strategy and a flow from response strategy to dialogue history. Specifically, the flow from dialogue history to response strategy shows that the bidirectional attention flow network uses the contextual information in the dialogue history to select supporting strategies related to the dialogue history, and then constrains the response strategy features according to the dialogue history; the flow from response strategy to dialogue history shows that the bidirectional attention flow network pays attention to the context related to the adopted strategy in the decoding stage, and then constrains the dialogue history features according to the response strategy. Further, in step (6), the updated dialogue history features are input into the decoder to obtain the conditional probability distribution of the prediction results, and the standard cross entropy loss function is used to calculate the loss value of the system reply; the updated response strategy features are input into the supervised MLP model to predict the response strategy distribution, and the negative log-likelihood loss between it and the true strategy is calculated to obtain the loss value of the strategy selection task.
[0027] The present invention also provides an emotion support dialogue generation system based on a heterogeneous graph network, comprising:
[0028] The data acquisition module is used to collect and preprocess the dialogue data set; wherein any dialogue data includes multiple rounds of user speech and system responses, as well as the support strategies adopted by the system in each round of responses;
[0029] The data processing module is used to obtain multiple rounds of dialogue history, strategy history, common sense knowledge and the system response sequence to be generated based on each pre-processed dialogue data;
[0030] The knowledge enhancement module is used to input the user utterances of each round into the pre-built common sense knowledge generator to obtain the external common sense knowledge of each round;
[0031] The encoding module is used to encode the dialogue history sequence, strategy history sequence, and common sense knowledge sequence of the current round and convert them into a high-dimensional feature vector representation;
[0032] The heterogeneous graph network module is used to input the feature vectors of the current round of dialogue history, strategy history and common sense knowledge into the heterogeneous graph network to obtain the response strategy features used to guide the system reply generation;
[0033] A two-way feedback module is used to input the current round of dialogue history features and response strategy features into the two-way attention flow network to obtain the response strategy features related to the dialogue history and the dialogue history features related to the response strategy;
[0034] A supervised MLP model, used to input the response strategy related to the dialogue history into the supervised MLP model, fit the support strategy adopted by the system reply, and calculate the negative log-likelihood loss value of the strategy selection task;
[0035] A response generation decoder, used for inputting the dialogue history features related to the response strategy into the response generation decoder, generating a response for the emotion support dialogue to be generated, and calculating a cross entropy loss value of the system reply;
[0036] A training module, configured to select a loss value and a system response loss value according to a strategy, and train the emotion support dialogue model until convergence;
[0037] The prediction module takes the conversation data to be processed as the input of the converged model and predicts the corresponding emotional support dialogue system response.
[0038] The present invention also provides a storage medium storing a computer program for generating an emotion support dialogue based on a heterogeneous graph network, wherein the computer program enables a computer to execute the emotion support dialogue generation based on a heterogeneous graph network.
[0039] The present invention also provides an electronic device, comprising one or more processors and memories and one or more programs, wherein the one or more programs are stored in the memories and are configured to be executed by the one or more processors, and the programs include executing the emotional support dialogue generation based on a heterogeneous graph network.
[0040] Compared with the prior art, the beneficial technical effects of the present invention are:
[0041] 1. This paper designs a heterogeneous graph network based on the attention mechanism to graph the structured dialogue process and obtain the feature representation of the support strategy based on the system response. Compared with the traditional emotional support dialogue generation method, this method does not generate the system response strategy based on rules or templates, which solves the problem of being unable to effectively capture and model the common sense knowledge state in the user's speech during the dialogue process and the dynamic changes of the system's historical strategy as the dialogue progresses.
[0042] 2. Compared with the traditional emotion-supported dialogue method that directly generates dialogue strategies by splicing dialogue history, user emotions and other features, the present invention aims to focus on the system support strategy for each round of dialogue. The present invention models the relationship between the system historical responses and their support strategies in a heterogeneous graph network, and dynamically captures the fine-grained strategy status in the current dialogue process.
[0043] 3. In order to obtain the context information most relevant to the response strategy and the response strategy information related to the context, the present invention introduces a bidirectional attention flow network to solve the problems of low response strategy prediction accuracy generated by traditional emotional support dialogue methods and low correlation between system responses and selected strategies.
[0044] 4. The state information provided by user discourse is limited. Judging the user state directly through user discourse will reduce the accuracy and interpretability of the heterogeneous graph modeling dialogue process. To address this situation, the present invention introduces a method for constructing a common sense knowledge generator based on a pre-trained knowledge graph to obtain external common sense knowledge in each round of user discourse and expand the user state information obtained by the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.
[0046] Figure 1 A block diagram of a method for generating emotion support dialogue based on a heterogeneous graph network according to the present invention;
[0047] Figure 2 It is a structural block diagram of a method for generating emotional support dialogue based on a heterogeneous graph network according to the present invention;
[0048] Figure 3 is the multi-source encoder model structure in the present invention;
[0049] Figure 4 The strategy-aware heterogeneous graph network model structure in the present invention;
[0050] Figure 5 This is the bidirectional feedback decoder model structure in the present invention. DETAILED DESCRIPTION
[0051] The specific implementation of the present invention is described below in conjunction with the accompanying drawings and examples, but the following examples are only used to explain the present invention in detail and do not limit the scope of the present invention in any way. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0052] Example 1
[0053] The embodiment of the present invention provides a method for generating emotional support dialogue based on a heterogeneous graph network. Figure 2 As shown, the emotion support dialogue generation model includes a multi-source encoder model, a strategy-aware heterogeneous graph network model and a bidirectional feedback decoder model. The present invention is described in detail below in conjunction with the embodiments. Figure 1 As shown, the specific steps include:
[0054] Step 1: Collect and preprocess the dialogue dataset; any dialogue data includes multiple rounds of user speech and system responses, as well as the support strategies adopted by the system in each round of responses.
[0055] In this embodiment, the support strategy is represented by {u1, r1, s1, ..., u i ,r i ,s i ,…,u n ,r n s n}, n is the total number of dialogue rounds, u i is the user utterance in the i-th round of dialogue, r i is the system response in the i-th round of dialogue, s i is the emotional support strategy adopted by the system in the i-th round of dialogue; in this embodiment, a total of 8 emotional support strategies are set: "asking questions", "retelling or paraphrasing", "emotional response", "self-disclosure", "affirmation and comfort", "providing suggestions", "obtaining information" and "others". Among them, in the early stage of the dialogue, supporters usually adopt exploratory strategies such as questions; after understanding the situation of the help-seeker, supporters tend to provide their opinions and suggestions; when the help-seeker is emotionally extreme, supporters will adopt comforting strategies such as empathy.
[0056] Step 2: Input each user utterance in the dialogue dataset into the common sense knowledge generator to obtain external common sense knowledge of each user utterance;
[0057] The embodiment of the present invention refers to the automatic common sense knowledge base construction model COMET, and selects a seed knowledge training set in the ATOMIC knowledge base for pre-training to construct a corresponding common sense knowledge generator.
[0058] like Figure 3 As shown, the user speech u i and a set of common sense relations N r The common sense knowledge generator generates multiple types of additional information such as psychological state and cognitive state in the form of text blocks, and obtains a common sense knowledge set to expand the information provided by the user's speech. The specific calculation formula is as follows:
[0059]
[0060] Among them, N r represents the common sense relationship set. There are five common sense relationships in the embodiment of the present invention: "psychological state", "cognitive state", "intention state", "demand state" and "influence state". j represents the jth specific relation, COMET represents the common sense knowledge generator, and B u Represents user utterance u i The corresponding set of common sense knowledge is
[0061] Splice the common sense knowledge set B i , obtain the common sense knowledge sequence k in the user's utterance i :
[0062]
[0063] Among them, each common sense knowledge is headed by its corresponding common sense relationship label. The correspondence between the common sense relationship and its label is: psychological state label [xReact], intention state label [xIntent], cognitive state label [xWant], demand state label [xNeed], and influence state label [xEffect].
[0064] Step 3: Based on each preprocessed dialogue data, obtain multiple rounds of dialogue history, strategy history, common sense knowledge, and the system response sequence to be generated. Figure 3 As shown in the figure, the multi-source encoder obtains the dialogue history features, strategy history features, and common sense knowledge features of each round of dialogue. The calculation process is as follows:
[0065] 1) According to each dialogue data in the dialogue data set, multiple rounds of dialogue data are obtained. For each round of the system's reply, the previous dialogue history between the user and the system is spliced together as the dialogue history H of this round i , and stitch together the supporting strategies adopted by the system as the strategy history S for this round i , expressed as:
[0066] H i =u1[EOS]r1[EOS]u2[EOS]r2…[EOS]u i-1 [EOS]r i-1 [EOS]u i [EOS]
[0067] S i =s1s2s3…s i-1
[0068] Among them, U i is the i-th round of dialogue history; S iis the strategy history for the i-th round; the label [EOS] indicates the interval or end of the sentence.
[0069] 2) Convert the dialogue history, policy history, and common sense knowledge in the form of labels into continuous vector representations to capture the semantic and syntactic information of the original labels:
[0070] H″i,S″ i ,K″ i =Embedding(H i ,S i ,K i )
[0071] Among them, the function of the Embedding function is to convert the input sequence into a continuously trainable embedding matrix.
[0072] 3) Use Transformer encoder to encode the dialogue history and capture the contextual information in the dialogue sequence:
[0073] H i ′ = TransformerEncoder(H i ″ )
[0074] Among them, TransformerEncoder is initialized using the pre-trained large language model BlenderBot. is the conversation history feature, d h is the hidden layer size of the encoder.
[0075] 4) Given a conversation history H i ′ , by focusing on the common sense knowledge related to the conversation history, we can obtain the common sense knowledge features related to the historical conversation content:
[0076] K′ i =TransformerEncoder(Softmax((K″ i ) T H′ i )K″ i )
[0077] Among them, Softmax is a commonly used method to convert high-dimensional vectors into probability distributions. is the common sense knowledge feature, d i is its characteristic dimension, . T Represents matrix transpose.
[0078] 5) Integrate the location features of the strategy history into the vector features:
[0079] S′i =S″ i +α*PositionEmbedding(S i )
[0080] Among them, PositionEmbedding is used to encode the position features of the strategy history, α is a trainable weight, For the strategy history characteristics, Its characteristic dimension.
[0081] Step 4: Figure 4 As shown, a heterogeneous graph network based on the attention mechanism is constructed to model the instance state and relationship information between the context features, historical strategy features and common sense knowledge features of each round of dialogue, and generate response strategy features for emotional support replies.
[0082] The heterogeneous graph network aggregates information from each node’s neighbors and obtains strategic features to guide supportive response generation based on a graph attention network:
[0083]
[0084] Among them, α t represents the weight of each node's neighbors, calculated by the attention mechanism; s t The response strategy features that guide the system's reply are obtained by concatenating the strategy node states output by the heterogeneous graph.
[0085] The specific structure of the HeteGraph heterogeneous graph network is as follows:
[0086] G=(v,ε)
[0087] Among them, v represents the four types of nodes in the graph, and ε represents the three types of edges in the graph.
[0088] The node information in the graph includes the user's historical discourse u i , System history reply i , the system's historical strategies i and external common sense knowledge k i , initialize the node features according to the corresponding encoding feature vector, expressed as:
[0089]
[0090] Wherein, N represents the number of conversation turns contained in the current conversation data.
[0091] The edge information in the graph, including conversation connections, strategy connections, and state connections, is expressed as:
[0092] 1) Dialogue connection: connect user utterances and system responses to each other to maintain the information flow between dialogues. The definition of this edge is as follows:
[0093] ε c ={(u i ,r i ),(r i ,u i+1 )|i∈[1,N-1]}
[0094] 2) Strategy connection: connect the strategy histories sequentially to maintain the information flow between strategy histories. The definition of this edge is as follows:
[0095] ε s ={(s i ,s i+1 )|i∈[1,N-2]}
[0096] 3) State connection: Connect the current user utterance with its corresponding common sense knowledge, and connect the historical system response with its corresponding support strategy to capture the state of the dialogue process. The definition of this edge is as follows:
[0097] ε state ={(u i ,k i ),(r j ,s j )|i∈[1,N],j∈[1,N-1]}
[0098] Wherein, N represents the number of conversation turns contained in the current conversation data.
[0099] The heterogeneous graph network uses a graph attention network (GATConv) to extract high-dimensional features of nodes and edges.
[0100] Specifically, the embodiment of the present invention adopts a three-layer heterogeneous graph network, that is, T=3, and the nodes in the t-th layer graph network are and The node will update its own state according to the edge and adjacent node states in the graph to obtain the node state of the t+1th layer. The update process uses the attention mechanism to determine the importance of adjacent nodes. The calculation process is as follows:
[0101]
[0102]
[0103] Where t={1,…,T}, N n v i The adjacent nodes of and is a trainable weight matrix, d is the dimension of the hidden layer, LeakyReLU is a nonlinear activation function, and ∥ represents a matrix concatenation operation.
[0104] The heterogeneous graph network obtains the response strategy features for emotional support replies by splicing all historical strategy nodes in the top-level graph.
[0105] Step 5: Use the bidirectional attention flow network to fully interact with the conversation history features H i ′ and response strategy characteristics Establish the flow from dialogue history to response strategy and the flow from response strategy to dialogue history. Specifically, the flow from dialogue history to response strategy shows that the model can use the contextual information in the dialogue history to select the response strategy related to the dialogue history; the flow from response strategy to dialogue history shows that the model can focus on the contextual features related to the supporting strategy adopted in the decoding stage, thereby generating a response constrained by the strategy. The calculation process of the flow from response strategy to dialogue history is as follows:
[0106]
[0107] in, Indicates the response strategy The transposed matrix of the j-th dimension historical strategy features.
[0108] Similarly, according to the dialogue history to response strategy flow, the response strategy feature s of the current round system is obtained N .
[0109] Step 6: Input the conversation history features related to the response strategy into the response generation decoder, generate a system response for the emotion support dialogue to be generated, and calculate the cross entropy loss value of the system response:
[0110] p(y z |y <z ,h′ i )=TransformerDecoder(W y<z ,h′ i )
[0111]
[0112] Among them, TransformerDecoder is initialized using the pre-trained large language model BlenderBot.
[0113] The response strategy related to the dialogue history is input into the supervised MLP model, the support strategy adopted by the system reply is fitted, and the negative log-likelihood loss value of the strategy selection task is calculated:
[0114] p(o)=Softmax(MLP(Mean-pooling(s N )))
[0115]
[0116] Among them, Mean-pooling is a commonly used pooling method, and MLP is a standard multi-layer perceptron.
[0117] Step 7: Fusing the loss value of the system response and the loss value of the strategy selection task, training the emotion support dialogue generation model until the model converges or reaches a specified number of training times:
[0118]
[0119] in, is the total loss value of the emotion support dialogue model.
[0120] Step 8: The processed conversation data is used as input to the converged model to predict emotionally supportive system responses.
[0121] Example 2
[0122] This embodiment provides an emotion support dialogue generation system based on a heterogeneous graph network, including:
[0123] The data acquisition module is used to collect and preprocess the dialogue data set; wherein any dialogue data includes multiple rounds of user speech and system responses, as well as the support strategies adopted by the system in each round of responses;
[0124] The data processing module is used to obtain multiple rounds of dialogue history, strategy history, common sense knowledge and the system response sequence to be generated based on each pre-processed dialogue data;
[0125] The knowledge enhancement module is used to input the user utterances of each round into the pre-built common sense knowledge generator to obtain the external common sense knowledge of each round;
[0126] The encoding module is used to encode the dialogue history sequence, strategy history sequence, and common sense knowledge sequence of the current round and convert them into a high-dimensional feature vector representation;
[0127] The heterogeneous graph network module is used to input the feature vectors of the current round of dialogue history, strategy history and common sense knowledge into the heterogeneous graph network to obtain the response strategy features used to guide the system reply generation;
[0128] A two-way feedback module is used to input the current round of dialogue history features and response strategy features into the two-way attention flow network to obtain the response strategy features related to the dialogue history and the dialogue history features related to the response strategy;
[0129] A supervised MLP model, used to input the response strategy related to the dialogue history into the supervised MLP model, fit the support strategy adopted by the system reply, and calculate the negative log-likelihood loss value of the strategy selection task;
[0130] A response generation decoder, used for inputting the dialogue history features related to the response strategy into the response generation decoder, generating a response for the emotion support dialogue to be generated, and calculating a cross entropy loss value of the system reply;
[0131] A training module, configured to select a loss value and a system response loss value according to a strategy, and train the emotion support dialogue model until convergence;
[0132] The prediction module takes the conversation data to be processed as the input of the converged model and predicts the corresponding emotional support dialogue system response.
[0133] Example 3
[0134] This embodiment provides a storage medium, including: a storage medium storing a computer program for generating emotion support dialogues based on a heterogeneous graph network, wherein the computer program enables a computer to execute a method for generating emotion support dialogues based on a heterogeneous graph network as described in Example 1.
[0135] Example 4
[0136] This embodiment provides an electronic device, including one or more processors and memories and one or more programs, wherein the one or more programs are stored in the memories and are configured to be executed by the one or more processors, and the programs are used to execute the method for generating emotion support dialogues based on heterogeneous graph networks described in Example 1.
[0137] In order to more fully verify and deeply understand the effectiveness of the embodiments of the present invention, experiments are carried out in this embodiment to verify the effectiveness of the embodiments of the present invention.
[0138] Specifically, in this embodiment, the public dataset ESConv is used to evaluate the emotion support dialogue generation technology based on the heterogeneous graph network. The dataset contains 1053 multi-round dialogue data. After preprocessing, 12266 dialogue data are obtained, of which 10598 are used as training sets, 834 are used as test sets, and 834 are used as validation sets.
[0139] Specifically, each piece of data consists of conversation history, strategy history, and common sense knowledge. Emotional support strategies include "asking questions", "retelling or paraphrasing", "emotional response", "self-disclosure", "affirmation and comfort", "providing advice", "obtaining information", and "others"; common sense knowledge categories include "psychological state", "cognitive state", "intention state", "need state", and "influence state".
[0140] Specifically, this embodiment is based on the ATOMIC pre-trained knowledge base, COMET common sense knowledge generator and BlenderBot pre-trained large language model, and uses the AdamW optimizer to minimize the total loss of the emotional support dialogue model. The initial learning rate is 3e-5, the learning rate warm-up step is 100 steps, the total number of training rounds is set to 10 rounds, and the model is fully trained once using all the data in the training set. The training batch size is set to 16, the batch size is 64 during verification, the encoder input sequence is up to 160, and the decoder input sequence is up to 40.
[0141] Specifically, the hardware configuration of the experiment in this embodiment is as shown in Table 1 below:
[0142] Table 1 Hardware Configuration Table
[0143] CPU Intel Xeon Gold 6130 GPU NVIDIA RTX A6000 48GB Memory 128GB Simulation Tools Python Modeling Platform Pytorch Platform
[0144] In Table 1, the CPU, GPU, memory, simulation tool, and modeling platform used in this embodiment are defined, which provides a hardware foundation for experimentally verifying the supportive response generation capability of the emotion support dialogue model.
[0145] In this embodiment, seven methods are introduced to verify the improvement of the response strategy prediction accuracy and supportive response generation capability of the emotional support model in the embodiment of the present invention, including:
[0146] Multi-Task Transformer: This model is a variant of Transformer, adding a task of predicting emotions;
[0147] MoEL: This model is a Transformer-based model that combines representations from multiple decoders to improve the empathy of responses;
[0148] MIME: This model is a Transformer-based model that uses emotion polarity and imitation for empathy generation;
[0149] BlenderBot-Joint: This model is an open-domain conversational agent that is fine-tuned to have multiple communication strategies and appends a special strategy token before the response utterance to represent the current round strategy;
[0150] GLHG: A global-to-local hierarchical graph network proposed in this paper to capture multi-source information and model hierarchical relationships for emotional support response generation;
[0151] MISC: This paper designs a hybrid strategy-aware model to introduce external knowledge to capture the user's immediate mental state and uses hybrid strategy representation to generate supportive responses;
[0152] FADO: This paper proposes a method for formulating strategic dispatch and generating strategically constrained responses in emotion-support dialogues.
[0153] In this embodiment, eight indicators are introduced to verify the reliability of the emotion support model in the embodiment of the present invention. The eight methods are: support strategy prediction accuracy (ACC), perplexity level (PPL), 2-gram-based BLEU evaluation (B-2), 3-gram-based BLEU evaluation (B-3), 4-gram-based BLEU evaluation (B-4), longest common subsequence (RL), 1-gram-based Distinct evaluation (D-
[0154] 1) and 2-gram based Distinct evaluation (D-2).
[0155] Table 2 Automatic evaluation indicators of the selected models on the ESConv dataset
[0156]
[0157] Specifically, a comparison of multiple indicators of this embodiment and other algorithms is shown in Table 2. As shown in Table 2, while most indicators of this embodiment are not inferior to the most popular models at present, its response strategy accuracy ACC, perplexity PPL and diversity D-2 indicators are better than the above 8 methods, proving the superiority of the present invention in predicting supportive response strategies.
[0158] The embodiments of the present invention improve the accuracy of response strategy prediction in the field of emotion support dialogues and the diversity and similarity of system responses. In the response generation process, external common sense knowledge in each round of user discourse is obtained, and the emotion support dialogue process is graph-structured. The high-level features of the dynamic change process of historical strategies are obtained through the graph attention mechanism, which solves the problem that the model cannot correctly obtain the relationship features between user status and historical strategies and contextual dialogues, resulting in the system adopting incorrect emotion support strategies and generating responses lacking emotion support.
[0159] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations of these embodiments including components are made without departing from the principles and spirit of the present invention, and still fall within the scope of protection of the present invention.
Claims
1. A method for generating emotional support dialogue based on heterogeneous graph networks, characterized in that: The method comprises the following steps: S1, collect and preprocess the dialogue dataset; S2, inputting each user utterance in the dialogue dataset into a common sense knowledge generator to obtain external common sense knowledge of each user utterance; S3. Based on each preprocessed dialogue data, obtain the dialogue history, strategy history, common sense knowledge and the system response to be generated in the form of multiple rounds of label sequences, and obtain the dialogue history features, strategy history features and common sense knowledge features of each round of dialogue through a multi-source encoder; S4. Constructing a heterogeneous graph network based on an attention mechanism, and modeling the graph structure information between the conversation history features, the strategy history features, and the common sense knowledge features through the heterogeneous graph network, so as to generate the response strategy features required for the emotional support reply; S5. Interacting the dialogue history features and the response strategy features through a bidirectional attention flow network, using the dialogue history features to select the response strategy related to the dialogue history, and using the response strategy features to select the dialogue history features related to the response strategy; S6, inputting the dialogue history features corresponding to the response strategy into the response generation decoder, generating a system response for the emotion support dialogue to be generated, and calculating the loss value of the system response; at the same time, inputting the response strategy features corresponding to the dialogue history into the supervised MLP model to fit the support strategy adopted by the system response, and calculating the loss value of the strategy selection task; S7. Fusing the loss value of the system response and the loss value of the strategy selection task, training the emotion support dialogue generation model formed by steps S1 to S6 until the emotion support dialogue generation model converges or reaches a specified number of training times.
2. The method for generating emotional support dialogue based on heterogeneous graph network according to claim 1, characterized in that: Any dialogue data in the dialogue data set includes multiple rounds of user utterances and system responses, as well as support strategies adopted in each round of system responses.
3. The method for generating emotional support dialogue based on heterogeneous graph network according to claim 1, characterized in that: The common sense generator is based on a pre-trained knowledge graph model, and the external common sense knowledge refers to inputting user discourse and common sense relationships into the common sense knowledge generator to obtain the user's psychological state and cognitive state in the form of text blocks to expand the information provided by the user discourse.
4. The method for generating emotional support dialogue based on heterogeneous graph network according to claim 3, characterized in that: The dialogue history is the concatenation of the user utterances of the current round and the previous round and the system responses of the previous round as the dialogue history of this round; the strategy history is the concatenation of the support strategies adopted by the system responses of the previous round as the strategy history of this round; the common sense knowledge is the set of external common sense knowledge obtained from the user utterances of the current round as the common sense knowledge of this round; The system response to be generated is to use the current round of system response as the system response to be generated in this round.
5. The method for generating emotional support dialogue based on heterogeneous graph network according to claim 4, characterized in that: In step S3, the method for obtaining multi-round label sequences is: converting the dialogue history, strategy history and common sense knowledge in character form into label sequence representation.
6. The method for generating emotional support dialogue based on heterogeneous graph network according to claim 1, characterized in that: The heterogeneous graph network includes four types of nodes: user's historical discourse, system's historical responses, system's historical strategies, and external common sense knowledge, and three types of edges: dialogue connections, strategy connections, and state connections.
7. The method for generating emotional support dialogue based on heterogeneous graph network according to claim 6, characterized in that: In step S4, the information of each node and its neighbors is aggregated through the heterogeneous graph network, and the strategy features guiding the generation of supportive responses are obtained based on the graph attention network: Among them, α t represents the weight of each neighbor of the node, which is calculated by the attention mechanism; s t The response strategy features that guide the system's reply are obtained by concatenating the strategy node states output by the heterogeneous graph; The HeteGraph heterogeneous graph network structure is as follows: in, Represents four types of nodes in the graph; ε represents three types of edges in the graph; The heterogeneous graph network uses a graph attention network to extract high-dimensional features of nodes and edges, and obtains response strategy features for emotional support dialogue by fusing the top-level historical strategy node features.
8. The method for generating emotional support dialogue based on heterogeneous graph network according to claim 7, characterized in that: In step S5, the process from the dialogue history to the response strategy shows that the bidirectional attention flow network uses the context information in the dialogue history to select the support strategy related to the dialogue history, and then constrains the response strategy characteristics according to the dialogue history; The flow from response strategy to dialogue history shows that the bidirectional attention flow network pays attention to the context related to the adopted strategy in the decoding stage, and then constrains the dialogue history features according to the response strategy.
9. The method for generating emotional support dialogue based on heterogeneous graph network according to claim 8, characterized in that: In step S6, The updated conversation history features are input into the decoder to obtain the conditional probability distribution of the prediction results, and the loss value of the system reply is calculated using the standard cross entropy loss function; The updated response strategy features are input into the supervised MLP model to predict the response strategy distribution, and the negative log-likelihood loss between it and the true strategy is calculated to obtain the loss value of the strategy selection task.
10. An emotion support dialogue generation system based on heterogeneous graph network, characterized in that: include: The data acquisition module is used to collect and preprocess the dialogue data set; wherein any dialogue data includes multiple rounds of user speech and system responses, as well as the support strategies adopted by the system in each round of responses; The data processing module is used to obtain multiple rounds of dialogue history, strategy history, common sense knowledge and the system response sequence to be generated based on each pre-processed dialogue data; The knowledge enhancement module is used to input the user utterances of each round into the pre-built common sense knowledge generator to obtain the external common sense knowledge of each round; The encoding module is used to encode the dialogue history sequence, strategy history sequence, and common sense knowledge sequence of the current round and convert them into a high-dimensional feature vector representation; The heterogeneous graph network module is used to input the feature vectors of the current round of dialogue history, strategy history and common sense knowledge into the heterogeneous graph network to obtain the response strategy features used to guide the system reply generation; A two-way feedback module is used to input the current round of dialogue history features and response strategy features into the two-way attention flow network to obtain the response strategy features related to the dialogue history and the dialogue history features related to the response strategy; A supervised MLP model, used to input the response strategy related to the dialogue history into the supervised MLP model, fit the support strategy adopted by the system reply, and calculate the negative log-likelihood loss value of the strategy selection task; A response generation decoder, used for inputting the dialogue history features related to the response strategy into the response generation decoder, generating a response for the emotion support dialogue to be generated, and calculating a cross entropy loss value of the system reply; A training module, configured to select a loss value and a system response loss value according to a strategy, and train the emotion support dialogue model until convergence; The prediction module takes the conversation data to be processed as the input of the converged model and predicts the corresponding emotional support dialogue system response.
11. A storage medium, characterized in that: It stores a computer program for an emotion support dialogue generation method based on a heterogeneous graph network, wherein the computer program enables a computer to execute an emotion support dialogue generation method based on a heterogeneous graph network as described in any one of claims 1 to 9.
12. An electronic device, characterized in that: include: One or more processors and memories and one or more programs, wherein the one or more programs are stored in the memories and are configured to be executed by the one or more processors, and the programs include a method for executing an emotion support dialogue generation method based on a heterogeneous graph network as described in any one of claims 1 to 9.
Citation Information
Cited By
IP network visual intercom emergency emotion recognition method and system based on artificial intelligence
CN120032409A
