A method for emotional dialogue generation based on multi-resolution emotions and multi-type knowledge

Abstract

The present invention discloses a method for generating an emotional dialogue based on multi-resolution emotions and multi-type knowledge, including receiving sentence texts in a dialogue and marking them; establishing a multi-resolution emotion module; the multi-resolution emotion module converts emotion words into vector representations to generate a final multi-resolution emotion context; establishing a multi-type knowledge interaction module; the multi-type knowledge interaction module constructs an emotion-enhanced context through joint interaction between the dialogue context and multi-type knowledge; generating an interactive vector and extracting an emotion signal; the decoder responds, generates an emotion response and constitutes an emotion dialogue together with the user input. The present invention uses multiple types of knowledge to clearly understand and express emotions in the generation of emotional dialogues, uses multiple types of knowledge to obtain an emotion-enhanced context map, uses multi-resolution emotions to better capture subtle changes in user emotions, and fuses multiple vectors and extracts emotion signals to guide the generation of emotional responses.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, and in particular relates to an emotional dialogue generation method based on multi-resolution emotions and multi-type knowledge. Background Art

[0002] Dialogue systems can be roughly divided into two categories: one is task-based dialogue systems, which mainly help users complete specific tasks in a specific field; the other is open-domain dialogue systems, also known as chatbots, which are characterized by chatting without purpose and without domain constraints. With the increasing demand for intelligent dialogue systems, more and more problems need to be addressed. Although dialogue systems have been widely used as an important way of human-computer interaction in recent years and have developed rapidly, existing dialogue systems focus on solving semantic consistency and content richness, while ignoring the impact of emotional state on response generation. Emotional dialogue is conducive to improving user satisfaction, but current research on improving human-computer interaction and human-computer resonance is not highly concerned.

[0003] Emotional dialogue is an important part of human dialogue. Humans have rich emotions, and emotions such as joy, anger, sadness, and happiness can be easily expressed in dialogue. As the most important manifestation of human behavior, emotion is of great significance to human daily communication. Emotional intelligence is also a key factor for the success of open domain dialogue systems. However, the existing related technologies still have the following problems: (1) The existing emotional dialogue model specifies the emotion label to generate a specific emotional response, but this can only generate responses with emotions and cannot make appropriate responses based on the user's emotions; (2) Dialogues that lack emotional dependence are difficult to resonate with humans, and it is difficult to use more natural language to generate responses to a certain question, and it is impossible to achieve the effect of simulating humans to have natural and meaningful conversations; (3) It is difficult to consider emotions naturally and coherently, and it is difficult to strike a balance between grammar and emotional expression; (4) Emotional expression is difficult to perceive, and is often implicit. Simply embedding emotional information into existing neural models cannot produce satisfactory emotions. Summary of the invention

[0004] The purpose of the present invention is to provide an emotional dialogue generation method based on multi-resolution emotions and multi-type knowledge, which can generate accurate and coherent emotional responses through multi-resolution emotions and multi-type knowledge.

[0005] The method for generating emotional dialogue based on multi-resolution emotions and multi-type knowledge provided by the present invention comprises the following steps:

[0006] S1. Receive the text of the sentences in the conversation and mark them;

[0007] S2. Establish a multi-resolution emotion module; the multi-resolution emotion module uses the word embedding layer, position embedding layer and emotion state embedding layer to convert the emotion words extracted from the dialogue sentence text into vector representations and generate the final multi-resolution emotion context;

[0008] S3. Establish a multi-type knowledge interaction module; the multi-type knowledge interaction module uses multi-type knowledge to enrich the dialogue sentence text and constructs the emotion-enhanced context through the joint interaction between the dialogue context and multi-type knowledge;

[0009] S4. Fusing the final multi-resolution emotion context generated by the multi-resolution emotion module with the final emotion knowledge enhanced dialogue context generated by the multi-type knowledge interaction module to generate an interactive vector;

[0010] S5. Extract emotional signals using interactive vectors;

[0011] S6. The decoder responds according to the emotional signal and generates an emotional response through a joint loss function; the generated emotional response and the user input together constitute an emotional dialogue.

[0012] The step S1 includes inputting a dialog context X=[X1, X2, ..., X I ,...,X M ], the common sense knowledge base ConceptNet and the sentiment dictionary NRC_VAD, a dialogue context consists of M sentences between two interlocutors, where X I represents the Ith sentence before the tag; concatenate the M sentences in the dialogue context X in order, use the GloVe word vector to represent each word and symbol in it, and get the tag sequence X = [x1, x2, ..., x i ,...,x m ], x i ∈X,x i represents the word vector of the i-th word after the token, and m represents the total number of words and symbols in the entire conversation context.

[0013] The step S2, establishing a multi-resolution emotion module, comprises the following steps:

[0014] A1. Use the sentiment dictionary NRC_VAD to extract fine-grained sentiment words from the conversation context X and generate the sentiment context μ; represent the sentiment context μ as the sentiment context embedding [w1,w2,...,w f ,...,w F ], where w f represents the sentiment embedding of the fth sentiment word, f∈1,2,...,F, where F represents the total number of sentiment words contained in the conversation context X;

[0015] A2. Use the word embedding layer, position embedding layer and emotional state embedding layer to represent each emotional word in the emotional context μ with a vector, and the emotional context is embedded in [w1,w2,...,w f ,...,w F ] includes word embedding, position embedding and sentiment state embedding, then w f It is expressed as: Among them, [;] represents vector concatenation; represents the fth sentiment word w f word embeddings; represents the fth sentiment word w f Position embedding; represents the fth sentiment word w f 's emotional state embedding; d represents the embedding dimension; R represents a real number;

[0016] A3. Encode the multi-resolution sentiment context μ using the Transformer encoder:

[0017]

[0018]

[0019] Among them, LayerNorm(·) represents the layer normalization operation; MHAtt(·) represents the multi-head self-attention layer composed of F attention heads; FFH(·) represents a two-layer feed-forward network with ReLU as the hidden activation function; Represents the fth sentiment word w output by the l-1th layer f Emotion embedding; l∈1,2,...,L, L represents the total number of stacked layers of Transformer encoder; represents the hidden state of the lth layer; represents the fth sentiment word w output by the lth layer f The Transformer encoder is stacked L times to represent the final multi-resolution emotional context as:

[0020] Calculate the attention output Attention(Φ,K,V):

[0021]

[0022] Where Φ represents the query matrix; K represents the key matrix, V represents the value matrix; d k is the dimension of the key matrix K;

[0023] The multi-head attention layer includes several adaptive dot product attentions, which map and concatenate the query matrix Φ, key matrix K, and value matrix V through several different linear transformations, and then pass through a linear layer to obtain the multi-head attention output MultiHead(Φ,K,V):

[0024] MultiHead(Φ,K,V)=Concat(head1,head2,...,head f ,...,head F )W O

[0025]

[0026] Where Φ represents the query matrix; K represents the key matrix, V represents the value matrix; Concat(·) represents the concatenation function; W O Represents the linear mapping weight matrix; head f represents the weighted matrix obtained by the module of the f-th attention head; Represents the weight matrix corresponding to the query matrix Φ; Represents the weight matrix corresponding to the key matrix K; Represents the weight matrix corresponding to the value matrix V.

[0027] The step S3, establishing a multi-type knowledge interaction module, includes the following steps:

[0028] B1. Use the common sense knowledge base ConceptNet and the sentiment dictionary NRC_VAD to enrich the conversation context; specifically, represent each tuple of the common sense knowledge base ConceptNet as τ = (x, r, c, s), where τ represents the tuple; x represents the start node; r represents the relationship; c represents the end node, and s represents the confidence score, s∈[1,10]; use min-max normalization to scale the features to the normalized confidence score (min-max)(s), where the value of (min-max)(s) is between 0 and 1:

[0029]

[0030] Wherein, (min-max)(·) indicates min-max normalization; min(s) indicates the lowest confidence score, min(s)=1; max(s) indicates the highest confidence score, max(s)=10;

[0031] The sentiment dictionary NRC_VAD is used to calculate the sentiment intensity values ​​of the queried words and related concepts:

[0032]

[0033] Among them, ||·||2 represents the L2 norm; V'(x i ) represents the word vector x of the i-th word after the tag i The value of the Valence dimension in the VAD vector; A'(x i ) represents the word vector x of the i-th word after the tag i The value of the Arousal dimension in the VAD vector;

[0034] B2. From the labeled sequence X = [x1, x2, ..., x i ,...,x m ], the token sequence X = [x1, x2, ..., x i ,...,x m' ], the word vector x for the i-th word i , select the concepts that are sentiment words and whose sentiment intensity values ​​are not lower than the set sentiment intensity threshold, and construct the word vector x of the i-th word after marking i Candidate tuples of Confidence value filtering, relationship filtering and concept sorting filtering are used to extract emotion-related knowledge; i ∈X, i∈1,2,...,m', m' represents the total number of non-stop words after the entire conversation token; Represents the word vector x of the i-th word after the tag i and the candidate tuple consisting of the k-th concept; Represents the word vector x of the i-th word after the tag i The relationship between the k-th concept; Indicates the i-th word x after the token i The k-th concept of Represents the word vector x of the i-th word after the tag i The confidence value of the k-th concept;

[0035] B3. Tuples filtered using concept sorting Enrich the conversation context and generate the final emotion knowledge enhanced conversation context C Q , and obtain the emotional context graph G enhanced with emotional knowledge.

[0036] In step B2: confidence value filtering, including the word vector x of the i-th word after tagging i The kth concept Perform confidence value filtering to filter out the word vector x of the i-th word after marking i The confidence value of the kth concept Tuples that are less than or equal to the preset confidence threshold α;

[0037] Relation filtering, including the word vector x of the i-th word after the tagi The kth concept Perform relationship filtering to filter out tuples with irrelevant relationships, including filtering out A tuple of L ir Represents a list of unrelated relations;

[0038] Concept sorting and filtering, including the word vector x of the i-th word after tagging i The kth concept Perform concept sorting and filtering according to the word vector x of the i-th word after tagging i Candidate tuples of The sentiment, semantics and confidence of each candidate tuple are used to calculate the comprehensive score

[0039]

[0040] in, Represents the word vector x of the i-th word after the tag i The kth concept The sentiment intensity value; cos(·) represents the cosine similarity function, which is used to calculate the word vector x of the i-th word after tagging i And the word vector x of the i-th word after tagging i The kth concept semantic similarity; Represents the word vector x of the i-th word after the tag i The confidence value of the k-th concept; according to the comprehensive score Arrange in descending order, and use the first K' candidate tuples as the word vector x of the i-th word after marking. i Sentiment knowledge, obtain the tuples after concept sorting and filtering

[0041] The step B3 comprises the following steps:

[0042] C1. The token sequence of the conversation context {x i} i＝1,2,...,m' ,concept and the sentence start marker CLS constitute the node N of the sentiment context graph G = {n q} q＝1,...,Q , Represents the word vector x of the i-th word after tagging i The kth concept of q Represents the word vector x of the i-th word after tagging i The qth node of ; m' represents the total number of non-stop words after the entire conversation tag; K' represents the word vector x of the i-th word after the conversation context tag iThe total number of concepts; Q represents the total number of nodes;

[0043] C2. Use the word embedding layer, position embedding layer and dialogue state embedding layer to embed the qth node n q Convert to the qth node n q Word embedding The qth node n q Position embedding and the qth node n q Conversation state embedding Among them, [;] represents vector concatenation;

[0044] C3. Use multi-head graph attention mechanism to update node representation with sentiment knowledge, and then use the adjacent vertices of the node to contextualize each node n. q :

[0045]

[0046]

[0047]

[0048] in, represents the concatenation of attention heads, F represents the total number of attention heads; A q Represents the qth node n in the adjacency matrix A q The subscript set of the neighboring nodes of θ represents the self-attention mechanism of the θth head; and Both represent linear transformation; d f represents the dimension of the f-th head; Represents the contextualized qth node n q vector; n b Represents the qth node n q The b-th node in the neighborhood of ; Represents the weight matrix corresponding to the value matrix; Indicates that the obtained weights are normalized using the softmax function; Represents the qth node n q The query matrix The corresponding weight matrix; Indicates that the qth node n in the adjacency matrix q The bond matrix k of the bth node in the neighborhood of b , or the qth node n in the adjacency matrix q The bond matrix k of the zth node in the neighborhood of z , the corresponding weight matrix;

[0049] C4. Use the Transformer layer to transform all nodes Global informatization, including:

[0050]

[0051]

[0052] in, represents the hidden state of the qth node in the lth layer; LayerNorm(·) represents the layer normalization operation; represents the vector of the qth node after the global informatization of the lth layer output; MHAtt(·) represents a multi-head self-attention layer, which includes F attention heads; FNN(·) represents a two-layer feedforward network with ReLU as the hidden activation function; l∈1,2,...,L, L represents the total number of stacked Transformer encoder layers, and the final emotional knowledge enhanced dialogue context is expressed as Represents the qth node n after emotional knowledge enhancement q The vector representation of , Q represents the total number of nodes;

[0053] In step S4, the interactive vector C is specifically: C = Concat (C F ,C Q ), Concat(·) represents the concatenation function; C F Represents the final multi-resolution emotional context; C Q Represents the final sentiment knowledge-enhanced conversation context.

[0054] The step S5 is to obtain the context of emotion enhancement:

[0055]

[0056] Among them, c newe Contexts that indicate emotional enhancement; Represents the qth node n q The sentiment intensity value; Q represents the total number of nodes; Represents the sentiment word w in the sentiment context q The emotional intensity value; Represents the qth node n after emotional knowledge enhancement q The vector representation of η nb Represents the qth node n q The bth node n in the neighborhood of b The emotional intensity value; Represents the qth sentiment word The bth node w in the neighborhood of bThe sentiment intensity value; F represents the total number of sentiment words in the sentiment context;

[0057] A linear layer with a softmax operation transforms the vector c newe Emotion category distribution P projected onto the coarse-grained emotion label e e In the example above, we obtain the intermediate sentiment vector e p , used to identify the emotional signals expressed by users; the intermediate emotional vector e p As an input to the decoder as an emotional feature, it is used to generate emotional responses:

[0058] e p =W e c newe +b e

[0059] P e (e|C)=softmax(e p )

[0060] Among them, W e ∈R q×d , b e ∈R q ;c newe Indicates the context of emotional enhancement; W e represents a real vector of q×d dimensions; b e represents a q-dimensional real vector; q represents the total number of sentiment categories; d represents the embedding dimension; C represents the interactive vector; softmax(·) represents the softmax regression function;

[0061] According to the true emotion label of the conversation context * , negative log-likelihood is used as the loss function for parameter learning during training: L emo = -log(P e (e * |C)), where L emo represents the negative log-likelihood loss function; log(·) represents the logarithm for easy calculation; P e represents the emotion category distribution on the coarse-grained emotion label e; C represents the interactive vector.

[0062] The step S6 comprises the following steps:

[0063] D1. The intermediate sentiment vector e p Transformed into a linear intermediate emotion vector e through linear transformation p ', where e p ∈R 1×q , e p '∈R 1×d , q represents the total number of emotion categories; R1×q represents a q-dimensional real vector; R 1×d represents a real vector of d dimensions;

[0064] D2. The linear intermediate sentiment vector e p 'With the decoder input token y 1:j-1 The embedding vector is connected as the input Y of the decoder emb ={y t} t＝0,1,…,j-1 , used for the t-th output; y0 represents the emotional signal input to the decoder, and y0 = e p ', the decoder input Y emb Input to the response generator with L Transformer layers, for each Transformer decoding layer, the decoder input Y emb Update to the new vector representation Y;

[0065] D3. Introducing emotional cross-attention mechanism to convert emotional signal c newe Integrate it into the response generation process and use the sentiment dependency vector generated by the sentiment cross-attention mechanism to generate a more appropriate sentiment-enhanced context vector:

[0066] D=Y+W[E-CatM(Y,C)]

[0067] E-CatM(Y,C)=[CMHAtt(Y,C)||c newe ]

[0068]

[0069]

[0070] Among them, the emotional cross-attention mechanism is used to replace the cross-attention sublayer in the Transformer decoding layer. The emotional cross-attention mechanism E-CatM combines the context vector and the emotional vector c generated by the cross multi-head attention mechanism CMHAtt newe Fusion, get the emotion dependency vector; D represents the vector of predicted response; Y represents the input vector of the decoder; W represents the linear mapping weight matrix; E-CatM(·) represents the emotion cross attention mechanism; C represents the interactive vector; CMHAtt(·) represents the cross multi-head attention mechanism; Represents the predicted sentiment vector after layer normalization operation; LayerNorm(·) represents the layer normalization operation; represents the predicted sentiment vector after the second layer normalization operation; FFN(·) represents a two-layer feed-forward network with ReLU as the hidden activation function;

[0071] D4. The response generator generates the distribution of the jth token over the vocabulary:

[0072]

[0073] in, represents the distribution of the jth word generated by the response generator over the vocabulary; y j represents the jth word; softmax(·) represents the softmax regression function; W o Represents the weight matrix corresponding to the j-th word generated; represents the jth word generated; y ＜j represents the first j-1 words; C represents the interactive vector;

[0074] D5. Copying nodes in the context graph And calculate and generate the jth word y j The probability distribution of is:

[0075]

[0076]

[0077] in, Indicates copying a node from the context graph The probability of; σ(·) represents the sigmoid activation function; W c Represents the jth word generated The corresponding weight matrix; b c represents the bias term; p(y j ) represents the final j-th word y j The probability distribution of represents the distribution of the jth word generated by the response generator over the vocabulary; represents the probability of copying the conversation context with rich sentiment knowledge, By attention weight The connection is obtained, Represents the qth node n after emotional knowledge enhancement q The vector representation of C represents the interactive vector;

[0078] D6. Use standard maximum likelihood estimation as the optimization objective:

[0079] L gen = -log p(y j |y ＜j ,C)

[0080] Among them, L genrepresents the standard maximum likelihood evaluation method; log(·) represents the logarithm, which is used for simple calculation; p(y j ) represents the final j-th word y j The probability distribution of y ＜j represents the first j-1 words; C represents the interactive vector;

[0081] D7. Define the joint loss function L g =βL emo +γL gen , and finally generate an emotional response Y={y1,y2,...,y n}, the emotional response generated by the system and the user's input together constitute the emotional dialogue; where β represents the control emotion generation loss function L emo The hyperparameter of the weight; γ represents the control response generation loss function L gen Hyperparameters of weights; y n Indicates the nth character in the generated dialogue.

[0082] The emotional dialogue generation method based on multi-resolution emotions and multi-type knowledge provided by the present invention uses multiple types of knowledge to clearly understand and express emotions in emotional dialogue generation, uses multi-type knowledge to obtain an emotion-enhanced context map, uses multi-resolution emotions to better capture subtle changes in user emotions, and fuses multi-resolution emotion representation vectors and emotion-enhanced context representation vectors after fusing multi-type knowledge to extract emotion signals to guide the generation of emotional responses. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 It is a schematic diagram of the process of the present invention.

[0084] Figure 2 Schematic diagram of the overall architecture of an embodiment of the present invention. DETAILED DESCRIPTION

[0085] like Figure 1 The structure diagram of the system of the present invention is as follows: The method for generating emotional dialogue based on multi-resolution emotions and multi-type knowledge provided by the present invention comprises the following steps:

[0086] S1. Receive the text of the sentences in the conversation and mark them;

[0087] S2. Establish a multi-resolution emotion module; the multi-resolution emotion module uses the word embedding layer, position embedding layer and emotion state embedding layer to convert the emotion words extracted from the dialogue sentence text into vector representations and generate the final multi-resolution emotion context;

[0088] S3. Establish a multi-type knowledge interaction module; the multi-type knowledge interaction module uses multi-type knowledge to enrich the dialogue sentence text and constructs the emotion-enhanced context through the joint interaction between the dialogue context and multi-type knowledge;

[0089] S4. Fusing the final multi-resolution emotion context generated by the multi-resolution emotion module with the final emotion knowledge enhanced dialogue context generated by the multi-type knowledge interaction module to generate an interactive vector;

[0090] S5. Extract emotional signals using interactive vectors;

[0091] S6. The decoder responds according to the emotional signal and generates an emotional response through a joint loss function; the generated emotional response and the user input together constitute an emotional dialogue.

[0092] The step S1 includes performing preliminary preprocessing operations on the data, converting all English characters into lowercase, and using the pre-trained Glove vector to initialize word embedding. Input dialogue context X = [X1, X2, ..., X I ,...,X M ], the common sense knowledge base ConceptNet and the sentiment dictionary NRC_VAD, a dialogue context consists of M sentences between two interlocutors, where X I represents the Ith utterance before the tag, I∈1,2,...,M, M represents the length of the entire dialogue context, that is, a multi-round dialogue context X consists of M sentences between two interlocutors; the M sentences in the dialogue context X are concatenated in sequence, and each word and symbol in it is represented by the Glove word vector, and the tag sequence X=[x1,x2,...,x i ,...,x m ], x i ∈X,x i represents the word vector of the i-th word after the token, and m represents the total number of words and symbols in the entire conversation context.

[0093] The step S2, establishing a multi-resolution emotion module, comprises the following steps:

[0094] A1. Use the sentiment dictionary NRC_VAD to extract fine-grained sentiment words from the conversation context X and generate the sentiment context μ; represent the sentiment context μ as the sentiment context embedding [w1,w2,...,w f ,...,w F ], where w f represents the sentiment embedding of the fth sentiment word, f∈1,2,...,F, where F represents the total number of sentiment words contained in the conversation context X;

[0095] A2. Use the word embedding layer, position embedding layer and emotional state embedding layer to represent each emotional word in the emotional context μ with a vector, and the emotional context is embedded in [w1,w2,...,w f ,...,w F ] includes word embedding, position embedding and sentiment state embedding, then w f It is expressed as: Among them, [;] represents vector concatenation; represents the fth sentiment word w f word embeddings; represents the fth sentiment word w f Position embedding; represents the fth sentiment word w f The emotional state embedding of the encoder facilitates the distinction between emotional context and emotional environment enhanced by multi-type knowledge; d represents the embedding dimension; R represents a real number;

[0096] A3. Encode the multi-resolution sentiment context μ using the Transformer encoder:

[0097]

[0098]

[0099] Among them, LayerNorm(·) represents the layer normalization operation; MHAtt(·) represents the multi-head self-attention layer composed of F attention heads; FFH(·) represents a two-layer feed-forward network with ReLU as the hidden activation function; Represents the fth sentiment word w output by the l-1th layer f sentiment embedding; l∈1,2,...,L, L represents the total number of stacked layers of Transformer encoder; h f represents the output of the lth layer; represents the fth sentiment word w output by the lth layer f The Transformer encoder is stacked L times to represent the final multi-resolution emotional context as:

[0100] Calculate the attention output Attention(Φ,K,V):

[0101]

[0102] Where Φ represents the query matrix; K represents the key matrix, and V represents the value matrix; the dimensions of Φ and K are d k ; The dimension of V is d v ;d kis the dimension of the key matrix K;

[0103] The multi-head attention layer includes several adaptive dot product attentions, that is, the query matrix, key matrix, and value matrix are mapped through several different linear transformations, and finally the different attention mechanisms are spliced ​​together, and then a linear layer is passed to obtain the multi-head attention output MultiHead(Φ,K,V):

[0104] MultiHead(Φ,K,V)=Concat(head1,head2,...,head f ,...,head F )W O

[0105]

[0106] Where Φ represents the query matrix; K represents the key matrix, V represents the value matrix; Concat(·) represents the concatenation function; W O Represents the linear mapping weight matrix; head f represents the weighted matrix obtained by the module of the f-th attention head; Represents the weight matrix corresponding to the query matrix Φ; Represents the weight matrix corresponding to the key matrix K; Represents the weight matrix corresponding to the value matrix V.

[0107] The step S3, establishing a multi-type knowledge interaction module, includes the following steps:

[0108] B1. Use the common sense knowledge base ConceptNet and the sentiment dictionary NRC_VAD as the knowledge source of the model, and then use sentiment knowledge to enrich the conversation context; specifically: ConceptNet is a large-scale language semantic graph that describes human common sense in natural language, aiming to help computers understand the meaning of words people use. The nodes in ConceptNet are concepts and the edges are relationships. Entities in the common sense knowledge base are represented in the form of triples.<concept1,relation,concept2> , each triple is an assertion, and each assertion is associated with a confidence score. For English data, ConceptNet contains 5.9M assertions, 38 relations, and 3.1M concepts. This method represents each tuple as τ = (x, r, c, s), where τ represents a tuple consisting of x, r, c, and s; x represents the start node (query); r represents the relation (edge); c represents the end node (concept), and s represents the confidence score, s∈[1,10]; min-max normalization is used to scale the features to the standardized confidence score (min-max)(s), where the value of (min-max)(s) is between 0 and 1:

[0109]

[0110] Wherein, (min-max)(·) indicates min-max normalization; min(s) indicates the lowest confidence score, min(s)=1; max(s) indicates the highest confidence score, max(s)=10;

[0111] The sentiment dictionary NRC_VAD is a list of English words and their VAD scores, which contains about 20k English words. The present invention uses NRC_VAD to calculate the sentiment intensity values ​​of sentiment words and related concepts:

[0112]

[0113] Among them, ||·||2 represents the L2 norm; V'(x i ) represents the word vector x of the i-th word after the tag i The value of the Valence dimension in the VAD vector; A'(x i ) represents the word vector x of the i-th word after the tag i The value of the Arousal dimension in the VAD vector; if the word vector x of the i-th word after the tag does not exist in the sentiment dictionary NRC_VAD i , then V'(x i ) and A'(x i ) are all set to 0.5;

[0114] B2. From the labeled sequence X = [x1, x2, ..., x i ,...,x m ], the token sequence X = [x1, x2, ..., x i ,...,x m' ], the word vector x for the i-th word i , select concepts that are sentiment words and whose sentiment intensity value is not less than 0.6, and construct the word vector x of the i-th word after marking i Candidate tuples of Confidence value filtering, relationship filtering and concept sorting filtering are used to extract emotion-related knowledge; i ∈X, i∈1,2,...,m', m' represents the total number of non-stop words after the entire conversation token; Represents the word vector x of the i-th word after the tag i and the candidate tuple consisting of the k-th concept; Represents the word vector x of the i-th word after the tag i The relationship between the k-th concept; Represents the word vector x of the i-th word after the tag i The k-th concept of Represents the word vector x of the i-th word after the tag i The confidence value of the k-th concept;

[0115] Confidence value filtering, including the word vector x of the i-th word after marking i The kth concept Perform confidence value filtering to filter out the word vector x of the i-th word after marking i The confidence value of the kth concept Tuples that are less than or equal to the preset confidence threshold α;

[0116] Relation filtering, including the word vector x of the i-th word after the tag i The kth concept Perform relation filtering to filter out tuples that are irrelevant to the relationship, that is, filter out A tuple of L irRepresents a list of irrelevant relations. In this embodiment, the list of irrelevant relations includes Antonym, NotDesires, FormOf, NotHasProperty, dbpedia, NotCapableOf, DistinctFrom, ExternalURL, SymbolOf, DerivedFrom, SymbolOf, CreatedBy, Synonym, MadeOf, EtymologicallyDerivedFrom, EtymologicallyRelatedTo, and AtLocation.

[0117] Concept sorting and filtering, including the word vector x of the i-th word after tagging i The kth concept Perform concept sorting and filtering according to the word vector x of the i-th word after tagging i Candidate tuples of The sentiment, semantics and confidence of each candidate tuple are used to calculate the comprehensive score

[0118]

[0119] in, Represents the word vector x of the i-th word after the tag i The kth concept The sentiment intensity value; cos(·) represents the cosine similarity function, which is used to calculate the word vector x of the i-th word after tagging i And the word vector x of the i-th word after tagging i The kth concept semantic similarity; Represents the word vector x of the i-th word after the tag i The confidence value of the k-th concept; according to the comprehensive score Arrange in descending order, and use the first K' candidate tuples as the word vector x of the i-th word after marking. i Sentiment knowledge, obtain the tuples after concept sorting and filtering In this embodiment, K'=10;

[0120] B3. Tuples filtered using concept sorting Enrich the conversation context and generate the final emotion knowledge enhanced conversation context C Q , to obtain the emotional context graph G enhanced by emotional knowledge, including the following steps:

[0121] C1. The token sequence of the conversation context {x i} i＝1,2,...,m' ,concept and the sentence start marker CLS constitute the node N of the sentiment context graph G = {n q} q＝1,...,Q , Represents the word vector x of the i-th word after tagging i The kth concept of q Represents the word vector x of the i-th word after tagging i The qth node of ; m' represents the total number of non-stop words after the entire conversation tag; K' represents the word vector x of the i-th word after the conversation context tag i The total number of concepts; Q represents the total number of nodes;

[0122] C2. Use the word embedding layer, position embedding layer and dialogue state embedding layer to embed the qth node n q Convert to Vector vector and vector Represents the qth node n q word embeddings; Represents the qth node n q Position embedding; Represents the qth node n q The dialogue state embedding is used to distinguish the nodes of different discourses in multi-round dialogues, that is, Among them, [;] represents vector concatenation; R represents a real number; d represents the embedding dimension. In a multi-round dialogue, there are multiple emotions in different rounds. Enables the model to distinguish between different rounds.

[0123] C3. Use multi-head graph attention mechanism to update node representation with sentiment knowledge, and then use the adjacent vertices of the node to contextualize each node n. q :

[0124]

[0125]

[0126]

[0127] in, represents the concatenation of attention heads, F represents the total number of attention heads; A q Represents the qth node n in the adjacency matrix A q The subscript set of the neighboring nodes of θ represents the self-attention mechanism of the θth head; and Both represent linear transformation; d f represents the dimension of the f-th head; Represents the contextualized qth node nq vector; n b Represents the qth node n q The b-th node in the neighborhood of ; Represents the weight matrix corresponding to the value matrix; Indicates that the obtained weights are normalized using the softmax function; Represents the qth node n q The query matrix The corresponding weight matrix; Indicates that the qth node n in the adjacency matrix q The bond matrix k of the bth node in the neighborhood of b , or the qth node n in the adjacency matrix q The bond matrix k of the zth node in the neighborhood of z , the corresponding weight matrix;

[0128] C4. Use the Transformer layer to transform all nodes Global informatization, that is, using global context information to update the representation of nodes and model nodes between different discourses, including:

[0129]

[0130]

[0131] in, represents the hidden state of the qth node in the lth layer; LayerNorm(·) represents the layer normalization operation; represents the vector of the qth node after the global informatization of the lth layer output; MHAtt(·) represents a multi-head self-attention layer, which includes F attention heads; FNN(·) represents a two-layer feedforward network with ReLU as the hidden activation function; l∈1,2,...,L, L represents the total number of stacked Transformer encoder layers, and the final emotional knowledge enhanced dialogue context is expressed as Represents the qth node n after emotional knowledge enhancement q The vector representation of , Q represents the total number of nodes;

[0132] In step S4, the interactive vector C is specifically: C = Concat (C F ,C Q ), Concat(·) represents the concatenation function; C F Represents the final multi-resolution emotional context; C Q Represents the final sentiment knowledge-enhanced conversation context.

[0133] In step S5, in order to perceive emotional information from the multi-resolution emotional context and the multi-type knowledge-rich context, it is necessary to obtain the emotional signal from the interactive vector fused by the two to identify the emotional information expressed by the user, and then guide the generation of emotional response, and use the weighted sum of all nodes in the emotional words in the multi-resolution emotional context and the multi-type knowledge-rich context to represent the emotional enhanced context:

[0134]

[0135] Among them, c newe Contexts that indicate emotional enhancement; Represents the qth node n q The sentiment intensity value; Q represents the total number of nodes; Represents the sentiment word w in the sentiment context q The emotional intensity value; Represents the qth node n after emotional knowledge enhancement q The vector representation of ; Represents the qth node n q The bth node n in the neighborhood of b The emotional intensity value; Represents the qth sentiment word The bth node w in the neighborhood of b The sentiment intensity value; F represents the total number of sentiment words in the sentiment context;

[0136] A linear layer with a softmax operation transforms the vector c newe Emotion category distribution P projected onto the coarse-grained emotion label e e In the example above, we obtain the intermediate sentiment vector e p , used to identify the emotional signals expressed by users; the intermediate emotional vector e p As an important emotional feature input to the decoder, it guides the generation of emotional response:

[0137] e p =W e c newe +b e

[0138] P e (e|C)=softmax(e p )

[0139] Among them, W e ∈R q×d , b e ∈R q ;c newe Indicates the context of emotional enhancement; W e represents a real vector of q×d dimensions; be represents a q-dimensional real vector; q represents the total number of sentiment categories; d represents the embedding dimension; C represents the interactive vector; softmax(·) represents the softmax regression function;

[0140] According to the true emotion label of the conversation context * , negative log-likelihood is used as the loss function for parameter learning during training: L emo = -log(P e (e * |C)), where L emo represents the negative log-likelihood loss function; log(·) represents the logarithm for easy calculation; P e represents the emotion category distribution on the coarse-grained emotion label e; C represents the interactive vector.

[0141] The step S6 comprises the following steps:

[0142] D1. The intermediate sentiment vector e p Transformed into a linear intermediate emotion vector e through linear transformation p ', where e p ∈R 1×q , e p '∈R 1×d , q represents the total number of emotion categories; R 1×q represents a q-dimensional real vector; R 1×d represents a d-dimensional real vector; d represents the embedding dimension;

[0143] D2. The linear intermediate sentiment vector e p 'With the decoder input token y 1:j-1 The embedding vector is connected as the input Y of the decoder emb ={y t} t＝0,1,...,j-1 , used for the t-th output y t ; y0 represents the emotional signal input to the decoder, and y0 = e p '; The decoder input Y emb Input to the response generator with L Transformer layers, that is, for each Transformer decoding layer, the decoder input Y emb Update to the new vector representation Y;

[0144] D3. Introducing emotional cross-attention mechanism to convert emotional signal c newe Integrate it into the response generation process and use the sentiment dependency vector generated by the sentiment cross-attention mechanism to generate a more appropriate sentiment-enhanced context vector:

[0145] D=Y+W[E-CatM(Y,C)]

[0146] E-CatM(Y,C)=[CMHAtt(Y,C)||c newe ]

[0147]

[0148]

[0149] Among them, the emotional cross-attention mechanism is used to replace the cross-attention sublayer in the Transformer decoding layer. The emotional cross-attention mechanism E-CatM combines the context vector and the emotional vector c generated by the cross multi-head attention mechanism CMHAtt newe Fusion, get the emotion dependency vector; D represents the vector of predicted response; Y represents the input vector of the decoder; W represents the linear mapping weight matrix; E-CatM(·) represents the emotion cross attention mechanism; C represents the interactive vector; CMHAtt(·) represents the cross multi-head attention mechanism; Represents the predicted sentiment vector after layer normalization operation; LayerNorm(·) represents the layer normalization operation; represents the predicted sentiment vector after the second layer normalization operation; FFN(·) represents a two-layer feed-forward network with ReLU as the hidden activation function;

[0150] D4. The response generator generates the distribution of the jth token over the vocabulary:

[0151]

[0152] in, represents the distribution of the jth word generated by the response generator over the vocabulary; y j represents the jth word; softmax(·) represents the softmax regression function; W o Represents the weight matrix corresponding to the j-th word generated; represents the jth word generated; y ＜j represents the first j-1 words; C represents the interactive vector;

[0153] D5. Copying nodes in the context graph And calculate and generate the jth word y j The probability distribution of is:

[0154]

[0155]

[0156] in, Indicates copying a node from the context graph The probability of; σ(·) represents the sigmoid activation function; W c Represents the jth word generated The corresponding weight matrix; b c represents the bias term; p(y j ) represents the final j-th word y j The probability distribution of represents the distribution of the jth word generated by the response generator over the vocabulary; represents the probability of copying the conversation context with rich sentiment knowledge, By attention weight The connection is obtained, Represents the qth node n after emotional knowledge enhancement q The vector representation of C represents the interactive vector;

[0157] D6. Use standard maximum likelihood estimation as the optimization objective:

[0158] L gen = -log p(y j |y ＜j ,C)

[0159] Among them, L gen represents the standard maximum likelihood evaluation method; log(·) represents the logarithm, which is used for simple calculation; p(y j ) represents the final j-th word y j The probability distribution of y ＜j represents the first j-1 words; C represents the interactive vector;

[0160] D7. Define the joint loss function L g =βL emo +γL gen , and finally generate an emotional response Y={y1,y2,...,y n}, the emotional response generated by the system and the user's input together constitute the emotional dialogue; where β represents the control emotion generation loss function L emo The hyperparameter of the weight; γ represents the control response generation loss function L gen Hyperparameters of weights; y n Indicates the nth character in the generated dialogue.

[0161] like Figure 2Schematic diagram of the overall architecture of an embodiment of the present invention. In this embodiment, an English empathic dialogue example from the EMPATHETIC DIALOGUES dataset is selected for display. The dataset contains coarse-grained emotion tags and multi-round empathic dialogues, and contains a total of 25k one-to-one open domain dialogues. The EMPATHETIC DIALOGUES dataset contains 32 evenly distributed emotion tags (including surprised, excited, annoyed, proud, angry, sad, grateful, lonely, impressed, afraid, disgusted, confident, terrified, hopeful, anxious, disappointed, joyful, prepared, guilty, furious, nostalgic, jealous, anticipating, embarrassed, content, devastated, sentimental, caring, trusting, ashamed, apprehensive, faithful).

[0162] The sentiment words in the context are extracted using the NRC_VAD external sentiment dictionary to represent fine-grained sentiment. The common sense knowledge base ConceptNet and the sentiment dictionary NRC_VAD are used as the knowledge sources of the model, and then the sentiment knowledge is used to enrich the conversation context.

[0163] During training, the conversation context and coarse-grained emotion labels are input into the system, where the coarse-grained emotion labels of the conversation context are used as supervisory signals; during testing, only the conversation context is input into the system, and the empathy capabilities of all conversation models are evaluated. According to the above steps S1-S6, a large number of experiments are carried out on a benchmark data set. This embodiment uses two automatic evaluation indicators to evaluate the model: Accuracy (emotional accuracy) and Perplexity (perplexity). Perplexity measures the high-level quality of the generated model. The lower the perplexity value, the better the effect. Emotional accuracy evaluates the model at the emotional level. It reflects the consistency between the basic real emotion label and the emotion label predicted by the generator. The higher the accuracy value, the better the effect. Experimental results show that the emotional dialogue generation model of this embodiment achieves better results than existing models.

[0164] Table 1 Automatic index evaluation results of baseline and proposed model

[0165] Model Accuracy Perplexity Transformer - 37.73 EmoPrepend-1 0.3452 38.30 MK-EDG 0.3711 37.19 MKR-EDG (the present invention) 0.3888 37.14

[0166] It can be observed from Table 1 that the present invention outperforms the baseline model in terms of accuracy and perplexity evaluation indicators, and significantly exceeds the latest baseline model MK-EDG in terms of accuracy. This significant improvement shows that the framework of multi-resolution emotion and multi-type knowledge fusion is more effective in emotional expression, so the emotional dialogue generation model proposed in the present invention is effective. The beneficial effects of the present invention are as follows: This method is suitable for the analysis of English emotional dialogue data, and helps to use external common sense knowledge bases and emotional vocabulary to obtain context-aware emotion-related common sense, enrich the dialogue context, and promote the detection of emotions in the dialogue, so as to understand the user's feelings and make appropriate responses. In the encoding process, the present method encodes the acquired multi-resolution emotional context and the emotion-enhanced context graph of multi-type knowledge fusion, uses the obtained emotion-enhanced context representation vector to extract emotional signals, guides the generation of responses, and the decoder decodes the emotional context that integrates the emotional context and multi-type knowledge enhancement to generate a natural emotional response.

Claims

1. A method for generating emotional dialogue based on multi-resolution emotions and multi-type knowledge, characterized in that The steps include: S1. Receive the text of the sentences in the conversation and mark them; S2. Establish a multi-resolution emotion module; the multi-resolution emotion module uses the word embedding layer, position embedding layer and emotion state embedding layer to convert the emotion words extracted from the dialogue sentence text into vector representations and generate the final multi-resolution emotion context; S3. Establish multi-type knowledge interaction modules; The multi-type knowledge interaction module enriches the dialogue text with multi-type knowledge and constructs the emotion-enhanced context through the joint interaction between the dialogue context and multi-type knowledge; S4. Fusing the final multi-resolution emotion context generated by the multi-resolution emotion module with the final emotion knowledge enhanced dialogue context generated by the multi-type knowledge interaction module to generate an interactive vector; S5. Extract emotional signals using interactive vectors; S6. The decoder responds according to the emotional signal and generates an emotional response through a joint loss function; the generated emotional response and the user input together constitute an emotional dialogue.

2. The method for generating emotional dialogue based on multi-resolution emotions and multi-type knowledge according to claim 1 is characterized in that The step S1 includes inputting a dialog context X=[X1, X2, ..., X I ,...,X M ], the common sense knowledge base ConceptNet and the sentiment dictionary NRC_VAD, a dialogue context consists of M sentences between two interlocutors, where X I represents the Ith sentence before the tag; concatenate the M sentences in the dialogue context X in order, use the GloVe word vector to represent each word and symbol in it, and get the tag sequence X = [x1, x2, ..., x i ,...,x m ], x i ∈X,x i represents the word vector of the i-th word after the token, and m represents the total number of words and symbols in the entire conversation context.

3. The method for generating emotional dialogue based on multi-resolution emotions and multi-type knowledge according to claim 2 is characterized in that The step S2, establishing a multi-resolution emotion module, comprises the following steps: A1. Use the sentiment dictionary NRC_VAD to extract fine-grained sentiment words from the conversation context X and generate the sentiment context μ; represent the sentiment context μ as the sentiment context embedding [w1,w2,...,w f ,...,w F ], where w f represents the sentiment embedding of the fth sentiment word, f∈1,2,...,F, where F represents the total number of sentiment words contained in the conversation context X; A2. Use the word embedding layer, position embedding layer and emotional state embedding layer to represent each emotional word in the emotional context μ with a vector, and the emotional context is embedded in [w1,w2,...,w f ,...,w F ] includes word embedding, position embedding and sentiment state embedding, then w f It is expressed as: Among them, [;] represents vector concatenation; represents the fth sentiment word w f word embeddings; represents the fth sentiment word w f Position embedding; represents the fth sentiment word w f 's emotional state embedding; d represents the embedding dimension; R represents a real number; A3. Encode the multi-resolution sentiment context μ using the Transformer encoder: Among them, LayerNorm(·) represents the layer normalization operation; MHAtt(·) represents the multi-head self-attention layer composed of F attention heads; FFH(·) represents a two-layer feed-forward network with ReLU as the hidden activation function; Represents the fth sentiment word w output by the l-1th layer f Emotion embedding; l∈1,2,...,L, L represents the total number of stacked layers of Transformer encoder; represents the hidden state of the lth layer; represents the fth sentiment word w output by the lth layer f The Transformer encoder is stacked L times to represent the final multi-resolution emotional context as: Calculate the attention output Attention(Φ,K,V): Where Φ represents the query matrix; K represents the key matrix, V represents the value matrix; d k is the dimension of the key matrix K; The multi-head attention layer includes several adaptive dot product attentions, which map and concatenate the query matrix Φ, key matrix K, and value matrix V through several different linear transformations, and then pass through a linear layer to obtain the multi-head attention output MultiHead(Φ,K,V): MultiHead(Φ,K,V)=Concat(head1,head2,...,head f ,...,head F )W O Where Φ represents the query matrix; K represents the key matrix, V represents the value matrix; Concat(·) represents the concatenation function; W O Represents the linear mapping weight matrix; head f represents the weighted matrix obtained by the module of the f-th attention head; Represents the weight matrix corresponding to the query matrix Φ; Represents the weight matrix corresponding to the key matrix K; Represents the weight matrix corresponding to the value matrix V.

4. The method for generating emotional dialogue based on multi-resolution emotions and multi-type knowledge according to claim 3 is characterized in that The step S3, establishing a multi-type knowledge interaction module, includes the following steps: B1. Use the common sense knowledge base ConceptNet and the sentiment dictionary NRC_VAD to enrich the conversation context; specifically, represent each tuple of the common sense knowledge base ConceptNet as τ = (x, r, c, s), where τ represents the tuple; x represents the start node; r represents the relationship; c represents the end node, and s represents the confidence score, s∈[1,10]; use min-max normalization to scale the features to the normalized confidence score (min-max)(s), where the value of (min-max)(s) is between 0 and 1: Wherein, (min-max)(·) indicates min-max normalization; min(s) indicates the lowest confidence score, min(s)=1; max(s) indicates the highest confidence score, max(s)=10; The sentiment dictionary NRC_VAD is used to calculate the sentiment intensity values ​​of the queried words and related concepts: Among them, ||·||2 represents the L2 norm; V'(x i ) represents the word vector x of the i-th word after the tag i The value of the Valence dimension in the VAD vector; A'(x i ) represents the word vector x of the i-th word after the tag i The value of the Arousal dimension in the VAD vector; B2. From the labeled sequence X = [x1, x2, ..., x i ,...,x m ], the token sequence X = [x1, x2, ..., x i ,...,x m' ], the word vector x for the i-th word i , select the concepts that are sentiment words and whose sentiment intensity values ​​are not lower than the set sentiment intensity threshold, and construct the word vector x of the i-th word after marking i Candidate tuples of Confidence value filtering, relationship filtering and concept sorting filtering are used to extract emotion-related knowledge; i ∈X, i∈1,2,...,m', m' represents the total number of non-stop words after the entire conversation token; Represents the word vector x of the i-th word after the tag i and the candidate tuple consisting of the k-th concept; Represents the word vector x of the i-th word after the tag i The relationship between the k-th concept; Indicates the i-th word x after the token i The k-th concept of Represents the word vector x of the i-th word after the tag i The confidence value of the k-th concept; B3. Tuples filtered using concept sorting Enrich the conversation context and generate the final emotion knowledge enhanced conversation context C Q , and obtain the emotional context graph G enhanced with emotional knowledge.

5. The method for generating emotional dialogue based on multi-resolution emotions and multi-type knowledge according to claim 4 is characterized in that In step B2: confidence value filtering, including the word vector x of the i-th word after tagging i The kth concept Perform confidence value filtering to filter out the word vector x of the i-th word after marking i The confidence value of the kth concept Tuples that are less than or equal to the preset confidence threshold α; Relation filtering, including the word vector x of the i-th word after the tag i The kth concept Perform relationship filtering to filter out tuples with irrelevant relationships, including filtering out A tuple of L ir Represents a list of unrelated relations; Concept sorting and filtering, including the word vector x of the i-th word after tagging i The kth concept Perform concept sorting and filtering according to the word vector x of the i-th word after tagging i Candidate tuples of The sentiment, semantics and confidence of each candidate tuple are used to calculate the comprehensive score in, Represents the word vector x of the i-th word after the tag i The kth concept The sentiment intensity value; cos(·) represents the cosine similarity function, which is used to calculate the word vector x of the i-th word after tagging i And the word vector x of the i-th word after tagging i The kth concept semantic similarity; Represents the word vector x of the i-th word after the tag i The confidence value of the k-th concept; according to the comprehensive score Arrange in descending order, and use the first K' candidate tuples as the word vector x of the i-th word after marking. i Sentiment knowledge, obtain the tuples after concept sorting and filtering 6. The method for generating emotional dialogue based on multi-resolution emotions and multi-type knowledge according to claim 5 is characterized in that The step B3 comprises the following steps: C1. The token sequence of the conversation context {x i } i＝1,2,...,m' ,concept and the sentence start marker CLS constitute the node N of the sentiment context graph G = {n q } q＝1,...,Q , Represents the word vector x of the i-th word after tagging i The kth concept of q Represents the word vector x of the i-th word after tagging i The qth node of ; m' represents the total number of non-stop words after the entire conversation is marked; K' represents the word vector x of the i-th word after the conversation context is marked i The total number of concepts; Q represents the total number of nodes; C2. Use the word embedding layer, position embedding layer and dialogue state embedding layer to embed the qth node n q Convert to the qth node n q Word embedding The qth node n q Position embedding and the qth node n q Conversation state embedding Among them, [;] represents vector concatenation; C3. Use multi-head graph attention mechanism to update node representation with sentiment knowledge, and then use the adjacent vertices of the node to contextualize each node n. q : in, represents the concatenation of attention heads, F represents the total number of attention heads; A q Represents the qth node n in the adjacency matrix A q The subscript set of the neighboring nodes of θ represents the self-attention mechanism of the θth head; and Both represent linear transformation; d f represents the dimension of the f-th head; Represents the contextualized qth node n q vector; n b Represents the qth node n q The b-th node in the neighborhood of ; Represents the weight matrix corresponding to the value matrix; Indicates that the obtained weights are normalized using the softmax function; Represents the qth node n q The query matrix The corresponding weight matrix; Indicates that the qth node n in the adjacency matrix q The bond matrix k of the bth node in the neighborhood of b , or the qth node n in the adjacency matrix q The bond matrix k of the zth node in the neighborhood of z , the corresponding weight matrix; C4. Use the Transformer layer to transform all nodes Global informatization, including: in, represents the hidden state of the qth node in the lth layer; LayerNorm(·) represents the layer normalization operation; represents the vector of the qth node after the global informatization of the lth layer output; MHAtt(·) represents a multi-head self-attention layer, which includes F attention heads; FNN(·) represents a two-layer feedforward network with ReLU as the hidden activation function; l∈1,2,...,L, L represents the total number of stacked Transformer encoder layers, and the final emotional knowledge enhanced dialogue context is expressed as Represents the qth node n after emotional knowledge enhancement q The vector representation of , Q represents the total number of nodes; 7. The method for generating emotional dialogue based on multi-resolution emotions and multi-type knowledge according to claim 6 is characterized in that In step S4, the interactive vector C is specifically: C = Concat (C F ,C Q ), Concat(·) represents the concatenation function; C F Represents the final multi-resolution emotional context; C Q Represents the final sentiment knowledge-enhanced conversation context.

8. The method for generating emotional dialogue based on multi-resolution emotions and multi-type knowledge according to claim 7, characterized in that The step S5 is to obtain the context of emotion enhancement: Among them, c newe Contexts that indicate emotional enhancement; Represents the qth node n q The sentiment intensity value; Q represents the total number of nodes; Represents the sentiment word w in the sentiment context q The emotional intensity value; Represents the qth node n after emotional knowledge enhancement q The vector representation of ; Represents the qth node n q The bth node n in the neighborhood of b The emotional intensity value; Represents the qth sentiment word The bth node w in the neighborhood of b The sentiment intensity value; F represents the total number of sentiment words in the sentiment context; A linear layer with a softmax operation transforms the vector c newe Emotion category distribution P projected onto the coarse-grained emotion label e e In the example above, we obtain the intermediate sentiment vector e p , used to identify the emotional signals expressed by users; the intermediate emotional vector e p As an input to the decoder as an emotional feature, it is used to generate emotional responses: e p =W e c newe +b e P e (e|C)=softmax(e p ) Among them, W e ∈R q×d , b e ∈R q ;c newe Indicates the context of emotional enhancement; W e represents a real vector of q×d dimensions; b e represents a q-dimensional real vector; q represents the total number of sentiment categories; d represents the embedding dimension; C represents the interactive vector; softmax(·) represents the softmax regression function; According to the true emotion label of the conversation context * , negative log-likelihood is used as the loss function for parameter learning during training: L emo = -log(P e (e * |C)), where L emo represents the negative log-likelihood loss function; log(·) represents the logarithm for easy calculation; P e represents the emotion category distribution on the coarse-grained emotion label e; C represents the interactive vector.

9. The method for generating emotional dialogue based on multi-resolution emotions and multi-type knowledge according to claim 8, characterized in that The step S6 comprises the following steps: D1. The intermediate sentiment vector e p Transformed into a linear intermediate emotion vector e through linear transformation p ', where e p ∈R 1×q , e p '∈R 1×d , q represents the total number of emotion categories; R 1×q represents a q-dimensional real vector; R 1×d represents a real vector of d dimensions; D2. The linear intermediate sentiment vector e p 'With the decoder input token y 1:j-1 The embedding vector is connected as the input Y of the decoder emb ={y t } t＝0,1,...,j-1 , used for the t-th output; y0 represents the emotional signal input to the decoder, and y0 = e p ', the decoder input Y emb Input to the response generator with L Transformer layers, for each Transformer decoding layer, the decoder input Y emb Update to the new vector representation Y; D3. Introducing emotional cross-attention mechanism to convert emotional signals into newe Integrate it into the response generation process and use the sentiment dependency vector generated by the sentiment cross-attention mechanism to generate a more appropriate sentiment-enhanced context vector: D=Y+W[E-CatM(Y,C)] E-CatM(Y,C)=[CMHAtt(Y,C)||c newe ] Among them, the emotional cross-attention mechanism is used to replace the cross-attention sublayer in the Transformer decoding layer. The emotional cross-attention mechanism E-CatM combines the context vector and the emotional vector c generated by the cross multi-head attention mechanism CMHAtt newe Fusion, get the emotion dependency vector; D represents the vector of predicted response; Y represents the input vector of the decoder; W represents the linear mapping weight matrix; E-CatM(·) represents the emotion cross attention mechanism; C represents the interactive vector; CMHAtt(·) represents the cross multi-head attention mechanism; Represents the predicted sentiment vector after layer normalization operation; LayerNorm(·) represents the layer normalization operation; represents the predicted sentiment vector after the second layer normalization operation; FFN(·) represents a two-layer feed-forward network with ReLU as the hidden activation function; D4. The response generator generates the distribution of the jth token over the vocabulary: in, represents the distribution of the jth word generated by the response generator over the vocabulary; y j represents the jth word; softmax(·) represents the softmax regression function; W o Represents the weight matrix corresponding to the j-th word generated; represents the jth word generated; y<j represents the first j-1 words; C represents the interactive vector; D5. Copying nodes in the context graph And calculate and generate the jth word y j The probability distribution of is: in, Indicates copying a node from the context graph The probability of; σ(·) represents the sigmoid activation function; W c Represents the jth word generated The corresponding weight matrix; b c represents the bias term; p(y j ) represents the final j-th word y j The probability distribution of represents the distribution of the jth word generated by the response generator over the vocabulary; represents the probability of copying the conversation context with rich sentiment knowledge, By attention weight The connection is obtained, Represents the qth node n after emotional knowledge enhancement q The vector representation of C represents the interactive vector; D6. Use standard maximum likelihood estimation as the optimization objective: L gen =-logp(y j |y<j,C) Among them, L gen represents the standard maximum likelihood evaluation method; log(·) represents the logarithm, which is used for simple calculation; p(y j ) represents the final j-th word y j The probability distribution of ; y<j represents the first j-1 words; C represents the interactive vector; D7. Define the joint loss function L g =βL emo +γL gen , and finally generate an emotional response Y={y1,y2,...,y n }, the emotional response generated by the system and the user's input together constitute the emotional dialogue; where β represents the control emotion generation loss function L emo The hyperparameter of the weight; γ represents the control response generation loss function L gen Hyperparameters of weights; y n Indicates the nth character in the generated dialogue.

Images