An emotion-based multi-turn dialogue model and dialogue method

Abstract

The application discloses a multi-turn dialogue model and a dialogue method based on emotion, and the multi-turn dialogue model comprises a dialogue history embedding module, an emotion prediction module and an emotion embedding module; the dialogue history embedding module comprises a word level encoder and a dialogue history self-attention module, and is used for inputting dialogue history information into the multi-turn dialogue model; different dialogue history information represents different multi-turn dialogues; the emotion prediction module is used for predicting the emotion of a reply, and the emotion category predicted by the emotion prediction module replaces the manually specified emotion category; the emotion prediction module obtains the emotion category through dialogue history, wherein the input of the emotion prediction module is a sentence vector of a dialogue history, and the emotion prediction module outputs an emotion category; the emotion embedding module is used for generating a reply and embedding a specified emotion type in the reply, the embedded emotion type is provided by the emotion prediction module, and finally the multi-turn dialogue model generates a reply with emotion, thereby improving the quality of the reply.

Description

Technical Field

[0001] This invention relates to the fields of multi-turn dialogue and deep learning technology, and in particular to an emotion-based multi-turn dialogue modeling method. Background Technology

[0002] With the rapid development of artificial intelligence technology, computing power, and computing infrastructure, the possibilities for intelligent dialogue systems are gradually increasing. Currently, dialogue systems are becoming increasingly prevalent in social life. Various intelligent assistants on smart terminal devices can complete simple tasks based on user input (including voice and text), such as checking the weather, making phone calls, adding schedules, and engaging in simple conversation. Intelligent customer service systems in various industries can also provide detailed explanations of solutions and terms based on user inquiries. These intelligent dialogue systems not only meet user needs but also save platform providers significant manpower costs and improve overall productivity.

[0003] While all dialogue systems essentially respond to user input, they can be broadly categorized into two types based on their specific applications: task-oriented dialogue systems and open-domain dialogue systems. Task-oriented dialogue systems require users to complete a specific task through dialogue. Open-domain dialogue systems, unlike other applications, do not aim to help users complete a specific task but rather interact with users through coherent and engaging dialogue. The responses generated by these systems need to be as close as possible to human responses. [1] .

[0004] Compared to task-oriented dialogue systems, open-domain dialogue systems have more abstract goals, and the responses generated by the system lack clear standard answers. The lack of explicit mathematical evaluation criteria makes system optimization more difficult. Therefore, open-domain dialogue systems are more challenging, requiring a deep understanding of user context and emotional needs. Furthermore, because the responses generated by open-domain dialogue systems are context-dependent, consistency must be considered regarding dialogue history and the speaker's personality and attributes. While the dialogue effectiveness of open-domain dialogue systems has significantly improved, there is still a gap between the system-generated responses and human responses. Therefore, research on open-domain dialogue systems is more difficult, and their application scope is relatively smaller. However, they are not independent of task-oriented dialogue systems, which can also have everyday conversational functions. Researching open-domain dialogue systems can improve the user experience of dialogue systems.

[0005] Compared to single-turn dialogues, multi-turn dialogues can simulate human language behavior in multi-turn conversations. These models need to consider contextual information, dialogue history, and other factors to determine the dialogue system's responses. Furthermore, multi-turn dialogues more closely resemble real-life human conversations. Recent work on multi-turn dialogue generation primarily incorporates dialogue history into the model. In 2016, Serban et al.... [5] The HRED model was proposed, which differs from ordinary dialogue models by using recurrent neural networks to fully consider contextual information. It was first proposed in 2017 by Vaswani et al. [6] After proposing the Transformer model, multi-turn dialogue systems began to function. [8,9] Most consider using attention mechanisms for processing.

[0006] Secondly, emotional communication is an important issue in human-computer dialogue systems. Numerous studies have shown... [2,3 [4] In human-computer dialogue systems, besides the content of the responses, emotional communication between machines and humans is also an important issue. Humans can express their emotions through language to gain comfort. For a dialogue system to engage in effective emotional communication with humans, it must possess emotional understanding capabilities, including recognizing user emotions and expressing appropriate emotions. If a dialogue system lacks emotional elements, the user will find the responses very stiff. Therefore, enabling machines to possess the ability to understand and express emotions is a new opportunity and challenge in the field of human-computer dialogue. Emotional dialogue systems, exemplified by Zhou H et al. (2018)...

[10] The proposed ECM model, for example, can mostly generate responses with specific emotions by specifying the emotions involved.

[0007] Incorporating emotional factors into multi-turn dialogue systems and learning from emotional changes during these interactions allows the system to generate responses that are more human-like, thereby improving the user experience. Furthermore, learned emotional changes, compared to generating responses based on specified emotions, are more consistent with the patterns of human conversation and contribute to more natural responses from the dialogue system.

[0008] [References]

[0009] [1]Chen H, Liu

[0010] [2]Hazarika D,Poria S,Mihalcea R,et al.Icon:Interactiveconversational memory network for multimodal emotion detection[C] / / Proceedings of the 2018conference on empirical methods in natural languageprocessing.2018:2594-2604.

[0011] [3]Zhou H,Huang M,Zhang T,et al.Emotional chatting machine:Emotionalconversation generation with internal and external memory[C] / / Proceedings ofthe AAAI Conference on Artificial Intelligence.2018,32(1).

[0012] [4]Asghar N,Poupart P,Hoey J,et al.Affective neural responsegeneration[C] / / European Conference on Information Retrieval.Springer,Cham,2018:154-166.

[0013] [5]Serban I,Sordoni A,Bengio Y,et al.Building end-to-end dialoguesystems using generative hierarchical neural network models[C] / / Proceedingsof the AAAI Conference on Artificial Intelligence.2016,30(1).

[0014] [6]Vaswani A,Shazeer N,Parmar N,et al.Attention is all you need[J].Advances in neural information processing systems,2017,30.

[0015] [7]Sun B,Feng S,Li Y,et al.Generating relevant and coherent dialogueresponses using self-separated conditional variational autoencoders[J].arXivpreprint arXiv:2106.03410,2021.

[0016] [8]Paulhus,D.L.,Vazire,S.2007.The self-report method.Handbook ofresearch methods in personality psychology,1,224-239.

[0017] [9]Xu J,Lei Z,Wang H,et a1.Discovering dialog structure graph forcoherent dialog generation[C] / / Proceedings of the 59th Annual Meeting of theAssociation for Computational Linguistics and the llth International JointConference on Natural Language Processing(Volume 1:Long Papers).2021:1726-1739.

[0018]

[10] Zhou H,Huang M,Zhang T,et al.Emotional chatting machine:Emotionalconversation generation with internal and external memory[C] / / Proceedings ofthe AAAI Conference on Artificial Intelligence.2018,32(1). Summary of the Invention

[0019] The purpose of this invention is to overcome the shortcomings of the prior art and provide an emotion-based multi-turn dialogue model and dialogue method, that is, to add emotional factors to the multi-turn dialogue system in order to generate more reasonable responses.

[0020] The objective of this invention is achieved through the following technical solution:

[0021] An emotion-based multi-turn dialogue model includes a dialogue history embedding module, an emotion prediction module, and an emotion embedding module;

[0022] The dialogue history embedding module, including a word-level encoder and a dialogue history self-attention module, is used to input dialogue history information into the emotion-based multi-turn dialogue model; different dialogue history information represents different multi-turn dialogues.

[0023] The sentiment prediction module is used to predict the sentiment of the response. The sentiment category predicted by the sentiment prediction module replaces the manually specified sentiment category. The sentiment prediction module obtains the sentiment category through the dialogue history. The input of the sentiment prediction module is a sentence vector of a dialogue history and the output is a sentiment category.

[0024] The sentiment embedding module is used to generate responses and embed specified sentiment types into them. The sentiment types are provided by the sentiment prediction module, which ultimately enables the multi-turn dialogue model to generate responses with sentiment, thereby improving the quality of the responses.

[0025] Furthermore, this includes the following steps:

[0026] (1) Define the dialogue history as C1 = {s1, ..., s2} N C1 and C2 = {t1, ..., t2} N}, where C1 and C2 represent the dialogue history of speaker one and speaker two, respectively, s i and t i These represent sentences from the dialogue history of the two speakers, where i represents the i-th sentence, i = 1, 2, ..., N; and s represents sentences from the dialogue history of the two speakers. i For example, s i ={x1, ..., x M}, where x k Represents the words in the sentence; calculates the context encoding h. M The process is as follows:

[0027] i k =σ(W i [h k-1 ω k ])

[0028] f k =σ(W f [h k-1 ω k ])

[0029] o k =σ(W o [h k-1 ω k ])

[0030] l k =tanh(W l [h k-1 ω k ])

[0031] c k =f k c k-1 +i k l k

[0032] h k =o k tanh(c k )

[0033] where i k f k and o k These are the input gate, memory gate, and output gate, respectively; ω k It corresponds to the word x k Word embedding, h k h represents the hidden variable corresponding to word k in the LSTM. k By combining ω k and h k-1 Calculation yielded; c k This represents the state of the nerve cell at time k; σ represents the sigmoid function, W i W f W o and W l The parameter matrix is ​​initialized with given random numbers and then continuously adjusted through training.

[0034] (2) For a given matrix with n query vectors keys matrix and values ​​matrix The formula for calculating dot product attention is as follows:

[0035]

[0036] Where d is the number of hidden units in the dialogue history self-attention module, the dialogue self-attention module consists of H heads, each head focusing on different parts of the channel; for the i-th head, using and These represent the parameters learned through continuous adjustment during training, corresponding to queries, keys, and values, respectively. The specific implementation of the dialogue history self-attention module is expressed by the following formula:

[0037]

[0038] After each parallel head is calculated, all vectors are concatenated to form a single vector. The calculation process is as follows:

[0039] M = Contact(M1, ..., M) H )

[0040] O = MW

[0041] in and O represents the output obtained by passing the dialogue history vectors of the two speakers through a self-attention module. The output of speaker one is represented by O1, and the output of speaker two is represented by O2. For O1 and O2, feedforward neural networks are used to output the dialogue history representations of the speakers, respectively. and The calculation process is as follows:

[0042]

[0043]

[0044] FNN stands for feedforward neural network;

[0045] (3) Input the vector representation g of the dialogue history into the fully connected layer to map the vector of the hidden layer dimension to a vector of the number of categories. The calculation process is as follows:

[0046] h = ReLU(W1g + b1)

[0047] Where ReLU is the activation function, W1 and b1 are the learnable parameters of the fully connected layer, and h is the obtained latent vector. The vector h is then processed through a softmax layer to obtain E = {e1, ..., e}. M} represents the probability of the sentiment category corresponding to the dialogue history. The sentiment category with the highest probability z is taken as the final result. The calculation process is as follows:

[0048]

[0049] Where e i Let w be the probability that the current dialogue history information belongs to the i-th sentiment category. i and b i These are the weights and biases of the i-th category, respectively;

[0050] (4) Construct a speaker attention decoder;

[0051] use As the key and value in the attention mechanism, the hidden representation of the response O r As a query, the output of the speaker-attention decoder is used with O d1 The calculation process is as follows:

[0052]

[0053] Where Attention is used for attention calculation; As the key and value in the attention mechanism, the output O of the speaker-attention decoder... d1 As a query, the output of the speaker-second attention decoder is O d2 The calculation process is as follows:

[0054]

[0055] Attention is calculated; the output of the last speaker's attention decoder is O. d2 Obtained using a feedforward neural network The calculation process is as follows:

[0056]

[0057] FNN stands for feedforward neural network;

[0058] (5) Emotional embedding;

[0059] In the emotion embedding module's state update, the state of the emotion-based multi-turn dialogue model is updated by inputting the emotion vector. The calculation process of the emotion embedding module is as follows:

[0060] h j =LSTM([Emb(y j-1 );e j ], h j-1 )

[0061] Where e j Representing the sentiment vector, Emb(y) j-1 ) represents the word vector representation of the previous predicted word, which is calculated using the word2vec method. h j-1 This represents the state of the LSTM at the previous time step, where the previous time step of h1 is... h1's Emb(y) j-1 () is the sentence start identifier <eos>The word vector representation; [.;.] indicates the operation of concatenating two feature vectors, separated by ";"; the sentiment vector representation is e. j The sentiment category z predicted by the sentiment prediction module is determined by the sentiment vector representation e. j The specific calculation process is as follows:

[0062]

[0063]

[0064]

[0065] in Let z represent the k-th word in the emotion dictionary, and T represent the emotion category. z The emotion category in the emotion dictionary is T. z The total number of words, α, β, and γ are all learnable parameters in the sentiment embedding module; a jk This represents the attention at time step j, specifically the attention to the k-th word in the emotion dictionary, where attention a is the focus. jk Through c jk The calculated result is derived from the hidden state h from the previous time step. j-1 Speaker attention mechanism output O d The word vector representation of the k-th word of category z in the sentiment lexicon. Sure.

[0066] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the emotion-based multi-turn dialogue method.

[0067] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the emotion-based multi-turn dialogue method.

[0068] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0069] 1. By using a dialogue history method based on a self-attention mechanism to distinguish speakers, the responses generated by this invention can better reference dialogue history information, and the generated responses are more reasonable and diverse, enabling the model to distinguish the differences between speakers and effectively improving the quality of the responses generated by the model.

[0070] 2. By using the sentiment prediction module and the sentiment embedding module, this invention enables multi-turn dialogue models to understand and express emotions, allowing the model to automatically generate appropriate emotions based on the dialogue history, and generate responses with emotions without logical or grammatical errors.

[0071] 3. Compared to previous work that often embedded dialogue history as a long text, this invention designs a dialogue history method based on a self-attention mechanism to distinguish speakers. The multi-turn dialogue model of this invention is more conducive to distinguishing speakers and can generate more reasonable responses. Attached Figure Description

[0072] Figure 1 This is a schematic diagram of the overall framework of the emotion-based multi-turn dialogue method of the present invention. Detailed Implementation

[0073] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0074] This embodiment provides an emotion-based multi-turn dialogue model, which includes three parts: a dialogue history embedding module, an emotion prediction module, and an emotion embedding module.

[0075] The dialogue history embedding module is used to input dialogue history information into the emotion-based multi-turn dialogue model. Different dialogue history information represents different multi-turn dialogues. Compared with previous works that often embedded dialogue history as a long text, this embodiment designs a method for distinguishing the speaker's dialogue history based on a self-attention mechanism.

[0076] The sentiment prediction module is used to predict the sentiment of the response. The sentiment category predicted by the sentiment prediction module replaces the manually specified sentiment category. The sentiment prediction module obtains the sentiment category through the dialogue history. The sentiment prediction module task is similar to the text sentiment classification task. The text sentiment classification task takes a sentence vector as input and outputs the sentiment category of the sentence. The sentiment prediction task takes a sentence vector of a dialogue history as input and outputs a sentiment category. Therefore, this embodiment refers to the text sentiment classification task.

[0077] The sentiment embedding module is used to generate responses and embed specified sentiment types into them. The sentiment types to be embedded are provided by the sentiment prediction module, which ultimately enables the model to generate responses with appropriate sentiments and improves the quality of the responses.

[0078] Specifically, such as Figure 1 As shown, the workflow of the multi-turn dialogue method in this embodiment is as follows:

[0079] 1. Dialogue History Embedded Module

[0080] 1.1 Word-level encoder

[0081] Define the dialogue history as C1 = {s1, ..., s2} N C1 and C2 = {t1, ..., t2} N }, where C1 and C2 represent the dialogue history of speaker one and speaker two, respectively, s i and t i These represent sentences from the dialogue between the two speakers, where i represents the i-th sentence. Let s... i For example, s i ={x1, ..., x M }, where x k This represents the words in the sentence. The context encoding h is calculated. M The process is as follows:

[0082] i k =σ(W i [h k-1 ω k ])

[0083] f k =σ(W f [h k-1 ω k ])

[0084] o k =σ(W o [h k-1 ω k ])

[0085] l k =tanh(W l [h k-1 ω k ])

[0086] c k =f k c k-1 +i k l k

[0087] h k =o k tanh(c k )

[0088] i k f k and o k These are the input, memory, and output gates, respectively. ω k It corresponds to the word x k Word embedding, h k h represents the hidden variable corresponding to word k in the LSTM. k By combining ω k and h k-1 Calculated. k This represents the state of the nerve cell at time k. σ represents the sigmoid function, and W... i W f W o and W l The parameter matrix is ​​initialized with a given random number and then its parameters are continuously adjusted through training.

[0089] 1.2. Dialogue History Self-Attention Module

[0090] Dialogue self-attention mechanisms can effectively capture long-range dependency information and are also highly computationally efficient. The specific steps include:

[0091] Given a matrix with n query vectors keys matrix and values ​​matrix The formula for calculating dot product attention is as follows:

[0092]

[0093] Where d is the number of hidden units in the dialogue history self-attention module, the dialogue self-attention module consists of H heads, each head focusing on different parts of the channel. For the i-th head, this invention uses... and These represent the parameters learned through continuous adjustment during training, corresponding to queries, keys, and values, respectively. The specific implementation of the dialogue history self-attention module is expressed by the following formula:

[0094]

[0095] After each parallel head is calculated, all vectors are concatenated to form a single vector. The calculation process is as follows:

[0096] M = Contact(M1, ..., M) H )

[0097] O = MW

[0098] in and Here, O represents the output obtained by passing the dialogue history vectors of the two speakers through a self-attention module. The output of speaker one is represented by O1, and the output of speaker two is represented by O2. Feedforward neural networks are used to output the dialogue history representations of the speakers for both O1 and O2. and The calculation process is as follows:

[0099]

[0100]

[0101] FNN stands for Feedforward Neural Network.

[0102] 2. Sentiment Prediction Module

[0103] The vector representation g of the dialogue history represents the vector input to the fully connected layer that maps the hidden layer dimension to a vector with dimension equal to the number of categories. The calculation process is as follows:

[0104] h = ReLU(W1g + b1)

[0105] Where ReLU is the activation function, W1 and b1 are the learnable parameters of the fully connected layer, and h is the obtained latent vector. The vector h is then processed through a softmax layer to obtain E = {e1, ..., e}. M } represents the probability of the sentiment category corresponding to the dialogue history. The sentiment category with the highest probability z is taken as the final result. The calculation process is as follows:

[0106]

[0107] Where e i Let w be the probability that the current dialogue history information belongs to the i-th sentiment category. i and b i These are the weights and biases for the i-th category, respectively.

[0108] 3. Emotion Embedding Module

[0109] 3.1 Speaker Attention Decoder Construction

[0110] In this invention, As the key and value in the attention mechanism, the hidden representation of the response O r As a query, the output of the speaker-attention decoder is used with O d1 The calculation process is as follows:

[0111]

[0112] Attention is used for attention calculation. As the key and value in the attention mechanism, the output O of the speaker-attention decoder... d1 As a query, the output of the speaker-second attention decoder is O d2 The calculation process is as follows:

[0113]

[0114] Similarly, Attention is used for attention calculation. Finally, the output of the speaker-2 attention decoder is O. d2 Obtained using a feedforward neural network The calculation process is as follows:

[0115]

[0116] FNN stands for Feedforward Neural Network.

[0117] 3.2 Emotional Embedding

[0118] This main part is an LSTM model. Compared to a regular LSTM model, the sentiment embedding module's state update method requires an input sentiment vector to update the model state. The calculation process of this module is shown in the formula:

[0119] h j =LSTM([Emb(y j-1 );e j ], h j-1 )

[0120] Where e j Representing the sentiment vector, Emb(y) j-1 ) represents the word vector representation of the previous predicted word, which is calculated using the word2vec method. h j-1 This represents the state of the LSTM at the previous time step, where the previous time step of h1 is... h1's Emb(y) j-1 () is the sentence start identifier <eos>The word vector representation. [.;.] indicates the operation of concatenating two feature vectors, separated by a semicolon (;). The sentiment vector representation is e. j The sentiment category z predicted by the sentiment prediction module is determined by the sentiment vector representation e. j The specific calculation process is as follows:

[0121]

[0122]

[0123]

[0124] in Let z represent the k-th word in the emotion dictionary, and T represent the emotion category. z The emotion category in the emotion dictionary is T. z The total number of words, α, β, and γ are all learnable parameters in the sentiment embedding module.

[0125] An LSTM model is used to calculate the attention score. jk This represents the attention at time step j, specifically the attention to the k-th word in the emotion dictionary, where attention a is the focus. jk Through c jk The calculated result is derived from the hidden state h from the previous time step. j-1 The speaker attention mechanism output Od and the word vector representation of the k-th word of category z in the sentiment lexicon. Confirmed. In the word generation process of the response, a vocabulary-based attention mechanism can be used to achieve this. The more relevant a word in the sentiment vocabulary is to the given sentiment, the greater its influence on the sentiment vector. This method ensures that more relevant sentiment words have a higher generation probability, thus achieving the response sentiment embedding.

[0126] This implementation was conducted on two real-world datasets: Daily-Dialog and MELD. The Daily-Dialog dataset was collected from an English learning website. Because the data originated from a learning website, the dialogues in the Daily-Dialog dataset are grammatically more standardized. The Daily-Dialog dataset contains 13,000 dialogues, each with an average of eight pairs of sentences, covering more than ten topics, including work, health, travel, economics, and politics. In addition, the Daily-Dialog dataset is rich in emotional data, with emotion labels categorized into seven classes: neutral, surprise, happiness, anger, sadness, fear, and disgust. The MELD dataset was publicly released by Poria et al. in 2018. The MELD data comes from excerpts from the movie Friends. The MELD dataset is a multimodal dataset, containing not only dialogue text but also corresponding audio and video information. Furthermore, the MELD dataset text contains seven emotions: anger, disgust, sadness, happiness, neutral, surprise, and fear. The MELD dataset contains over 1400 dialogues, totaling more than 13000 sentences. Table 1 summarizes the characteristics of these two datasets.

[0127] Table 1. Dataset Information Statistics

[0128]

[0129] For evaluating the responses generated by the multi-turn dialogue model, two methods are used: automatic evaluation metrics and manual evaluation metrics. This embodiment uses two automatic evaluation metrics: one based on the language model (PPL) and the other based on word overlap (BLEU, distinct-1, distinct-2). The formula for calculating PPL (Perplexity) is as follows:

[0130]

[0131] in The perplexity score represents the probability that the generated sentence conforms to the language model. Therefore, the lower the perplexity score, the more grammatically correct and grammatically correct the response generated by the multi-turn dialogue model is, and the better it conforms to the language model. However, in dialogue tasks, some common responses such as "I don't know" have the lowest perplexity score, so it is not possible to truly judge the quality of a dialogue model's generated response.

[0132] The BLEU metric uses word overlap calculation to evaluate the model. Specifically, it assesses the similarity between the generated response and responses in the test set, with results ranging from 0 to 1. Generally, it can be divided into multiple evaluation metrics based on n-grams, where n is typically 1, 2, 3, or 4, representing the maximum number of words selected in each calculation. A higher overlap ratio results in a higher BLEU score, and smaller n values ​​lead to higher BLEU scores than larger n values. In this example, n is set to 2 for calculation.

[0133] The distinct metric represents the ratio of the number of distinct statements in the generated sentences to the total number of sentences. This metric aims to assess the diversity and originality of the generated sentences. Because generic responses are generated in dialogue tasks, but these generic responses generally have no specific meaning in the dialogue, although they may score well on other metrics, the distinct metric can be used to assess whether the model generates too many generic or repetitive responses. Its calculation formula is shown in the following equation:

[0134]

[0135] Where U represents the number of n-gram types and G represents the total number of n-grams, the larger the distinct value, the more n-gram types there are in the response and the fewer repetitions. Therefore, the distinct index can be used to evaluate the diversity of the model's response.

[0136] In the human evaluation, this invention selects two dimensions for assessment: content and sentiment. The specific operation is as follows: 50 dialogues are randomly selected for each sentiment category. The sentiment category and dialogue history are input into the model of this invention and other baseline models, respectively. Each response is obtained, and the generated response, dialogue history, and specified sentiment type are then submitted to four researchers in the field for scoring in both content and sentiment aspects. The content score ranges from 0 to 2, and the sentiment score ranges from 0 to 1. The content evaluation index is used to assess whether the response is coherent and relevant to the dialogue history. A score of 0 indicates a grammatical problem and inconsistency with the dialogue history; 1 indicates grammatically correct but meaningless; and 2 indicates a grammatically correct response, relevant to the dialogue history, and completely consistent with previous statements. The sentiment evaluation index is used to assess whether the generated response is consistent with the specified sentiment type. A score of 0 indicates the sentiment of the response is inconsistent with the specified sentiment category; and 1 indicates the sentiment of the generated response is consistent with the specified sentiment category.

[0137] Tables 2 and 3 show the experimental results of the automatic evaluation metrics of this invention on the Daily-Dialog dataset and the MELD dataset, respectively. Tables 4 and 5 show the human evaluation results of the multi-turn dialogue model of this invention on the Daily-Dialog dataset and the MELD dataset, respectively. The corresponding results are compared with existing traditional emotion dialogue models Seq2Seq and ECM.

[0138] Table 2 Automatic Evaluation of the Daily-Dialog Dataset

[0139]

[0140]

[0141] Table 3 Automatic Evaluation of MELD Dataset

[0142]

[0143] Table 4. Human evaluation of the Daily-Dialog dataset

[0144]

[0145] Table 5. Manual evaluation of the MELD dataset.

[0146]

[0147] Experimental results from automatic evaluation show that, in terms of automatic evaluation metrics that do not reflect emotion, this invention outperforms the two baseline models in the emotion-based multi-turn dialogue modeling task across both datasets. Regarding human evaluation, although the ranking of model scores may vary across different emotion categories in the Daily-Dialog and MELD datasets, the invention ultimately achieves the best results in both content and emotion metrics after calculating the arithmetic average of scores across all categories.

[0148] Preferably, embodiments of this application also provide a specific implementation of an electronic device capable of implementing all steps of the emotion-based multi-turn dialogue method in the above embodiments. The electronic device specifically includes the following:

[0149] Processor, memory, communications interface, and bus;

[0150] The processor, memory, and communication interface communicate with each other via a bus; the communication interface is used to realize information transmission between server-side devices, metering devices, and user-side devices.

[0151] The processor is used to call a computer program in memory, and when the processor executes the computer program, it implements all the steps in the emotion-based multi-turn dialogue method in the above embodiments.

[0152] Embodiments of this application also provide a computer-readable storage medium capable of implementing all steps of the emotion-based multi-turn dialogue method in the above embodiments. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all steps of the emotion-based multi-turn dialogue method in the above embodiments.

[0153] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, hardware + program embodiments are relatively simple in description because they are fundamentally similar to method embodiments; relevant parts can be referred to the descriptions in the method embodiments.

[0154] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0155] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0156] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0157] This invention is not limited to the embodiments described above. The above description of specific embodiments is intended to illustrate and explain the technical solutions of this invention. The specific embodiments described above are merely illustrative and not restrictive. Without departing from the spirit and scope of the claims, those skilled in the art can make many specific modifications based on the teachings of this invention, and these modifications all fall within the scope of protection of this invention.< / eos> < / eos>

Claims

1. An emotion-based multi-turn dialogue method, based on a multi-turn dialogue device, the multi-turn dialogue device including a dialogue history embedding module, an emotion prediction module, and an emotion embedding module; The dialogue history embedding module, including a word-level encoder and a dialogue history self-attention module, is used to input dialogue history information into the emotion-based multi-turn dialogue model; different dialogue history information represents different multi-turn dialogues. The sentiment prediction module is used to predict the sentiment of the response. The sentiment category predicted by the sentiment prediction module replaces the manually specified sentiment category. The sentiment prediction module obtains the sentiment category through the dialogue history. The input of the sentiment prediction module is a sentence vector of a dialogue history and the output is a sentiment category. The emotion embedding module is used to generate responses and embed a specified emotion type into the responses. The embedded emotion type is provided by the emotion prediction module, ultimately enabling the multi-turn dialogue model to generate responses with emotion, thereby improving the quality of the responses. It is characterized by including the following steps: (1) Define the history of dialogue as and ,in and These represent the dialogue history of speaker one and speaker two, respectively. and These represent sentences from the dialogue history of the two speakers, where... Representing the In short, =1, 2…N; with For example, ,in Representing words in a sentence; calculating context encoding The process is as follows: ； ； ； ； ； ； Among them , and These are the input gate, memory gate, and output gate, respectively. The corresponding words Word embedding, Representing words in LSTM The corresponding hidden variables, By combination and Calculated; Indicates in The state of nerve cells at any given moment; Represents the sigmoid function. , , and The parameter matrix is ​​initialized with given random numbers and then continuously adjusted through training. (2) For a given condition, A matrix of query vectors keys matrix and values ​​matrix The formula for calculating dot product attention is as follows: ； in It is the number of hidden units in the dialogue history self-attention module. The dialogue self-attention module consists of... Each head is composed of several parts, and each head focuses on a different part of the channel; for the first... Size, using , and These represent the parameters learned through continuous adjustment during training, corresponding to queries, keys, and values, respectively. The specific implementation of the dialogue history self-attention module is expressed by the following formula: ； After each parallel head is calculated, all vectors are concatenated to form a single vector. The calculation process is as follows: ； ； in and , among them To process the dialogue history vectors of two speakers through a self-attention module, the output for speaker one is... To represent, the output of speaker two is used To represent; for and Each speaker's dialogue history is output using a feedforward neural network. and The calculation process is as follows: ； ； in Represents a feedforward neural network; (3) Represent the dialogue history as a vector The input vectors to the fully connected layer are mapped to vectors with dimensions equal to the number of classes. The calculation process is as follows: ； in It is an activation function. and These are the learnable parameters of the fully connected layer. To obtain the implicit vector, further refine the vector. pass The layers are calculated to obtain the final result. The most likely sentiment category is the one that corresponds to the dialogue history. The final result is calculated as follows: ； in The current dialogue history belongs to the first... The probability of each sentiment category. and The first Weights and biases for each category; (4) Construct a speaker attention decoder; use The hidden representation of the response serves as the key and value in the attention mechanism. As a query, the output of the speaker-attention decoder is used... The calculation process is as follows: ； in For attention calculation; using The output of the speaker-attention decoder serves as both the key and value in the attention mechanism. As a query, the output of the speaker-2 attention decoder is used for The calculation process is as follows: ； For attention calculation; the output of the last speaker 2 attention decoder Obtained using a feedforward neural network The calculation process is as follows: ； in Represents a feedforward neural network; (5) Emotional embedding; In the emotion embedding module's state update, the state of the emotion-based multi-turn dialogue model is updated by inputting the emotion vector. The calculation process of the emotion embedding module is as follows: ； in Representing the sentiment vector, This represents the word vector representation of the previous predicted word, which is calculated using the word2vec method. This represents the state of the LSTM at the previous time step, where The previous time step was , of Sentence start identifier <eos>Word vector representation; This represents the operation of concatenating two eigenvectors, where the two eigenvectors are represented by "". "Segmented; Emotional Vector Representation" The sentiment category is predicted by the sentiment prediction module. Decision, Emotion Vector Representation The specific calculation process is as follows:< / eos> ； ； ； in Indicates sentiment category For the k-th word in the emotion dictionary, The emotion category in the emotion dictionary is Total number of words, , and These are all learnable parameters within the emotion embedding module; This represents the attention at time step j, specifically the attention to the k-th word in the emotion dictionary. pass The calculated result is the hidden state from the previous time step. Speaker's attention mechanism output The word vector representation of the k-th word of category z in the sentiment lexicon. Sure.

2. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the emotion-based multi-turn dialogue method of claim 1.

3. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the emotion-based multi-turn dialogue method of claim 1.

Images