A human-computer conversation method and device, electronic equipment and storage medium
By using a global perspective processing and emotion fusion module, the problem of naturalness and emotion fusion in existing human-computer dialogue methods is solved, generating more natural and emotionally appropriate responses.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SOUTH CHINA NORMAL UNIV
- Filing Date
- 2023-04-23
- Publication Date
- 2026-05-29
AI Technical Summary
Existing human-computer dialogue methods lack naturalness and emotional integration, lack conversational knowledge, and their responses do not match the user's emotions.
It adopts a global perspective to process knowledge, combines it with an emotion fusion module, uses emotion graphs and background knowledge feature representations, encodes them with GRU neural networks and multi-head graph attention, generates emotion topic guidance vectors, and combines generation and replication mechanisms to generate responses.
It achieves more accurate knowledge acquisition and emotion integration, resulting in more natural responses that are consistent with the context and the user's emotions.
Smart Images
Figure CN116467420B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of natural language processing technology, and in particular to a human-computer dialogue method, apparatus, electronic device, and storage medium. Background Technology
[0002] Natural Language Processing (NLP) studies models of language ability and applications, refining, evaluating, and ultimately applying them to various real-world systems. It investigates theories and methods for achieving effective communication between humans and computers, encompassing research areas such as automatic summarization, machine writing, relation extraction, and dialogue systems. It is a crucial research area within artificial intelligence. Dialogue systems, in particular, primarily study how to achieve language interaction between humans and computers.
[0003] Existing human-computer dialogue methods and models mainly suffer from the following problems:
[0004] (1) Although the extraction-based generation method is rich in information, it lacks naturalness and cannot meet the requirements of conversation.
[0005] (2) The above research is basically about filtering knowledge between conversation and unstructured knowledge or between conversation and structured knowledge, which may result in a lack of conversational knowledge.
[0006] (3) The emotions expressed in the response contradict the emotions expressed by the user. Summary of the Invention
[0007] Based on this, the purpose of the present invention is to provide a human-computer dialogue method, device, electronic device and storage medium that processes knowledge from a global perspective, takes into account the context of the dialogue, and can acquire knowledge more accurately; and uses an emotion fusion module to make the conversation emotional with an emotion knowledge enhancement library and integrates it with the context theme to generate responses.
[0008] In a first aspect, the present invention provides a human-computer dialogue method, comprising the following steps:
[0009] Based on the conversation for which a response is to be generated, obtain the background knowledge corresponding to the conversation;
[0010] The session is encoded to obtain a session feature representation;
[0011] The conversation is combined with a commonsense database and a sentiment lexicon to form a sentiment graph, and the sentiment graph is encoded to obtain a sentiment graph feature representation;
[0012] The background knowledge is encoded to obtain a background knowledge feature representation;
[0013] The conversation feature representation, the sentiment graph feature representation, and the background knowledge features are matched and concatenated to obtain the sentiment topic guidance vector.
[0014] The emotional topic guidance vector is decoded to obtain the response corresponding to the conversation.
[0015] Further, the conversation feature representation, the sentiment graph feature representation, and the background knowledge features are matched and concatenated to obtain the sentiment topic guidance vector, including the following steps:
[0016] Using two separate matching matrices, the conversation feature representation and the sentiment graph feature representation are matched with the background knowledge feature representation to obtain the sentiment-background weight vector and the conversation-background weight vector, respectively.
[0017] A sliding window is used to perform knowledge selection processing on the emotion-background weight vector and the conversation-background weight vector to obtain the emotion topic conversion vector.
[0018] The sentiment theme transformation vector is concatenated with the sentiment graph feature representation to obtain the sentiment theme guidance vector.
[0019] Furthermore, the session and the background knowledge are encoded using a GRU neural network.
[0020] Furthermore, the sentiment graph is encoded using multi-head graph attention and an MLP layer to obtain a sentiment graph feature representation.
[0021] Furthermore, the conversation is combined with a commonsense database and a sentiment lexicon, including the following steps:
[0022] By combining words from the dialogue with keywords from the commonsense database ConceptNet, a series of tuples are constructed. Each tuple is represented as t = (x, r, c, s), where c, r, x, and s represent the head concept, relation, tail concept, and confidence score of the tuple, respectively; i represents the i-th tuple; k represents the keyword index; and K represents the number of tuples obtained through keyword retrieval.
[0023] The sentiment intensity value of each tuple is calculated using the sentiment lexicon NRC_VAD.
[0024] Based on the following rules, construct an emotion graph using the n tuples with the highest emotion intensity values and the dialogue:
[0025] Adjacent words point sequentially to the next word;
[0026] The selected candidate sentiment words point to their corresponding keyword c. i .
[0027] Further, the emotional topic guidance vector is decoded to obtain the response corresponding to the conversation, including the following steps:
[0028] Use the following formula to calculate the probability of generating from the vocabulary;
[0029]
[0030] Among them, P v (r t W represents the probability of being generated from the vocabulary. v These are learnable parameters; The emotion theme transformation vector is concatenated with the background knowledge vector, and a readout layer is used to obtain the overall feature vector.
[0031] Use the following formula to calculate the probability of directly copying from background knowledge:
[0032]
[0033] Among them, P k (r t ) represents the probability of directly copying from background knowledge; It is the emotional theme guiding vector; h K Background knowledge vector;
[0034] Use the following formula to calculate the probability of generation based on the sentiment theme transition vector in each time period:
[0035] P(r t )=gP v ( t )+(1-g) k ( t )
[0036] Among them, P v (r t ) and P k (r t ) represent the probability of generating from the vocabulary and the probability of copying directly from background knowledge, respectively; g is a soft gate, which depends on the decoding state;
[0037] The final response is generated by selecting words from the vocabulary list based on probability distribution.
[0038] Secondly, the present invention also provides a human-computer dialogue device, comprising:
[0039] The background knowledge acquisition module is used to obtain the background knowledge corresponding to the conversation based on the conversation for which a reply is to be generated;
[0040] A session encoding module is used to encode the session to obtain a session feature representation;
[0041] The sentiment graph encoding module is used to combine the conversation with a common sense database and a sentiment lexicon to form a sentiment graph, and to encode the sentiment graph to obtain a sentiment graph feature representation;
[0042] The background knowledge encoding module is used to encode the background knowledge to obtain a background knowledge feature representation;
[0043] The sentiment topic guidance vector generation module is used to match and concatenate the conversation feature representation, the sentiment graph feature representation, and the background knowledge features to obtain the sentiment topic guidance vector.
[0044] The response generation module is used to decode the emotional topic guidance vector to obtain the response corresponding to the conversation.
[0045] Thirdly, the present invention also provides an electronic device, comprising:
[0046] At least one memory and at least one processor;
[0047] The memory is used to store one or more programs;
[0048] When the one or more programs are executed by the at least one processor, the at least one processor performs the steps of a human-computer dialogue method as described in any of the first aspects of the present invention.
[0049] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a human-computer dialogue method as described in any of the first aspects of the present invention.
[0050] This invention provides a human-computer dialogue method, device, electronic device, and storage medium. By introducing structured and unstructured knowledge and processing knowledge from a global perspective, it can take into account the context of the dialogue and acquire knowledge more accurately. By using an emotion fusion module, which combines generation and replication mechanisms, it solves the problem of insufficient knowledge in traditional conversations. It uses an emotion knowledge enhancement library to make the conversation emotional and integrates it with the context and topic to generate responses.
[0051] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description
[0052] Figure 1 A schematic diagram of the structure of a human-computer dialogue device provided by the present invention;
[0053] Figure 2A schematic diagram illustrating the steps of a human-computer dialogue method provided by the present invention;
[0054] Figure 3 A comparison of three different methods for generating responses for a single dialogue scenario is presented, where EGKS is the response generated using a human-computer dialogue method provided by this invention. Detailed Implementation
[0055] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0056] In a specific application scenario, the human-computer dialogue method provided by this invention relies on intelligent devices such as computers. Users input the dialogue to be generated through the dialogue input window of the intelligent device.
[0057] Please also refer to Figure 1 and Figure 2 ,in, Figure 1 This is a schematic diagram of the modules of the human-computer dialogue device of the present invention. Figure 2 yes Figure 1 The flowchart shown illustrates the execution of a human-computer dialogue device. The human-computer dialogue method device of the present invention includes: a dialogue encoding module 10, a background knowledge encoding module 20, an emotion graph encoding module 30, an emotion topic guidance module 40, a response module 50, a common sense database, and an emotion lexicon.
[0058] Specifically, the dialogue encoding module 10 is used to perform step S10: encoding the dialogue to obtain a dialogue feature representation.
[0059] In this embodiment, a GRU neural network is used to encode the session.
[0060] Taking the current session C as an example, the session feature representation is obtained using the following formula:
[0061]
[0062] in, This represents the feature representation of the t-th token in the current session, e(c t () represents the embedding vector of the t-th token in the session. This indicates the previous output of the BIGRU. Initialize to 0.
[0063] The background knowledge encoding module 20 includes a background knowledge acquisition submodule 21 and a background knowledge encoding submodule 22. The background knowledge refers to external document information relevant to the current session. Incorporating background knowledge into responses enhances the ability to transfer knowledge.
[0064] Background knowledge acquisition submodule 21 is used to perform step S21: retrieving relevant external information as background knowledge based on the dialogue. In some examples, keywords in the current dialogue are first extracted, and then the background knowledge is obtained through retrieval. In other examples, background knowledge can also be retrieved from the user's historical conversations.
[0065] Background knowledge encoding submodule 22 is used to perform step S22: encoding the background knowledge to obtain a background knowledge feature representation. In this embodiment, a GRU neural network is also used to encode the session, and then a high-speed transformation is performed to obtain its aggregate vector. Specifically, the background knowledge feature representation is obtained using the following formula:
[0066]
[0067] g k =σ(W g [h kt hx || ]+b)
[0068] in, hx represents the feature representation after aggregation of background knowledge. |x| W represents the output of the last layer of the BIGRU. l W nl W g These are learnable parameters.
[0069] The sentiment graph encoding module 30 is used to perform step S30: combining the dialogue with a common sense database and a sentiment lexicon to form a sentiment graph, and encoding the sentiment graph to obtain a sentiment graph feature representation. Specifically, it includes a tuple construction submodule 31, a sentiment intensity calculation submodule 32, a sentiment graph construction submodule 33, and a sentiment graph encoding submodule 34.
[0070] Tuple construction submodule 31 is used to perform step S31: combining keywords from the conversation with keywords from the commonsense database ConceptNet to construct a series of tuples. Here, each tuple is represented as t = (x, r, c, s), where t represents the head concept, relation, tail concept, and confidence score of the tuple, respectively; i represents the i-th tuple, k represents the keyword index selected; and K represents the number of keywords selected through x. i The number of tuples in a keyword search.
[0071] In one specific embodiment, the commonsense database is ConceptNet. ConceptNet is a large-scale knowledge graph that describes general human natural language knowledge and has been demonstrated in previous research to play an important role in emotion-related tasks. It consists of 5.9 million tuples, 3.1 million concepts, and 38 relations. Each tuple consists of four elements (head concept, relation, tail concept, confidence score) denoted as t = (x, r, c, s), which in this model is used to augment the knowledge of the previous conversation.
[0072] The sentiment intensity calculation submodule 32 is used to perform step S32: using the sentiment lexicon NRC_VAD, calculate the sentiment intensity value of each tuple.
[0073] In this embodiment, the sentiment lexicon is the sentiment lexicon NRC_VAD. NRC_VAD is a three-dimensional vector knowledge base with three attributes (Valence, Arousal, Dominance), representing the degree of positivity, excitement, and subjectivity, respectively, and each attribute takes a value of [0, 1].
[0074] Specifically, the emotional intensity value is calculated using the following formula.
[0075]
[0076] in, V represents the k-th word in the tuple retrieved from the i-th word search. a () represents the score for level of activity, A r () represents the excitement level score, min-max() represents the maximum and minimum value standardization, and ||2 represents the L2 norm.
[0077] The sentiment graph construction submodule 33 is used to perform step S33: construct a sentiment graph using the n tuples with the highest sentiment intensity values and the dialogue according to the following rules:
[0078] (1) Two adjacent words point to the next word in sequence;
[0079] (2) The selected candidate sentiment words point to their corresponding keywords c i .
[0080] The resulting sentiment graph is stored as an adjacency matrix.
[0081] The sentiment graph encoding submodule 34 is used to perform step S34: encoding the sentiment graph. In a specific embodiment, a multi-head graph attention operation followed by an MLP layer is used as a graph encoder to encode the sentiment graph and obtain the sentiment feature vector. To reduce the number of model parameters, the number of heads in the multi-head attention operation is two, and the number of layers is also two.
[0082] For a sentiment graph, the word embeddings, position embeddings, and vertex embeddings of the graph are used as inputs to the graph encoder, which will eventually yield the sentiment graph feature representation.
[0083] Embedding refers to Word2Vec, which involves embedding a high-dimensional space containing the number of words into a continuous vector space with a much lower dimension, where each word or phrase is mapped to a vector in the real number field.
[0084] Word embedding refers to converting words into corresponding vectors, position embedding refers to converting the positions of words into corresponding vectors, and graph embedding refers to a process of mapping graph data (usually a high-dimensional dense matrix) into low-dimensional dense vectors.
[0085] The sentiment topic guidance module 40 is used to perform step S40: matching and concatenating the conversation feature representation, the sentiment graph feature representation, and the background knowledge features to obtain a sentiment topic guidance vector. The sentiment topic guidance module 40 includes a weight vector generation submodule 41, a sentiment topic transformation vector generation submodule 42, and a sentiment topic guidance vector generation submodule 43.
[0086] The weight vector generation submodule 41 is used to perform step S41: using two separate matching matrices, the conversation feature representation and the sentiment graph feature representation are matched with the background knowledge feature representation to obtain the sentiment-background weight vector and the conversation-background weight vector, respectively.
[0087] For these two matching matrices, a max pooling layer is applied along the X-axis to obtain two background-aware weighted feature representation vectors: the sentiment-background weight vector and the conversation-background weight vector. Each element in the feature vector represents a weight related to the background; a larger weight indicates a greater relevance.
[0088] The emotion topic conversion vector generation submodule 42 is used to perform step S42: using a sliding window to perform knowledge selection processing on the emotion-background weight vector and the conversation-background weight vector to obtain the emotion topic conversion vector.
[0089] The matching matrix is constructed as shown in the following formula:
[0090] M kc [i,j]=V M tanh(Wm1 H ki +W m2 H cj )
[0091] Among them, V M It is a learnable vector, W m1 and W m2 These are learnable parameters. H C and H K Potential representations of historical conversations and background knowledge.
[0092] Inspired by GLKS, this application adopts the concept of a sliding window for global knowledge selection processing.
[0093] The sliding window algorithm performs the required operations on an array or string given a specific window size. In a specific embodiment, it involves expanding and summing the background-aware weight feature representation vector using a window of size m, and performing attention operations on the expanded vector using a window of size m. The purpose of summing the conversation sentiment weight vector is to extract overall semantic information using a sliding window of size m; the attention operation on background knowledge is used to calculate sliding attention weights, promoting partial knowledge attention.
[0094] The sentiment theme guidance vector generation submodule 43 is used to perform step S43: concatenating the sentiment theme transformation vector with the sentiment graph feature representation to obtain the sentiment theme guidance vector.
[0095] The response module 50 is used to perform step S50: decoding the sentiment topic guidance vector to obtain the response corresponding to the session. In one embodiment, a decoder is used to decode the sentiment topic guidance vector. The decoder combines a generation mechanism and a replication mechanism, determining the generation probability based on the sentiment topic transformation vector at each time period.
[0096] Preferably, the response module 50 includes: a vocabulary probability calculation submodule 51, a background knowledge probability calculation submodule 52, an emotion topic probability calculation submodule 53, and a response generation submodule 54.
[0097] The vocabulary probability calculation submodule 51 is used to perform S51: calculate the probability generated from the vocabulary using the following formula;
[0098]
[0099] Among them, P v (r t W represents the probability of being generated from the vocabulary. v These are learnable parameters; The sentiment topic transformation vector is concatenated with the background knowledge vector, and a readout layer is used to obtain the overall feature vector. The vocabulary used is glove.6B.300d, and word embeddings and other transformations are also based on this document, which records the vector corresponding to each word.
[0100] Background knowledge probability calculation submodule 52 is used to perform step S52: using the following formula, calculate the probability of directly copying from background knowledge:
[0101]
[0102] Among them, P k (r t ) represents the probability of directly copying from background knowledge; It is the emotional theme guiding vector; h K This is a background knowledge vector.
[0103] The sentiment topic probability calculation submodule 53 is used to perform step S53: using the following formula, calculate the probability of generation based on the sentiment topic transformation vector in each time period:
[0104] P(r t )=gP v (r t )+(1-g)P k (r t )
[0105] Among them, P v (r t ) and P k (r t ) represent the probability of generating from the vocabulary and the probability of copying directly from background knowledge, respectively; g is a soft gate, which depends on the decoding state;
[0106] The response generation submodule 54 is used to perform step S54: select words from the vocabulary according to the probability distribution to generate the final response.
[0107] like Figure 3 As shown, Figure 3 This paper compares the responses generated using three different methods for a single dialogue scenario, with EGKS representing the response generated using a human-computer dialogue method provided by this invention. The italicized parts of the responses in the figure represent the correct answers extracted from background knowledge, while the bold parts represent the parts generated by the model. It can be seen that all three models have strong knowledge selection capabilities, and EGKS (this technology) generates a more natural and reasonable response.
[0108] This application also provides an electronic device, including:
[0109] At least one memory and at least one processor;
[0110] The memory is used to store one or more programs;
[0111] When the one or more programs are executed by the at least one processor, the at least one processor performs the steps of a human-computer dialogue method as described above.
[0112] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0113] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the human-computer dialogue method described above.
[0114] Computer storage media include permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.
[0115] This invention provides a human-computer dialogue method, device, electronic device, and storage medium. By introducing structured and unstructured knowledge and processing knowledge from a global perspective, it can take into account the context of the dialogue and acquire knowledge more accurately. By using an emotion fusion module, which combines generation and replication mechanisms, it solves the problem of insufficient knowledge in traditional conversations. It uses an emotion knowledge enhancement library to make the conversation emotional and integrates it with the context and topic to generate responses.
[0116] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.
Claims
1. A human-computer dialogue method, characterized in that, Includes the following steps: Based on the conversation for which a response is to be generated, obtain the background knowledge corresponding to the conversation; The session is encoded to obtain a session feature representation; The conversation is combined with a commonsense database and a sentiment lexicon to form a sentiment graph, and the sentiment graph is encoded to obtain a sentiment graph feature representation; The background knowledge is encoded to obtain a background knowledge feature representation; The conversation feature representation, the sentiment graph feature representation, and the background knowledge features are matched and concatenated to obtain the sentiment topic guidance vector. The emotional topic guidance vector is decoded to obtain the response corresponding to the conversation; Combining the conversation with a commonsense database and a sentiment lexicon includes the following steps: By combining words from the dialogue with keywords from the commonsense database ConceptNet, a series of tuples are constructed. Each tuple is represented as t=(x, r, c, s), where x, r, c, and s represent the head concept, relation, tail concept, and confidence score of the tuple, respectively; i represents the i-th tuple; k represents the keyword index; and K represents the number of tuples obtained through keyword retrieval. The sentiment intensity value of each tuple is calculated using the sentiment lexicon NRC_VAD. Based on the following rules, construct an emotion graph using the n tuples with the highest emotion intensity values and the dialogue: Adjacent words point sequentially to the next word; The selected candidate sentiment words point to their corresponding keywords. ; The process of matching and concatenating the conversation feature representation, the sentiment graph feature representation, and the background knowledge features to obtain the sentiment topic guidance vector includes the following steps: Using two separate matching matrices, the conversation feature representation and the sentiment graph feature representation are matched with the background knowledge feature representation to obtain the sentiment-background weight vector and the conversation-background weight vector, respectively. A sliding window is used to perform knowledge selection processing on the emotion-background weight vector and the conversation-background weight vector to obtain the emotion topic conversion vector. The sentiment theme transformation vector is concatenated with the sentiment graph feature representation to obtain the sentiment theme guidance vector; Decoding the sentiment topic guidance vector to obtain the response corresponding to the conversation includes the following steps: Use the following formula to calculate the probability of generating from the vocabulary; in, This indicates the probability of generating from the vocabulary. These are learnable parameters; The emotion theme transformation vector is concatenated with the background knowledge vector, and a readout layer is used to obtain the overall feature vector; Use the following formula to calculate the probability of directly copying from background knowledge: in, The probability of directly copying from background knowledge; It is an emotional theme guiding vector; Background knowledge vector; Use the following formula to calculate the probability of generation based on the sentiment theme transition vector in each time period: in, and These represent the probabilities of generating from the vocabulary and directly copying from background knowledge, respectively; g is a soft gate that depends on the decoding state. The final response is generated by selecting words from the vocabulary list based on probability distribution.
2. The human-computer dialogue method according to claim 1, characterized in that: The session and the background knowledge are encoded using a GRU neural network.
3. The human-computer dialogue method according to claim 1, characterized in that: The sentiment graph is encoded using multi-head graph attention and an MLP layer to obtain a sentiment graph feature representation.
4. A human-computer dialogue device for performing a human-computer dialogue method as described in claim 1, characterized in that, include: The background knowledge acquisition module is used to obtain the background knowledge corresponding to the conversation based on the conversation for which a reply is to be generated; A session encoding module is used to encode the session to obtain a session feature representation; The sentiment graph encoding module is used to combine the conversation with a common sense database and a sentiment lexicon to form a sentiment graph, and to encode the sentiment graph to obtain a sentiment graph feature representation; The background knowledge encoding module is used to encode the background knowledge to obtain a background knowledge feature representation; The sentiment topic guidance vector generation module is used to match and concatenate the conversation feature representation, the sentiment graph feature representation, and the background knowledge features to obtain the sentiment topic guidance vector. The response generation module is used to decode the emotional topic guidance vector to obtain the response corresponding to the conversation.
5. An electronic device, characterized in that, include: At least one memory and at least one processor; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the at least one processor implements the steps of a human-computer dialogue method as described in any one of claims 1-3.
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a human-computer dialogue method as described in any one of claims 1-3.