Multi-turn Dialogue Response Generation Method and Terminal Device Based on Dual-channel Semantic Enhancement

Abstract

The present invention discloses a multi-turn dialogue response generation method and a terminal device based on dual-channel semantic enhancement. The method includes obtaining an initial word vector of a dialogue text; obtaining an ordered semantic representation of the initial word vector, including obtaining a sentence semantic vector at the utterance level of the initial word vector, determining a sentence semantic vector at the dialogue level of the initial word vector according to the sentence semantic vector at the utterance level, and denoting the sentence semantic vector at the dialogue level as the ordered semantic representation; obtaining a graph domain semantic representation of the initial word vector on the graph domain; performing semantic enhancement on the dialogue text according to the ordered semantic representation and the graph domain semantic representation to obtain an enhanced semantic representation; and generating a response text according to the enhanced semantic representation. The present invention aims to integrate the semantic advantages in different structural modellings to obtain a larger-span information association and semantic reasoning. The model of the present invention performs excellently on the benchmark model and alleviates the problem of long-distance semantic dependence.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence, and particularly relates to a multi-turn dialogue response generation method and a terminal device based on dual-channel semantic enhancement. Background Art

[0002] With the rise of the Internet of Everything and human-computer interaction, as a widely used communication medium, the dialogue system has penetrated vertically into many scenarios such as intelligent customer service, AI speakers, and intelligent cockpits. At the same time, due to its great advantages in improving the information service experience and assisting voice command interaction, the dialogue system has great research value and application value, among which the multi-turn dialogue system is the most prominent. Multi-turn dialogue response generation is a generative dialogue that focuses on continuous dialogue and complex semantic interaction, and can generate meaningful and diverse fluent responses to users according to the interaction text between the user and the agent within a certain period of time. In recent years, it has been gradually investigated and concerned by researchers in various countries.

[0003] Most of today's intelligent dialogue systems are developed based on end-to-end deep neural network technology. With the popularization of application scenarios, the responses of the dialogue system cannot keep up with the times, the form is relatively single, and the content lacks scene value. Among them, although the research on the multi-turn dialogue system of continuous interaction has improved the response quality by introducing common sense or fixed sentence patterns, the main challenge is to effectively model the context and obtain an accurate semantic representation.

[0004] In the multi-turn dialogue response generation in the prior art, during the process of semantic information extraction, due to the limitations of the model structure and the long sequence structure of the conversation history, it is difficult to obtain accurate query information, and it is easy to introduce semantic noise to interfere with the generated response, resulting in an unsatisfactory response and poor robustness. Summary of the Invention

[0005] The present invention provides a multi-turn dialogue response generation method and a terminal device based on dual-channel semantic enhancement, which solves the technical problems of poor response quality and poor robustness existing in the multi-turn dialogue response method in the prior art.

[0006] The first aspect of the present invention discloses a multi-turn dialogue response generation method based on dual-channel semantic enhancement, including:

[0007] Obtain the initial word vector of the dialogue text;

[0008] Obtain the sequential semantic representation of the initial word vector, including obtaining the sentence-level semantic vector of the initial word vector at the utterance level, determining the sentence-level semantic vector of the initial word vector at the dialogue level according to the sentence-level semantic vector at the utterance level, and recording the sentence-level semantic vector at the dialogue level as the sequential semantic representation;

[0009] Obtain the graph domain semantic representation of the initial word vector on the graph domain;

[0010] Semantically enhance the dialogue text according to the sequential semantic representation and the graph domain semantic representation to obtain an enhanced semantic representation;

[0011] Generate a response text according to the enhanced semantic representation.

[0012] Preferably, obtaining the utterance-level sentence semantic vector of the initial word vector specifically includes:

[0013] Sequentially input the initial word vector into a sentence-level encoder and a word attention module to obtain an utterance-level sentence semantic vector;

[0014] Determining the dialogue-level sentence semantic vector of the initial word vector according to the utterance-level sentence semantic vector specifically includes:

[0015] Sequentially input the utterance-level sentence semantic vector into a context encoder and a sentence attention module to obtain a dialogue-level sentence semantic vector;

[0016] Both the sentence-level encoder and the context encoder are bidirectional gated neural networks;

[0017] The mechanisms used in both the word attention module and the sentence attention module are attention mechanisms.

[0018] Preferably, obtaining the graph domain semantic representation of the initial word vector on the graph domain specifically includes:

[0019] Obtain the topic keywords of the initial word vector, and determine the nodes of the heterogeneous cognitive graph according to the topic keywords. The nodes include "topic - sentence cluster" nodes, dialogue query nodes, and ordinary nodes (dialogue sentence clusters without topics);

[0020] Determine the edges of the heterogeneous cognitive graph and the weights of each edge according to the nodes of the heterogeneous cognitive graph. The weights are determined according to the degree of topic overlap between sentences in the dialogue text corresponding to the initial word vector;

[0021] Use a graph neural network to learn the vector representations of the nodes in the heterogeneous cognitive graph to obtain the graph domain semantic representation of the initial word vector on the graph domain.

[0022] Preferably, semantically enhancing the dialogue text according to the sequential semantic representation and the graph domain semantic representation specifically includes:

[0023] Semantically enhance the dialogue text according to the first formula. The first formula is:

[0024]

[0025] where cfinal is the enhanced semantic representation, is the sequential semantic representation, is the graph domain semantic representation, δ is the number of semantics in the sequential semantic representation, and (1 - δ) is the number of semantics in the graph domain semantic representation.

[0026] Preferably, according to the enhanced semantic representation, a response text is generated, specifically including:

[0027] Input the enhanced semantic representation into a one-way gated neural network to obtain the hidden state of each word in the generated response text;

[0028] According to the hidden state, determine the generation probability of each word, and determine the response text according to the generation probability.

[0029] Preferably, inputting the enhanced semantic representation into a one-way gated neural network to obtain the hidden state of each word in the generated response text specifically includes:

[0030] Generate the hidden state of each word in the response text according to the second formula, and the second formula is:

[0031]

[0032] In the formula, y i is the i-th word in the generated response text during the training phase, y i-1 is the (i - 1)-th word in the generated response text during the training phase, is the hidden state of y i , GRU(·) means inputting the parameters therein into the gated neural network, is the hidden state of y i-1 , c final is the enhanced semantic representation.

[0033] Preferably, determining the generation probability of each word according to the hidden state specifically includes:

[0034] Determine the generation probability of each word according to the third formula, and the third formula is:

[0035]

[0036] In the formula, is the i-th word in the generated response text during the prediction phase, is 's generation probability, and are respectively the generation probabilities of the i-th word in the topic keyword vocabulary and the response text vocabulary during the prediction phase of.

[0037] Preferably, the is determined according to the fourth formula, and the fourth formula is:

[0038]

[0039] In the formula, η(·) is the non - linear function tanh, V is the response text vocabulary, K is the topic keyword vocabulary, is the hidden state of the i - th word y i in the response text generated during the training phase, y i-1 is the (i - 1) - th word in the response text generated during the training phase, c final is the enhanced semantic representation, and vocab represents the variable i.

[0040] Preferably, the is determined according to the fifth formula, and the fifth formula is:

[0041]

[0042] In the formula, η(·) is the non - linear function tanh, V is the response text vocabulary, K is the topic keyword vocabulary, is the hidden state of the i - th word y i in the response text generated during the training phase, y i-1 is the (i - 1) - th word in the response text generated during the training phase, c final is the enhanced semantic representation, and vocab represents the variable i.

[0043] The second aspect of the present invention discloses a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above - mentioned method are implemented.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The present invention is a method for sequential and graph - domain dual - channel collaborative semantic modeling and reasoning, aiming to integrate the semantic advantages in different structural modeling to obtain a larger - span information association and semantic reasoning. Specifically, on the one hand, the present invention constructs a dialogue - level heterogeneous cognitive graph, where the graph nodes are the integration of topic semantics and sentence - cluster semantics, and the edges in the graph are the degree of topic overlap between sentences. Then, a dual - gated graph neural network is used for deep learning to obtain the semantic representation of the dialogue context in the graph domain. On the other hand, a hierarchical attention mechanism is embedded in the remaining sequential channel to obtain the sequential semantic representation of the dialogue context. Finally, the information contributions of the two semantic representations are coordinated for prediction. The model of the present invention performs excellently on the benchmark model and alleviates the long - distance semantic dependence problem.

[0046] The present invention helps to promote the further development of multi-turn dialogue generation, enables the system to better understand the high-level semantic information of the context, and can also regain new cognition from the reconstructed heterogeneous cognitive map structure, helping to generate diverse and valuable information with good robustness, thus improving the satisfaction and efficiency of users in using information services. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a schematic flow chart of the multi-turn dialogue response generation method based on dual-channel semantic enhancement according to an embodiment of the present invention;

[0048] Figure 2 is a detailed flow chart of the multi-turn dialogue response generation method based on dual-channel semantic enhancement according to an embodiment of the present invention;

[0049] Figure 3 is a schematic structural diagram of the neural network of the present invention;

[0050] Figure 4 is a schematic diagram of node semantic representation encoding according to an embodiment of the present invention;

[0051] Figure 5 is the aggregation strategy of semantic features according to a specific embodiment of the present invention;

[0052] Figure 6 is a performance graph of the SGDC model and the baseline model of the present invention on test samples with different context lengths. Figure 6 In (a), it is the PPL value of the dataset DailyDialog, (b) is the PPL value of the dataset MuTual, (c) is the Dist-2 value of the dataset DailyDialog, (d) is the Dist-2 value of the dataset MuTual, (e) is the EA value of the dataset DailyDialog, and (f) is the EA value of the dataset MuTual. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] The technical solutions of the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation cases. It should be understood that the following embodiments are only for illustrative and explanatory purposes of the present invention and should not be construed as limiting the protection scope of the present invention. All technologies implemented based on the above content of the present invention are covered within the scope of protection intended by the present invention.

[0054] Substantially, multi-turn dialogue response generation is a problem of predicting a sequence based on a sequence. The present invention tasks the modeling of multi-turn dialogue response generation, which belongs to the natural language text generation task. A multi-turn dialogue sequence Diag contains M>2 turns of utterances, where m∈(1,M] represents the m-th turn. The dialogue sequence can be represented as Diag = {U1,......U m-1 ,U M}, where the dialogue history discourse sequence (U1,......U m-2 ) represents the context information of the entire dialogue context, and the dialogue query discourse U m-1 represents the current dialogue progress status, and U M is the target response to be generated in the multi-turn dialogue response generation task of the present invention.

[0055] Most of today's intelligent dialogue systems are developed based on end-to-end deep neural network technology. With the popularization of application scenarios, the responses of dialogue systems cannot keep up with the times, the forms are relatively single, and the content lacks scene value. Among them, although the research on multi-turn dialogue systems with continuous interaction has improved the response quality by introducing common sense or fixed sentence patterns, the main challenge is to effectively model the context and obtain an accurate semantic representation. The currently most commonly used hierarchical encoder-decoder framework ignores the fact that dialogue is generated in a coherent process, and any two utterances are semantically related and complementary or subtractive to each other. When encoding each utterance separately without considering their internal relationships, the hierarchical model may not be able to capture the discourse coherence in the context and ultimately generate an unsatisfactory response. Therefore, the encoder-decoder framework based on the hierarchical model still needs to distinguish the contribution degrees of different sentences and perform differential encoding in the context semantic modeling of the dialogue history.

[0056] In multi-turn dialogue response generation, the topic information is the high-level semantic feature extracted from the dialogue history. The model improves the information content and relevance of the response by integrating the topic semantics. However, existing research only introduces or selects topics to alleviate semantic sparsity. When representing the topic vector, the semantic interaction with the specific dialogue context where the topic is located is not considered. This context-free method may lead to inaccurate topic representations and discourse sentence representations due to the inherent polysemy of natural language, thereby damaging the effect of response generation.

[0057] The vector representation learning of multi-turn dialogue text input has gradually stabilized from the unordered bag-of-words structure to the sequence structure, and the semantic modeling method has also progressed from machine learning methods to deep learning methods represented by recurrent neural networks and attention mechanisms. However, due to the neural network learning mode of the sequence structure, semantic modeling is still difficult to better solve the long-distance dependence problem. With the wide application of graph neural networks in various NLP sub-tasks, the task of multi-turn dialogue response generation also urgently needs to break the shackles of non-Euclidean space, deeply explore the graph structure existing in its own input, and use this to model the context to assist the current sequence structure modeling.

[0058] The first aspect of the present invention provides a multi-turn dialogue response generation method based on dual-channel semantic enhancement, such asFigure 1 and Figure 2 As shown in and

[0059] , it includes:

[0059] Step 1, obtain the initial word vector of the dialogue text.

[0060] Step 2, obtain the sequential semantic representation of the initial word vector, including obtaining the utterance-level sentence semantic vector of the initial word vector, determining the dialogue-level sentence semantic vector of the initial word vector according to the utterance-level sentence semantic vector, and denoting the dialogue-level sentence semantic vector as the sequential semantic representation.

[0061] The main purpose of this step is to model the context of the dialogue text in the temporal development through the semantic analysis of the sequential channel and form the semantic vector representation of the dialogue context in the sequential channel. In order to reduce the loss of important semantics during the multi-turn dialogue process, this step adds hierarchical attention in the encoder of the hierarchical architecture to focus on important semantics at different granularities. The hierarchical architecture encoder includes a sentence-level encoder and a context encoder, and the hierarchical attention includes word attention and sentence attention.

[0062] Step 2 specifically includes:

[0063] Step 21, input the initial word vector into the sentence-level encoder and the word attention module in sequence to obtain the utterance-level sentence semantic vector. The sentence-level encoder is a bidirectional gated neural network, and the mechanism used by the word attention module is the attention mechanism.

[0064] The sentence-level encoder and the word attention learn the utterance-level sentence semantic vector based on the initialized word vector representation of the dialogue context, as shown in formula (1):

[0065]

[0066] In the formula, taking the utterance-level sentence semantic vector learning of U i as an example, is a bidirectional gated recurrent unit (Bidirectional GatedRecurrentUnit, ), Num i is the total number of words in sentence U i , is the adjacent hidden state output of the bidirectional gated recurrent unit when learning word x j,i , w j,i is the initial vector of word x i at the j-th position in sentence U j,i , is the latest vector representation of word x i at the j-th position in sentence U j,i , and also represents the adjacent hidden state output of the bidirectional gated recurrent unit when learning word x j,iThe hidden state output after

[0067] Different from the encoder of the ordinary hierarchical architecture, in the present invention, the last hidden state is not used as the semantic vector of sentence U i , but instead, the similarity between s t-1 in the decoding step and the hidden state sequence is calculated to determine the respective weights {α 1,i , α 2,i ... α j,i} attached to each hidden state, and then the semantic vector of U i is obtained by weighted summation As shown in Formulas (2) and (3):

[0068]

[0069]

[0070] In the formula, η(·) represents the Relu function, which can save calculations, alleviate overfitting and the vanishing gradient problem. Thus, the present invention can obtain a sequence of sentence semantic vectors at the utterance level

[0071] Step 22: Input the sentence semantic vectors at the utterance level into the context encoder and the sentence attention module in sequence to obtain the sentence semantic vectors at the dialogue level, where the context encoder is a bidirectional gated neural network, and the mechanism used in the sentence attention module is the attention mechanism.

[0072] The context encoder and the sentence attention are used to learn the sentence semantic vectors at the dialogue level based on the sentence semantic vectors at the utterance level. Similar to the above calculation:

[0073]

[0074]

[0075] Among them, η(·) represents the Relu function, is the bidirectional gated neural network (Bidirectional Gated Recurrent Unit, ), is the output of the adjacent hidden state when the bidirectional gated neural network is learning the vocabulary x t,i , β i,t is the weight of the vocabulary x i at the t-th position of sentence U t,i , is the vocabulary x i at the t-th position of sentence U t,iThe latest vector representation also represents the hidden state output of the bidirectional gated neural network after learning the vocabulary x t,i and is the hidden state vector of the output layer obtained by the context encoder and is the sentence-level semantic vector representation at the dialogue level obtained after sentence attention weighted calculation is obtained by weighted aggregation after calculating the association weights between the decoding state and each hidden state of the context encoder, and can adjust the final semantic representation of the entire dialogue context based on the real-time application status of the text. The present invention generally refers to as referred to as the dialogue-level sentence semantic vector. This final semantic vector of the dialogue context represents the context after complex semantic interaction learning in the sequential channel and is an important reference and input for the decoder

[0076] Step 3: Obtain the graph domain semantic representation of the initial word vector on the graph domain

[0077] The main objective of the present invention is to model the medium and long-distance semantic associations of dialogue texts across time series through explicit and implicit semantic analysis of the graph domain channel, and to learn the semantic vector representation of the dialogue context in the graph domain channel. First, a graph is constructed according to the explicit and implicit relationships in the dialogue context, then a new type of graph neural network is designed to learn the vector representation of nodes, and finally the final semantic vector representation is obtained through pooling calculation. In order to reduce the loss of important semantics during the multi-round dialogue transfer process, the present invention designs a double-gated filtering mechanism in the traditional graph neural network layer. The double-gated mechanism can reduce the semantic noise during node information update

[0078] Compared with the simplest fully connected neural network (MLP), the graph neural network technology updates node information on the graph domain structure. During calculation, in addition to the weight matrix, there is also an adjacency matrix A for aggregation calculation, as Figure 3 shown. In the research, there are three major categories of common graph neural networks, namely

[0079] Graph Convolution Networks (GCN)

[0080] Graph Convolutional Neural Networks (GCNs) are divided into spectral-domain-based GCNs and spatial-domain-based GCNs. The latter is the most widely used in GCN applications. Therefore, this invention focuses on introducing spatial-domain-based GCN (hereinafter referred to as GCN). Similar to the convolutional calculation of Euclidean data by traditional CNNs, GCN performs convolution based on the node relationships in graph-domain data, aggregates the representation of the central node with the representations of its neighbors along the edges to update the vector representation of the central node, can adapt to different positions and structures, and can also share weights during node calculations. Information transfer calculation between nodes:

[0081]

[0082] where M k (·) and U k (·) are functions with learnable parameters, and generally a multi-layer perceptron (MLP) is used. represents the node vector of node v at the k-th layer, and u ∈ N v represents the set of neighbor nodes of node v.

[0083] Graph Attention Networks (GAT)

[0084] Graph Attention Networks (GAT) support amplifying the influence of the most important parts of neighbor nodes. It uses an attention mechanism during the aggregation process to determine the weights of neighborhood nodes, control the semantic information of multiple neighbor node representation vectors input to the central node, and generate a random walk representation oriented towards important targets. Information transfer calculation between nodes:

[0085]

[0086] where W k (·) and U k (·) are functions with learnable parameters, and generally a multi-layer perceptron (MLP) is used. α(·) is an attention function that can adaptively control the contribution of neighbor nodes to the semantic information of node v.

[0087] Graph Spatial-Temporal Networks (GSTN)

[0088] Graph Spatial-Temporal Networks (GSTN) perform excellently in spatio-temporal correlation, such as application scenarios like node prediction in traffic networks. GSTN can predict future node values or labels, as well as predict spatio-temporal graph labels, and follows two methods based on RNN and based on CNN in construction. Taking the RNN-based method as an example, adding graph convolutional units can capture spatio-temporal dependencies, and the node update calculation is:

[0089]

[0090] Among them, Gconv(·) is a graph convolution unit, and A v is the adjacency matrix of the central node at the k-th layer, representing the association between neighbor nodes and the central node; RNN(·) is a classical recurrent neural network calculation, and the detailed introduction is as follows:

[0091] h t = σ(U·x t + W·h t-1 ) (9)

[0092] Based on this, step 3 specifically includes:

[0093] Step 31: Obtain the topic keywords of the initial word vectors, and determine the nodes of the heterogeneous cognitive map according to the topic keywords. The nodes include topic nodes, non-topic nodes, and query nodes.

[0094] In the present invention, the segmented dialogue content is explicitly and implicitly connected by extracting topic keywords to construct a heterogeneous cognitive map. First, the entire dialogue context {U1, U2... U m-1 , U m} is segmented into three parts {U, Q, R}, namely the dialogue history sentences {U1, U2... U m-2}, the dialogue query sentence {U m-1}, and the response text {U m}. Among them, the dialogue history sentences are far from the response text, representing the global historical information of the dialogue. The dialogue query sentence is adjacent to the response text, representing the short-term intention information of the dialogue, and both belong to coarse-grained semantic information. In addition, different from long text analysis, there is often content in the dialogue context that has nothing to do with the direction of the entire dialogue flow. For example, "Yes, I understand". Therefore, the present invention extracts topic keywords to better understand the dialogue context. Topic keywords are special named entities, important entities distributed throughout the dialogue context, highly recognizable, representing fine-grained semantic information, and can be used to model the semantic flow association of the dialogue.

[0095] The present invention uses the term frequency-inverse document frequency algorithm (TF-IDF algorithm) to extract topic keywords. Topic keywords are high-frequency words in the dialogue text that can represent the dialogue context.

[0096] Step 32: Determine the edges of the heterogeneous cognitive map and the weight of each edge according to the nodes of the heterogeneous cognitive map. The weight is determined according to the degree of topic overlap between sentences in the dialogue text corresponding to the initial word vectors.

[0097] The implementation algorithms of step 31 and step 32 are shown in Table 1.

[0098] Table 1 Heterogeneous Cognitive Map Construction Algorithm

[0099]

[0100] Algorithm 1 elaborates on the process of constructing a heterogeneous cognitive map from dialogue text. The establishment of the heterogeneous cognitive map is to support cognitive reasoning based on the graph domain channel. Specifically, it can utilize the collaborative information in dialogue queries, dialogue history, and topic keywords to perform multi-hop reasoning to obtain stronger semantic interaction. Among them, this invention uses Stanford CoreNLP (https: / / stanfordnlp.github.io / CoreNLP) for data processing such as word segmentation and part-of-speech tagging, but it is not sufficient to represent the semantics of dialogue sentences. Therefore, the TF-IDF algorithm is used to extract topic keywords, which are high-frequency words in the dialogue text that can represent the dialogue context.

[0101] After obtaining the set of topic keywords K in this invention, this invention constructs graph nodes based on the status of topic keywords in dialogue sentences. The set of sentences containing a certain topic k and the topic k itself form the first type of important nodes in the heterogeneous cognitive map, called v k . It can be noted that a certain sentence may contain several topic keywords, which indicates that the current sentence has rich semantic information and can establish a connection channel for information interaction with other similar sentences. When a sentence does not contain topic keywords, this invention believes that the semantic role of the current sentence in the entire dialogue is relatively small and will be classified into a special node v empty . At the same time, since the dialogue query sentence is closest to the reply text, this invention believes that its semantic role is the most important and classifies it into another special node v Q .

[0102] When establishing the connections between nodes in the graph, according to the explicit and implicit relationships, an edge set E = {e K ,v empty ,v Q} is constructed from the above three types of nodes {v i,j}. v k is the "topic-sentence cluster" node, v empty is the ordinary node (the cluster of dialogue sentences without topics), and v Q is the dialogue query node. Here, it should be noted that this invention takes the heterogeneous node features into account in the subsequent node representation, and only the connection weights need to be considered when establishing edges. In steps 13-17 of the algorithm, this invention can see that when node v i and node v j share a sentence, this invention adds an edge e i,j, the more sentences they jointly own, the closer the relationship between the two nodes, and the greater the weight. In addition, due to the association of two types of special nodes, the present invention directly connects the two special nodes to construct a special edge e Q,E , which can learn important relevant information in semantic noise from the inspiration of query sentences.

[0103] Step 33: Use a graph neural network to learn the vector representation of nodes in the heterogeneous cognitive graph and obtain the graph-domain semantic representation of the initial word vectors in the graph domain.

[0104] Node semantic representation encoding

[0105] Inference on the heterogeneous cognitive graph is based on the update and learning of the graph node representation, and the initial representation of the graph node can provide correct guidance for subsequent graph neural network learning. Taking Figure 4 as an example, the present invention calculates the initial vector representations of three types of nodes:

[0106] ① For node v empty , the present invention performs average pooling on the initial semantic vectors of the sentences belonging to node v empty to obtain the node vector v e . When node v empty has no sentence set, the present invention performs average pooling on all dialogue history sentences:

[0107]

[0108] ② For node v Q , the present invention directly uses the initial vector representation of the dialogue query sentence as the node vector v Q :

[0109]

[0110] ③ For node v k , the present invention performs a concatenation operation on the topic vector and sentence vector belonging to this node, and performs a dimension transformation through a single-layer fully connected network. Taking the node where the k1 topic is located as an example:

[0111]

[0112] After obtaining the initial vector representations of the three types of nodes, for the convenience of subsequent graph neural network calculations, the present invention refers to the node vector representations {v Q , v e , v K} as {v1, v2... v m}, where m = K + 2.

[0113] Transmission and update of node information

[0114] Message passing between graph nodes is achieved through two steps: information aggregation and information combination, and this process can be carried out multiple times (usually referred to as layers or hops). Information aggregation is to converge the semantic interaction information of adjacent nodes in the same layer; information combination is to update and combine the information of the same node in different layers.

[0115] Information aggregation focuses on how a certain node collects the semantic information of adjacent nodes. The present invention notices that when the conversation lasts for several rounds, the number of adjacent nodes of a certain node is relatively large and not all of them can bring equivalent semantic information to the central node, and some even bring semantic noise. Therefore, different from ordinary graph neural networks, when updating the nodes in the l-th layer, the present invention selects GRU units to filter the information content of the adjacent node clusters to alleviate semantic noise. Specifically, the reset gate R in the gating mechanism t will control the information flow from neighbor node v j to v i :

[0116]

[0117] where R is the set of all types of edges, is the neighbor cluster of node v with edge type r i , is a certain adjacent node v j 's node representation in the l-th layer. |·| represents the size of the adjacent node cluster. The GRU unit defines the conversion process of aggregating adjacent information. The conversion of adjacent node representations can be achieved through a multi-layer perceptron (MLP). represents the aggregated information of node v i in the l-th layer. Considering the complex connection of the graph domain structure and the large number of nodes, the present invention adds a residual connection to avoid gradient disappearance while retaining its own important semantics:

[0118]

[0119] where, f s is achieved through a multi-layer perceptron (MLP).

[0120] Information combination focuses on updating and combining the representations of the same node in different layers to obtain the information content after multi-hop cognition. However, existing research shows that graph neural networks are prone to smoothing problems in inter-layer reasoning, and the smoothing problems will lead to similar node representations, thus losing the ability to distinguish information. To solve this problem, the present invention will control the size of the information flow of node v i from the l-th layer to the l + 1-th layer from different source points, and this is to add a Gate weight in the information combination:

[0121]

[0122]

[0123] Among them, sigmoid(·) determines the weight by quantifying the contribution degree of different information sources of the same node to the inter-layer information update. Specifically, determines the amount of information from the original node representation and the updated node representation when combining information, similar to a flexible residual mechanism. η(·) is the non-linear activation function Leaky ReLU, ⊙ represents element-wise multiplication, and f s and f g are both implemented using a single-layer MLP. After multiple rounds of message passing, all nodes will have their finally updated node representations.

[0124] Step 4: According to the sequential semantic representation and the graph-domain semantic representation, perform semantic enhancement on the dialogue text to obtain the enhanced semantic representation.

[0125] The sequential channel can obtain the semantic representation on the sequential dialogue data through hierarchical attention and progressive modeling of the encoder. The graph-domain channel can perform multi-hop reasoning on the dialogue intention and semantics on the graph through the heterogeneous cognitive graph constructed by the present invention and the designed dual-gated GNN, and establish multiple semantic association representations in the medium and long distances. The semantic results of the two channels complement each other, and through information collaboration, high-level semantic cognition of the entire dialogue context can be achieved.

[0126] In the graph-domain channel, the semantic representation of each node is obtained through multi-hop reasoning. This semantic representation aggregates long-distance information transmission between layers, and after the present invention obtains it, in order to facilitate collaboration with the semantics of the sequential channel in the decoder, the weight score i is used to manage the semantic information of each node:

[0127]

[0128]

[0129] Among them, is the semantic representation of the dialogue query sentence that has not entered the dual-channel, representing the initial dialogue intention, which has a good guiding effect on the generated reply. Therefore, the present invention uses it to calculate the information management weight of the node information; NumL is the total number of semantic nodes.

[0130] In the information collaboration module of the dual-channel, the present invention also uses a Gate mechanism to control the influence of the semantic information of the two channels on the decoding process of the generated reply:

[0131]

[0132]

[0133] Among them, δ is the number of semantics in the sequence channel The number of semantics delivered to the decoder, and 1 - δ represents the graph domain channel The number of semantics delivered to the decoder. The two parts of semantics are combined into the final semantics-enhanced semantic representation c through addition final . c final is the integrated semantics learned from the development of sequence semantics and the association of graph domain semantics. It emphasizes the dialogue direction of the dialogue query sentence in information integration and can assist the decoder to accurately decode and generate new words

[0134] Step 5: Generate a response text according to the enhanced semantic representation, specifically including

[0135] Step 51: Input the enhanced semantic representation into a unidirectional gated neural network to obtain the hidden state of each word in the generated response text

[0136] In the decoder module part, the present invention uses a unidirectional GRU to decode and generate the latest hidden state to update the semantic vector of the entire decoding layer, so as to obtain the hidden state to obtain the probability distribution of the decoding vocabulary

[0137]

[0138] Among them is the hidden state of the decoding layer when generating the i-th word y i in the response text, c final is the semantic representation after the collaboration of dual-channel information, representing the semantic inspiration after the dialogue idea is clear, and y i-1 is the vector representation of the (i - 1)-th word in the response text of the present invention during training, and is represented by the vector representation of the (i - 1)-th word in the predicted text during prediction can be used to ensure the consistency of the response text

[0139] Step 52: Determine the generation probability of each word according to the hidden state, and determine the response text according to the generation probability

[0140] When decoding and generating text, the present invention believes that the generated response tends to extend from the topic keywords. Therefore, a topic bias probability is added to distinguish from the previous encoder, forcing the model to consider the topic development. The corresponding generation probability is calculated as

[0141]

[0142] Among them, K and V respectively represent the topic keyword vocabulary and the response text vocabulary. Correspondingly, p V and pK The probability values are all obtained by Softmax normalization:

[0143]

[0144]

[0145] Among them, η(·) is the nonlinear function tanh. During the training process, the present invention defines θ as the trainable parameter, divides the training text into batches for training, and obtains the best model effect by optimizing the cross-entropy loss function based on negative log-likelihood. The learning parameter is the process of gradient backpropagation and update descent, where:

[0146]

[0147] The present invention first proposes a neural network model for a fine-grained information interaction method based on topic-enhanced dialogue history understanding. On the one hand, the model of the present invention uses the topic semantics and each sentence for fine-grained semantic interaction to obtain an enhanced semantic representation of the dialogue history sentences; on the other hand, it uses the dialogue query sentence to guide the fusion of the topic matrix to obtain the dialogue intention semantic representation. The two operations aim to use the topic semantics to enhance the understanding of the context, thereby breaking through the disadvantages of using topic information indiscriminately in the past;

[0148] The present invention first breaks the fixed thinking of the sequence structure modeling of the dialogue history context content, and proposes a model based on the collaborative semantic modeling and reasoning method of the sequential and graph domains. Drawing on the "twin towers" model in the recommendation system, our dual-channel model can understand and train the entire dialogue context from the perspective of the graph domain. At the same time, the dual-channel collaborative semantic modeling can maximize the semantic value, broadening the research ideas of the multi-turn dialogue system.

[0149] The multi-turn dialogue response generation method based on dual-channel semantic enhancement of the present invention can be applied to scenarios such as intelligent customer service systems on e-commerce platforms, voice interaction modules of bionic AI robots, and new-style retrieval on portal websites. In addition, it can also be embedded in military information services to achieve scenarios such as intelligent planning and analysis of battle situation texts and assisting in command and decision-making, improving the efficiency and interaction experience of information services.

[0150] The method of the present invention improves the fluency, diversity, and rationality of the multi-turn dialogue generation response, enhances the robustness of the generative dialogue, and optimizes the ability effect of text semantic modeling.

[0151] Next, the present application will be described in more specific embodiments.

[0152] Experimental preparation:

[0153] 1. Research questions

[0154] In the Sequential and Graph Dual-channel Collaborative (SGDC) semantic modeling and reasoning model proposed in the present invention, the following three research questions are proposed in this embodiment to guide subsequent experiments:

[0155] RQ1: Does the SGDC model of this application perform better than other baseline models in terms of fluency, relevance, and diversity?

[0156] RQ2: What is the impact of the length (number of turns) of the entire dialogue context on the performance of our SGDC model in generating multi-turn dialogue responses?

[0157] RQ3: When the model decodes and predicts the response text, does the collaborative method of the sequential and graph domain dual channels affect the overall performance of our SGDC model?

[0158] 2. Datasets

[0159] In this embodiment, the DailyDialog dataset and the MuTual dataset are selected for experiments.

[0160] The DailyDialog dataset was collected by scholars in the field from daily life. There are a total of 13,118 conversations, covering various topics such as education, travel, weather, and shopping, which can reflect most of the communication between us humans. The semantic structure of DailyDialog is more standardized and formal, more valuable in terms of theme, and the number of speakers is reasonable, and the number of dialogue turns is not redundant, making it more valuable for research and application.

[0161] The MuTual dataset is a high-quality manually annotated multi-turn dialogue reasoning dataset, which contains a total of 8,860 manually annotated conversations based on the English listening comprehension test for Chinese students. Compared with previous dialogue benchmark datasets, MuTual is more challenging in terms of reasoning. The situations of the above two datasets are shown in Table 2.

[0162] Table 2 Dataset Information

[0163] DailyDialog MuTual Number of dialogues 13118 8860 Average number of turns in a dialogue 7.9 4.73 Average number of words in a sentence 14.6 19.57

[0164] 3. Experimental control baseline models

[0165] In this embodiment, five relevant multi-turn dialogue generation models are selected as the baseline algorithm models for comparison with the algorithm model of this application in terms of overall performance, and the experimental effects are discussed and analyzed. The brief introduction of the baseline algorithm models is as follows:

[0166] S2S-Att: The most popular encoder-decoder framework. The encoder encodes the input sequence into an intermediate state, and then the decoder is used for decoding and generation. Both the encoder and the decoder adopt the gated neural network GRU. Meanwhile, the Attention mechanism is added to the decoder input at each time step to ensure that the word predicted each time is the most relevant to the input text.

[0167] HRED: The first hierarchical context modeling method for response generation. It uses utterance-level GRUs to encode each sentence and uses dialogue-level GRUs to transform the utterance vectors into dialogue-level vector representations. Compared with the ordinary S2S framework, it considers the three-level semantic progression of "vocabulary-utterance-dialogue", which can help the aggregation and propagation of information at each level, thus realizing the modeling of multi-turn dialogue history.

[0168] THRED: The first model introducing topic awareness in the field of multi-turn dialogue generation. Specifically, THRED introduces topic awareness on the basis of the HRED model and uses the joint attention mechanism of topic-context to guide the decoding of responses.

[0169] ReCoSa: It uses the Self-Attention mechanism to associate with the dialogue context most closely related to the response text. It is an improved hybrid model of Transformer and HRED, with attention mechanisms embedded in both word-level and utterance-level encoders for hierarchical modeling, and currently has the most advanced performance in the field of multi-turn dialogue generation.

[0170] In addition, to explore the impact of the collaborative method of dual-channel semantics on model performance, we constructed three baseline models according to common semantic feature aggregation techniques, which are briefly introduced as follows:

[0171] -Avg: A dual-channel model adopting the mean aggregation strategy, focusing on background semantics;

[0172] -Max: A dual-channel model adopting the maximum aggregation strategy, focusing on foreground semantics;

[0173] -Concat: A dual-channel model adopting the mean aggregation strategy, focusing on global semantics.

[0174] The aggregation strategy of semantic features is an important strategy adopted when aggregating several semantic vectors or converting a semantic matrix into a fixed-length vector representation. As Figure 5 shown, when semantic features are aggregated, it can reduce information redundancy, converge on focal semantics, and prevent overfitting in training, which is similar to the role of the pooling layer in convolutional neural networks. Common aggregation strategies include maximum aggregation, mean aggregation, concatenation aggregation, and gated aggregation.

[0175] Maximum aggregation strategy

[0176] The operation of the maximum aggregation strategy is to take the maximum value of the vector element values in each dimension as the vector element value of the same dimension in the new semantic vector representation. According to the process of the maximum aggregation strategy, the other element values of the same dimension will not be passed to the next layer of semantic vectors. Such aggregation enables the model to focus on the most prominent and shallow parts of the semantics. Texts with single vocabulary, simple syntax, and shallow semantics can adopt this aggregation strategy. The maximum pooling layer can be used for the aggregation operation. The maximum aggregation formula is:

[0177] h max = max{h1, h2... h i} (26)

[0178] Mean aggregation strategy

[0179] The operation of the mean aggregation strategy is to take the average of the vector element values in each dimension as the vector element value of the same dimension in the new semantic vector representation. The mean aggregation strategy believes that each element in each dimension contains equal semantic value, and semantic neutralization can be achieved through averaging, reducing the deviation of the semantic direction. It is a strategy that can improve the robustness of the model. In the later positions of a deep neural network, the semantic information is rich and balanced, making it more suitable for the mean aggregation strategy. The mean aggregation formula is:

[0180]

[0181] Connection aggregation strategy

[0182] The connection aggregation strategy is an aggregation strategy derived by taking into account the advantages of both the mean aggregation and the maximum aggregation. Taking two feature vectors v1 ∈ R n and v2 ∈ R m as an example, the common operation of the connection aggregation strategy is to concatenate the vector elements. The most obvious thing is that the dimension increases to v concat ∈ R n+m . Connection aggregation is to attach semantic information violently, retaining semantic information to the greatest extent. It is a direct but very effective strategy. After concatenation, a linear mapping is often used to transform to the required number of dimensions. The formula for connection aggregation is:

[0183]

[0184] Gated aggregation strategy

[0185] The gated aggregation strategy supports flexible information collaboration and controls the input level of each dimension or vector semantics through Gate calculation. The specific operation of the gated aggregation strategy will learn the Gate value through the activation function sigmoid and gradient update in deep learning, and then combine the Gate value and each state through the Hadamard product to aggregate the final vector representation. The gated aggregation strategy supports deep learning training updates, and can confirm the important weights of each dimension aggregation based on the neural network. It is suitable for models with complex semantics and reasoning tasks. The formula for gated aggregation is:

[0186]

[0187] 4. Experimental evaluation criteria and indicators

[0188] The automated evaluation of multi-round dialogue response generation mainly considers the fluency, diversity, and topic relevance of the generated text. The fluency of the text represents the reasonable compliance of the grammatical and semantic associations of words, that is, the probability of the deep learning model in the part-of-speech sequence and vocabulary collocation distance is maximized, which meets the requirements of human-readable and understandable text and does not have ambiguous understanding. The diversity of the text represents the semantic richness of the dialogue context and has extensible characteristics. It is not a meaningless response of "I don't know", which is a necessary requirement for a chat-type dialogue system. The topic relevance refers to whether the response has practical significance and conforms to the topic continuation or transition of the dialogue scene. It is an important direction for judging the response generation task.

[0189] In terms of specific implementation, in order to better evaluate the pros and cons of the baseline algorithm and the research work of this application, the perplexity PPL, dist-1 and dist-2 and the sentence relevance indicators based on Embedding are used to evaluate the fluency, diversity and relevance of the generated responses according to the literature (completion).

[0190] PPL: According to the reference, we use language model perplexity (PPL) to evaluate the fluency of generated text. The lower the PPL value, the higher the probability of the generated reply text, the more reasonable the vocabulary arrangement and collocation, and the easier it is to understand and flow. The formula is:

[0191]

[0192] Distinct: This article uses the automated evaluation indicators Distinct-1 and Distinct-2 to evaluate the content diversity of the generated text. The higher the Distinct-n score, the higher the proportion of n-grams in the sentence, which means that the generated text is richer in content and the response effect is better. The calculation formula is as follows:

[0193]

[0194] Embedding: Different from calculating the overlap or co-occurrence degree between prediction and truth in the ngram way, the Embedding-based evaluation method transfers the text into low-dimensional semantic representations, and then measures the correlation degree through text similarity. In this embodiment, Greedy Matching (GM), Embedding Average (EA), and Vector Extrema (VE) are used for evaluation. The larger the values of the three evaluation metrics, the closer the semantic correlation between the predicted text and the true text, and the more relevant the response is.

[0195] Greedy Matching (GM). The embedding greedy value metric uses greedy search to generate as many words or semantics similar to the keywords in the true text as possible, considering word-level alignment in a more fine-grained manner, and is more accurate for long text evaluation. The formula calculation is as follows:

[0196]

[0197]

[0198] Embedding Average (EA). The embedding average metric is widely used to measure text similarity. Cosine similarity is used to measure the semantic vectors of the predicted and true texts, where the semantic vector is a method of calculating the phrase meaning by averaging the vector representations of its constituent words. The formula calculation is as follows:

[0199]

[0200] Vector Extrema (VE). The embedding extrema metric uses the extrema of each dimension of the word vector when calculating the text vector, and is measured and compared using cosine similarity as above. It should be noted that this evaluation metric focuses on information extrema, that is, topic information, so it can be used to measure topic relevance. The formula calculation is as follows:

[0201]

[0202] Parameter settings and implementation environment

[0203] To fairly compare the baseline algorithm and the algorithm model of the present invention, the embodiments of the present invention all adopt the Adam optimizer and the Pytorch framework. During training, the word embedding vectors are randomly initialized and the model is updated. The dimension is 512. The input and output hidden dimensions of all recurrent neural network (GRU and BIGRU) units are also 512. The model learning rate is set to 0.0001 for gradient clipping. The number of samples (batch size) participating in training for each iteration is set to 64, and optimization training and validation prediction are both carried out on a workstation with NVIDIA TITAN RTX GPU.

[0204] In addition, the topics in the model of this embodiment are extracted by TF-IDF. To accelerate the training process and prevent error accumulation caused by large model errors in the early training process, a teacher forcing mechanism is introduced to force the input of the decoder to be the target token, thereby reducing error transmission in the model and ensuring that the parameters can be updated normally.

[0205] The software and hardware implementation configurations for this work are shown in Table 3 below:

[0206] Table 3 Software and Hardware Implementation Configurations

[0207]

[0208] Analysis and Discussion of Experimental Results:

[0209] 1. Overall Performance Compared with the Baseline Model

[0210] To explore Research Question 1 (RQ1), in this embodiment, the SGDC model is compared with the baseline model of the DialogueRNN framework on two datasets, MuTual and DailyDialog. The evaluation results of fluency, relevance, and diversity are shown in Table 4. Among them, the index values of the best-performing baseline models are underlined, and the best evaluation index values are in bold.

[0211] Table 4 Performance Comparison between the SGDC Model and the Baseline Model of the DialogueRNN Framework

[0212]

[0213]

[0214] It can be seen from the experimental results in Table 4 that:

[0215] 1 Obviously, the SGDC model for dual-channel semantic modeling in this embodiment outperforms other baseline models in most evaluations on the two datasets. The significantly superior model performance on the two datasets demonstrates the effectiveness of the sequential and graph-domain dual-channel semantic collaborative modeling approach, which can obtain semantic associations and inference effects that cannot be mined by the DialogueRNN framework.

[0216] 2 In the evaluation of the relevance dimension, we found an interesting phenomenon: taking the MuTual dataset as an example, the SGDC model obtained relatively high scores in all three metrics evaluated for relevance. Compared with the best baseline model, the VE and EA scores were about 2% higher, and the GM score was 3.5% higher. Compared with all other baseline models, the VE, EA, and GM scores were all 5% higher. Similarly, the same was true for the DailyDialog dataset, but the improvement was not as large as that of the MuTual dataset. To explain this ability gap in relevance, it is important to understand the essence of these models and datasets. The MuTual dataset is a more rigorously annotated dataset focusing on multi-turn dialogue reasoning. The SGDC model mines semantic connections from different structural perspectives. The sequential channel can capture general progressive semantics, and the graph-domain channel can obtain long-distance information dependencies across distance barriers through edge connections in multi-turn dialogues. The collaborative modeling advantage of this dual-channel is obviously superior to other baseline models of the DialogueRNN framework.

[0217] 3 The score of this embodiment is significantly higher than that of ReCoSa. This may be because the graph-domain channel of the present invention is constructed based on the relationship of "topic-sentence cluster" nodes. Semantic reasoning on the heterogeneous graph is more effective than the prior mechanism of ReCoSa that only focuses on question-answer correspondence. This also shows that the model of the present invention can accurately perceive the topic direction of the dialogue, so a similarity score closer to the real response text can be obtained, and the dialogue trajectory on the topic can be maintained.

[0218] 4 Surprisingly, there is not much difference between the model of the present invention and other benchmark models in terms of the diversity Dist-1 and Dist-2 scores. This may be because the semantic modeling of the present invention pursues semantic connections too much and also uses topic bias probabilities, thus losing a certain amount of text diversity. This is a point that is difficult to balance for dialogue generation models. However, from the perspective of scenario applications, sacrificing controllable diversity in exchange for more closely related responses is also worthwhile.

[0219] The impact of dialogue context length on performance

[0220] To explore RQ2, this embodiment analyzes the performance of the SGDC model and the baseline model on test samples with different context lengths, that is, the number of turns in the multi-turn conversation is different. This embodiment manually divides the sampled test set into three groups according to the conversation context length, namely short conversations (the number of conversation turns is less than 5), medium conversations (the number of conversation turns is between 6 and 10), and long conversations (the number of conversation turns is greater than 10). Use some objective evaluation metrics to evaluate the performance of each model, and draw the results as Figure 6 . In each model in the figure, the three bar charts from left to right represent short, medium, and long respectively.

[0221] It can be seen from the experimental results that:

[0222] 1 Whether it is the SGDC model or the baseline model, as the conversation length increases, the perplexity score shows a monotonous increase to varying degrees. This shows that the longer the conversation length, the more complex the semantic modeling of the conversation, the more difficult it is to capture information association, and the model is easily affected by irrelevant semantic noise, resulting in a decline in prediction ability.

[0223] 2 The SGDC model of the present invention performs better than the baseline model in both short, medium, and long conversation test sets, which proves the robustness of the model of the present invention. In addition, compared with the best benchmark model, SGDC has the largest improvement in long conversations, which shows that the graph domain channel of SGDC indeed plays a role in capturing long-distance semantic dependencies and has a unique ability advantage in long conversation semantic modeling.

[0224] The impact of the collaborative method of dual-channel semantics on performance

[0225] To explore RQ3, this embodiment changes the way of dual-channel information collaboration in the SGDC model and designs three variant models to explore the best aggregation strategy for semantic information in the two channels. The three variant models are:

[0226] · SGDC Avg : The SGDC Avg model selects the averaging aggregation strategy. This collaborative method assumes that the semantic information in the two channels is equal. Therefore, for the two semantic vectors, the semantic representation of the entire context is obtained through average pooling;

[0227] · SGDC Max : The SGDC Max model focuses on selecting the most important semantic features and adopts the maximization aggregation strategy. This collaborative method assumes that the maximum value in the semantic vector representation can reflect important semantics. Therefore, for the two semantic vectors, the semantic representation of the entire context is obtained through max pooling;

[0228] · SGDC Concat: SGDC Concat The model believes that the semantic information of the two channels is equally important and there should be no loss. Therefore, the semantic representation of the entire context is obtained by directly combining the semantic vectors;

[0229] Table 5 Results of the impact of collaboration methods on performance

[0230]

[0231] For the convenience of comparison, we call the model used in the above experiment SGDC Gate , and record SGDC Gate and the generation effects of its three variants on the MuTual and DailyDialog datasets. Among them, the evaluation metrics of semantic relevance (GM\EA\VE) are selected in this embodiment. It can be found from the differences in the evaluation metrics in Table 5 that:

[0232] ① SGDC Gate is significantly better than SGDC Avg and SGDC Max on the three metrics of semantic relevance. This shows that the semantic information of the two channels has its own uniqueness and importance. Only by finding the best aggregation strategy through the Gate mechanism can the benefit of semantic information aggregation be maximized. The averaging aggregation strategy of SGDC Avg may cause the loss of important semantics respectively, and the maximization aggregation strategy of SGDC Max is too focused on important semantics and belongs to local relevance, making it difficult to capture the balance of semantic associations and thus losing the overall semantic relevance of the response text.

[0233] ② SGDC Gate has little difference from SGDC Concat in the evaluations of EA and VE, but is significantly better than SGDC Concat in GM. This curious phenomenon can be explained from the specific details of the evaluation metrics. EA and VE measure the average level and extreme value level of the word embedding similarity between the predicted text and the real text respectively. SGDC Gate finds the balance point of aggregation through the Gate mechanism, while SGDC Concat accepts everything. The semantic enhancement of the graph domain channel itself captures long-distance dependencies. Therefore, they are different ways to achieve the same goal of accurate solution and brute-force solution, and there is not much difference in the average level and extreme value level of the predicted text. The GM value not only considers the similarity of word embeddings, but also takes into account the alignment between words, so it is a more fine-grained evaluation item. At this time, the advantage of the Gate mechanism is reflected, and SGDC Gate can filter more fine-grained semantic noise than SGDC Concat .

[0234] Drawing on the rich research ideas of graph neural network technology, the present invention proposes a collaborative semantic modeling and reasoning method based on a dual-channel of sequence and graph domains, and designs a dual-gated graph neural network based on nodes on the graph. Moreover, the model of the present invention has been experimentally verified on both open-domain datasets and dialogue reasoning datasets. The experimental results show the advantages of the collaborative semantic modeling and reasoning method of the dual-channel. At the same time, as the number of dialogue turns increases, the model of the present invention still has good robustness.

[0235] Starting from the dissection and reuse of content features and the dual-channel enhancement of structural features, the model solution proposed by the present invention has relatively good practical application value:

[0236] (1) The generative dialogue method only considers the progressive semantics of the sequence structure and ignores the strong correlation interaction of distant contexts. To effectively and comprehensively utilize context information, the present invention adopts the most direct method, that is, to break the sequence structure, draw on the thinking mode of repeated thinking in human conversations, and use context information to design a heterogeneous graph that can perform cognitive reasoning. This idea of "breaking and then establishing" can expand the effectiveness of research.

[0237] (2) In the information transmission process of updating node representations in graph neural networks, the present invention designs a dual-gated GNN to filter and screen information transmission through GRU units for information aggregation and a Gate mechanism for information combination. This design can effectively filter semantic noise and capture key semantic information, which is an attempt in the field of dialogue generation.

[0238] The second aspect of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0239] The present invention proposes a method for collaborative semantic modeling and reasoning of a dual-channel of sequence and graph domains, aiming to integrate the semantic advantages in different structural modeling to obtain a greater span of information association and semantic reasoning. Specifically, on the one hand, the present invention constructs a dialogue-level heterogeneous cognitive graph, where the graph nodes are the integration of topic semantics and sentence cluster semantics, and the edges in the graph are the degree of topic overlap between sentences. Then, a dual-gated graph neural network is used for deep learning to obtain the semantic representation of the dialogue context in the graph domain. On the other hand, a hierarchical attention mechanism is embedded in the remaining sequence channel to obtain the sequential semantic representation of the dialogue context. Finally, the information contributions of the two semantic representations are coordinated for prediction. The model of the present invention performs excellently on the benchmark model and alleviates the long-distance semantic dependence problem.

[0240] The above are only the preferred embodiments of the present invention and do not impose any form of limitation on the present invention. Although the present invention has been disclosed above in its preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can, without departing from the scope of the technical solution of the present invention, make many possible changes and modifications to the technical solution of the present invention by using the technical content disclosed above, or modify it into equivalent embodiments with equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention.

Claims

1. A method for generating multi-turn dialogue responses based on dual-channel semantic enhancement, characterized in that, Including: Obtain the initial word vector of the dialogue text; Obtain the sequential semantic representation of the initial word vector, including obtaining the utterance-level sentence semantic vector of the initial word vector, determining the dialogue-level sentence semantic vector of the initial word vector according to the utterance-level sentence semantic vector, and denoting the dialogue-level sentence semantic vector as the sequential semantic representation; Obtain the graph domain semantic representation of the initial word vector on the graph domain; Semantically enhance the dialogue text according to the sequential semantic representation and the graph domain semantic representation to obtain an enhanced semantic representation; Generate a response text according to the enhanced semantic representation; Obtain the graph domain semantic representation of the initial word vector on the graph domain, specifically including: Obtain the topic keywords of the initial word vector, and determine the nodes of the heterogeneous cognitive graph according to the topic keywords, where the nodes include topic-sentence cluster nodes, dialogue query nodes, and ordinary nodes; Determine the edges of the heterogeneous cognitive graph and the weight of each edge according to the nodes of the heterogeneous cognitive graph, where the weight is determined according to the topic coincidence degree between sentences in the dialogue text corresponding to the initial word vector; Use a graph neural network to learn the vector representation of the nodes in the heterogeneous cognitive graph to obtain the graph domain semantic representation of the initial word vector on the graph domain; Semantically enhance the dialogue text according to the sequential semantic representation and the graph domain semantic representation, specifically including: Semantically enhance the dialogue text according to the first formula, and the first formula is: where c final is the enhanced semantic representation, is the sequential semantic representation, is the graph domain semantic representation, δ is the number of semantics in the sequential semantic representation, and (1 - δ) is the number of semantics in the graph domain semantic representation; Generate a response text according to the enhanced semantic representation, specifically including: Input the enhanced semantic representation into a unidirectional gated neural network to obtain the hidden state of each word in the generated response text; Determine the generation probability of each word according to the hidden state, and determine the response text according to the generation probability.

2. The method according to claim 1, characterized in that, Obtain the utterance-level sentence semantic vector of the initial word vector, specifically including: Input the initial word vector into a sentence-level encoder and a word attention module in sequence to obtain the utterance-level sentence semantic vector; Determine the dialogue-level sentence semantic vector of the initial word vector according to the utterance-level sentence semantic vector, specifically including: Input the utterance-level sentence semantic vector into a context encoder and a sentence attention module in sequence to obtain the dialogue-level sentence semantic vector; Both the sentence-level encoder and the context encoder are bidirectional gated neural networks; The mechanisms used in both the word attention module and the sentence attention module are attention mechanisms.

3. The method according to claim 1, characterized in that, Input the enhanced semantic representation into a unidirectional gated neural network to obtain the hidden state of each word in the generated response text, specifically including: Generate the hidden state of each word in the response text according to the second formula, and the second formula is: where y i is the i-th word in the generated response text during the training phase, and y i-1 is the (i - 1)-th word in the generated response text during the training phase, is the hidden state of y i , GRU(·) represents inputting the parameters therein into a gated neural network, is the hidden state of y i-1 , and c final is the enhanced semantic representation.

4. The method according to claim 1, characterized in that, Determine the generation probability of each word according to the hidden state, specifically including: Determine the generation probability of each word according to the third formula, and the third formula is: In the formula, is the i-th word in the generated response text during the prediction phase, is the generation probability of, and are the generation probabilities of the i-th word in the topic keyword vocabulary and the response text vocabulary during the prediction phase, respectively. of.

5. The method according to claim 4, characterized in that, The said Determined according to the fourth formula, the fourth formula is: where η(·) is the non - linear function tanh, V is the vocabulary of response texts, K is the vocabulary of topic keywords, is the hidden state of the i - th word y in the response text generated during the training phase, i y i-1 is the (i - 1) - th word in the response text generated during the training phase, c final is the enhanced semantic representation, and vocab represents the variable i.

6. The method according to claim 4, characterized in that, The determined according to the fifth formula, the fifth formula being: where η(·) is the non - linear function tanh, V is the vocabulary of response texts, and K is the vocabulary of topic keywords. is the hidden state of the i - th word y i in the response text generated during the training phase, and y i-1 is the (i - 1) - th word in the response text generated during the training phase, c final is the enhanced semantic representation, and vocab represents the variable i.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

Images