Digital human continuous dialogue and memory enhancement method based on knowledge graph

Abstract

The invention relates to the field of artificial intelligence, natural language processing and knowledge graph modeling, in particular to a digital human continuous dialogue and memory enhancement method based on a knowledge graph. Comprising the following steps: receiving a dialogue text and a historical dialogue context input by a user in the current round, and processing through a multi-semantic channel to obtain a channel semantic representation vector and a semantic vector of a dialogue in the current round; performing language multi-dimensional scoring on each node in the global knowledge graph to obtain a multi-dimensional score of the node, and forming a knowledge graph sub-graph used by the current round of dialogue; for each activated node, calculating memory intensity; and performing weighted fusion on the node embedded representation vector, generating a semantic enhancement vector fusing the current context information and the historical memory content, and generating a natural language reply conforming to the current context. The technical problems that an existing digital human dialogue lacks long-term memory and context consistency in multiple rounds of dialogues, and a knowledge graph is difficult to dynamically update are solved.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence, natural language processing and knowledge graph modeling, and in particular to a method for continuous dialogue and memory enhancement of digital humans based on knowledge graphs. Background Art

[0002] With the rapid development of artificial intelligence (AI), digital humans have become an essential tool for natural user interaction. By integrating natural language processing, computer vision, and speech recognition technologies, digital humans can simulate conversations and interactions with humans, and are widely used in a variety of fields, including customer service, healthcare, education, and entertainment. However, most existing digital human conversational approaches rely on traditional rule-based question-and-answer models or deep learning models. While these technologies have achieved good results in short-term conversations, they often exhibit significant limitations when faced with more complex, multi-round conversational tasks.

[0003] Most existing digital human conversation methods and technologies struggle to effectively address the contextual consistency issues encountered during multi-round conversations. Existing methods typically only handle single-round conversations or maintain only short-term contextual memory, failing to maintain consistency across multiple conversation rounds. This can lead to information loss or misunderstandings during multi-round conversations, and makes them incapable of handling complex, long-term conversational tasks. Furthermore, while knowledge graphs provide digital humans with rich background knowledge, existing technologies often rely on static knowledge retrieval and fail to effectively dynamically adjust graph content based on conversational context, resulting in inflexible and inefficient knowledge application.

[0004] Most current digital human dialogue methods lack the ability to recognize and memorize personalized user needs. They are unable to adjust responses based on historical conversation content and user preferences, making digital human interactions appear mechanical and monotonous, lacking sufficient adaptability. Furthermore, while some methods have attempted to introduce memory enhancement mechanisms, these are often limited to simple information storage and fail to form intelligent memory management models based on context and historical interaction content. This can lead to outdated knowledge or incorrect memory path updates, compromising the intelligence and accuracy of dialogue.

[0005] Therefore, there is an urgent need for a method that can maintain contextual coherence, dynamically schedule knowledge graphs, and enhance memory capabilities in multi-round conversations to improve the conversation quality and intelligence level of digital humans. Summary of the Invention

[0006] The present invention provides a method for continuous dialogue and memory enhancement of digital humans based on knowledge graphs to solve the technical problems of existing digital human dialogues, such as lack of long-term memory and context consistency in multi-round dialogues, difficulty in dynamic updating of knowledge graphs, insufficient semantic understanding, and insufficient personalized responses.

[0007] The present invention provides a method for continuous conversation and memory enhancement of digital humans based on knowledge graphs, specifically including the following technical solutions: A method for continuous conversation and memory enhancement of digital humans based on knowledge graphs, comprising the following steps: S1. Receive the user's current round of conversation text and historical conversation context, process them through multiple semantic channels, and obtain a channel semantic representation vector. Based on the channel semantic representation vector, obtain the semantic vector of the current round of conversation. S2. Based on the semantic vector of the current round of dialogue, perform a multi-dimensional score on each node in the global knowledge graph to calculate the multi-dimensional score of the node; based on the multi-dimensional score, construct the knowledge graph subgraph used for the current round of dialogue; S3. For each node in the activated knowledge graph subgraph in the current dialogue round, calculate its memory strength in the current context; S4. Perform weighted fusion on the node embedding representation vectors based on the memory strength of each activated node to generate a semantically enhanced vector that combines the current context information with the historical memory content. S5. The semantic vector of the current round of dialogue, the semantic enhancement vector, and the historical dialogue context encoding vector are input into the response generation module to generate a natural language response that conforms to the current context.

[0008] Preferably, the S1 specifically includes: In each semantic channel, the current input text, historical conversation context and corresponding graph structure are jointly modeled to extract the channel semantic representation vector. The channel semantic representation vector is then weighted according to its relevance in the current context to obtain the weighted channel semantic representation vector.

[0009] Preferably, the S1 specifically includes: All weighted channel semantic representation vectors are aggregated through the fusion function to obtain the semantic vector of the current round of dialogue.

[0010] Preferably, the S2 specifically includes: In the implementation of multi-dimensional scoring, the following three factors are used as scoring factors: first, the semantic similarity between the semantic vector of the current round of dialogue and the node embedding representation vector; second, the mutual information between the node embedding representation vector and the historical dialogue context; third, the graph structure similarity between the structural position of the node in the global knowledge graph and the nodes related to the historical dialogue context.

[0011] Preferably, the S2 specifically includes: The three score factors are weighted and summed to calculate the multi-dimensional score of each node, and the node with a multi-dimensional score higher than a set activation judgment threshold is an activated node, and the activated node forms a knowledge graph subgraph used in the current round of dialogue.

[0012] Preferably, the S3 specifically comprises: The modeling of memory strength combines the multi-dimensional score of the activated node, the frequency of the activated node being called in the historical dialogue and the time interval experienced by the activated node from the last call, and after the combination, the activated function is normalized to obtain the memory strength of each activated node in the current round of dialogue.

[0013] Preferably, the S4 specifically comprises: In the process of weighted fusion of node embedding representation vectors, each node embedding representation vector is multiplied by its corresponding memory strength, and then the weighted results of all nodes are summed to obtain a semantic enhanced vector integrating current context information and historical memory content.

[0014] Preferably, the S5 specifically comprises: The response generation module is a multi-layer natural language generation model, and in the generation process of the natural language generation model, the response generation model integrates the current user semantics, the historical dialogue context and the knowledge memory content to generate a natural language reply conforming to the current context.

[0015] The technical scheme of the application has the following advantages: 1. The application can simultaneously capture the structured features, entity association and context co-occurrence mode of dialogue text through parallel modeling and dynamic weighted fusion of multiple semantic channels, and significantly improves the analysis ability of complex user intent compared with a single semantic channel.

[0016] 2. The application can dynamically select the most relevant knowledge graph subgraph based on the multi-dimensional scoring mechanism, and enhance the adaptability and retrieval accuracy of the knowledge graph in the dialogue context.

[0017] 3. The application proposes a memory modeling method combining access frequency and time decay, which weights the nodes in the activated subgraph through a self-defined memory strength function, and has the ability to distinguish between "short-term hotspots" and "long-term importance", significantly improving the dialogue memory regulation effect.

[0018] 4. The application designs an enhanced representation generation mechanism for semantic and memory fusion, which fuses and calculates the node embedding representation vector and the memory strength to construct a semantic enhanced vector for guiding response generation, improving the context consistency and knowledge relevance of the output content.

[0019] 5、The application fuses the semantic vector of the current round of dialogue, the semantic enhancement vector and the historical dialogue context coding vector, and generates a natural language reply conforming to the current context. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A method flowchart for digital human continuous dialogue and memory enhancement based on a knowledge graph according to the application. DETAILED DESCRIPTION

[0021] In order to further illustrate the technical means and effects taken by the present application to achieve the predetermined object of the application, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0023] The specific scheme of the method for digital human continuous dialogue and memory enhancement based on a knowledge graph provided by the present application will be specifically described below in combination with the drawings.

[0024] Referring to the drawings Figure 1 It shows a method flowchart for digital human continuous dialogue and memory enhancement based on a knowledge graph according to an embodiment of the present application, which comprises the following steps: S1. Receiving the dialogue text input by the user in the current round and the historical dialogue context, processing through a multi-semantics channel to obtain a channel semantic representation vector; obtaining a semantic vector of the dialogue in the current round based on the channel semantic representation vector; The conversation text and historical conversation context input by the user in the current round are received and processed separately through multiple semantic channels. Each semantic channel represents a specific type of knowledge graph structure, such as a grammatical structure graph, an entity relationship graph, or a context co-occurrence graph. In each semantic channel, based on the Graph Attention Network (GAT) mechanism in the existing Graph Neural Network (GNN), an improved method of multi-semantic channel parallel modeling and channel weighted fusion is proposed to jointly model the current input text, historical conversation context, and the corresponding graph structure, and extract the channel semantic representation vector. Subsequently, the channel semantic representation vector is weighted according to its relevance in the current context to obtain the weighted channel semantic representation vector. Finally, all weighted channel semantic representation vectors are aggregated through the fusion function to obtain the semantic vector of the current round of conversation. , as the basic input for subsequent steps, the calculation formula is as follows: , in: Represents the semantic vector of the current round of dialogue; Indicates the index number of the current round of dialogue; Indicates that the user is in the current round (i.e. Dialogue text entered by round); Refers to the Turn Between rounds, the conversation text input by the user and the digital human’s response constitute a multi-round conversation record set, which is formally represented as , that is, the historical conversation context, It is The digital human in the round responds to the user's input dialogue text the generated natural language response; Represents the semantic channel index; represents the total number of semantic channels; Indicates the The adaptive fusion weight of the semantic channel is used to reflect the importance of the semantic channel in the current context and is obtained by dynamic calculation of the channel attention mechanism. Specifically, The channel semantic representation vector output by the semantic channel is jointly scored with the historical conversation context to obtain the channel relevance score. The function normalizes the correlation scores of all channels to obtain , its value range is , and meet ; Indicates the The knowledge graph structure corresponding to each semantic channel, such as grammatical structure graph, entity graph or context co-occurrence graph; The graph attention network is The encoding function on the semantic channel, the input is the dialogue text of the current round , historical conversation context and The knowledge graph structure corresponding to the semantic channel , the output is Semantic sub-vectors under semantic channels; Is a function placeholder, indicating "unified fusion processing of the output of multiple semantic channels", and its prototype is commonly found in Fusion, gating mechanism or attention mechanism, the specific choice depends on the user design requirements.

[0025] S2. Based on the semantic vector of the current round of dialogue, perform a multi-dimensional score on each node in the global knowledge graph to calculate the multi-dimensional score of the node; based on the multi-dimensional score, construct the knowledge graph subgraph used for the current round of dialogue; Semantic vector based on the current round of dialogue , for the global knowledge graph Each node in the global knowledge graph is scored multidimensionally to determine whether it is closely related to the current conversation context. A node is a knowledge graph element in the global knowledge graph that represents a semantic entity or conceptual unit. It has a clear physical or semantic reference and originates from a structured corpus or knowledge base. It represents a core object closely related to the conversation topic, such as a person, place, event, topic, or emotion. The multidimensional scoring process comprehensively considers the following three factors as scoring factors: first, the semantic similarity between the semantic vector of the current conversation round and the node embedding representation vector; second, the mutual information between the node embedding representation vector and the historical conversation context, which measures the node's semantic coverage of the historical context; and third, the graph structural similarity between the node's structural position in the global knowledge graph and related nodes in the historical conversation context, such as adjacency structure or path similarity. The three scoring factors are weighted and summed to calculate the multidimensional score of each node. Only nodes with a multidimensional score above the set activation threshold (the weighted value of the mean and standard deviation of the multidimensional scores of all nodes) are activated to form the knowledge graph subgraph used in the current conversation round. The calculation formula is as follows: , in: Represents the knowledge graph subgraph in the current round of dialogue, which is composed of all multi-dimensional scores exceeding the activation judgment threshold The node composition of Indicates the activation threshold, which is fixed by expert experience. When the node Activated and added to the knowledge graph subgraph ; Representing the global knowledge graph The nodes; Represents the global knowledge graph, which contains a set of entity nodes and their relationship edges, serving as the supporting structure of knowledge semantics; Representation node The multi-dimensional scoring is used to determine whether the node should be activated; , Represents the weighting coefficients of the three scoring factors, which are used to control the relative importance of the semantic similarity between the semantic vector of the current round of dialogue and the node embedding representation vector, the mutual information between the node embedding representation vector and the historical dialogue context, and the structural similarity between the structural position of the node in the global knowledge graph and the nodes related to the historical dialogue context in the multi-dimensional scoring. It is set or trained by expert experience. The training method includes supervised learning methods based on labeled data (such as cross entropy or ranking loss optimization) and meta-learning mechanisms (such as channel attention or gated weighted networks) to achieve dynamic adaptive updates. The actual training method used depends on the availability of data annotation, the complexity of the dialogue task, the real-time response requirements of the system, and the requirements for model interpretability. After normalization, the value range is , and meet ; Representation node Embedding representation vector, obtained through pre-trained knowledge graph embedding (such as TransE, GAT, etc.); Representation node The embedding representation vector and the semantic vector of the current round of dialogue The semantic similarity between nodes is greater. The closer the semantic content of the node is to the topic semantics of the current round of dialogue, the The knowledge graph subgraph that is more semantically consistent with the current input and should be activated and included in the current round of dialogue ; Representation node The embedding representation vector and historical conversation context The mutual information between them, specifically, refers to the mutual information strength. A larger value indicates that the semantic content has a stronger correlation with the historical conversation content, and is therefore more suitable for being activated for knowledge reasoning and memory enhancement in the current round of conversation. Representation node The graph structure similarity between the structural position in the global knowledge graph and the nodes related to the historical conversation context. This similarity can be estimated by the adjacency relationship, path overlap, co-occurrence frequency at the same layer, or the distance based on the graph embedding structure. The larger the value, the closer the node is to the previous one. In terms of graph structure, it is closer to the node structure involved in historical topics and has stronger potential semantic relevance, so it is more suitable to be activated for knowledge reasoning and response generation in the current round of dialogue.

[0026] S3. For each node in the activated knowledge graph subgraph in the current dialogue round, calculate its memory strength in the current context; For the knowledge graph subgraph in the current round of dialogue activated in step S2 For each node in the conversation, its memory strength in the current context is further calculated as the weight basis for subsequent semantic fusion. The modeling of memory strength integrates the following three aspects: first, the activation score of the activated node in the current round, that is, the multi-dimensional score of the node, which reflects its relevance to the current semantics; second, the frequency of the activated node being called in the historical conversation, and the weight of the frequently used node should be increased; third, the time interval between the activated node and the last call, and the time decay mechanism is used to weaken the memory of inactive old nodes. After combining the above three aspects and normalizing them through the activation function, the memory strength of each activated node in the current round of conversation is obtained. , the calculation formula is as follows: , in: Indicates the activated node In the current round (i.e. round) memory strength during conversation; Represents the frequency enhancement coefficient, which is used to control the effect of historical access frequency on memory strength. Its value can be set according to expert experience and the range is ; Indicates the activated node The historical access frequency of a function, which represents the cumulative number of times it has been called or activated in the past rounds of conversations; Represents the time decay coefficient, which is used to control the forgetting speed. Its value can be set according to expert experience and the range is between; Indicates the activated node The time interval or number of rounds difference since the last call or activation.

[0027] S4. Perform weighted fusion on the node embedding representation vectors based on the memory strength of each activated node to generate a semantically enhanced vector that combines the current context information with the historical memory content; According to the memory strength of each activated node in the current round of dialogue calculated in step S3, the node embedding representation vector is weighted and combined. Specifically, each node embedding representation vector is multiplied by its corresponding memory strength, and then the weighted results of all nodes are summed to obtain a semantic enhancement vector that integrates the current context information and historical memory content. , the calculation formula is as follows: , Semantic Enhancement Vector It not only reflects the semantic intention of the current user input, but also integrates the knowledge graph content and long-term memory information related to the conversation, and has stronger contextual expression capabilities.

[0028] S5. The semantic vector of the current round of dialogue, the semantic enhancement vector, and the historical dialogue context encoding vector are input into the response generation module to generate a natural language response that conforms to the current context.

[0029] The semantic vector of the current dialogue, the semantic enhancement vector that integrates the current context information and historical memory content, and the encoding vector of the historical dialogue context are input into the response generation module. The response generation module can be a multi-layered natural language generation model, such as a decoder with an attention mechanism or a Transformer structure. During the generation process of the natural language generation model, the response generation model integrates the current user semantics, historical conversation context, and knowledge memory content to generate a natural language response that fits the current context. , complete a round of dialogue: , in: Indicates the In a round-by-round dialogue, the semantic vector based on the current round of dialogue , semantic enhancement vector and the historical conversation context encoding vector Comprehensively generated natural language response results; Represents historical conversation context The encoding vector represents "what was discussed before", and supports a variety of structures based on different modeling accuracy requirements, including lightweight average pooling methods, RNN encoders with strong sequence modeling capabilities, or RNNs with long-distance dependency perception capabilities. The specific encoding method used by the encoder can be flexibly configured based on the balance between the actual conversation history modeling accuracy, conversation length processing capability, and computing resources. Represents a vector concatenation operation, which is used to concatenate multiple vectors into a long vector according to the dimension (channel dimension or feature dimension); by the semantic vector of the current round of dialogue , semantic enhancement vector and the historical conversation context encoding vector Perform vector concatenation to generate a fused semantic information vector; Represents a decoder, which is used to decode the fused semantic information vector into a natural language response result .

[0030] In summary, a method for continuous dialogue and memory enhancement of digital humans based on knowledge graphs has been completed.

[0031] The order in which the embodiments of the invention are presented is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0032] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0033] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for continuous dialogue and memory enhancement of digital humans based on knowledge graph, characterized in that: The following steps are involved: S1. Receive the user's current round of conversation text and historical conversation context, process them through multiple semantic channels, and obtain a channel semantic representation vector. Based on the channel semantic representation vector, obtain the semantic vector of the current round of conversation. S2. Based on the semantic vector of the current round of dialogue, perform a multi-dimensional score on each node in the global knowledge graph to calculate the multi-dimensional score of the node; based on the multi-dimensional score, construct the knowledge graph subgraph used for the current round of dialogue; S3. For each node in the activated knowledge graph subgraph in the current dialogue round, calculate its memory strength in the current context; S4. Perform weighted fusion on the node embedding representation vectors based on the memory strength of each activated node to generate a semantically enhanced vector that combines the current context information with the historical memory content. S5. The semantic vector of the current round of dialogue, the semantic enhancement vector, and the historical dialogue context encoding vector are input into the response generation module to generate a natural language response that conforms to the current context.

2. The method for continuous conversation and memory enhancement of digital humans based on knowledge graph according to claim 1 is characterized in that: Said S1 specifically includes: In each semantic channel, the current input text, historical conversation context and corresponding graph structure are jointly modeled to extract the channel semantic representation vector. The channel semantic representation vector is then weighted according to its relevance in the current context to obtain the weighted channel semantic representation vector.

3. The method for continuous conversation and memory enhancement of digital humans based on knowledge graph according to claim 2 is characterized in that: Said S1 specifically includes: All weighted channel semantic representation vectors are aggregated through the fusion function to obtain the semantic vector of the current round of dialogue.

4. The method for continuous conversation and memory enhancement of digital humans based on knowledge graph according to claim 1 is characterized in that: Said S2 specifically includes: In the implementation of multi-dimensional scoring, the following three factors are used as scoring factors: first, the semantic similarity between the semantic vector of the current round of dialogue and the node embedding representation vector; second, the mutual information between the node embedding representation vector and the historical dialogue context; third, the graph structure similarity between the structural position of the node in the global knowledge graph and the nodes related to the historical dialogue context.

5. The method for continuous conversation and memory enhancement of digital humans based on knowledge graph according to claim 4 is characterized in that: Said S2 specifically includes: The above three scoring factors are weighted and summed up to calculate the multi-dimensional score of each node. The nodes with multi-dimensional scores higher than the set activation judgment threshold are regarded as activated nodes, and the activated nodes constitute the knowledge graph subgraph used in the current round of dialogue.

6. The method for continuous conversation and memory enhancement of digital humans based on knowledge graph according to claim 1 is characterized in that: Said S3 specifically includes: The memory strength modeling combines the multi-dimensional scores of the activated node, the frequency of the activated node being called in historical conversations, and the time interval between the activated node and the last call. After the combination, it is normalized using the activation function to obtain the memory strength of each activated node in the current round of conversation.

7. The method for continuous conversation and memory enhancement of digital humans based on knowledge graph according to claim 1 is characterized in that: Said S4 specifically includes: In the process of weighted fusion of node embedding representation vectors, each node embedding representation vector is multiplied by its corresponding memory strength, and then the weighted results of all nodes are summed up to obtain a semantic enhancement vector that integrates current context information and historical memory content.

8. The method for continuous conversation and memory enhancement of digital humans based on knowledge graph according to claim 1 is characterized in that: Said S5 specifically includes: The response generation module is a multi-layered natural language generation model. During the generation process of the natural language generation model, the response generation model integrates the current user semantics, historical conversation context, and knowledge memory content to generate a natural language response that conforms to the current context.

Images