Intelligent customer service dialogue generation system and method based on knowledge graph

Abstract

The invention relates to the technical field of artificial intelligence, in particular to an intelligent customer service dialogue generation system and method based on a knowledge graph, and the system comprises a language model, a knowledge graph module, a query module, a matching module, a reasoning module, a strategy generator, a reinforcement learning unit and a language generation module. The module comprises a knowledge subgraph isomorphic mapping unit, a graph entropy gradient reasoning unit and a Markov decision optimization unit, and the three units form a progressive processing link. The knowledge sub-graph isomorphic mapping unit maps the retrieval sub-graph to an isomorphic sub-graph in a knowledge graph topological space; the map entropy gradient reasoning unit constructs an entropy gradient field to realize reasoning path optimization from a high-entropy region to a low-entropy region; the Markov decision optimization unit models an inference process into a Markov decision process, a reinforcement learning method optimizes an inference strategy, and effective inference from dominant knowledge to implicit knowledge is realized in combination with a knowledge graph and a deep reinforcement learning technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to an intelligent customer service dialogue generation system and method based on a knowledge graph, which is applied to intelligent dialogue scenarios such as enterprise intelligent customer service and online consultation. BACKGROUND

[0002] With the development of artificial intelligence technology, intelligent customer service systems have been widely used in customer service of various industries. Traditional intelligent customer service systems mainly use rule-based methods or simple retrieval models. These methods often perform poorly when dealing with complex problems, especially when implicit knowledge needs to be understood, multi-step reasoning needs to be performed, or non-standardized problems need to be handled.

[0003] The current market intelligent customer service system mainly has the following problems: first, the understanding ability of complex problems is limited, and it is difficult to handle problems that require deep reasoning; second, the knowledge application is not flexible, and only direct matching problems can be handled, and the mining ability of implicit knowledge is insufficient; third, the dialogue generation lacks personalization, and it is difficult to provide differentiated services according to user characteristics and dialogue scenarios; finally, the system is difficult to continuously learn and optimize from user interaction.

[0004] In the prior art, although some systems have tried to combine knowledge graphs and deep learning methods, most systems still remain at the level of simple knowledge retrieval, lacking effective knowledge reasoning mechanisms and self-adaptive optimization capabilities. Therefore, how to design an intelligent customer service system that can perform deep knowledge reasoning and continuously optimize dialogue strategies is an important challenge facing the industry. SUMMARY

[0005] The purpose of the present application is to provide an intelligent customer service dialogue generation system and method based on a knowledge graph, which aims to solve the problems of insufficient reasoning ability, inflexible knowledge application, lack of personalization and self-adaptive optimization capability of existing intelligent customer service systems when dealing with complex problems.

[0006] The present application proposes an intelligent customer service dialogue generation system based on a knowledge graph, which includes:

[0007] A language model for processing user input problem text, extracting semantic information, and generating a set of reply candidates;

[0008] A knowledge graph module for storing enterprise business knowledge and common sense, the knowledge graph module constructs a knowledge base in the form of entity, relationship and attribute triplets;

[0009] A query module connected to the knowledge graph module for converting the problem semantic information processed by the language model into a subgraph retrieval task on the knowledge graph, and generating a retrieval subgraph;

[0010] The matching module is connected with the query module and the knowledge graph module, and is configured to perform entity, relationship and attribute matching on the search subgraph to generate a matching degree sequence.

[0011] The reasoning module is connected with the matching module and the knowledge graph module, and is configured to receive the matching degree sequence. The reasoning module includes a knowledge subgraph isomorphism mapping unit, a graph entropy gradient reasoning unit and a Markov decision optimization unit. Through progressive processing of the three units, implicit knowledge is mined from the knowledge graph based on the search subgraph to generate a reasoning subgraph.

[0012] The strategy generator is connected with the language model and the reasoning module, and is configured to fuse the reply candidate set of the language model and the knowledge of the reasoning subgraph to generate a reply strategy.

[0013] The reinforcement learning unit is connected with the strategy generator, and is configured to continuously optimize the reply strategy based on user feedback to select an optimal reply.

[0014] The language generation module is connected with the strategy generator, and is configured to convert the reply strategy into a natural language reply and output the natural language reply to the user.

[0015] Preferably, the knowledge subgraph isomorphism mapping unit is configured to:

[0016] extract topological features of the search subgraph, the topological features including node degree, clustering coefficient, centrality measure and structural role;

[0017] identify nodes with similar topological features in the knowledge graph to construct a candidate mapping set;

[0018] expand the mapping by an incremental structure matching strategy while maintaining the topological relationship unchanged;

[0019] select an optimal mapping as an initial reasoning subgraph based on structure preservation degree, semantic similarity and coverage.

[0020] Preferably, the graph entropy gradient reasoning unit is configured to:

[0021] calculate a local entropy value of each node in the knowledge graph, the local entropy value being based on structural entropy, attribute entropy and historical query entropy of the node;

[0022] construct an entropy gradient field to indicate the direction in which the entropy value decreases fastest;

[0023] extend the subgraph along the entropy gradient direction from the initial reasoning subgraph generated by the knowledge subgraph isomorphism mapping unit to generate multiple candidate reasoning paths;

[0024] select an optimal reasoning path based on entropy reduction, path length and semantic coherence to expand the reasoning subgraph.

[0025] As preferred, the Markov decision optimization unit is used for:

[0026] modeling the reasoning process as a Markov decision process, defining the state space as the state representation of the current reasoning subgraph, and defining the action space as the set of possible reasoning operations;

[0027] designing a reward function, considering the change of similarity between the reasoning subgraph and the retrieval subgraph, the change of information entropy of the reasoning subgraph, and the computational complexity of the operation;

[0028] controlling the reasoning process based on the state space, the action space, and the reward function by a policy optimization method;

[0029] evaluating the integrity, accuracy, and conciseness of the final reasoning subgraph, and generating a reasoning confidence distribution.

[0030] As preferred, the query module is used for:

[0031] converting the user question text into a word vector;

[0032] mapping the word vector into a node set of the knowledge graph;

[0033] calculating the semantic similarity between the node and the question text;

[0034] based on the semantic similarity, selecting nodes with high similarity to construct a retrieval subgraph;

[0035] the retrieval subgraph contains entity nodes, relationship edges, and attribute information directly related to the question text.

[0036] As preferred, the matching module is used for:

[0037] receiving the retrieval subgraph generated by the query module;

[0038] performing entity matching on the entity nodes in the retrieval subgraph;

[0039] performing relationship matching on the relationship edges in the retrieval subgraph;

[0040] performing attribute matching on the attribute information in the retrieval subgraph;

[0041] based on the results of the entity matching, the relationship matching, and the attribute matching, generating a matching degree sequence, which contains matching entities, relationships, attributes, and their corresponding confidence.

[0042] As preferred, the strategy generator is used for:

[0043] receive the reply candidate set generated by the language model and the inference subgraph generated by the inference module;

[0044] extract key knowledge points from the inference subgraph;

[0045] reorder and optimize the reply candidate set according to the key knowledge points;

[0046] generate a reply strategy that integrates the language model output and the knowledge graph inference result;

[0047] pass the reply strategy and user feedback to the reinforcement learning unit for optimization.

[0048] As a preferred, the reinforcement learning unit is used for:

[0049] receive the reply strategy provided by the strategy generator and user feedback;

[0050] define the state space as the representation of the dialogue history and the current question;

[0051] define the action space as the possible reply selection;

[0052] design a reward function considering the accuracy, relevance and user satisfaction of the reply;

[0053] constantly optimize the reply strategy by policy gradient method and select the optimal reply;

[0054] pass the optimized reply strategy to the language generation module.

[0055] As a preferred, the system further comprises:

[0056] a user portrait module for storing the user's historical consultation data, user type, user level and behavior characteristics;

[0057] a scene classification module connected with the user portrait module, for judging the dialogue scene type according to the user portrait and the current question;

[0058] a multi-modal input module for processing various forms of user input such as text, voice and image;

[0059] a multi-modal output module connected with the language generation module, for outputting the generated reply to the user in the form of text, voice or image.

[0060] The intelligent customer service dialogue generation method based on knowledge graph comprises the following steps:

[0061] receive the user input question text and extract semantic information through the language model;

[0062] transforming the question semantic information into a subgraph retrieval task on the knowledge graph, and generating a retrieval subgraph;

[0063] performing entity, relation and attribute matching on the retrieval subgraph, and generating a matching degree sequence;

[0064] based on the matching degree sequence, mining implicit knowledge from the knowledge graph through progressive processing of a knowledge subgraph isomorphism mapping unit, a graph entropy gradient reasoning unit and a Markov decision optimization unit, and generating a reasoning subgraph, wherein:

[0065] the knowledge subgraph isomorphism mapping unit extracts the topological features of the retrieval subgraph, identifies nodes with similar topological features in the knowledge graph, constructs a candidate mapping set, expands the mapping through an incremental structure matching strategy, and selects the optimal mapping as an initial reasoning subgraph;

[0066] the graph entropy gradient reasoning unit calculates the local entropy value of each node in the knowledge graph, constructs an entropy gradient field, expands the subgraph along the entropy gradient direction, generates multiple candidate reasoning paths, selects the optimal reasoning path, and expands the reasoning subgraph;

[0067] the Markov decision optimization unit models the reasoning process as a Markov decision process, controls the reasoning process through a strategy optimization method, evaluates the quality of the final reasoning subgraph, and generates a reasoning confidence distribution;

[0068] generating a reply strategy based on the reply candidate set generated by the language model and the knowledge of the reasoning subgraph;

[0069] constantly optimizing the reply strategy based on user feedback through a reinforcement learning unit, and selecting the optimal reply;

[0070] transforming the optimal reply into a natural language reply and outputting it to the user.

[0071] The present application combines knowledge graph and deep reinforcement learning technology innovatively, especially introduces a topological perception reasoning mechanism, realizes effective reasoning from explicit knowledge to implicit knowledge, and significantly improves the ability of the system to handle complex problems. The present application has the following beneficial effects:

[0072] 1. Improved complex problem processing capability: through the topological perception reasoning mechanism, the system can perform effective reasoning depth of 4-5 layers, which is significantly higher than the reasoning depth of 1-2 layers of traditional systems, and greatly improves the ability to handle complex logical relationships.

[0073] 2. Enhanced implicit knowledge mining capability: the coverage of implicit knowledge of the system is improved from 35% of the traditional system to 78%, which greatly reduces the number of unknown class answers, and can answer questions that are not explicitly stored in the knowledge base but can be obtained through reasoning.

[0074] 3. Improved response accuracy: In complex problem handling, the accuracy of reasoning results increased from 62% of traditional systems to 89%, and the error reasoning rate was significantly reduced.

[0075] 4. Improved user experience: One-time answer resolution rate increased from 45% to 76%, user satisfaction score increased from 3.2 to 4.5 (full score 5), and average conversation turns decreased from 4.5 to 2.3.

[0076] 5. Adaptive optimization: Through the reinforcement learning unit, the system can continuously learn and optimize the dialogue strategy from user interaction, continuously improving service quality. BRIEF DESCRIPTION OF DRAWINGS

[0077] Figure 1 The overall architecture diagram of the intelligent customer service dialogue generation system based on a knowledge graph provided by the embodiments of the present application is shown in the figure.

[0078] Figure 2 The internal structure diagram of the topological perception reasoning module in the embodiments of the present application is shown in the figure.

[0079] Figure 3 The processing flowchart of the knowledge subgraph isomorphism mapping unit in the embodiments of the present application is shown in the figure.

[0080] Figure 4 The processing flowchart of the graph entropy gradient reasoning unit in the embodiments of the present application is shown in the figure.

[0081] Figure 5 The processing flowchart of the Markov decision optimization unit in the embodiments of the present application is shown in the figure.

[0082] Figure 6 The overall workflow diagram of the system in the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0083] Please refer to the accompanying Figures 1-6 The scope of protection of the present application is not limited to the following.

[0084] Referring to Figure 1 The intelligent customer service dialogue generation system based on a knowledge graph of the present application includes a language model 1, a knowledge graph module 2, a query module 3, a matching module 4, a reasoning module 5, a strategy generator 6, a reinforcement learning unit 7, and a language generation module 8.

[0085] Preferably, the system further includes a user portrait module 9, a scene classification module 10, a multi-modal input module 11, and a multi-modal output module 12.

[0086] The language model 1 is used to process the question text of the user input, extract semantic information, and generate a set of reply candidates. The present application can use a pre-trained language model such as BERT, GPT, etc., and select a suitable model according to the specific application scenario. The language model 1 receives the question text of the user input, extracts the semantic representation of the text through a deep learning network, and generates multiple possible reply candidates as the input of the strategy generator 6.

[0087] The knowledge graph module 2 is used to store enterprise business knowledge and common sense, and construct a knowledge base in the form of entity, relationship and attribute triplets. In an embodiment of the present application, the knowledge graph module 2 uses a graph database such as Neo4j for storage, which facilitates efficient querying and reasoning. Each node in the knowledge graph represents an entity, containing entity ID, type and attribute list; each edge represents the relationship between entities, containing start point, end point, relationship type and weight. Preferably, the knowledge graph module 2 supports subgraph retrieval similarity based on semantic vector calculation, providing a basis for subsequent reasoning and matching.

[0088] The query module 3 is connected with the knowledge graph module 2, and is used to convert the semantic information of the question processed by the language model 1 into a subgraph retrieval task on the knowledge graph, and generate a retrieval subgraph. Specifically, the query module 3 converts the user question text into word vectors, maps these word vectors to the node set of the knowledge graph, calculates the semantic similarity between the nodes and the question text, and selects relevant nodes based on the similarity to construct a retrieval subgraph.

[0089] The matching module 4 is connected with the query module 3 and the knowledge graph module 2, and is used to match entities, relationships and attributes of the retrieval subgraph, and generate a matching degree sequence. The matching module 4 receives the retrieval subgraph generated by the query module 3, and performs entity matching, relationship matching and attribute matching respectively, to generate a matching degree sequence containing matching entities, relationships, attributes and their corresponding confidence.

[0090] The reasoning module 5 is the core innovative part of the system, and is connected with the matching module 4 and the knowledge graph module 2, and is used to receive the matching degree sequence, and mine implicit knowledge through deep knowledge reasoning to generate a reasoning subgraph. As shown in Figure 2 The reasoning module 5 includes a knowledge subgraph isomorphism mapping unit 51, a graph entropy gradient reasoning unit 52 and a Markov decision optimization unit 53, which form a progressive processing link to realize the knowledge transformation from the retrieval subgraph to the reasoning subgraph.

[0091] The strategy generator 6 is connected with the language model 1 and the reasoning module 5, and is used to fuse the reply candidate set of the language model 1 and the reasoning subgraph of the reasoning module 5, and generate a reply strategy. The strategy generator 6 extracts key knowledge points from the reasoning subgraph, reorders and optimizes the reply candidate set according to these knowledge points, and generates a reply strategy that fuses the language model output and the knowledge graph reasoning result.

[0092] The reinforcement learning unit 7 is connected with the strategy generator 6 for continuously optimizing the reply strategy based on user feedback and selecting the optimal reply. The reinforcement learning unit 7 receives the reply strategy provided by the strategy generator 6 and user feedback, continuously optimizes the reply strategy through the policy gradient method, and selects the reply that best meets the user's needs.

[0093] The language generation module 8 is connected with the strategy generator 6 for converting the reply strategy into a natural language reply and outputting it to the user. The language generation module 8 generates fluent and natural reply text according to the reply strategy, ensuring the readability and user-friendliness of the reply.

[0094] Reference Figure 3 The knowledge subgraph isomorphism mapping unit 51 is used to map the retrieval subgraph to an isomorphic subgraph in the knowledge graph topology space, providing an initial reasoning subgraph for subsequent reasoning. The processing flow of the knowledge subgraph isomorphism mapping unit 51 includes topology feature extraction, isomorphic candidate identification, isomorphic mapping construction, and optimal mapping selection.

[0095] In the topology feature extraction stage, the knowledge subgraph isomorphism mapping unit 51 extracts local topology features for each node in the retrieval subgraph, including node degree, clustering coefficient, centrality measure, and structural role. The node degree represents the number of directly connected nodes, the clustering coefficient represents the connection density between neighbor nodes, the centrality measure represents the importance of the node in the graph, and the structural role represents the functional positioning of the node in the topology structure. Preferably, the structural role is divided into three categories: core node, bridge node, and edge node, which are classified according to the centrality and connection pattern of the node.

[0096] In the isomorphic candidate identification stage, the knowledge subgraph isomorphism mapping unit 51 screens a subset of nodes with similar topology features in the knowledge graph, constructing a feature similarity matrix SM, where SM[i,j] represents the feature similarity between node i in the retrieval subgraph and node j in the knowledge graph. The feature similarity calculation formula is:

[0097] ,

[0098] wherein, is the feature similarity between node and node , with a value range of [0, 1]; is the degree similarity between node and node , with a value range of [0, 1]; is the clustering coefficient similarity between node and node , with a value range of [0, 1]; is the centrality similarity between node and node the centrality similarity of the node is the structural role similarity of the node and the node , with a value range of [0, 1]; and are weight coefficients, satisfying , preferably, can be set to 0.25, 0.2, 0.3 and 0.25. Then, the knowledge subgraph isomorphism mapping unit 51 applies a hierarchical clustering algorithm to identify high-similarity node clusters, selects a representative node from each node cluster, and constructs a candidate mapping set .

[0099] In the isomorphic mapping construction phase, the knowledge subgraph isomorphism mapping unit 51 starts from the candidate mapping set CM, expands the mapping by an incremental structure matching strategy, and maintains the topological relationship unchanged. For each candidate mapping m∈CM, initialize the mapping function f, map the starting node to the candidate node, then recursively expand the mapping, preferably use a ripple expansion strategy, expand layer by layer from the core node outward, while maintaining the topological relationship unchanged. Calculate the semantic similarity during the expansion process, and retain the mapping that meets the minimum similarity threshold, preferably, the minimum similarity threshold can be set to 0.6.

[0100] In the optimal mapping selection phase, the knowledge subgraph isomorphism mapping unit 51 calculates the comprehensive score of the generated mapping set, and selects the mapping with the highest score as the final mapping result. The comprehensive score calculation formula is:

[0101] ,

[0102] wherein, is the comprehensive score of the mapping , with a value range of [0, 1]; is the structure preservation degree of the mapping , indicating the consistency of the structure after mapping with the original structure, with a value range of [0, 1]; is the semantic similarity of the mapping , indicating the semantic similarity of the mapped node and the original node, with a value range of [0, 1]; is the coverage rate of the mapping , indicating the proportion of the mapped coverage of the original retrieval subgraph, with a value range of [0, 1]; , and are weight coefficients, satisfying ​​, preferably, can be set to 0.4, 0.4 and 0.2. In this way, the knowledge subgraph isomorphism mapping unit 51 generates an initial reasoning subgraph as the starting point of subsequent reasoning.

[0103] With reference to Figure 4 , the graph entropy gradient reasoning unit 52 is used to construct the entropy gradient field of the knowledge graph, and realize the optimization of the reasoning path from the high-entropy area to the low-entropy area. The processing flow of the graph entropy gradient reasoning unit 52 includes node entropy value calculation, entropy gradient field construction, reasoning path generation and path evaluation selection.

[0104] In the node entropy value calculation stage, the graph entropy gradient reasoning unit 52 calculates the local entropy value of each node in the knowledge graph, reflecting the uncertainty of the knowledge around the node. The node entropy value calculation considers three dimensions: structure entropy, attribute entropy and historical query entropy. The structure entropy reflects the uncertainty of the connection mode of the node, the attribute entropy reflects the integrity and certainty of the attribute value of the node, and the historical query entropy reflects the ambiguity degree of the node in the historical dialogue. The node entropy value calculation formula is:

[0105] ,

[0106] wherein, is the entropy value of the node , the value range is [0, 1], the larger the value, the higher the uncertainty; is the structure entropy of the node , the value range is [0, 1]; is the attribute entropy of the node , the value range is [0, 1]; is the historical query entropy of the node , the value range is [0, 1]; , and are weight coefficients, satisfying , preferably, can be set to 0.3, 0.4 and 0.3.

[0107] The structure entropy calculation formula is:

[0108] ,

[0109] wherein, is the structure entropy of the node ; is the set of relationship types of the node ; is the probability of the relationship type , calculated as the number of occurrences of the relationship type divided by the total number of relationships of the node ; is the natural logarithm function.

[0110] The attribute entropy calculation formula is:

[0111] ,

[0112] wherein, is the attribute entropy of the node ; is the complete attribute number of the node , that is, the attribute number with specific values; is the total attribute number of the node , including all attributes with values and without values.

[0113] The historical query entropy calculation formula is:

[0114] ,

[0115] wherein, is the historical query entropy of the node ; is the historical query set involving the node ; is the probability of the query , calculated as the number of occurrences of the query divided by the total number of queries involving the node ; is the natural logarithm function.

[0116] In the entropy gradient field construction stage, the graph entropy gradient reasoning unit 52 constructs the entropy distribution graph of the entire knowledge graph based on the node entropy value, calculates the entropy gradient of each node, points to the direction with the fastest decreasing entropy value around, and forms the entropy gradient vector field of the entire graph. The entropy gradient calculation formula of the node is:

[0117] ,

[0118] wherein, is the entropy gradient of the node , which is a vector; is the neighbor node set of the node ; is the size of the set , that is, the number of neighbor nodes; is the entropy value of the node ; is the entropy value of the node ; is the distance between the node and the node The distance between two nodes, usually taking the value of 1 (directly connected nodes); The from node The unit vector pointing to the node .

[0119] In the reasoning path generation phase, the graph entropy gradient reasoning unit 52 takes the initial reasoning subgraph generated by the knowledge subgraph isomorphism mapping unit 51 as the starting point and expands the subgraph along the entropy gradient direction. In each expansion, the direction with the steepest entropy gradient, the node with high semantic correlation, and the path with high historical reasoning success rate are selected to generate multiple candidate reasoning paths. Preferably, a beam search strategy is adopted to maintain The top 5 optimal paths, which can be set to 5.

[0120] In the path evaluation and selection phase, the graph entropy gradient reasoning unit 52 calculates the quality score of each reasoning path and selects the path with the highest score as the optimal reasoning path. The path quality score calculation formula is:

[0121] ,

[0122] where Score is the quality score of the path ; EntropyReduction is the entropy reduction, representing the entropy value difference between the start and end points of the path, calculated as , where is the start node of the path, is the end node of the path; Length is the path length, representing the number of nodes contained in the path; SemanticCoherence is the semantic coherence, representing the semantic association degree between nodes on the path, with a value range of [0, 1]; , and are weight coefficients, which can be set to 0.5, 0.2, and 0.3, respectively. In this way, the graph entropy gradient reasoning unit 52 generates an expanded reasoning subgraph, further enriching the reasoning result.

[0123] Referring to Figure 5 , the Markov decision optimization unit 53 is used to model the reasoning process as a Markov decision process and optimize the reasoning strategy through reinforcement learning. The processing flow of the Markov decision optimization unit 53 includes MDP model construction, strategy optimization, adaptive reasoning control, and reasoning result evaluation.

[0124] In the MDP model construction phase, the Markov decision optimization unit 53 defines the state space S as the state representation of the current inference subgraph, defines the action space A as the set of possible inference operations, and designs the reward function R(s, a, s'). The state representation uses a graph neural network to extract the structured representation of the inference subgraph, and the state vector contains graph structure features, node content features, and inference history features. The action space includes expansion operations, refinement operations, abstraction operations, and termination operations, where the expansion operation is subdivided into relationship expansion, attribute expansion, and semantic expansion. The reward function considers the change in similarity between the inference subgraph and the retrieval subgraph, the change in information entropy of the inference subgraph, and the computational complexity of the operation.

[0125] In the policy optimization phase, the Markov decision optimization unit 53 initializes the policy network π(a|s) and the value network V(s), and optimizes the inference strategy through deep reinforcement learning methods. The policy network π(a|s) is used to determine the probability of selecting action a in state s, and the value network V(s) is used to evaluate the long-term value of state s. Preferably, the advantage actor-critic (A2C) algorithm is used, and the policy network update formula is:

[0126] ,

[0127] where, is the gradient of the policy network parameter ; represents the expectation with respect to state s and action a; is the gradient of the policy network output probability with respect to parameter ; is the probability of the policy network selecting action a in state s under parameter ; is the advantage function, representing the advantage of selecting action a relative to the average performance.

[0128] The advantage function calculation formula is:

[0129] ,

[0130] where, is the advantage function; is the state-action value function, representing the long-term value of selecting action a in state s; is the state value function, representing the long-term value of state s.

[0131] The value network update formula is:

[0132] ,

[0133] where, is the gradient of the value network parameter ; represents the expectation about the state ; is an immediate reward; is a discount factor, representing the discount rate of future rewards, with a value range of [0, 1], preferably, it can be set to 0.9; is a parameter under which the value of the new state ; is a parameter under which the value of the current state .

[0134] In the adaptive inference control stage, the Markov decision optimization unit 53 controls the inference process according to the optimized strategy π, dynamically adjusts the inference depth, balances exploration and utilization, and timely terminates invalid inference. Specifically, at each inference step, the strategy network π is used to select the inference action a according to the current state s, the action a is executed, and then the new state s' and the reward r are observed, and the experience pool is updated. In order to balance exploration and utilization, an ε-greedy strategy is adopted, with an ε probability of randomly selecting an action and a 1-ε probability of selecting the action with the highest probability output by the strategy network, and ε gradually decreases from the initial value 0.3 to 0.05 as the training progresses.

[0135] In the inference result evaluation stage, the Markov decision optimization unit 53 calculates the quality indicators of the final inference subgraph, including completeness, accuracy and conciseness, and generates an inference confidence distribution. The completeness indicator measures the degree to which the inference subgraph covers the knowledge required for the problem, the accuracy indicator measures the confidence of the correctness of the knowledge, and the conciseness indicator measures the redundancy of the knowledge representation. The inference confidence distribution represents the uncertainty of the inference result, helping the strategy generator to handle knowledge uncertainty. In this way, the Markov decision optimization unit 53 generates the final inference subgraph to provide knowledge support for reply generation.

[0136] The query module 3 is used to convert the problem text into a subgraph retrieval task on the knowledge graph. The query module 3 first preprocesses the problem text, including word segmentation, stop word removal, etc., and then converts the text into a word vector representation. Preferably, a word embedding model such as Word2Vec or GloVe can be used to convert each word into a fixed-dimensional vector, such as a 300-dimensional vector. The word vector representation calculation formula is:

[0137] ,

[0138] wherein, is the vector representation of the text, which is a 300-dimensional vector; is the set of words in the text; is the size of the set , i.e. the number of words; is the word is a 300-dimensional vector; represents a vector sum operation.

[0139] Then, the query module 3 maps the word vector into the node set of the knowledge graph, and calculates the semantic similarity between the node and the question text. The formula for calculating the semantic similarity between the node and the text is:

[0140] ,

[0141] where Sim (text) is the semantic similarity between the node and the text, and the value range is [-1, 1]; is the vector representation of the node , which is a 300-dimensional vector; is the vector representation of the text, which is a 300-dimensional vector; and cos represents the cosine similarity function, which calculates the cosine value of the included angle between two vectors.

[0142] Finally, the query module 3 selects nodes with high similarity based on the semantic similarity to construct a retrieval subgraph. Preferably, nodes with a similarity greater than a threshold τ can be selected, and τ can be set to 0.5. At the same time, the relationship between nodes is considered. If there is a direct relationship between two nodes, the relationship is also included in the retrieval subgraph.

[0143] The matching module 4 is used for entity, relationship and attribute matching of the retrieval subgraph to generate a matching degree sequence. The matching module 4 first performs entity matching on the entity nodes in the retrieval subgraph, and calculates the matching degree of the entity nodes and the entities in the knowledge graph. The formula for calculating the entity matching degree is:

[0144] MatchEntity( , ) = SimName( , ) + SimType( , ) + SimAttr( , ),

[0145] where MatchEntity( , ) is the matching degree of the entity and the entity , and the value range is [0, 1]; SimName( , ) is the name similarity, and its value range is [0, 1]; SimType( , ) is the type similarity, and its value range is [0, 1]; SimAttr( , ) is the attribute similarity, and its value range is [0, 1]; 、 and are weight coefficients, and satisfy , preferably, can be set as 0.4, 0.3 and 0.3.

[0146] Then, the matching module 4 performs relation matching on the relation edges in the search sub-graph, and calculates the matching degree of the relation edges and the relations in the knowledge graph. The relation matching degree calculation formula is:

[0147] MatchRelation( , ) = SimType( , ) + SimEntity( .src, .src) + SimEntity( .dst, .dst),

[0148] wherein, MatchRelation( , ) is the matching degree of the relation and the relation , and its value range is [0, 1]; SimType( , ) is the relation type similarity, and its value range is [0, 1]; SimEntity( .src, .src) is the source entity similarity, and its value range is [0, 1]; SimEntity( .dst, .dst) is the target entity similarity, and its value range is [0, 1]; .src is the source entity of the relation ; .dst is the target entity of the relation ; 、 and are weight coefficients, and satisfy , preferably, can be set to 0.4, 0.3 and 0.3.

[0149] Finally, the matching module 4 performs attribute matching on the attribute information in the retrieved subgraph, and calculates the matching degree of the attribute information and the attributes in the knowledge graph. The attribute matching degree calculation formula is:

[0150] MatchAttr( , ) = SimName( , ) + SimValue( , ) + SimEntity( .entity, .entity),

[0151] wherein, MatchAttr( , ) is the matching degree of attribute a1 and attribute a2, the value range is [0, 1]; SimName( , ) is the attribute name similarity, the value range is [0, 1]; SimValue( , ) is the attribute value similarity, the value range is [0, 1]; SimEntity( .entity, .entity) is the similarity of the entity to which the attribute belongs, the value range is [0, 1]; .entity is the entity to which the attribute a belongs; w1, w2 and w3 are weight coefficients, satisfying w1 + w2 + w3 = 1, preferably, can be set to 0.3, 0.4 and 0.3.

[0152] Based on the above matching results, the matching module 4 generates a matching degree sequence containing matching entities, relationships, attributes and their corresponding confidence. The matching degree sequence is used as the input of the reasoning module 5 for subsequent knowledge reasoning.

[0153] The strategy generator 6 is used to fuse the reply candidate set of the language model 1 and the reasoning subgraph of the reasoning module 5 to generate a reply strategy. The strategy generator 6 first extracts key knowledge points from the reasoning subgraph, including entity, relationship and attribute information. The key knowledge point extraction adopts an importance scoring mechanism, and the importance score calculation formula is:

[0154] Importance(k) = w1 · Relevance(k, q) + w2 · Centrality(k) + w3 ·Confidence(k),

[0155] wherein, Importance(k) is the importance score of the knowledge point k, the value range is [0, 1]; Relevance(k, q) is the relevance of the knowledge point k and the question q, the value range is [0, 1]; Centrality(k) is the centrality of the knowledge point k in the reasoning subgraph, the value range is [0, 1]; Confidence(k) is the confidence of the knowledge point k, the value range is [0, 1]; w1, w2 and w3 are weight coefficients, satisfying w1 + w2 + w3 = 1, preferably, can be set as 0.5, 0.3 and 0.2. The top K high importance scores are selected as the key knowledge points, and K can be set as 5.

[0156] Then, the strategy generator 6 reorders and optimizes the reply candidate set according to the key knowledge points. For each reply candidate r, the knowledge coverage is calculated, and the formula for calculating the knowledge coverage is:

[0157]

[0158] wherein, KnowledgeCoverage(r) is the knowledge coverage of the reply candidate r, the value range is [0, 1]; K is the set of key knowledge points; is the importance weight of the knowledge point k, that is, the importance score; Contains(r, k) is an indicator function, indicating whether the reply candidate r contains the knowledge point k, 1 for containing and 0 for not containing; denotes the summation operation.

[0159] Finally, the strategy generator 6 generates a reply strategy that fuses the language model output and the reasoning result of the knowledge graph. The formula for calculating the reply strategy score is:

[0160]

[0161] wherein, Score(r) is the comprehensive score of the reply candidate r, the value range is [0, 1]; LanguageModelScore(r) is the language model score, indicating the fluency and naturalness of the reply, the value range is [0, 1]; KnowledgeCoverage(r) is the knowledge coverage, indicating the degree of containing key knowledge points in the reply, the value range is [0, 1]; Coherence(r) is the coherence of the reply, indicating the consistency of the reply with the dialogue history, the value range is [0, 1];​​ , and are weight coefficients, satisfying , preferably, can be set to 0.3, 0.4 and 0.3. The reply with the highest score is selected as the initial reply strategy, which is passed to the reinforcement learning unit 7 for further optimization.

[0162] The reinforcement learning unit 7 is used to continuously optimize the reply strategy based on user feedback and select the optimal reply. The reinforcement learning unit 7 defines the state space as the representation of the dialogue history and the current question, defines the action space as the possible reply choices, and designs the reward function to consider the accuracy, relevance and user satisfaction of the reply.

[0163] The state representation adopts a vector representation of the dialogue history and the current question, and the calculation formula is:

[0164] ,

[0165] wherein, is the state at time step , which is a vector; is the vector representation of the dialogue history; is the vector representation of the current question; and

[0166] The action representation adopts a vector representation of the reply candidate, and each action corresponds to a possible reply. The reward function design includes immediate reward and delayed reward, the immediate reward is based on the intrinsic quality of the reply, and the delayed reward is based on user feedback. The calculation formula of the reward function is:

[0167] ,

[0168] wherein, is the reward of selecting action in state ; is the accuracy reward, indicating the factual accuracy of the reply, with a value range of [0, 1]; is the relevance reward, indicating the relevance of the reply to the question, with a value range of [0, 1]; is the user feedback reward, indicating the user's satisfaction with the reply, with a value range of [0, 1]; and are weight coefficients, satisfying , preferably, can be set to 0.3, 0.3 and 0.4.

[0169] The reinforcement learning unit 7 adopts the policy gradient method to continuously optimize the reply strategy. Preferably, the Proximal Policy Optimization (PPO) algorithm is adopted, and the objective function is:

[0170] ,

[0171] wherein, is the objective function of the PPO algorithm; represents the expectation about the time step ; is the probability ratio, calculated as , wherein is the probability of selecting action in state under the current policy, is the probability of selecting action in state under the old policy; is the advantage function, representing the advantage of action relative to the average performance; clip is a clipping function that limits the value within a specified range; is the clipping parameter, controlling the step size of policy update, preferably, it can be set to 0.2. By continuously optimizing the objective function, the reinforcement learning unit 7 can learn a better reply strategy and improve the quality of the system dialogue.

[0172] Referring to Figure 6 , the workflow of the intelligent customer service dialogue generation system based on the knowledge graph of the application includes the following steps:

[0173] Step S1, receiving the question text input by the user, extracting semantic information through the language model 1;

[0174] Step S2, converting the question semantic information into a subgraph retrieval task on the knowledge graph, generating a retrieval subgraph;

[0175] Step S3, matching entities, relationships and attributes of the retrieval subgraph, generating a matching degree sequence;

[0176] Step S4, based on the matching degree sequence, through the progressive processing of the knowledge subgraph isomorphism mapping unit 51, the graph entropy gradient reasoning unit 52 and the Markov decision optimization unit 53, mining the implicit knowledge from the knowledge graph, generating a reasoning subgraph;

[0177] Step S5, generating a reply strategy based on the reply candidate set generated by the language model 1 and the knowledge of the reasoning subgraph;

[0178] Step S6, continuously optimizing the reply strategy based on user feedback through the reinforcement learning unit 7, selecting the optimal reply;

[0179] Step S7, converting the optimal reply into a natural language reply and outputting it to the user.

[0180] Preferably, before step S1, a user portrait construction step can also be included, which constructs a user portrait according to user historical interaction data to provide personalized information for subsequent processing. Between step S1 and step S2, a scene classification step can also be included, which judges the dialogue scene type according to the user portrait and the current question to provide scene-related information for subsequent processing.

[0181] The intelligent customer service dialogue generation system based on the knowledge graph of the application is applicable to various intelligent customer service scenarios, such as financial consultation, technical support, policy consultation, etc. Taking financial consultation as an example, when a user asks how to calculate the after-tax yield of this financial product, the traditional system can only provide simple product information, but cannot explain the specific calculation method. However, the system can infer tax rate rules, calculation methods and other implicit knowledge from the knowledge graph through the topological perception reasoning mechanism, and give detailed calculation steps and results, greatly improving the user experience.

[0182] The system of the application has shown significant performance improvement in the test evaluation of multiple industry fields. In the financial consultation scene test, the coverage rate of implicit knowledge of the system is improved from 35% of the traditional system to 78%, the accuracy rate of reasoning results is improved from 62% to 89%, the one-time answer solving rate is improved from 45% to 76%, the user satisfaction score is improved from 3.2 points to 4.5 points (full score 5 points), and the average dialogue round is reduced from 4.5 rounds to 2.3 rounds. These indicators fully prove the significant advantages of the system in handling complex problems, mining implicit knowledge and improving user experience.

[0183] In addition, the system also has good performance, with an average reasoning time of less than 200ms, supporting high concurrency processing, with a single server supporting 500QPS, meeting the actual application requirements.

[0184] In summary, the intelligent customer service dialogue generation system based on the knowledge graph provided by the application innovatively combines knowledge graph and deep reinforcement learning technology, especially introduces the topological perception reasoning mechanism, realizes effective reasoning from explicit knowledge to implicit knowledge, significantly improves the system's ability to handle complex problems, and provides a new technical solution for the field of intelligent customer service.

[0185] The above only describes the preferred embodiments of the application and is not intended to limit the application. The application can have various changes and modifications for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the application shall be included in the protection scope of the application.

Claims

1. The intelligent customer service dialogue generation system based on knowledge graph is characterized by: include: Language model, used to process the question text entered by the user, extract semantic information, and generate a set of answer candidates; A knowledge graph module is used to store enterprise business knowledge and common sense. The knowledge graph module constructs a knowledge base in the form of entity, relationship and attribute triples; A query module, connected to the knowledge graph module, is used to convert the question semantic information processed by the language model into a subgraph retrieval task on the knowledge graph and generate a retrieval subgraph; A matching module, connected to the query module and the knowledge graph module, for matching entities, relationships, and attributes of the retrieval subgraph to generate a matching degree sequence; An inference module, connected to the matching module and the knowledge graph module, for receiving the matching degree sequence. The inference module includes a knowledge subgraph isomorphism mapping unit, a graph entropy gradient inference unit, and a Markov decision optimization unit. Through the progressive processing of the three units, implicit knowledge is mined from the knowledge graph based on the retrieval subgraph to generate an inference subgraph. a strategy generator, connected to the language model and the reasoning module, for fusing the response candidate set of the language model and the knowledge of the reasoning subgraph to generate a response strategy; A reinforcement learning unit, connected to the strategy generator, for continuously optimizing the reply strategy based on user feedback and selecting the optimal reply; The language generation module is connected to the strategy generator and is used to convert the response strategy into a natural language response and output it to the user.

2. The intelligent customer service dialogue generation system according to claim 1, characterized in that: The knowledge subgraph isomorphism mapping unit is used for: Extracting topological features of the search subgraph, wherein the topological features include node degree, clustering coefficient, centrality measure and structural role; Identifying nodes with similar topological features in the knowledge graph and constructing a candidate mapping set; Expand the mapping through an incremental structure matching strategy, keeping the topology unchanged; Based on structure preservation, semantic similarity and coverage, the optimal mapping is selected as the initial reasoning subgraph.

3. The intelligent customer service dialogue generation system according to claim 1, characterized in that: The graph entropy gradient inference unit is used to: Calculating a local entropy value for each node in the knowledge graph, wherein the local entropy value is based on the structural entropy, attribute entropy, and historical query entropy of the node; Construct an entropy gradient field to indicate the direction in which entropy decreases fastest; Starting from the initial reasoning subgraph generated by the knowledge subgraph isomorphism mapping unit, the subgraph is expanded along the entropy gradient direction to generate multiple candidate reasoning paths; Based on entropy reduction, path length, and semantic coherence, the optimal reasoning path is selected and the reasoning subgraph is expanded.

4. The intelligent customer service dialogue generation system according to claim 1, characterized in that: The Markov decision optimization unit is used to: The reasoning process is modeled as a Markov decision process, the state space is defined as the state representation of the current reasoning subgraph, and the action space is defined as the set of possible reasoning operations; Design a reward function that takes into account the change in similarity between the inference subgraph and the retrieval subgraph, the change in the information entropy of the inference subgraph, and the computational complexity of the operation; controlling the reasoning process by a policy optimization method based on the state space, the action space, and the reward function; Evaluate the completeness, accuracy, and simplicity of the final inference subgraph and generate an inference confidence distribution.

5. The intelligent customer service dialogue generation system according to claim 1, characterized in that: The query module is used to: Convert user question text into word vectors; Mapping the word vector to a node set of the knowledge graph; Calculating the semantic similarity between the node and the question text; Based on the semantic similarity, selecting nodes with high similarity to construct a retrieval subgraph; The retrieval subgraph includes entity nodes, relationship edges and attribute information directly related to the question text.

6. The intelligent customer service dialogue generation system according to claim 1, characterized in that: The matching module is used for: receiving a retrieval subgraph generated by the query module; Performing entity matching on entity nodes in the retrieval subgraph; Performing relationship matching on the relationship edges in the search subgraph; Performing attribute matching on the attribute information in the retrieval subgraph; Based on the results of the entity matching, the relationship matching, and the attribute matching, a matching degree sequence is generated, where the matching degree sequence includes matching entities, relationships, attributes, and their corresponding confidences.

7. The intelligent customer service dialogue generation system according to claim 1, characterized in that: The policy generator is used to: receiving a response candidate set generated by the language model and a reasoning subgraph generated by the reasoning module; Extracting key knowledge points from the reasoning subgraph; Reordering and optimizing the response candidate set according to the key knowledge points; Generate a response strategy that combines language model output and knowledge graph reasoning results; The response strategy and user feedback are passed to the reinforcement learning unit for optimization.

8. The intelligent customer service dialogue generation system according to claim 1, characterized in that: The reinforcement learning unit is used to: receiving a reply strategy and user feedback provided by the strategy generator; Define the state space as the representation of the dialogue history and the current question; Define the action space as possible response choices; Design a reward function that takes into account the accuracy, relevance, and user satisfaction of responses; Continuously optimize the response strategy through the policy gradient method and select the optimal response; The optimized reply strategy is passed to the language generation module.

9. The intelligent customer service dialogue generation system according to claim 1, characterized in that: The system further comprises: User portrait module, used to store users' historical consultation data, user type, user level and behavioral characteristics; A scenario classification module, connected to the user portrait module, is used to determine the type of conversation scenario based on the user portrait and the current question; Multimodal input module, used to process user input in various forms such as text, voice, and images; The multimodal output module is connected to the language generation module and is used to output the generated response to the user in the form of text, voice or image.

10. The method for generating intelligent customer service dialogue based on knowledge graph is characterized by: The following steps are involved: Receive the question text entered by the user and extract semantic information through the language model; Convert the semantic information of the question into a subgraph retrieval task on the knowledge graph to generate a retrieval subgraph; Matching entities, relationships, and attributes on the search subgraph to generate a matching degree sequence; Based on the matching degree sequence, implicit knowledge is mined from the knowledge graph through the progressive processing of the knowledge subgraph isomorphism mapping unit, the graph entropy gradient reasoning unit and the Markov decision optimization unit to generate an inference subgraph, where: The knowledge subgraph isomorphic mapping unit extracts the topological features of the retrieval subgraph, identifies nodes with similar topological features in the knowledge graph, constructs a candidate mapping set, expands the mapping through an incremental structure matching strategy, and selects the optimal mapping as the initial reasoning subgraph; The graph entropy gradient reasoning unit calculates the local entropy value of each node in the knowledge graph, constructs an entropy gradient field, expands the subgraph along the entropy gradient direction, generates multiple candidate reasoning paths, selects the optimal reasoning path, and expands the reasoning subgraph; The Markov decision optimization unit models the reasoning process as a Markov decision process, controls the reasoning process through a policy optimization method, evaluates the quality of the final reasoning subgraph, and generates a reasoning confidence distribution; generating a response strategy based on a response candidate set generated by the language model and knowledge of the reasoning subgraph; Continuously optimize the response strategy based on user feedback through a reinforcement learning unit to select the optimal response; The optimal response is converted into a natural language response and output to the user.

Images