Knowledge graph enhanced reasoning framework based on graph neural network and large language model

Abstract

The present invention discloses a knowledge graph enhanced reasoning framework based on graph neural network and large language model, which includes a vectorized representation module, a graph neural network encoding module, a large language model interaction module and a joint reasoning module; the vectorized representation module aligns the structured knowledge graph with the unstructured text and converts it into a unified vectorized representation, providing a basis for subsequent processing; the graph neural network encoding module uses the low-dimensional embedding of the graph neural network learning data to obtain the graph structure representation; the large language model interaction module uses the large language model to semantically complete the knowledge graph entity relationship and generate an enhanced context vector; the joint reasoning module integrates the results of the first two and outputs an enhanced knowledge graph quad. The present invention significantly improves the accuracy and completeness of knowledge graph reasoning through multi-module collaboration, and has broad application prospects in the fields of intelligent question answering, intelligent decision-making, etc.

Description

Technical Field

[0001] The present invention relates to the field of knowledge representation and reasoning technology, and specifically to a knowledge graph enhanced reasoning framework based on graph neural networks and large language models. Background Art

[0002] Knowledge graphs, as a structured knowledge representation method, describe the relationships between entities through graph structures and are widely used in fields such as intelligent question-answering, information retrieval, and recommendation systems. Traditional knowledge graph construction relies primarily on structured data, storing knowledge in the form of triples (head entity, relationship, tail entity). However, most real-world knowledge exists in the form of unstructured text, such as news reports and academic papers. This leads to problems such as incomplete information and delayed updates in knowledge graphs.

[0003] To acquire knowledge from unstructured text, researchers have begun using natural language processing techniques to extract entities and relationships from text and integrate them into knowledge graphs. However, due to the ambiguity and diversity of natural language, as well as the semantic gap between structured knowledge graphs and unstructured text, entity alignment and relationship extraction tasks face many challenges.

[0004] In terms of knowledge reasoning, early reasoning methods were primarily based on rules and logic. While highly accurate, these methods relied on domain experts to formulate the rules and struggled to handle large, complex knowledge graphs. With the development of deep learning, graph neural networks (GNNs) have gradually become the mainstream technology for knowledge graph reasoning. They can automatically learn features and relationships within graph structures, effectively improving reasoning performance. However, GNNs have limitations in handling long-range dependencies and understanding semantics. This is particularly true when there is missing information in the knowledge graph, significantly impacting the accuracy of the reasoning results.

[0005] At the same time, large language models (LLMs) have achieved remarkable results in natural language processing tasks thanks to their powerful language understanding and generation capabilities. Introducing LLMs into knowledge graph reasoning can leverage their semantic completion capabilities to compensate for information gaps in knowledge graphs.

[0006] Therefore, how to effectively combine large language models with graph neural networks to give full play to the advantages of both remains an urgent problem to be solved. Summary of the Invention

[0007] The purpose of the present invention is to provide a knowledge graph enhanced reasoning framework based on graph neural networks and large language models. Through the entity alignment algorithm, it realizes the precise alignment of structured knowledge graph entities and unstructured text entities, generates a joint vector representation, and effectively integrates multi-source knowledge. With the help of a multi-layer graph attention network, the relationship between entities is deeply learned, the graph structure representation is output, and the understanding of the graph structure information is improved. The large language model is used to semantically complete the missing information of the graph structure representation, generate a semantic completion vector, and fill the information gap of the knowledge graph. By combining the weighted graph structure representation and the semantic completion vector, an enhanced knowledge graph quadruple containing a confidence score is output to provide users with more reliable reasoning results. The framework of the present invention can effectively improve the quality and reasoning performance of the knowledge graph, and has important theoretical significance and practical application value.

[0008] To achieve the above objectives, the present invention provides a knowledge graph-enhanced reasoning framework based on graph neural networks and large language models, which includes the following modules: vectorized representation module, graph neural network encoding module, large language model interaction module and joint reasoning module;

[0009] The vectorized representation module aligns structured knowledge graph entities and unstructured text entities through an entity alignment algorithm to generate a joint vector representation;

[0010] The graph neural network encoding module uses a multi-layer graph attention network to learn the relationship between entities based on the joint vector representation and outputs a graph structure representation;

[0011] The large language model interaction module uses the large language model to perform semantic completion on the knowledge graph entity relationship to generate a semantic completion vector;

[0012] The joint reasoning module outputs an enhanced knowledge graph quadruple (head entity, relationship, tail entity, confidence) containing a confidence score by combining and weighting the graph structure representation and the semantic completion vector.

[0013] Furthermore, the entity alignment algorithm adopts a matching strategy based on cosine similarity, which is expressed as follows:

[0014] Suppose the set of structured knowledge graph entity vectors is , is the total number of structured knowledge graph entity vectors, each knowledge graph entity vector corresponds to a knowledge graph entity; the set of unstructured text entity vectors is , is the total number of unstructured text entity vectors; the cosine similarity formula is expressed as follows:

[0015]

[0016] in, is the first The vector representation of an entity, The first The vector representation of the entity; when the similarity is calculated If it exceeds the preset threshold, it is considered a match, otherwise it is not a match;

[0017] Furthermore, the joint vector representation is composed of a fusion vector and unmatched entity vectors of the structured knowledge graph;

[0018] The fusion vector The weighted fusion generation of the successfully matched structured knowledge graph entity vector and the unstructured text entity vector is as follows:

[0019]

[0020] in, is the first The vector representation of an entity, The first The vector representation of an entity, for and similarity.

[0021] Furthermore, the multi-layer graph attention network uses the vector in the joint vector representation as the initial feature vector of the node, which is expressed as follows:

[0022] Suppose the vector set in the joint vector representation is , is the total number of vectors in the joint vector representation; the update formula of the multi-layer graph attention network node feature vector is expressed as follows:

[0023]

[0024] in, is the updated feature vector of the node, For the node With neighboring nodes The attention weights between are the parameters of the multi-layer graph attention network, For nodes The set of neighbor nodes of is the activation function;

[0025] The multi-layer graph attention network updates all nodes through the multi-layer graph attention network node feature vector update formula, and updates the updated node feature vector As the input of the next layer of graph attention network, repeat the node update formula to perform feature extraction and update;

[0026] The graph structure representation node is composed of the feature vector of the final node in the multi-layer graph attention network.

[0027] Furthermore, the attention weight The calculation formula is:

[0028]

[0029] in, is the softmax activation function, is the LeakyReLU activation function; are the parameters of the multi-layer graph attention network; is the attention parameter vector; is the feature vector of node i, is the feature vector of node i’s neighbor node j; and Perform vector concatenation ; and a are updated via backpropagation.

[0030] Furthermore, the large language model interaction module receives the graph structure representation , is the total number of vectors in the joint vector representation, and the semantic completion vector is generated according to the following steps:

[0031] First, the entity pairs in the knowledge graph are and , the corresponding graph structure representation and , and the embedding vector corresponding to the unstructured text Concatenate to generate enhanced input vector ;

[0032] Then the enhanced input vector is input into the large language model to generate the enhanced relationship The probability distribution of ;

[0033] Finally, select the one with the highest probability The enhanced relationship , generate the semantic completion vector set corresponding to the enhanced input vector , Z is the preset threshold.

[0034] Furthermore, the combination weighting is expressed as follows:

[0035]

[0036] in, Characterize the structure of the map The corresponding reasoning score, is the semantic completion vector The corresponding reasoning score, is the preset threshold, Score the final reasoning;

[0037] This framework selects the corresponding relationship with the highest score y , generate knowledge graph quadruple after enhanced reasoning , and is the corresponding entity pair in the knowledge graph.

[0038] Compared with the prior art, the advantages of the present invention are:

[0039] (1) Multi-source knowledge fusion and precise alignment: This paper uses an entity alignment algorithm based on cosine similarity matching to break through the limitations of traditional reliance on a single knowledge source or simple text matching. It aligns structured knowledge graphs and unstructured text entities, generates a joint vector representation, and improves the accuracy and comprehensiveness of knowledge fusion.

[0040] (2) Graph feature learning and missing information completion: This paper uses a multi-layer graph attention network to learn entity relationships, output graph structure representations, and simultaneously uses a large language model to complete missing information. Compared with traditional technologies, it can better capture complex relationships and improve the accuracy and reliability of reasoning.

[0041] (3) Cross-model fusion and reasoning result optimization: This paper realizes the deep fusion of graph neural network and large language model by combining weighted fusion graph structure representation and semantic completion vector, and outputs enhanced knowledge graph quadruple with confidence score, providing valuable reference for user decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0043] Figure 1 It is a framework module diagram of the present invention;

[0044] Figure 2This is a structural diagram of the vectorized representation module of the present invention;

[0045] Figure 3 This is a structural diagram of the joint reasoning module of the present invention. DETAILED DESCRIPTION

[0046] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0047] Example 1: Please refer to Figure 1 As shown, the present embodiment describes a knowledge graph enhanced reasoning framework based on graph neural network and large language model, which includes a vectorized representation module, a graph neural network encoding module, a large language model interaction module and a joint reasoning module;

[0048] The vectorized representation module, reference Figure 1 As shown in the figure, the entity alignment algorithm is used to align structured knowledge graph entities with unstructured text entities to generate a joint vector representation;

[0049] The entity alignment algorithm adopts a matching strategy based on cosine similarity, which is expressed as follows:

[0050] Suppose the set of structured knowledge graph entity vectors is , is the total number of structured knowledge graph entity vectors, each knowledge graph entity vector corresponds to a knowledge graph entity; the set of unstructured text entity vectors is , is the total number of unstructured text entity vectors; the cosine similarity formula is expressed as follows:

[0051]

[0052] in, is the first The vector representation of an entity, The first The vector representation of the entity; when the similarity is calculated If it exceeds the preset threshold, it is considered a match, otherwise it is not a match;

[0053] The knowledge graph entity vector The knowledge graph embedding model ComplEx is used: This model is based on complex space and can handle symmetric and antisymmetric relationships in the knowledge graph. In ComplEx, entities and relationships are represented as complex vectors. Using a scoring function, various relationship patterns in the knowledge graph are captured, and entity vectors are learned.

[0054] The text entity vector This is obtained through the pre-trained BERT model, which uses a bidirectional Transformer architecture to encode the input text. When processing sentences containing text entities, the model can generate a vector representation corresponding to the entity based on contextual information.

[0055] The joint vector representation is composed of a fusion vector and unmatched entity vectors of the structured knowledge graph;

[0056] The fusion vector The weighted fusion generation of the successfully matched structured knowledge graph entity vector and the unstructured text entity vector is as follows:

[0057]

[0058] in, is the first The vector representation of an entity, The first The vector representation of an entity, for and similarity.

[0059] The graph neural network encoding module uses a multi-layer graph attention network to learn the relationship between entities based on the joint vector representation and outputs a graph structure representation;

[0060] The multi-layer graph attention network uses the vector in the joint vector representation as the initial feature vector of the node, which is expressed as follows:

[0061] Suppose the vector set in the joint vector representation is , is the total number of vectors in the joint vector representation; the update formula of the multi-layer graph attention network node feature vector is expressed as follows:

[0062]

[0063] in, is the updated feature vector of the node, For the node With neighboring nodes The attention weights between are the parameters of the multi-layer graph attention network, For nodes The set of neighbor nodes of is the activation function;

[0064] The multi-layer graph attention network updates all nodes through the multi-layer graph attention network node feature vector update formula, and updates the updated node feature vector As the input of the next layer of graph attention network, repeat the node update formula to perform feature extraction and update;

[0065] The graph structure representation node is composed of the feature vector of the final node in the multi-layer graph attention network.

[0066] The attention weight The calculation formula is:

[0067]

[0068] in, is the softmax activation function, is the LeakyReLU activation function; are the parameters of the multi-layer graph attention network; is the attention parameter vector; is the feature vector of node i, is the feature vector of node i’s neighbor node j; and Perform vector concatenation ;

[0069] parameter and The update is performed through the back-propagation algorithm and the optimizer. The update process is as follows:

[0070] Initialization parameters: Parameters of the multi-layer graph attention network before training begins and the attention parameter vector Initialization; this framework uses random initialization method to set parameters by randomly sampling from Gaussian distribution and The initial value of

[0071] Forward propagation: The vector set in the joint vector representation Input the multi-layer graph attention network as the initial feature vector of the node;

[0072] For each node , calculate its relationship with neighbor nodes The attention weight between :

[0073] First calculate and , and then concatenate them to get ; Then calculate and through The activation function performs nonlinear transformation; finally, the results of all neighbor nodes are transformed by The activation function is normalized to obtain the attention weight ;

[0074] Calculate the loss function: Define the loss function based on specific tasks including but not limited to node classification and link prediction. , the loss function is used to measure the difference between the model’s prediction results and the true label;

[0075] Backpropagation: Calculating the loss function Parameters and Gradient and This step uses the chain rule to start from the loss function and gradually calculate the gradient of each layer backward; for each layer, the node feature vector update formula is , the derivative of the activation function needs to be considered when calculating the gradient , attention weight right and The derivative of , and the neighbor node feature vector right The derivative of

[0076] Update parameters: Use the optimizer (Adam optimizer in this example) to calculate the gradient and Update parameters separately and ; The optimizer is based on the set learning rate , update the parameters as follows:

[0077]

[0078]

[0079] Repeat the training process: repeat the steps of forward propagation, calculating the loss function, backpropagation, and updating parameters until the model converges.

[0080] The large language model interaction module uses the large language model to perform semantic completion on the knowledge graph entity relationship to generate a semantic completion vector;

[0081] The large language model interaction module receives the graph structure representation , is the total number of vectors in the joint vector representation, and the semantic completion vector is generated according to the following steps:

[0082] First, the entity pairs in the knowledge graph are and , the corresponding graph structure representation and , and the embedding vector corresponding to the unstructured text Concatenate to generate enhanced input vector :

[0083]

[0084] Among them, emb represents the text embedding operation;

[0085] Then the enhanced input vector Input large language model to generate enhanced relations The probability distribution of :

[0086]

[0087] in, are the pre-trained classification layer model parameters, Represents the processing of a large language model;

[0088] Finally, select the one with the highest probability The enhanced relationship , through the feature fusion function Generate the semantic completion vector set corresponding to the enhanced input vector , Z is the preset threshold, and the formula is as follows:

[0089]

[0090] in, For the relationship The relationship vector obtained after the embedding operation;

[0091] The feature fusion function The gating mechanism is used to effectively fuse the target entity pairs. Spectral structural representation of 、 and the predicted relationship Embedded representation of , thereby generating a more representative and semantically rich semantic completion vector ;

[0092] The gating mechanism includes a gating coefficient Calculation and semantic completion vector Calculation of

[0093] The gating coefficient Calculation of: Gating coefficient For control of graph structure representation and The weight distribution when generating semantic completion vectors is calculated as follows: , is the learnable parameter in the gating mechanism;

[0094] The semantic completion vector The calculation formula is:

[0095]

[0096] in, is the gating coefficient, is the semantic completion vector, 、 The target entity pair Spectral structural representation of For the predicted relationship Embedding representation; the relationship between the weighted fusion result and the prediction Embedded representation of Add together to get the final semantic completion vector , relation embedding The semantic information of the predicted relationship is carried and combined with the weighted fusion result of the graph structure representation, so that the semantic completion vector can comprehensively consider the structural information of the entity pair and the semantic information of the predicted relationship;

[0097] The gating mechanism can be used to learn parameters It is learned through the back propagation algorithm during the training process of the model; specifically, during the training process of the entire framework, the loss value is calculated according to the loss function (cross entropy loss in this embodiment), and then the loss function is calculated through back propagation. Finally, the optimization algorithm (Adam in this example) is used to update the gradient The value of makes the loss function gradually decrease, allowing the model to learn the optimal gating mechanism parameters.

[0098] The joint reasoning module, reference Figure 3 As shown, by combining and weighting the graph structure representation and the semantic completion vector, an enhanced knowledge graph quadruple (head entity, relationship, tail entity, confidence) containing a confidence score is output;

[0099] The combination weighting is expressed as follows:

[0100]

[0101] in, Characterize the structure of the map The corresponding reasoning score, is the semantic completion vector The corresponding reasoning score, is the preset threshold, Score the final reasoning; and The calculation formula is as follows:

[0102]

[0103]

[0104] in, is the first multi-layer perceptron, is the second multi-layer perceptron, both are two-layer fully connected networks; the first multi-layer perceptron model parameters and the multi-layer graph attention network parameters and Joint training; the second multi-layer perceptron model parameters are trained through the classification layer parameters Come to train;

[0105] This framework selects the corresponding relationship with the highest score y , generate knowledge graph quadruple after enhanced reasoning , and is the corresponding entity pair in the knowledge graph.

[0106] The above formulas are dimensionless and use only numerical values ​​for calculations. These formulas are based on extensive data and software simulations, and are intended to be as close to actual conditions as possible. Those skilled in the art can adjust the preset parameters in the formulas based on their specific needs.

[0107] Throughout this specification, references to terms such as "one embodiment," "example," or "specific example" indicate that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0108] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to specific embodiments. Obviously, many modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A knowledge graph-enhanced reasoning framework based on graph neural networks and large language models, characterized by: include: The vectorized representation module aligns structured knowledge graph entities with unstructured text entities through an entity alignment algorithm to generate a joint vector representation; A graph neural network encoding module, which uses a multi-layer graph attention network to learn the relationship between entities based on the joint vector representation and outputs a graph structure representation; A large language model interaction module uses a large language model to perform semantic completion on the entity relationship of the knowledge graph and generate a semantic completion vector; The joint reasoning module combines the graph structure representation and the semantic completion vector in a weighted combination to output an enhanced knowledge graph quadruple (head entity, relationship, tail entity, confidence) containing a confidence score. The large language model interaction module receives the graph structure representation , is the total number of vectors in the joint vector representation, and the semantic completion vector is generated according to the following steps: First, the entity pairs in the knowledge graph are and , the corresponding graph structure representation and , and the embedding vector corresponding to the unstructured text Concatenate to generate enhanced input vector ; Then the enhanced input vector is input into the large language model to generate the enhanced relationship The probability distribution of ; Finally, select the one with the highest probability The enhanced relationship , generate the semantic completion vector set corresponding to the enhanced input vector , Z is the preset threshold.

2. The frame according to claim 1, characterized in that The entity alignment algorithm adopts a matching strategy based on cosine similarity, which is expressed as follows: Suppose the set of structured knowledge graph entity vectors is , is the total number of structured knowledge graph entity vectors, each knowledge graph entity vector corresponds to a knowledge graph entity; the set of unstructured text entity vectors is , is the total number of unstructured text entity vectors; the cosine similarity formula is expressed as follows: in, is the first The vector representation of an entity, The first The vector representation of the entity; when the similarity is calculated If the value exceeds the preset threshold, it is considered a match, otherwise it is not a match.

3. The frame according to claim 2, characterized in that The joint vector representation is composed of a fusion vector and unmatched entity vectors of the structured knowledge graph; The fusion vector The weighted fusion generation of the successfully matched structured knowledge graph entity vector and the unstructured text entity vector is as follows: in, is the first The vector representation of an entity, The first The vector representation of an entity, for and similarity.

4. The frame according to claim 1, wherein The multi-layer graph attention network uses the vector in the joint vector representation as the initial feature vector of the node, which is expressed as follows: Suppose the vector set in the joint vector representation is , is the total number of vectors in the joint vector representation; the update formula of the multi-layer graph attention network node feature vector is expressed as follows: in, is the updated feature vector of the node, For the node With neighboring nodes The attention weights between are the parameters of the multi-layer graph attention network, For nodes The set of neighbor nodes of is the activation function; The multi-layer graph attention network updates all nodes through the multi-layer graph attention network node feature vector update formula, and updates the updated node feature vector As the input of the next layer of graph attention network, repeat the node update formula to perform feature extraction and update; The graph structure representation nodes are composed of updated feature vectors of all nodes in the multi-layer graph attention network.

5. The frame according to claim 4, characterized in that The attention weight The calculation formula is: in, is the softmax activation function, is the LeakyReLU activation function; are the parameters of the multi-layer graph attention network; is the attention parameter vector; is the feature vector of node i, is the feature vector of node i’s neighbor node j; and Perform vector concatenation ; and a are updated via backpropagation.

6. The frame according to claim 1, wherein: The combination weighting is expressed as follows: in, Characterize the structure of the map The corresponding reasoning score, is the semantic completion vector The corresponding reasoning score, is the preset threshold, Score the final reasoning; This framework selects the corresponding relationship with the highest score y , generate knowledge graph quadruple after enhanced reasoning , and is the corresponding entity pair in the knowledge graph.

Images