Knowledge graph completion method based on large language model

Abstract

The invention discloses a knowledge graph completion method based on a large language model. The knowledge graph completion method comprises the following steps: constructing a multi-granularity index of an entity sub-graph and a relation mode; performing mixed retrieval based on the query triad to obtain related local sub-graphs and global relation modes; using a graph neural network and a projector to convert the retrieved structured knowledge into a soft prompt vector, and injecting the soft prompt vector into a large language model; and finally, calculating the semantic similarity between the model output and the candidate entity through a semantic matching module to finish prediction. According to the method, the structure information of the knowledge graph and the semantic comprehension ability of the large language model are effectively integrated through a retrieval enhancement mechanism (RAKGC), the problems of hallusion, text mismatching, weak graph structure perception ability and the like of the large language model in a knowledge graph completion task are solved, and the completion accuracy and robustness are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and knowledge engineering technology, and in particular relates to a knowledge graph completion method that combines large language models and retrieval enhancement technology. Background Technology

[0002] Knowledge graph completion aims to predict missing entities or relations based on the existing knowledge graph structure. With the development of Large Language Models (LLMs), leveraging their rich world knowledge and powerful language understanding capabilities for knowledge graph completion has become a novel approach. However, existing methods face several challenges: First, knowledge graphs often lack rich textual descriptions, making LLMs prone to "illusions"; second, the predictions generated by LLMs are often in natural language form, leading to "text mismatches" with the normalized entity names in the knowledge graph (e.g., generating "NYC" while the graph shows "New York City"), resulting in misjudgments; finally, existing methods often simply linearize triples into text, ignoring the graph's topological structure information and limiting the model's reasoning ability. Furthermore, the limited context window length of LLMs prevents direct input of large-scale graph data. Therefore, effectively selecting and injecting structured knowledge to enhance the reasoning ability of LLMs is a pressing issue that needs to be addressed. Summary of the Invention

[0003] The purpose of this invention is to propose a knowledge graph completion method (RAKGC) based on a large language model. By using a retrieval enhancement generation paradigm, the structured information of the knowledge graph is transformed into soft prompts and injected into the large language model. Combined with a semantic matching mechanism, this effectively solves the above-mentioned problems.

[0004] To achieve the above objectives, this invention employs the following technical solution: a knowledge graph completion method based on a large language model, comprising the following steps: Step 1. Subgraph Construction: Constructing an entity subgraph index, a relation pattern index, and a negative sample set for the knowledge graph. The entity subgraph index encodes neighborhoods with different hop counts using a graph neural network to capture multi-granularity structural features; the relation pattern index mines the global distribution patterns of relations through clustering; the negative sample set is used to guide the model in exploring new knowledge. Step 2. Hybrid Retrieval: For the query triple $(h, r, ?)$ to be completed, the entity subgraph most relevant to the head entity $h$ and the relation pattern most matching the relation $r$ are retrieved respectively. A consistency scoring mechanism is introduced to reorder the retrieval results, ensuring semantic consistency between the retrieved subgraphs and the target relation. Step 3. Knowledge Injection: Encoding the retrieved subgraphs using a graph neural network, and mapping structured knowledge to the embedding space of the large language model through a learnable projector to generate soft prompts. The soft hint is placed before the input sequence as a virtual token to guide the model's reasoning. Step 4. Semantic matching: Map the text output of the large language model to the candidate entity names in the knowledge graph to the same semantic vector space (e.g., using BERT), and align and sort them by calculating cosine similarity to solve the matching problem caused by differences in literal expression.

[0005] The present invention has the following advantages: Enhanced structure awareness: By retrieving and encoding entity subgraphs, large language models can leverage the local topological structure of knowledge graphs for reasoning, compensating for the shortcomings of pure text input.

[0006] Utilizing global schemas: Introducing relation schema indexes and leveraging global distribution constraints of relations (such as head and tail entity type constraints) to narrow the search space and improve inference accuracy.

[0007] Solving text mismatch: By using a semantic vector-based matching module, the performance loss caused by the inconsistency between the model-generated results and the literal names of the graph entities is resolved.

[0008] Efficiency and adaptability: It adopts a soft prompt injection method, which does not occupy too much context window and keeps the parameters of the large language model frozen, reducing training costs and having good generalization ability. Attached Figure Description

[0009] Figure 1 This is an overall framework diagram of the RAKGC method in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the workflow of the hybrid retrieval module in an embodiment of the present invention. Figure 3 This is a flowchart illustrating the workflow of the knowledge injection module in an embodiment of the present invention. Detailed Implementation

[0010] The present invention will be further described in detail below with reference to specific embodiments.

[0011] Step 1: Subgraph Construction Module This step aims to organize and index structured knowledge from a large-scale knowledge graph. (1) Entity Subgraph Index Construction: For any entity $e$ in the graph, define its $k$-hop neighborhood subgraph $G_e^k$. First, use a pre-trained language model (PLM) to text-encode nodes and edges to obtain the initial embedding $h^{(0)}$. Then, use an $L$-layer graph neural network (GNN) to aggregate neighborhood information. For the $l$-th layer, the update formula is: $h_e^{(l)} = GNN^{(l)}(h_e^{(l-1)}, {h_{e'}^{(l-1)}, h_r^{(l-1)} | (e, r, e') \in G_e^k})$. Finally, node embeddings of different granularities ${h_e^{(1)}, ..., h_e^{(L)}}$ are obtained and stored in the vector database. (2) Relational Pattern Index Construction: Cluster entities to obtain category labels. Statistically calculate the head / tail entity category distributions $P_r^{head}(c)$ and $P_r^{tail}(c)$ for each relation $r$. Construct a relational pattern vector $v_r^{pattern}$, which integrates the relation's semantic vector, category distribution vector, and typical example triple vector. (3) Negative Sample Set Construction: Extract known triples that are the same as the query relation but different from the tail entities. Use these as context during inference to explicitly inform the model which are "known" and guide the model to explore potential "unknown" answers.

[0012] Step 2: Hybrid Retrieval Module This step is responsible for dynamically retrieving relevant knowledge. (1) Query Vector Generation: Input the descriptions of the query head entity $h$ and relation $r$ into PLM to obtain query vectors $q_h$ and $q_r$. (2) Similarity Retrieval: Calculate the cosine similarity in the vector database to retrieve the Top-K$ entity subgraphs and the Top-M$ relation patterns. (3) Re-ranking: Calculate the consistency score between the retrieved subgraph $G$ and the relation pattern $r$: $match(G, r) = \frac{1}{|G|}\sum_{(h', r', t') \in G} I(c_{t'} \in C_r^{tail})$ where $C_r^{tail}$ is the set of candidate tail entity categories for relation $r$. Retain the $K'$ subgraphs with the highest matching degree based on the score.

[0013] Step 3: Knowledge Injection Module This step transforms the retrieved symbolic knowledge into soft hints that the model can understand. (1) Knowledge Encoding: The retrieved $K'$ subgraphs are re-encoded using GNN and aggregated to obtain entity knowledge representation $z_{entity}$; relation pattern vectors are directly extracted to obtain $z_{relation}$. The two are concatenated to obtain the total knowledge vector $z_{knowledge} = [z_{entity}; z_{relation}]$. (2) Soft Hint Generation: The knowledge vector is mapped to soft hints $s$ through the projector $f_{proj}$: $s = f_{proj}(z_{knowledge}) = W_2 \cdot ReLU(W_1z_{knowledge} + b_1) + b_2$ (3) Injection: $s$ is inserted at the beginning of the LLM input sequence: ${s, e_1,e_2, ..., e_T}$, where $e_i$ is the token embedding of the query text.

[0014] Step 4: Semantic Matching Module This step solves the alignment problem between the generated text and the entities in the graph. (1) Text Embedding: The response text generated by LLM is encoded using the BERT model to obtain the vector $e_{pre}$; at the same time, the names of all candidate entities in the graph are pre-encoded to obtain the vector set ${e_{kg}^i}$. (2) Similarity Calculation and Ranking: Calculate the cosine similarity: $similarity(e_{pre}, e_{kg}^i) = \frac{e_{pre} \cdot e_{kg}^i}{|e_{pre}| |e_{kg}^i|}$ The candidate entities are sorted in descending order according to the similarity, and the Top-K are selected as the final prediction results.

[0015] Furthermore, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the above-described method. The present invention also provides a computer-readable storage medium storing a computer program implementing the above-described method.

[0016] The above description is only a preferred embodiment of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A knowledge graph completion method based on a large language model, characterized in that, The process includes the following steps: Step 1. Subgraph Construction: Extract structured knowledge related to the query triples from the knowledge graph to construct an entity subgraph index and a relation pattern index; Step 2. Hybrid Retrieval: Based on the head entity and relation in the query triples, retrieve relevant contextual knowledge from the entity subgraph index and relation pattern index using vector similarity, and reorder the retrieval results; Step 3. Knowledge Injection: Use a graph neural network to structurally encode the retrieved contextual knowledge, and use a projector to transform the encoded knowledge into soft cue vectors, injecting them into the embedding space of the large language model; Step 4. Semantic matching: Obtain the response text generated by the large language model, use the semantic coding model to encode the response text and the candidate entities in the knowledge graph respectively, calculate the semantic similarity between the two, and obtain the final prediction result based on the similarity ranking.

2. The knowledge graph completion method based on a large language model according to claim 1, characterized in that, Step 1 specifically includes: constructing an entity subgraph index: for any entity $e$ in the knowledge graph, extract a subgraph $G_e^k$ containing nodes and connecting edges that are no more than $k$ hops away from it. Use a pre-trained language model to initially encode the nodes and edges in the subgraph, and use a graph neural network to aggregate neighborhood information to obtain multi-granularity node embeddings, which are then stored in a vector database; constructing a relation pattern index: cluster the entities in the knowledge graph, statistically analyze the distribution pattern of each relation $r$ in different entity category combinations, combine the semantic description of the relation with typical example triples, generate relation pattern vectors, and store them in a vector database; constructing a negative sample set: select triples from the entity subgraph that are the same as the query relation but have different tail entities, which are used to guide the model to filter known facts and explore potential new answers during the inference stage.

3. The knowledge graph completion method based on a large language model according to claim 1, characterized in that, Step 2 specifically includes: inputting the text descriptions of the head entity $h$ and relation $r$ in the query triple $(h, r, ?)$ into a pre-trained language model to generate a head entity query vector $q_h$ and a relation query vector $q_r$; calculating the cosine similarity between $q_h$ and the vectors in the entity subgraph index to retrieve the top $K$ entity subgraphs; calculating the similarity between $q_r$ and the vectors in the relation pattern index to retrieve the top $M$ relation patterns; introducing a consistency scoring mechanism to calculate the matching degree between the retrieved entity subgraphs and relation patterns, re-ranking the candidate entity subgraphs according to the matching degree, and selecting the $K'$ subgraphs with the highest matching degree as the final retrieval results.

4. The knowledge graph completion method based on a large language model according to claim 1, characterized in that, Step 3 specifically includes: encoding the retrieved $K'$ entity subgraphs using a graph neural network and aggregating them to obtain an entity knowledge vector $z_{entity}$; extracting the vector representations of the retrieved $M$ relation patterns and concatenating them to obtain a relation knowledge vector $z_{relation}$; inputting the concatenated $z_{entity}$ and $z_{relation}$ into a projector, which is composed of a multilayer perceptron, to map the knowledge vector into a soft cue vector $s$ with the same dimension as the large language model; and placing the soft cue vector $s$ before the token embedding sequence of the query text as the input context of the large language model.

5. The knowledge graph completion method based on a large language model according to claim 1, characterized in that, Step 4 specifically includes: encoding the response text generated by the large language model using the BERT model to obtain the response vector $e_{pre}$; encoding the names of all candidate entities in the knowledge graph using the BERT model to obtain the entity vector set $E_{kg}$; calculating the cosine similarity between the response vector $e_{pre}$ and each vector in the entity vector set $E_{kg}$, sorting the candidate entities according to the similarity from high to low, and generating an ordered list as the completion result.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the knowledge graph completion method based on a large language model as described in any one of claims 1 to 5.

7. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the knowledge graph completion method based on a large language model as described in any one of claims 1 to 5.

Images