A knowledge graph question answering method combining a large model and a graph neural network

Abstract

The application relates to a knowledge graph question answering method combining a large model and a graph neural network, comprising a question decomposition module, a sub-question solving module and an answer reasoning module, wherein an iterative question decomposition method is proposed in the question decomposition module to solve the difficulty that the size of a subgraph in a multi-hop question cannot be determined by using a graph neural network method, the sub-question solving module is composed of a subgraph retrieval stage and a reasoning path construction stage, a similarity definition method is given in the subgraph retrieval stage, and a Personalized PageRank algorithm is used for calculation, and in order to improve the accuracy of GNN answers, the subgraph of a subsequent question is combined with the subgraph of a previous question to realize dynamic expansion of the subgraph. In the reasoning path construction stage, a candidate answer is obtained by using a pre-trained graph neural network, and a BFS algorithm is used to construct a reasoning path. Finally, the rationality of the reasoning path is judged by using the answer reasoning module to give a sub-question answer. The application provides a knowledge graph question answering framework combining a large model and a graph neural network, and effectively improves the performance of the model in a multi-hop question.

Description

Technical Field

[0001] This invention relates to a knowledge graph question answering method that combines large models and graph neural networks, belonging to the field of knowledge graph question answering technology. Background Technology

[0002] A knowledge graph is a database that stores information in a structured format, where knowledge in the real world is represented by triples (head entity, relation, tail entity). Knowledge graph question answering is a question-answering system based on knowledge graphs, aiming to answer natural language questions by querying structured knowledge graphs.

[0003] The mainstream approach to knowledge graph question answering is Retrieval Augmentation (RAG), which aims to mitigate the illusion of LLM by using up-to-date and accurate information (e.g., information from the knowledge graph) as context. In knowledge graph question answering, the goal is to use a large model to answer natural language questions based on information provided by the knowledge graph. For example, given the question "Which language do Jamaicans speak?", the knowledge retrieved from the knowledge graph using RAG is "Jamaica→language_spoken→English". Using this knowledge as context for the large model's answer allows it to access information from the knowledge graph. However, Retrieval Augmentation is a greedy strategy, only retrieving the next-hop entity with the highest score for the current entity, failing to directly derive the scores of entities not adjacent to the current entity, potentially leading to local optima. For example... Figure 1 As shown, suppose there is a question "Which primary school does John go to?". When searching for the next-hop entity with the highest score using the RAG method, "university" is found. Then, the search continues for the next-hop entity with the highest score for "university", resulting in "University of Oxford". The final score for this answer is 0.4 × 0.2 = 0.08. However, the actual path is to first select the lower-scoring entity "Oslo", and then select its highest-scoring entity "Gamlebyen Skole". The score for the actual answer is 0.2 × 0.8 = 0.16, which is higher than the score of the entity retrieved using the greedy strategy.

[0004] To address the issue of large models easily getting trapped in local optima during knowledge graph retrieval, previous work employed a paradigm combining large models with graph neural networks. By using graph neural networks in the path retrieval phase to capture the scores between any two entities, the shortcomings of large models during retrieval were mitigated. However, since knowledge graphs are generally very large, with millions or even hundreds of millions of entities, it is typically necessary to first retrieve a smaller subgraph specific to the current question from the knowledge graph, and then answer the question within that subgraph. Generally, the size of the subgraph is proportional to the complexity of the question, leading to significant differences in subgraph size between questions of different complexities. Previous methods used a fixed subgraph size; if the extracted subgraph is too large, it reduces the accuracy of the final answer prediction. If the extracted subgraph is too small, the final answer may not be within the subgraph. Especially for multi-hop problems, the final answer may be too far from the seed entity, resulting in low accuracy for previous methods on multi-hop problems. Summary of the Invention

[0005] To overcome the problems existing in the prior art, this invention provides a knowledge graph question answering method that combines large models and graph neural networks.

[0006] For multi-hop problems, this invention proposes an iterative problem decomposition method. First, the context learning capability of a large model is used to decompose the multi-hop problem, generating an initial single-hop sub-problem. Then, after solving the sub-problem, the solution steps are added to the context, and the enriched context is input back into the large model, allowing the large model to iteratively decompose the problem until the large model can no longer provide sub-problems. At this point, the last given answer is taken as the final answer.

[0007] After obtaining a single-hop sub-problem, the top-k key entities of the sub-problem are extracted as seed entities using the context learning capability of the large model. Then, starting from the seed entities, a sub-graph specific to the current problem is retrieved from the knowledge graph. The sub-graph and sub-problem are then input into a graph neural network (GNN) model to obtain all possible answer entities. Entities are sorted by probability from high to low, and entities with a cumulative probability greater than 95% are selected as candidate answers. Next, the Breadth-First Search (BFS) algorithm is used to obtain all paths between the seed entities and each candidate answer. These paths are then textualized to form explicit reasoning paths, which serve as the context for the large model's reasoning. Finally, the answer to the sub-problem is obtained through the large model's reasoning capability. After solving each single-hop sub-problem, the solution steps are added to the context to provide the large model with more problem information. The enriched context is then input back into the large model, iteratively allowing the large model to decompose the problem. If the large model cannot further decompose into sub-problems, the last answer is the final answer; if sub-problems are decomposed, the above steps are followed. The overall architecture diagram of the model is shown below. Figure 5 As shown in the diagram. Simultaneously, the subgraph for each subsequent question is merged with the subgraph for the previous question to achieve dynamic expansion of the subgraph, which is beneficial for answering subsequent questions. A diagram illustrating this dynamic expansion is shown below. Figure 4 As shown.

[0008] A knowledge graph question answering method combining large models and graph neural networks includes the following steps:

[0009] Step 1: Problem Decomposition: Utilize the context learning capabilities of large models to decompose multi-hop problems.

[0010] The specific steps of decomposition include: when a multi-hop problem is received, the context examples are concatenated with the multi-hop problem and then input into the large model. The large model generates an initial single-hop sub-problem, which is then input into the sub-problem solving module to obtain the inference path. Finally, the inference path is input into the inference module to obtain the answer.

[0011] The specific steps and results of the above decomposition are added into the context and provided to the large model. Then, the enriched context is input into the large model again, so that the large model can iteratively decompose the problem and solve it according to the above steps until the large model can no longer provide sub-problems.

[0012] Step 2: Sub-problem solving: After obtaining the sub-problem through Step 1, it is necessary to construct the reasoning path of the sub-problem using the GNN model, which specifically includes the problem subgraph retrieval stage and the reasoning path construction stage.

[0013] Step 3: Answer Reasoning: After obtaining all paths between seed entities and candidate answers through the question subgraph retrieval module, these paths are textualized to construct the context of the LLM. Then, the sub-questions are concatenated and input into the LLM. The reasoning ability of the large model is used to obtain the final answer to the sub-question.

[0014] The problem subgraph retrieval module in step two specifically includes:

[0015] Once a single-hop sub-problem derived from a multi-hop problem is obtained, the problem subgraph of the sub-problem is retrieved from the knowledge graph.

[0016] Knowledge graphs are composed of triples (s, r, o). For a given problem, selecting the triples that are truly relevant is crucial. From a probabilistic perspective, we define relevance as follows:

[0017]

[0018] Where Score(e,q,G) represents the relevance of triple e to question q in knowledge graph G, N represents the total number of random walks in knowledge graph starting from question q, times(s) represents the number of times triple s is visited during the random walk, and subject to start=q means that the constraint is to start from q.

[0019] Since question q is difficult to represent directly in the knowledge graph, a large model is used to extract the top-k key entities from the question and then map them to the corresponding entities in the knowledge graph, thus approximating the question. The relevance definition after approximation is as follows:

[0020]

[0021] Relevance can be calculated using the Personalized PageRank algorithm, ultimately selecting the 2000 most relevant triples as the problem subgraph for the current problem.

[0022] The reasoning path construction module in step two specifically includes:

[0023] The graph neural network part uses a pre-trained ReaRev graph neural network. The sub-problem and the problem subgraph are input into the ReaRev model to obtain the probability of each entity as an answer. The probabilities are then sorted from high to low, and entities with a cumulative probability greater than 95% are selected as candidate answers. The BFS algorithm is used to retrieve all paths between the seed entity and each candidate answer.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0025] To mitigate the problem of RAG methods easily getting trapped in local optima, this invention employs a GNN method in the question subgraph retrieval stage. Furthermore, to address the difficulty in determining the question subgraph size in multi-hop problems using GNNs, this invention proposes an iterative question decomposition method, resulting in smaller differences in the size of the question subgraphs between decomposed single-hop problems. Finally, to improve the accuracy of GNN answers, the question subgraphs of subsequent questions are merged with those of previous questions, achieving dynamic expansion of the question subgraph. This invention provides a knowledge graph question answering framework combining a large model and graph neural networks, effectively improving the model's performance on multi-hop problems.

[0026] This invention first decomposes multi-hop problems using the context learning capabilities of a large model, obtaining an initial single-hop sub-problem. Then, it solves and infers the answer to the sub-problem, ultimately obtaining the answer. The solution steps of this sub-problem are then added to the context to provide the large model with more problem information. Simultaneously, the enriched context is input into the large model again for iterative problem decomposition until the large model can no longer provide sub-problems, at which point the answer to the last sub-problem is taken as the final answer. In this way, the problem subgraph can dynamically approach the final answer entity. The final model is tested on the WebQSP knowledge graph question-answering dataset. Compared to the traditional graph neural network method ReaRev, this invention improves the hit@1 metric by 7 points and the F1 metric by 0.8 points. Compared to the Retrieval Augmentation Generation (RAG) method ROG, it improves the hit metric by 1.5 points, the hit@1 metric by 3.3 points, and the F1 metric by 0.9 points. See the complete results graph below. Figure 8 This invention has broad practical application value and can be applied in fields such as knowledge graphs, helping to better solve practical problems. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of the RAG method, a knowledge graph question answering method combining large models and graph neural networks, according to the present invention.

[0029] Figure 2 This is a schematic diagram of the GNN method for knowledge graph question answering that combines a large model and a graph neural network, according to the present invention.

[0030] Figure 3This is a contextual example diagram illustrating the question decomposition of a knowledge graph question answering method combining a large model and graph neural networks according to the present invention.

[0031] Figure 4 This is a schematic diagram illustrating the differences between the knowledge graph question answering method combining large models and graph neural networks of the present invention and the GNN method;

[0032] Figure 5 This is an architecture diagram of a knowledge graph question answering method that combines a large model and a graph neural network according to the present invention;

[0033] Figure 6 This is an example diagram of a large model prompt template for a knowledge graph question answering method that combines large models and graph neural networks according to the present invention.

[0034] Figure 7 This is an example diagram of the answer reasoning prompt for a knowledge graph question answering method combining large models and graph neural networks according to the present invention;

[0035] Figure 8 The figure shows the experimental results of a knowledge graph question answering method combining a large model and a graph neural network according to the present invention.

[0036] Figure 9 This is an example diagram of a prompt template for key entity extraction in a knowledge graph question answering method that combines a large model with a graph neural network, according to the present invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] Reference Figure 5 A knowledge graph question answering method that combines large models with graph neural networks includes:

[0039] Step 1: Problem Decomposition Phase

[0040] In the problem decomposition phase, this invention utilizes the context learning capability of a large model to decompose multi-hop problems, with specific examples of the context provided in the appendix. Figure 3 In this example, two learning examples, Q1 and Q2, are provided. Each learning example consists of several sub-problems (Subque), inference paths (Ra), and answers (Ans). Therefore, the final context example prompt is structured as follows:

[0041]

[0042] in This indicates text concatenation. The following explanation uses specific examples.

[0043] Suppose we have a multi-hop question, multi_ques, that asks "What language is spoken in Bolt's country?", then the input to the large model would be: The main model provides a subquestion: "What country is Usain Bolt from?". The subquestion is then calculated as follows:

[0044]

[0045] Where G(X) represents the input of X as content to the large model G for a response.

[0046] If the first subquestion has been solved, we can obtain the reasoning path Ra and the answer Ans for the first subquestion, concatenate them into the context, and then input the enriched context back into the large model to obtain the second subquestion. The new subquestion is calculated as follows:

[0047]

[0048] If the current subquestion is "Finished!", it means that the question-answering is complete, and the answer to the last question will be taken as the final answer.

[0049] Step Two: Sub-problem Solving Phase

[0050] After obtaining the sub-problem through step (1), the reasoning path of the sub-problem needs to be constructed through the GNN model, which includes two steps: the problem subgraph retrieval stage and the reasoning path construction stage.

[0051] Step 2.1: Problem Subgraph Retrieval Phase

[0052] Since knowledge graphs typically contain millions of facts and nodes, it is impractical to feed all nodes into a graph neural network simultaneously. Therefore, smaller, question-specific subgraphs G can be retrieved for question q. q Ideally, all correct answers to the question are contained in the retrieved subgraph, i.e., {a} i}∈G q Then, the retrieved problem subgraph G... q Used together with question q as input to the GNN model, causing the GNN model to output the correct answer.

[0053] Specifically, when a multi-hop problem is decomposed into multiple single-hop subproblems, we first leverage the context learning capability of the large model to extract the top-k key entities from the subproblems as seed entities for the current problem. See the specific context prompt for details. Figure 9 The prompt consists of two parts: an initial task introduction and three subsequent examples. After extracting the seed entity `seed_ent`, the system first searches the knowledge graph for triples (s, r, o) where either the head entity `s` or the tail entity `o` is the seed entity; these triples are denoted as `cand_ent`. Then, the Personalized PageRank algorithm is applied to these triples to select the 2000 triples most relevant to the seed entity in the problem, which are then used as the problem subgraph for that subproblem.

[0054] The specific steps of the Personalized PageRank algorithm are as follows:

[0055] A random walk begins from the seed node (seed_ent). At each node reached, the walk stops with a probability of 1-d (where d is 0.2) and restarts from the seed node, or continues with a probability of d. When continuing, a node is randomly selected from the nodes pointed to by the current node according to a uniform distribution and the walk continues. After many rounds of this process, the probability of each vertex being visited will stabilize. This probability is used as the correlation between the current triplet and the current subproblem. Finally, the top 2000 triplets with the highest probabilities are selected as the problem subgraph for the current subproblem. The probability calculation formula for each node is as follows:

[0056]

[0057] Where PPR(i) represents the probability of the i-th entity, and out(i) represents the out-degree of the i-th entity.

[0058] Step 2.2: Inference Path Construction Module

[0059] In knowledge graph question answering, the GNN model can be viewed as a node classification problem, where KG entities are classified as either answers or non-answers to a given question. GNNs aggregate data from each neighbor node V... ' Information To update the representation of node v at level l The specific GNN update expression is as follows:

[0060]

[0061] The function ω(·) measures the fact (v,r,v) 'The relationship between r and problem q is the degree of relevance. Ψ(·) is used to combine the representations between two consecutive layers.

[0062] This invention uses a pre-trained ReaRev graph neural network. The input to the graph neural network is a subproblem q and a problem subgraph G. q In the subproblem q and the problem subgraph G q Before inputting the subproblem q into the GNN model, it needs to be embedded. This invention uses the sbert model to embed the input. Assuming the subproblem is q, the embedded output is E. q .

[0063] E q With problem subgraph G q The entities are input into the GNN model to obtain entity scores (ans_score). Ans_score represents the probability of each entity being an answer. The entity scores (ans_score) are sorted from highest to lowest, and then those entities with a cumulative probability of 95% are selected as candidate answers (cand_ans) from highest to lowest. Figure 5 The answers highlighted in yellow are Jamaica, America, and English.

[0064] Having the seed entity `seed_ent` and candidate entities `cand_ans`, the Breadth-First Search (BFS) algorithm is used to sequentially search all paths between each seed entity and each candidate answer in the problem subgraph. For example... Figure 5 There are 3 paths starting from "Usain bolt" and ending at "Jamaica", "America", and "English", resulting in [['born_in'],['enter_competition','USAC','Host_place'],['spoken_language']].

[0065] Step 3: Answer Reasoning Module

[0066] To facilitate understanding by larger models, the retrieved paths are textualized, and the textualized result is "Usain".

[0067] bolt->born_in->Jamaica", "Usain

[0068] The results “bolt->enter_competition->USAC->Host_place->America” and “Usainbolt->spoken_language->English” are referred to as the inference path.

[0069] Since the reasoning path retrieves all paths from all starting entities to the ending entity, there may be cases where the ending entity itself is not the correct answer, or cases where the ending entity is the correct answer but the reasoning path is incorrect. Therefore, further filtering of the reasoning path is required.

[0070] To alleviate the two problems mentioned above, this invention employs contextual reasoning of a large model in the final stage, relying on the reasoning ability of the large model to determine whether the answer and reasoning path are reasonable.

[0071] The prompt is divided into 3 parts, as detailed in the attached document. Figure 6 As shown, these are the inference paths / Prompts. reason Task command Prompt instruct With the question Prompt question Among them, Prompt reason With Prompt question It will dynamically modify depending on the problem, while Prompt instruct The three prompt parts remain unchanged. Finally, the three parts are concatenated to form the contextual cue prompt for GPT. ICL .

[0072] Prompt ICL =prompt reason +Prompt instruct +Prompt question

[0073] A complete example is attached. Figure 7 As shown.

[0074] Finally, Prompt ICL Provide it to a large model, and use the reasoning ability of the large model to determine the final answer.

[0075] The final model experimental results are shown below. Figure 8 In this experiment, k is set to 1 in the top-k range. The hit metric measures whether any of the generated answers is correct; the hit@1 metric measures whether the answer with the highest probability among the generated answers is correct. The F1 score considers both recall and precision of the generated answers.

[0076] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A knowledge graph question answering method combining large-scale models and graph neural networks, characterized in that: Includes the following steps: Step 1: Problem Decomposition: Utilize the context learning capabilities of large models to decompose multi-hop problems. The specific steps of decomposition include: when a multi-hop problem is received, the context examples are concatenated with the multi-hop problem and then input into the large model. The large model generates an initial single-hop sub-problem, which is then input into the sub-problem solving module to obtain the inference path. Finally, the inference path is input into the inference module to obtain the answer. The results of the specific steps of the above decomposition are added into the context and provided to the large model. Then, the enriched context is input into the large model again, so that the large model can iteratively decompose the problem and solve it according to the above steps until the large model can no longer provide sub-problems. Step Two: Sub-problem Solving: After obtaining the sub-problems through Step One, it is necessary to construct the reasoning path for the sub-problems using the GNN model. This specifically includes the problem subgraph retrieval stage and the reasoning path construction stage. Step 3: Answer Reasoning: After obtaining all paths between seed entities and candidate answers through the question subgraph retrieval module, these paths are textualized to construct the context of the LLM. Then, the sub-questions are concatenated and input into the LLM. The reasoning ability of the large model is used to obtain the final answer to the sub-question. The problem subgraph retrieval stage in step two specifically includes: Once the single-hop subproblems decomposed from the multi-hop problem are obtained, the problem subgraph of the single-hop subproblems is retrieved in the knowledge graph; Knowledge graphs are composed of triples, and the relevance is defined from a probabilistic perspective: Where Score(e,q,G) represents the relevance of triple e to question q in knowledge graph G, N represents the total number of random walks in knowledge graph starting from question q, times(s) represents the number of times triple s is visited during the random walk, and subject to start=q means that the constraint is to start from q. Since question q is difficult to represent directly in a knowledge graph, we extract the top-k key entities from the question using a large model as seed entities to approximate question q. The relevance definition after approximation is as follows: Finally, this relevance was calculated using the Personalized PageRank algorithm, and the 2000 triples most relevant to the problem were selected as the problem subgraph for the current problem. The reasoning path construction stage in step two specifically includes: In knowledge graph question answering, the GNN model is considered a working method for node classification, where KG entities are classified as answers or non-answers to a given question. First, candidate answers to sub-questions are obtained through a graph neural network, where the graph neural network part uses a pre-trained ReaRev graph neural network. The sub-questions and question subgraphs are input into the ReaRev model to obtain the probability of each entity being an answer. Then, the probabilities are sorted from high to low, and entities with a cumulative probability greater than 95% are selected as candidate answers. The BFS algorithm is used to retrieve all paths between the seed entity and each candidate answer.

Images