Knowledge graph question and answer method combining large model and graph neural network

Abstract

The invention relates to a knowledge graph question answering method combining a large model and a graph neural network, which comprises a question decomposition module, a sub-question solving module and an answer reasoning module, the method is used for solving the problem that the size of a problem sub-graph is difficult to determine in a multi-hop problem in a graph neural network method, a sub-problem solving module is composed of a problem sub-graph retrieval stage and a reasoning path construction stage, in the problem sub-graph retrieval stage, a similarity definition mode is given, calculation is carried out through a Personalized PageRank algorithm, and the problem sub-graph retrieval stage is used for solving the problem sub-graph size. And meanwhile, in order to improve the accuracy of GNN answering, the question sub-graphs of the subsequent questions are combined with the question sub-graphs of the previous questions, so that the dynamic expansion of the question sub-graphs is realized. In the inference path construction stage, candidate answers are obtained through a pre-trained graph neural network, and then an inference path is constructed by using a BFS algorithm. And finally, the reasonability of the reasoning path is judged through an answer reasoning module so as to give sub-question answers. The invention provides a knowledge graph question and answer framework combining a large model and a graph neural network, and the performance of the model on a multi-hop problem is effectively improved.

Description

Technical Field

[0001] The present invention relates to a knowledge graph question answering method combining a large model with a graph neural network, and belongs to the technical field of knowledge graph question answering. Background Art

[0002] A knowledge graph is a database that stores information in a structured form, where real-world knowledge is represented in the form of triples (head entity, relationship, tail entity). Knowledge graph question answering is a question answering system based on knowledge graphs, which aims to answer natural language questions by querying structured knowledge graphs.

[0003] The mainstream existing knowledge graph question answering method is the retrieval-augmented generation (RAG) method, which aims to alleviate the illusion of LLM by using the latest and accurate information (such as information obtained from KG) as context. In knowledge graph question answering, the goal is to use a large model to answer natural language questions based on the information provided by the knowledge graph. For example, there is a question "Which language do Jamaicans speak?" The knowledge retrieved on the knowledge graph through RAG technology is "Jamaica→language_spoken→English". This knowledge is used as the context for the large model to answer, and the large model can access the information of the knowledge graph to answer the question. However, the retrieval-augmented generation method is a greedy strategy. Each time, it can only retrieve the next-hop entity with the highest score of the current entity. It cannot directly derive the score of entities that are not adjacent to the entity, and may fall into a local optimal solution. For example Figure 1 As shown in the figure, suppose there is a question "Which primary school does John go to?" When searching for the next hop entity with the highest score through RAG, "university" is retrieved, and then the next hop entity with the highest score of "university" is retrieved, and "University of Oxford" is retrieved. The final answer score is 0.4×0.2=0.08 points. However, the actual path is to first select the entity with a lower score "Oslo", and then select its entity with the highest score "Gamlebyen Skole". The score of the actual answer is 0.2×0.8=0.16 points, which is higher than the score of the entity retrieved by the greedy strategy.

[0004] In order to solve the problem that large models are prone to fall into local optimal solutions when retrieving knowledge graphs, previous work has adopted the paradigm of combining large models with graph neural networks. By using graph neural networks in the path retrieval stage to capture the scores between any two entities, the defects of large models in retrieval are alleviated. However, since knowledge graphs are generally large and the number of entities is as high as millions or even hundreds of millions, it is generally necessary to first retrieve a smaller question subgraph specific to the current question on the knowledge graph, and then answer the question on the question subgraph. In general, the size of the question subgraph is often proportional to the complexity of the question, resulting in large differences in the size of the question subgraph between questions of different complexity. The previous method uses a fixed question subgraph size; if the extracted question subgraph is too large, the accuracy of the final answer prediction will be reduced. If the extracted question subgraph is too small, the final answer may not be in the question subgraph. In particular, for multi-hop questions, the final answer may be too far away from the seed entity, resulting in low accuracy of the previous method on multi-hop questions. Summary of the invention

[0005] In order to overcome the problems existing in the prior art, the present invention provides a knowledge graph question answering method that combines a large model with a graph neural network.

[0006] For multi-hop problems, the present invention proposes an iterative problem decomposition method. First, the context learning ability of the large model is used to decompose the multi-hop problem to generate an initial single-hop sub-problem. After solving the sub-problem, the solution steps are added to the context, and then the enriched context is input into the large model again, so that the large model iteratively decomposes the problem until the large model can no longer give a sub-problem. At this time, the answer to the problem given last time is taken as the final answer.

[0007] After obtaining the single-hop sub-problem, first use the context learning ability of the large model to extract the top-k key entities of the sub-problem as the seed entity of the problem, and then use the seed entity as the starting point to retrieve the problem sub-graph specific to the current problem in the knowledge graph. Then, input the problem sub-graph and sub-problem into the graph neural network model (GNN) to obtain all possible answer entities of the problem, and sort the entities from high to low according to probability and select all entities with cumulative probability greater than 95% as candidate answers. Then use the BFS algorithm to obtain all paths between the seed entity and each candidate answer, and then text the path to form a displayed reasoning path and use it as the context for the large model reasoning. Finally, the answer to the sub-problem is obtained through the reasoning ability of the large model. After solving each single-hop sub-problem, the solution steps of the sub-problem will be added to the context to provide the large model with more problem information. The enriched context is then input into the large model again, so that the large model can iteratively decompose the problem. If the large model cannot continue to decompose the sub-problem, the last answer will be the final answer. If the sub-problem is decomposed, it will be solved according to the above steps. The overall architecture diagram of the model is shown below. Figure 5 At the same time, the question subgraph of each subsequent question will be merged with the previous question subgraph to achieve dynamic expansion of the question subgraph, which is conducive to answering subsequent questions. The dynamic expansion diagram is shown in Figure 4 shown.

[0008] A knowledge graph question answering method combining a large model with a graph neural network includes the following steps:

[0009] Step 1: Problem decomposition: Use the context learning ability of the large model to decompose the multi-hop problem. When receiving a multi-hop problem, the context example is spliced ​​with the multi-hop problem and then input into the large model. The large model generates an initial single-hop sub-problem, which is input into the sub-problem solving module to obtain the reasoning path, and then the reasoning path is input into the reasoning module to obtain the answer. The results of the above steps are added to the context to provide the large model with more problem information; then the enriched context is input into the large model again, allowing the large model to iteratively decompose the problem and solve it according to the above steps until the large model can no longer give a sub-problem. At this time, the answer to the problem given last time is used as the final answer.

[0010] Step 2: Sub-problem solving: including problem sub-graph retrieval module and reasoning path construction module.

[0011] Step 3: Answer reasoning: After obtaining all the paths between the seed entity and the candidate answers through the question subgraph retrieval module, they are textualized to construct the context of the LLM, and then the sub-questions are spliced ​​and input into the LLM together, using the reasoning ability of the large model to obtain the final answer to the sub-question.

[0012] The problem subgraph retrieval module in step 2 specifically includes:

[0013] When a single-hop sub-problem is decomposed from a multi-hop problem, the problem sub-graph of the sub-problem is retrieved in the knowledge graph;

[0014] The knowledge graph is composed of triplets (s, r, o). For a problem, it is crucial to select the triplets that are truly relevant to the problem. The definition of relevance is given from a probability perspective:

[0015]

[0016] Where Score(e,q,G) represents the correlation between triple e and question q in the knowledge graph G, N represents the total number of random walks in the knowledge graph starting from question q, times(s) represents the number of visits to triple s during the random walk, and subject to start = q represents the constraint condition of starting from q.

[0017] Since question q is difficult to represent directly in the knowledge graph, the top-k key entities in the question are extracted through a large model and then mapped to the corresponding entities in the knowledge graph to approximate the question. The relevance after approximation is defined as follows:

[0018]

[0019] The relevance can be calculated by using the Personalized PageRank algorithm, and finally the 2000 triples most relevant to the question are screened out as the question subgraph of the current question.

[0020] The reasoning path construction module in step 2 specifically includes:

[0021] The graph neural network part uses the pre-trained ReaRev graph neural network. The sub-questions and question sub-graphs are input into the ReaRev model to obtain the probability of each entity as the answer. The probabilities are then sorted from high to low, and entities with a cumulative probability greater than 95% are selected from high to low as candidate answers. The BFS algorithm is used to retrieve all paths between the seed entity and each candidate answer.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] To alleviate the problem that the RAG method is prone to falling into local optimal solutions, the present invention adopts the GNN method in the question subgraph retrieval stage. At the same time, in order to alleviate the dilemma of the GNN method in determining the size of the question subgraph in multi-hop problems, the present invention proposes an iterative problem decomposition method, and the difference in the size of the problem subgraphs between the decomposed single-hop problems is small. Finally, in order to improve the accuracy of GNN answers, the question subgraphs of subsequent questions will be merged with the question subgraphs of previous questions to achieve dynamic expansion of the question subgraphs. The present invention provides a knowledge graph question-answering framework that combines a large model and a graph neural network, which effectively improves the performance of the model on multi-hop problems.

[0024] The present invention first decomposes the multi-hop problem through the context learning ability of the large model to obtain an initial single-hop sub-problem. After that, the sub-problem is solved and the answer is inferred to finally obtain the answer to the sub-problem. Then, the solution steps of the sub-problem are added to the context to provide the large model with more problem information. At the same time, the enriched context is input into the large model again for iterative problem decomposition. When the large model cannot give the sub-problem, the answer to the last sub-problem is taken as the final answer. In this way, the question subgraph can dynamically approach the final answer entity. The final model was tested on the knowledge graph question-answering dataset webqsp. Compared with the traditional graph neural network method ReaRev, the present invention improves the hit@1 index by 7 points and the F1 index by 0.8 points. Compared with the retrieval enhanced generation (RAG) method ROG, the hit index is improved by 1.5 points, the hit@1 index is improved by 3.3 points, and the F1 index is improved by 0.9 points. See the complete result graph for details. Figure 8 The present invention has a wide range of practical application value and can be applied in fields such as knowledge graphs, which helps to better solve practical problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A schematic diagram of a RAG method of the present invention that combines a large model with a knowledge graph question answering method of a graph neural network;

[0027] Figure 2 A schematic diagram of a GNN method of a knowledge graph question answering method combining a large model with a graph neural network according to the present invention;

[0028] Figure 3This is a context example diagram of question decomposition of a knowledge graph question answering method combining a large model and a graph neural network in the present invention;

[0029] Figure 4 A schematic diagram showing the difference between a knowledge graph question answering method combining a large model with a graph neural network and a GNN method in the present invention;

[0030] Figure 5 This is an architecture diagram of a knowledge graph question-answering method that combines a large model with a graph neural network in the present invention;

[0031] Figure 6 This is an example diagram of a large model prompt template of a knowledge graph question-answering method combining a large model with a graph neural network in the present invention;

[0032] Figure 7 This is an example diagram of an answer reasoning prompt for a knowledge graph question-answering method combining a large model with a graph neural network in the present invention;

[0033] Figure 8 This is a graph showing the experimental results of a knowledge graph question answering method combining a large model with a graph neural network in the present invention;

[0034] Fig. 9 This is an example diagram of a prompt template for key entity extraction of a knowledge graph question-answering method that combines a large model with a graph neural network in the present invention. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] Reference Figure 5 , a knowledge graph question answering method combining a large model with a graph neural network includes:

[0037] Step 1: Problem decomposition stage

[0038] In the problem decomposition stage, the present invention uses the context learning ability of the large model to decompose the multi-hop problem, wherein the context example is shown in the attached Figure 3 In this example, two learning examples Q1 and Q2 are given. Each learning example is composed of several sub-questions Subque, reasoning paths Ra, and answers Ans. Therefore, the final context example prompt is composed as follows:

[0039]

[0040] The ⊕ represents text concatenation. The following is an explanation with specific examples.

[0041] If there is a multi-hop question multi_ques called "What language is spoken in Bolt's country?", then the input to the big model is prompt⊕multi_ques, and the big model gives a subquestion called "What country is Usain Bolt from?". The subquestion is calculated as follows:

[0042] subquestion=G(prompt⊕multi_ques)

[0043] Here G(X) means inputting X as content into the large model G for answering.

[0044] If the first subquestion has been solved at this time, we can obtain the reasoning path Ra and the answer Ans of the first subquestion, splice it into the context and then input the enriched context into the large model again to get the second subquestion. The new subquestion is calculated as follows:

[0045] subquestion=G(prompt⊕multi_ques⊕Ra⊕Ans)

[0046] If the current subquestion is "Finished!", it means that the question answering is completed, and the answer to the last question is taken as the final answer.

[0047] Step 2: Sub-problem solving stage

[0048] After obtaining the sub-problem through step (1), it is necessary to construct the reasoning path of the sub-problem through the GNN model, which specifically includes two steps: the problem sub-graph retrieval stage and the reasoning path construction stage.

[0049] Step 2.1: Question subgraph retrieval phase

[0050] Since knowledge graphs usually contain millions of facts and nodes, it is not practical to put all nodes into a graph neural network at the same time. Therefore, a smaller problem-specific subgraph G can be retrieved for question q. q , ideally, all correct answers to the question are contained in the retrieved subgraph, i.e. {a i}∈G q . Then the retrieved question subgraph G q Together with the question q, it is used as the input of the GNN model, and the GNN model outputs the correct answer.

[0051] Specifically, when a multi-hop problem is decomposed into multiple single-hop sub-problems, we first use the context learning ability of the large model to extract the top-k key entities in the sub-problems as the seed entities of the current problem. The specific context prompt is shown in Fig. 9 The prompt consists of two parts: the initial task introduction and the subsequent three examples. After extracting the seed entity seed_ent, first use the seed entity seed_ent as the starting point to retrieve all triples (s, r, o) in the knowledge graph whose head entity s or tail entity o is the seed entity, denoted as cand_ent. Then, the Personalized PageRank algorithm is executed on the searched triples to filter out the 2000 triples most relevant to the seed entity in the question as the question subgraph of the subquestion.

[0052] The specific steps of the Personalized PageRank algorithm are as follows:

[0053] Start random walk from the seed entity seed_ent node. Stop walk with probability 1-d (where d is 0.2) at each node and restart from the seed entity seed_ent node, or continue walk with probability d. When choosing to continue walk, randomly select a node from the nodes pointed by the current node according to uniform distribution and walk downstream. After many rounds of walks, the probability of each vertex being visited will tend to be stable. This probability is used as the relevance of the current triple and the current sub-problem. Finally, the top 2000 triples are selected as the problem subgraph of the current sub-problem. The probability calculation formula for each node is as follows:

[0054]

[0055] Where PPR(i) represents the probability of the i-th entity, and out(i) represents the out-degree of the i-th entity.

[0056] Step 2.2: Reasoning Path Building Module

[0057] The GNN model in knowledge graph question answering can be viewed as a node classification problem, where KG entities are classified as answers or non-answers to a given question. GNN aggregates information from each neighbor node V' To update the representation of node v at layer l The specific GNN update expression is as follows:

[0058]

[0059] where the function ω(·) measures how relevant the relation r of the fact (v, r, v') is to the question q. Ψ(·) is used to combine the representations between two consecutive layers.

[0060] The present invention uses the pre-trained ReaRev graph neural network, the input of which is the sub-problem q and the problem sub-graph G. q . When subproblem q is combined with problem subgraph G q Before inputting into the GNN model, the sub-problem q needs to be embedded first. The present invention uses the sbert model to embed the input. Assuming the sub-problem is q, the output after embedding is E q .

[0061] E q With the problem subgraph G q They are input into the GNN model together to get the entity score ans_score. ans_score represents the probability of each entity being the answer. The entity scores ans_score are sorted from large to small, and then those entities with cumulative probabilities reaching 95% are selected from large to small as the candidate answers cand_ans given by the GNN model, for example Figure 5 The answers marked in yellow are Jamaica; America; English.

[0062] After we have the seed entity seed_ent and the candidate entity cand_ans, we use the BFS algorithm to retrieve all the paths between each seed entity and each candidate answer in the question subgraph. Figure 5 There are 3 paths starting from "Usain bolt" and ending at "Jamaica", "America", and "English", and the result is [['born_in'],['enter_competition','USAC','Host_place'],['spoken_language']].

[0063] Step 3: Answer Reasoning Module

[0064] In order to facilitate the understanding of the large model, the retrieved path is textualized, and the textualized result is "Usain

[0065] bolt->born_in->Jamaica", "Usain

[0066] bolt->enter_competition->USAC->Host_place->America” and “Usainbolt->spoken_language->English”, and this result is called the reasoning path.

[0067] Since the reasoning path retrieves all paths from all starting entities to the end entities, there may be cases where the end entity itself is not the correct answer and where the end entity is the correct answer but the reasoning path is wrong. Therefore, further screening of the reasoning path is required.

[0068] In order to alleviate the above-mentioned reasoning path error problem, the present invention adopts the contextual reasoning of the large model in the final stage, relying on the reasoning ability of the large model to determine whether the answer and the reasoning path are reasonable.

[0069] Divide the prompt into 3 parts, as shown in the attached Figure 6 As shown. They are respectively the reasoning path Prompt reason , Task Instruction Prompt instruct With Question Prompt question , where Prompt reason With Prompt question It will be modified dynamically according to the different questions, and Prompt instruct The three parts of the prompt are finally concatenated to serve as the GPT context prompt. ICL .

[0070] Prompt ICL =prompt reason +Prompt instruct +Prompt question

[0071] A complete example is attached Figure 7 shown.

[0072] Finally, Prompt ICL Provide it to the big model and use the reasoning ability of the big model to determine the final answer.

[0073] The final model experimental results are shown in Figure 8 In this experiment, k in top-k is 1. The hit indicator is used to measure whether there is any correct answer in the generated answer; the hit@1 indicator is used to measure whether the answer with the highest probability in the generated answer is the correct answer. The F1 indicator comprehensively considers the recall rate and precision of the generated answer.

[0074] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations of these embodiments are made without departing from the principles and spirit of the present invention, and still fall within the scope of protection of the present invention.

Claims

1. A knowledge graph question answering method combining a large model and a graph neural network, characterized by: The following steps are involved: Step 1: Problem decomposition: Use the context learning ability of the large model to decompose the multi-hop problem. When receiving a multi-hop problem, the context example is spliced ​​with the multi-hop problem and then input into the large model. The large model generates an initial single-hop sub-problem, which is input into the sub-problem solving module to obtain the reasoning path, and then the reasoning path is input into the reasoning module to obtain the answer. The results of the above steps are added to the context to provide more problem information to the large model. The enriched context is then input into the large model again, allowing the large model to iteratively decompose the problem and solve it according to the above steps until the large model can no longer give a sub-problem. Step 2: Sub-problem solving: including problem sub-graph retrieval module and reasoning path construction module. Step 3: Answer reasoning: After obtaining all the paths between the seed entity and the candidate answers through the question subgraph retrieval module, they are textualized to construct the context of the LLM, and then the sub-questions are spliced ​​and input into the LLM together, using the reasoning ability of the large model to obtain the final answer to the sub-question.

2. According to claim 1, a knowledge graph question answering method combining a large model and a graph neural network is characterized in that: The problem subgraph retrieval module in step 2 specifically includes: After obtaining the single-hop sub-problem decomposed from the multi-hop problem, the problem subgraph of the single-hop sub-problem is retrieved in the knowledge graph; The knowledge graph consists of triplets, and the definition of relevance is given from a probability perspective: Where Score(e,q,G) represents the correlation between triple e and question q in the knowledge graph G, N represents the total number of random walks in the knowledge graph starting from question q, times(s) represents the number of visits to triple s during the random walk, and subject to start = q represents the constraint condition of starting from q. Question q is difficult to represent directly in the knowledge graph. The top-k key entities in the question are extracted through a large model as the seed entities of the current question to approximate question q. The relevance after approximation is defined as follows: Finally, the relevance can be calculated by using the PersonalizedPageRank algorithm, and the 2000 triples most relevant to the question are finally screened out as the question subgraph of the current question.

3. According to claim 1, a knowledge graph question answering method combining a large model and a graph neural network is characterized in that: The reasoning path construction module in step 2 specifically includes: The GNN model in knowledge graph question answering is regarded as a node classification problem, in which KG entities are classified as answers or non-answers to a given question. The candidate answers to the sub-questions are first obtained through the graph neural network, where the graph neural network part uses the pre-trained ReaRev graph neural network. The sub-questions and question subgraphs are input into the ReaRev model together to obtain the probability of each entity being an answer. The probabilities are then sorted from high to low, and entities with a cumulative probability greater than 95% are selected from high to low as candidate answers. The BFS algorithm is used to retrieve all paths between the seed entity and each candidate answer.

Images