Large language model reasoning question and answer enhancement method for multi-hop question

Abstract

The invention discloses a multi-hop question-oriented big language model reasoning question and answer enhancement method, and relates to the technical field of reasoning enhancement of big language models. The method aims at solving the problem that an existing large language model reasoning question-answering method is low in reasoning accuracy. The method comprises the steps that questions are constructed into a question tree, and post-order traversal is conducted on the question tree; for the leaf nodes, obtaining candidate answers of corresponding sub-questions of the leaf nodes in the question tree and probability distribution of the candidate answers; and for an intermediate node in the question tree, carrying out beam combination on the sub-questions corresponding to the intermediate node, replacing placeholders in the questions with answers of the corresponding questions, and obtaining a candidate answer set and a distribution probability of the replaced questions. And carrying out beam combination on the sub-questions corresponding to the root nodes, replacing placeholders in the questions with answers of the corresponding questions, then outputting candidate answers and probability distribution of the replaced questions, and selecting the answer with the highest corresponding probability as a final answer. The method is used for obtaining the answer of the multi-hop question through reasoning.

Description

Technical Field

[0001] The present invention relates to the technical field of large language model reasoning enhancement, and in particular to a large language model reasoning question-answering enhancement method for multi-hop problems. Background Art

[0002] Since the release of chatGPT by openAI in November 2022, large language models have quickly attracted widespread attention from the global capital market. In the next less than two years, various technology giants have followed up and proposed their own models. In this global "hundred-model war", the model's problem-solving ability on complex problems is one of the most important indicators for judging the comprehensive ability of the model. Whether it is the GPT series model of OpenAI abroad, or Wenxin Yiyan of Baidu in China and Kimi of the Dark Side of the Moon, without exception, they refresh the model score data on various public reasoning evaluation data sets to prove whether their models are "smarter" than their competitors' models. Obviously, even the most advanced GPT series models still face some challenges when dealing with knowledge-intensive multi-hop problems. These problems not only limit the reasoning ability of the model, but also affect its reliability and effectiveness in practical applications.

[0003] For knowledge-intensive and complex problems that require multi-hop reasoning, there are two main directions to improve the reasoning ability of large language models. One direction is to improve the strategy for prompt words, and the other direction is to perform retrieval enhancement on the model. The most famous improvement strategy for prompt words is the chain of thought (CoT) prompt proposed by Wei et al. in 2022, which prompts LLM to generate reasoning processes before the final answer, which can significantly improve the model's reasoning ability and improve the accuracy of the answer. There are also some works that help the model reason without direct training by designing specific instructions or clustering demonstrations, that is, to perform effective task execution in the scenario of zero-shot learning. In addition, it has been proven through extensive experiments that self-integration strategies are also an effective way to improve the reasoning performance of large models. Representative methods include the use of probabilistic sampling methods for multiple reasoning proposed by Wang et al. in 2023, and the diversification of reasoning paths through multilingual chains of thought by Qin et al. in 2023. For solving complex problems, representative strategies include the idea of ​​decomposing the problem and solving it step by step proposed by Zhou et al. in 2023, and the strategy of modeling the reasoning process as BFS or DFS search on the reasoning tree proposed by Yao et al. in the same year.

[0004] At present, the most advanced large language model services, in addition to the setting of prompt word strategies, also use retrieval-enhanced reasoning methods when facing complex problems. Especially when the model faces knowledge-sensitive tasks, it is likely to have the problem of factual hallucination, and the retrieval-enhanced reasoning method can greatly alleviate this phenomenon. Most of the early works adopted single-shot retrieval schemes, but they found it difficult to collect all the necessary knowledge to answer complex questions, resulting in knowledge omissions. To solve this problem, iterative retrieval methods were further proposed. In 2022, Khattab et al. proposed the DSP scheme, which iteratively interacts between the retriever and the reader through a program-defined process. In 2023, Press et al. proposed the SelfAsk scheme, which iteratively decomposes problems and solves them through Google search. Also in 2023, Trivedi et al. proposed the IRCoT scheme, which uses each reasoning step as a query for retrieval until the final answer is obtained. Similarly, in 2023, Shao et al. proposed the ITER-RETGEN scheme, which iteratively retrieves by connecting the output of the previous round with the original question. In 2023, Jiang et al. proposed the FLARE scheme, which introduced a look-ahead mechanism to dynamically control the timing of retrieval according to the reasoning confidence. In 2023, Zhang et al. proposed the Beam Retrieval scheme, which introduced an end-to-end framework that aims to retrieve relevant paragraphs in each hop of each question through beam search. At the same time, some works achieve more accurate retrieval by decomposing the problem into QDMR format or decomposing the problem into a tree structure, while in 2023, Park et al. used the construction of a reasoning graph as a solution.

[0005] However, the above strategies still have significant problems. Iterative retrieval is difficult to achieve retrieval that is precisely aligned with model reasoning, while sub-question retrieval is difficult to accurately aggregate answers, which may lead to cascading errors, that is, an error in one sub-question will lead to a final error in the entire question. In addition, in an open domain setting, existing technologies only rely on knowledge from a single source for complex problems, which makes the large language model lack and incomplete knowledge in certain fields, resulting in insufficient basis and inaccuracy in the final reasoning answer. However, the introduction of multi-source knowledge may encounter knowledge conflicts, making effective collaboration difficult. In the process of reasoning, contradictory external auxiliary knowledge will seriously affect the reasoning judgment of the model, resulting in instability or even complete errors in the final result output. Therefore, the existing large language model reasoning question-answering methods still have the problem of low reasoning accuracy for multi-hop questions. Summary of the invention

[0006] The purpose of the present invention is to solve the problem of low reasoning accuracy in the existing large language model reasoning question answering method, and proposes a large language model reasoning question answering enhancement method for multi-hop problems.

[0007] The large language model reasoning question answering enhancement method for multi-hop questions is as follows:

[0008] Step 1: Construct the multi-hop problem Q into a problem tree;

[0009] Step 2: Initialize the node number i=1 in the problem tree and perform post-order traversal on the nodes of the problem tree;

[0010] Step 3: Determine node Q (i) Whether it is a leaf node;

[0011] If Q (i) If it is an intermediate node, execute step 4;

[0012] If Q (i) If it is the root node, execute step 4;

[0013] If Q (i) If it is a leaf node, directly execute step 5;

[0014] Step 4: Q (i) Perform beam combination on the corresponding sub-problems and convert Q (i) Replace the placeholders in the corresponding sub-questions with the corresponding candidate answers, and then let Q (i) Corresponding sub-problem q (i) = placeholders are replaced with the corresponding candidate answers, and q (i) Store in set Q (i) ', then execute step five;

[0015] Among them, Q (i) ′ is Q (i) The set Q corresponding to the specific problem (i) ;

[0016] Step 5: Use the LLM model to analyze Q from different knowledge sources (i) The specific questions in ′ are respectively subjected to complementary multi-source reasoning to obtain the answer set

[0017] If Q (i) is an intermediate node, for Q (i) The answer set of the corresponding sub-questions After probability aggregation, select the k answers with the highest probability and the corresponding probability as Q (i) Candidate answers to the corresponding sub-questions and the probability distribution of candidate answers p (i) , then set i=i+1 and return to step 3;

[0018] If Q (i) is a leaf node, for Q (i) The answer set of the corresponding sub-questions After the selection probability aggregation, the answer with the highest probability and the corresponding probability are taken as Q (i) Candidate answers to the corresponding sub-questions and the probability distribution of candidate answers p (i) , let i=i+1, and return to step 3;

[0019] If Q (i) is the root node, for Q (i) The answer set of the corresponding sub-questions After probability aggregation, select the k answers with the highest probability and the corresponding probability as Q (i) Candidate answers to the corresponding sub-questions and the probability distribution of candidate answers p (i) , then execute step 6;

[0020] Step 6: Output the candidate answer with the highest probability as the answer to Q, and end.

[0021] Furthermore, in step 1, the multi-hop problem Q is constructed as a problem tree, specifically:

[0022] First, the multi-hop question Q and the prompt words for question decomposition are input into the large language model for question decomposition to obtain N sub-questions;

[0023] The N sub-problems obtained by decomposition include sub-problems corresponding to leaf nodes, sub-problems corresponding to intermediate nodes, and sub-problems corresponding to root nodes;

[0024] Then, a problem tree is constructed based on the N sub-problems to obtain a problem tree sequence.

[0025] Further, the nodes in the problem tree are numbered in the order of post-order traversal;

[0026] There is a placeholder in the parent node to replace the answer to the corresponding question of the child node;

[0027] The placeholder is #i, where i represents the child node number.

[0028] Furthermore, the problem tree is constructed based on the N sub-problems to obtain a problem tree sequence, specifically:

[0029] Q decomp = {Q (1) ,Q (2) ,Q (i) ,...,Q (N)}

[0030] Q (i) = {q (i) ,a (i) ,p (i)}

[0031] Among them, q (i) is the subproblem corresponding to the i-th node, a (i) ∈R k is the set of candidate answers to the sub-question corresponding to the i-th node, p (i) ∈R k is the probability set of candidate answers for the sub-question corresponding to the i-th node, k is the preset number of candidate answers, i is the number of nodes in the question tree, i∈[1,N], Q (i) is the i-th node, Q (N) is the Nth node and Q (N) is the root node.

[0032] Furthermore, the step 4 of (i) Perform beam combination on the corresponding sub-problems and convert Q (i) Replace the placeholders in the corresponding sub-questions with the corresponding candidate answers, and then let Q (i) Corresponding sub-problem q (i) = placeholders are replaced with the corresponding candidate answers, and q (i) Store in set Q (i) ' and execute step 5, specifically:

[0033] If Q (i) Corresponding sub-problem q (i) If the number of placeholders in is 1, then q (i) The placeholders in the loop are replaced with the k candidate answers with the highest probability for the sub-problems corresponding to the placeholders, and a total of k replaced sub-problems are obtained. (i) = Sub-questions after the placeholders are replaced with the corresponding candidate answers, and the k replaced sub-questions are stored in the set Q (i) ' and execute step 5;

[0034] If Q (i) Corresponding sub-problem q (i) If the number of placeholders in is greater than 1, the Cartesian product of each candidate answer combination of the sub-questions corresponding to the placeholders is calculated, and the probability of the Cartesian product of each candidate answer combination is obtained. Finally, Q (i) Corresponding sub-problem q (i) The placeholders in are replaced with the corresponding candidate answer combinations, and then q is looped (i) = Sub-questions after the placeholders are replaced with the corresponding candidate answers, and all replaced sub-questions are stored in the set Q (i) ' and execute step five.

[0035] Furthermore, the probability of the Cartesian product of each candidate answer combination is obtained by the following formula:

[0036]

[0037] in, is the j'th candidate answer corresponding to question x1, is the j*th candidate answer to question xn, n is Q (i) The total number of placeholders in yes The corresponding probability is yes The corresponding probability, j, j'...j* is the number of the candidate answer.

[0038] Furthermore, the LLM model in step 5 is used to analyze Q from different knowledge sources. (i) The specific questions in ′ are respectively subjected to complementary multi-source reasoning to obtain the answer set Specifically:

[0039]

[0040] in, is the answer set, K s is the knowledge retrieved from different knowledge sources, s∈{implicit internal knowledge, explicit internal knowledge, Wikipedia knowledge, web search knowledge}, The k answers with the highest confidence for each specific sub-question reasoning are stored in .

[0041] Furthermore, the step 5 of (i) The answer set of the corresponding sub-questions After probability aggregation, select the k answers with the highest probability and the corresponding probability as Q (i) Candidate answers to the corresponding sub-questions and the probability distribution of candidate answers p (i) , specifically:

[0042] First, get The probability distribution of the answer in j ;

[0043] Then, according to p j choose The k most probable answers are Q (i) Candidate answers to the corresponding sub-questions At the same time, obtain the probability distribution p of the candidate answers (i) .

[0044] Furthermore, the acquisition The probability distribution of the answer in j , specifically:

[0045]

[0046] Among them, n,j∈[1,k], j,n are candidate answer labels, f j is the frequency of the jth candidate answer in set L, which is the set of k candidate answers with the highest confidence among all the answers to a specific question, g n is the frequency of the nth candidate answer in the set L, τ is the temperature, p j is the probability distribution of the jth candidate answer.

[0047] Further, according to p j choose The k most probable answers are Q (i) Candidate answers to the corresponding sub-questions At the same time, obtain the probability distribution p of the candidate answers (i) , specifically:

[0048] First, according to p j get The probability that one of the answers is y:

[0049]

[0050] Where y is One of the candidate answers, Q (i) ′ is Q (i) The specific question set obtained after the placeholders in the corresponding sub-questions are replaced with the corresponding candidate answers, P(m (i) ) is a specific question (i) The corresponding probability, P(y|x=m (i) ) is the specific problem m (i) The probability distribution of the answer being y when Q (i) The probability that the answer to the corresponding sub-question is y;

[0051] Then, yes Each candidate answer y in the equation is probability aggregated and sorted according to the probability of the answer from large to small. Arrange the answers in the formula and get the first k answers and their corresponding probabilities as Q (i) Candidate answers to the corresponding sub-questions and the corresponding probability p (i) .

[0052] The beneficial effects of the present invention are:

[0053] The present invention proposes a reasoning question-answering enhancement method for a large language model, Beam Aggregation Reasoning (BeamAggR), which is applied to the background system of a large language model dialogue service. The present invention assists the large language model in improving its ability to solve complex problems and improves the user experience. The present invention first decomposes a complex multi-hop problem into a tree structure, then performs bottom-up multi-source knowledge reasoning in a post-order traversal manner, and uses beam aggregation at the intermediate nodes until the root node is reached to obtain the final answer. In the entire bottom-up reasoning process, the present invention finely integrates diverse sources of knowledge, thereby reducing the model hallucination problem caused by insufficient knowledge. In addition, the beam aggregation of the present invention also merges answers in a probabilistic manner and gives priority to paths with higher probability, thereby reducing cascading errors, thereby improving the reasoning accuracy of the large language model reasoning question-answering. In addition, the present invention uses the complementarity of multi-source reasoning to reduce knowledge omissions and conflicts. The experimental results of the embodiment show that the method of the present invention is significantly better than the benchmark method on four data sets, with an average improvement rate of 8.5%, which is significantly improved compared with the existing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is the processing flow chart of BeamAggR;

[0055] Figure 2 Some specific prompt words are set for the problem decomposition module;

[0056] Figure 3 A schematic diagram is provided for some specific prompt words when using implicit internal knowledge for closed-book reasoning;

[0057] Figure 4 A schematic diagram of some specific prompt words for inducing large language models to generate parameter knowledge;

[0058] Figure 5 A schematic diagram of some specific cue words used when using explicit internal knowledge for reasoning;

[0059] Figure 6 A schematic diagram for setting up some specific prompt words when using Wikipedia for external knowledge reasoning;

[0060] Figure 7 A schematic diagram for setting up some specific prompt words when using a search engine to perform external knowledge reasoning;

[0061] Figure 8 Schematic diagram of the F1 score on each subset of the three methods of the present invention, FLARE, and ProTree to segment the original data of MuSiQue according to the number of inference hops. DETAILED DESCRIPTION

[0062] The present invention focuses on the following problem: During the reasoning process, how can the model adaptively select and integrate knowledge from different sources while reducing cascading errors in sub-problem aggregation? Based on this motivation, the present invention proposes a knowledge-intensive multi-hop reasoning method based on the BeamAggregation Reasoning (BeamAggR) framework. Specifically, the BeamAggR framework includes three steps: (i) Problem decomposition: complex problems are decomposed using LLMs and converted into a problem tree. The root node contains the original complex problem, the leaf nodes contain atomic problems, and the intermediate nodes contain compound problems that require combined (or comparative) reasoning to obtain answers. Then, the present invention traverses the problem tree in post-order and adopts a bottom-up reasoning method, which ensures that the LLM solves the atomic problem first and then gradually traverses more complex compound problems along the problem tree. (ii) Complementary multi-source reasoning: For atomic problems, the BeamAggR framework performs multi-source reasoning and then performs fine-grained answer aggregation to promote knowledge collaboration. The aggregated answers are normalized to probability distributions as candidate answers for beam aggregation. (iii) Beam aggregation: For a complex problem, the present invention lists the combination of its dependent sub-problems and performs reasoning. Finally, the reasoning results are probabilistically aggregated to select the most promising prediction results.

[0063] In order to improve the ability of large language models to solve complex problems, following the design and definition of complex problems in existing popular public datasets, this paper focuses on multi-hop question answering tasks in open domains. The input of the task is a multi-hop question Q that requires multiple steps of reasoning to solve, and each step of reasoning requires specific knowledge. Given Q as the input query, the retriever R retrieves relevant documents The retrieved document D is then input as context into the model for knowledge reasoning, and the answer y=LLM(Q,D) is obtained through model reasoning. The present invention uses large language models (LLMs) with a small number of hints and capable of external retrieval to solve these tasks. The present invention is described below in conjunction with specific embodiments.

[0064] Specific implementation method 1: Figure 1 As shown, the specific process of the large language model reasoning question answering enhancement method for multi-hop problems in this embodiment is as follows:

[0065] Step 1: Construct the complex multi-hop problem Q into a problem tree:

[0066] First, the multi-hop question Q and the corresponding prompt words for question decomposition are input into the large language model for question decomposition to obtain N = 4 sub-questions;

[0067] Multi-hop problems are those that require at least two steps of reasoning to obtain an answer.

[0068] The N sub-problems obtained by decomposition include sub-problems corresponding to leaf nodes, sub-problems corresponding to intermediate nodes, and sub-problems corresponding to root nodes;

[0069] The nodes in the question tree are numbered in the order of post-order traversal, and #i is used as a placeholder for the intermediate question to replace the answer to the sub-question numbered i in other questions; i is a natural number;

[0070] Then, a problem tree is constructed based on N sub-problems, and the post-order traversal sequence of the problem tree is as follows:

[0071] Q decomp = {Q (1) ,Q (2) ,Q (i) ,...,Q (N)}

[0072] Q (i) = {q (i) ,a (i) ,p (i)}

[0073] Where i is the node number in the problem tree, i∈[1,N], Q (i) is the i-th node, q (i) is the subproblem corresponding to the i-th node, a (i) ∈R k is the set of candidate answers for the sub-question corresponding to the i-th node, k is the number of preset candidate answers, and p (i) ∈R k is the probability set of candidate answers to the sub-question corresponding to the i-th node, Q (N) It is the Nth node and the root node.

[0074] In this step, the prompt word setting for problem decomposition is 24-shot; the prompt word for problem decomposition is as follows: Figure 2 As shown;

[0075] Since the multi-hop problem involves complex structures, such as bridging, comparison, combination and their integration, in order to solve this problem, the present invention parses the complex problem into a tree structure to express the reasoning structure, such as Figure 1 As shown in the figure, the complex problem Q is decomposed into a tree structure with four nodes containing four simpler sub-problems, each node is denoted by Q (1) To Q (4), there is a (combinatorial) dependency relationship between them, and the decomposition process is carried out by prompting the large language model. The root node represents the original complex problem, the leaf node represents the atomic sub-problem, and the intermediate nodes require combinatorial (or comparative) reasoning to obtain the answer. The present invention uses #i as a placeholder for the intermediate question, which can be replaced with the solved answer to the i-th sub-problem when traversing to the node. After decomposition, the present invention represents the sub-problem in the QDMR format proposed by Wolfson et al. in 2020. Complex problems are then processed step by step in a bottom-up post-order traversal manner. The tree structure is represented as a post-order traversal node sequence, each node containing a question, a candidate answer, and an associated probability.

[0076] Step 2: Initialize the node number i=1 of the problem tree and perform post-order traversal on the nodes of the problem tree;

[0077] Step 3: Determine node Q (i) Whether it is a leaf node;

[0078] If Q (i) If it is an intermediate node, execute step 4;

[0079] If Q (i) If it is the root node, execute step 4;

[0080] If Q (i) If it is a leaf node, directly execute step 5;

[0081] Step 4: Q (i) Perform beam combination on the corresponding sub-problems and convert Q (i) Replace the placeholders in the corresponding sub-questions with the corresponding candidate answers, and then let Q (i) Corresponding sub-problem q (i) = placeholders are replaced with the corresponding candidate answers, and q (i) Store in set Q (i) ', and execute step five:

[0082] If Q (i) Corresponding sub-problem q (i) If the number of placeholders in is 1, then q (i) The placeholders in the question are replaced with the k candidate answers with the highest probability for the sub-questions corresponding to the placeholders, and a total of k replaced sub-questions are obtained. (i) = Sub-questions after the placeholders are replaced with the corresponding candidate answers, and the k replaced sub-questions are stored in the set Q (i) ' and execute step 5;

[0083] Among them, Q (i) ′ is Q (i)A specific set of questions obtained after the placeholders in the corresponding sub-questions are replaced with the corresponding candidate answers;

[0084] If Q (i) Corresponding sub-problem q (i) If the number of placeholders in is greater than 1, the Cartesian product of each candidate answer combination of the sub-questions corresponding to the placeholders is calculated, and the probability of the Cartesian product of each candidate answer combination is obtained. Finally, Q (i) Corresponding sub-problem q (i) The placeholders in are replaced with the corresponding candidate answer combinations, and then q is looped (i) = Sub-questions after the placeholders are replaced with the corresponding candidate answers, and all replaced sub-questions are stored in the set Q (i) ' and execute step 5;

[0085] The probability of the Cartesian product of each candidate answer combination is:

[0086]

[0087] in, is the j'th candidate answer corresponding to question x1, is the j*th candidate answer to question xn, n is Q (i) The total number of placeholders in yes The corresponding probability is yes The corresponding probability;

[0088] In the process of beam aggregation, the present invention infers the specific problem obtained by replacing the candidate answer combination inherited from the sub-problem to expand the breadth of reasoning and the exploration space, and finally selects the reasoning path by maximizing the predicted marginal probability distribution. For each intermediate node, further reasoning is required based on the answer derived from the previous sub-problem, and each sub-problem is associated with a set of possible answers and corresponding probabilities, called candidates (or beams). The intermediate problem may depend on multiple sub-problems, so the present invention enumerates the combinations in the candidate items. However, in order to prevent combinatorial explosion, the present invention uses beam size (controls the number of candidates) to limit the exploration space. Afterwards, for each combination, the placeholder in the corresponding question of the intermediate node is replaced by the candidate answer, and the complementary multi-source reasoning method is used again for multi-source reasoning, that is, step four. Specifically, when calculating the probability of the corresponding question of the intermediate node, the Cartesian product of the candidate answers of the corresponding sub-problems of its son nodes is calculated.

[0089] Step 5: Use the LLM model to analyze Q from different knowledge sources (i) The specific problems in ′ are respectively subjected to complementary multi-source reasoning to obtain Q (i) The answer set of the corresponding sub-questions

[0090] If Q (i) is an intermediate node, for Q (i) The answer set of the corresponding sub-questions After probability aggregation, select the k answers with the highest probability and the corresponding probability as Q (i) Candidate answers to the corresponding sub-questions and the probability distribution of candidate answers p (i) , then set i=i+1 and return to step 3;

[0091] If Q (i) is a leaf node, for Q (i) The answer set of the corresponding sub-questions After the selection probability aggregation, the answer with the highest probability and the corresponding probability are taken as and p (i) , let i=i+1, and return to step 3;

[0092] If Q (i) is the root node, for Q (i) The answer set of the corresponding sub-questions After probability aggregation, select the k answers with the highest probability and the corresponding probabilities as and p (i) , then execute step 6;

[0093] Step 5: Q (i) The answer set of the corresponding sub-questions After probability aggregation, select the k answers with the highest probability and the corresponding probability as Q (i) Candidate answers to the corresponding sub-questions and the probability distribution of candidate answers p (i) , specifically:

[0094] Step 51: Use the LLM model to analyze node Q from different knowledge sources (i) Corresponding to Q (i) ′The specific questions in the set are respectively subjected to complementary multi-source reasoning to obtain a set of candidate answers

[0095]

[0096] in, It's Q (i) The answer set obtained after reasoning for each specific sub-question in ′, where the k answers with the highest confidence for each specific sub-question are added to middle.

[0097] In this step, implicit internal knowledge and explicit internal knowledge both belong to internal knowledge reasoning, and Wikipedia knowledge and web search knowledge both belong to external knowledge reasoning. The following will explain in detail from the following two categories.

[0098] Internal knowledge reasoning: For implicit internal knowledge reasoning, the present invention prompts the model to perform closed-book reasoning through chain-of-thought demonstrations. As for explicit internal knowledge, some studies have shown that the model's parameter knowledge can be used as a retrieval source. The so-called "model parameter knowledge" refers to the knowledge related to a specific task learned by the model during the training process. This knowledge is encoded in the model's parameters. Therefore, for explicit internal knowledge reasoning, the present invention first prompts the model to generate parameter knowledge related to the problem, and then uses the generated parameter knowledge to perform explicit logical reasoning.

[0099] For implicit internal knowledge, the present invention uses 20-shot closed-book reasoning, and some specific prompt words are set as follows: Figure 3 As for explicit internal knowledge, the present invention first directly uses prompt words to let the large language model (LLM) generate parameter knowledge. Some specific prompt words are set as follows: Figure 4 As shown, 5-shot is then used for knowledge reasoning. Some specific prompt words are set as follows Figure 5 shown.

[0100] External knowledge reasoning: The present invention uses Wikipedia and search engines as external retrieval for external knowledge reasoning. For Wikipedia, the present invention uses the BM25 algorithm to perform sparse retrieval in the complete Wikipedia data set. For search engines, the present invention calls the Google search API and uses natural search results as retrieval content (that is, it does not contain items marked as advertisements). After the retrieval, the present invention uses a few-shot prompt to reason about the retrieved documents.

[0101] For external knowledge reasoning using Wikipedia and search engines, the present invention uses 3-shot and 5-shot prompts respectively, and some specific prompt words are set as follows: Figure 6 and Figure 7 shown.

[0102] Step 52: Get The probability distribution of the answer in is obtained by the following formula:

[0103]

[0104] Among them, n,j∈[1,k], j,n are candidate answer labels, f jis the frequency of the jth candidate answer in set L, which is the set of k candidate answers with the highest confidence among all the answers to a specific question, g n is the frequency of the nth candidate answer in the set L, τ is the temperature, p j is the probability distribution of the jth candidate answer;

[0105] In this step, in order to avoid the hallucination problem caused by lack of knowledge, the present invention uses four reasoning strategies combined with answer standardization to merge information from different knowledge sources. After completing four sets of independent knowledge reasoning, the present invention merges them by voting to achieve knowledge aggregation. Specifically, the answers obtained from each knowledge source are counted, and the frequency of occurrence of each answer is counted. Afterwards, the present invention converts the corresponding frequency of the answer into a probability distribution through a normalization method, so as to facilitate the subsequent beam search aggregation.

[0106] Step 53: According to p j choose The k most probable answers are Q (i) Candidate answers to the corresponding sub-questions At the same time, obtain the probability distribution p of the candidate answers (i) :

[0107] First, according to p j get The probability that one of the answers is y:

[0108]

[0109] Where y is One of the candidate answers, Q (i) ′ is Q (i) The specific question set obtained after the placeholders in the corresponding sub-questions are replaced with the corresponding candidate answers, P(m (i) ) is a specific question (i) The corresponding probability, P(y|x=m (i) ) is the specific problem m (i) The probability distribution of the answer being y when Q (i) The probability that the answer to the corresponding sub-question is y;

[0110] When the specific problem is m (i) The corresponding answer is y, and y is in When the sequence number in is j, then P(y|x=m (i) )=p j .

[0111] Then, yes Each candidate answer y in the equation is probability aggregated and sorted according to the probability of the answer from large to small. Arrange the answers in the formula and get the first k answers and their corresponding probabilities as Q (i) Candidate answers to the corresponding sub-questions and the corresponding probability p (i) .

[0112] In this step, the present invention has explored many sub-question combinations in the beam combination and obtained multiple sets of questions, answers and corresponding probabilities. This step aggregates the above answers according to probability to determine the optimal reasoning path. This can be formally expressed as argmaxp(y). After the probability answers are aggregated, the present invention retains the k answers y and their corresponding probabilities p(y) that correspond to the maximum final prediction probability as candidates. The aggregated candidates will continue to propagate from bottom to top and participate in subsequent beam aggregation. The answer y here is used as the answer to a sub-question in the beam aggregation process. And p(y) is the probability of the answer to the corresponding sub-question Propagate upward until reaching the root node. Once reaching the root node, consider the answer with the highest probability as the final answer.

[0113] Step 6: Output the candidate answer with the highest probability as the answer to Q, and end.

[0114] Embodiment: In order to verify the beneficial effects of the present invention, the present invention carried out the following experiments:

[0115] The present invention uses a classic complex question "When did the city where Souvenir's performer was born become the capitol of the state Knowles was from?" as an example to conduct experiments according to the method described in the specific implementation method, such as Figure 1 As shown. First, the original problem is divided into a problem tree with four nodes, where the sub-problem Q (1) and subproblem Q (3) is a leaf node, Q (4) is the root node, Q (2) is the middle node, and then the problem tree is traversed in post-order, and the traversal order is Q (1) , Q (2) , Q (3) , Q (4) ; First, for node Q (1) The corresponding sub-problems are subjected to multi-source knowledge reasoning to obtain Q (1) For the answer to the corresponding question, set the candidate item set k = 2, vote to obtain the candidate answers Paul and Martin, and then obtain the probability distribution corresponding to each candidate answer; traverse the next node Q(2) , Q (2) is an intermediate node, and Q (2) Q (1) Placeholder #1 of node Q, so beam combining is performed and placeholder #1 is replaced by node Q (1) The k candidate answers with the highest probability are selected, and k specific questions are obtained after replacement. Then, multi-source knowledge reasoning and candidate answer probability aggregation are performed respectively. (4) When it depends on Q (2) and Q (3) . Get Q (2) and Q (3) The Cartesian product of the candidate answers and obtain the probability value corresponding to its Cartesian product Perform substitution of the placeholder #i in the middle question to derive the new question. For example, by using Replace #1 and use Replace #2, derive The corresponding probability of such a new question ("When did Austin become the capital of Texas?") is Then, multi-source reasoning and candidate answer probability aggregation are performed on all replaced questions, and node Q is selected (4) The candidate answer with the highest probability among the candidate answers is taken as the final answer.

[0116] The formal description of the entire framework process is shown in Algorithm 1 below:

[0117]

[0118]

[0119] To ensure fairness and quantifiable comparison, this embodiment evaluates the BeamAggR framework (the present invention) on four popular large-scale open-domain multi-hop reasoning datasets. They are the HotpotQA evaluation dataset proposed by Yang et al. in 2018; the 2WikiMQA evaluation dataset proposed by Ho et al. in 2020; the Bamboogle evaluation dataset containing two-hop reasoning proposed by Press et al. in 2023, and the MuSiQue evaluation dataset containing 2 to 4 hop reasoning proposed by Trivedi et al. in 2022. When conducting experiments on the four datasets, the present invention conducts experiments in an open-domain setting and uses the entire Wikipedia data dump and web search for retrieval.

[0120] In the experimental comparison, the present invention used 7 other well-known large language model reasoning enhancement technologies as the control group, and used GPT-3.5-turbo as the basic model to conduct experiments on four evaluation data sets. The 7 reasoning enhancement technologies used as controls include:

[0121] Standard prompting strategy (SP): directly generate the final answer through a few-shot examples.

[0122] Chain of Thought Prompt (COT): This strategy encourages large language models to generate reasoning steps before the final answer, improving the model's reasoning ability for complex problems.

[0123] One-shot retrieval (OneR) uses the original question as the query, concatenating sparse retrieval and Google search results into prompts to guide the model to perform Chain of Thought (CoT) reasoning.

[0124] Self-Ask uses an iterative approach to decompose complex problems. It iteratively generates sub-questions based on existing reasoning, and then retrieves and answers these questions until the final answer is obtained.

[0125] Interleaved Retrieval-to-Inference (IRCoT) alternates retrieval-enhanced reasoning and reasoning-enhanced retrieval until the retrieved information is sufficient to answer the question.

[0126] FLARE) dynamically adjusts the retrieval timing according to the confidence of the inference and performs retrieval based on the upcoming inference sentence.

[0127] ProbTree parses the problem into a tree structure and uses a sub-problem aggregation method based on logarithmic probability to obtain the final answer.

[0128] Table 1 shows the experimental results of the present invention and seven other control groups on four popular multi-hop reasoning datasets. It can be observed that one-time retrieval (OneR) can improve the performance on HotpotQA, 2WikiMQA and Bamboogle. However, when facing the more challenging MuSiQue, one-time retrieval even hurts the performance (from 24.7 to 16.4), which indicates that inaccurate retrieval and knowledge conflict may aggravate the hallucination phenomenon.

[0129] Table 1 Experimental results on four open domain multi-hop reasoning datasets

[0130]

[0131] Compared with One-shot Retrieval (OneR), the iterative retrieval method achieves significant improvement by providing more comprehensive knowledge. In addition, the sub-question retrieval based method has a more explicit query, which leads to a further improvement in retrieval accuracy and becomes the best performing baseline method.

[0132] As shown in Table 1, our method surpasses the previous best methods, namely ProbTree: HotpotQA (+2.5), MuSiQue (+4.0), 2WikiMQA (+3.7), and Bamboogle (+8.2) on all four datasets. The improvements of our method can be attributed to three aspects: (i) Question decomposition leads to more accurate retrieval. Compared with iterative retrieval based on generated content, sub-questions can serve as clearer retrieval queries. (ii) Complementary reasoning promotes collaboration between different knowledge sources. Our method dynamically selects and utilizes knowledge from multiple sources at a fine-grained level, thereby reducing knowledge conflicts and omissions. (iii) Probability-based beam aggregation mitigates cascading errors and optimizes reasoning trajectories through consistency.

[0133] Significant improvements can be observed from the experimental results, especially for comparison problems. Comparison problems involve interactions between sub-problems. In the beam aggregation step of the present invention, the model can enumerate various combinations during reasoning, thereby covering a wider range of possibilities (the performance is improved from 65.9 to 74.2 and 81.7 to 89.9, respectively). In addition, the method of the present invention is also good at handling more complex problems (3-hop or 4-hop reasoning problems), such as Figure 8 As shown, Figure 8 The comparison effect diagram of the present invention with FLARE and ProbTree is given. As the reasoning depth increases, the method of the present invention shows a slower performance degradation, indicating that it can accurately aggregate answers and effectively reduce cascading errors.

[0134] The present invention replaces the implementation methods of some key steps to observe the performance changes, and the optimization contribution results of each key step are shown in Table 1.

[0135] Impact of multi-source reasoning: The present invention attempts to remove one or more knowledge sources, as shown in (a) in Table 2. Removing any knowledge source will result in a certain performance degradation, which shows that each type of knowledge contributes to reasoning (1-4 in (a) in Table 2). In addition, when the internal knowledge source or the external knowledge source is completely removed, the downward trend (4.1% reduction when removing internal knowledge and 9.7% reduction when removing external knowledge) shows that external knowledge makes a greater contribution than internal knowledge. Reasoning using only internal knowledge will result in a serious performance degradation (5 in (a) in Table 2). Despite this, the method of the present invention is still better than the classic mind chain reasoning method, which reflects the effectiveness of the aggregate reasoning method. Moreover, the experimental results show that reasoning using only external knowledge will also result in a significant performance degradation, as shown in 6 in (a) in Table 2. This shows that internal knowledge can supplement external retrieval, proving the effectiveness of the present invention in knowledge collaboration.

[0136] Effect of beam aggregation: In order to verify the effectiveness of the beam aggregation module, the present invention is compared with three replacement strategies. First, the present invention is compared with the coarse-grained polarization aggregation strategy. The polarization aggregation strategy refers to the aggregation strategy used in the ProbTree framework proposed by Cao et al. in 2023. Each intermediate question is queried twice, one is a direct query to the model, and the other is a query to an external knowledge source. When the answer is aggregated, the answer with a higher probability is directly selected. The results are shown in 1 in (b) of Table 2. This polarization aggregation strategy leads to a significant decline, which shows that coarse-grained polarization aggregation is difficult to effectively integrate knowledge, thereby emphasizing the superiority of the beam aggregation of the present invention. In order to study the importance of probabilistic aggregation, the present invention regards the aggregated answers as equal weights as another control strategy (w / o probability), and the results are shown in 2 in (b) of Table 2. On the two data sets, this control strategy caused the results to drop by 5% and 4%, respectively, which shows that prioritizing reasoning trajectories can stimulate better reasoning. Finally, the present invention studies the impact of candidate size settings. Larger candidate sizes can bring wider reasoning breadth, but they also increase reasoning overhead. When it is set to 1, a greedy aggregation strategy is adopted. This is a cost-effective variant of the beam aggregation inference of the present invention. The results of the greedy aggregation strategy as the control group are shown in 3 in (b) of Table 2, which leads to a slight performance degradation, indicating that with the help of a wider inference scope, some erroneous inferences may be corrected during the inference process.

[0137] Table 2 Ablation results of multi-source knowledge reasoning and beam aggregation

[0138]

[0139] The above calculation examples of the present invention are only used to explain the calculation model and calculation process of the present invention in detail, and are not intended to limit the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. A large language model reasoning question answering enhancement method for multi-hop questions, characterized by: The specific process of the method is: Step 1: Construct the multi-hop problem Q into a problem tree; Step 2: Initialize the node number i=1 in the problem tree and perform post-order traversal on the nodes of the problem tree; Step 3: Determine node Q (i) Whether it is a leaf node; If Q (i) If it is an intermediate node, execute step 4; If Q (i) If it is the root node, execute step 4; If Q (i) If it is a leaf node, directly execute step 5; Step 4: Q (i) Perform beam combination on the corresponding sub-problems and convert Q (i) Replace the placeholders in the corresponding sub-questions with the corresponding candidate answers, and then let Q (i) Corresponding sub-problem q (i) = placeholders are replaced with the corresponding candidate answers, and q (i) Store in set Q (i) ', then execute step five; Among them, Q (i) ′ is Q (i) A collection of specific problems; Step 5: Use the LLM model to analyze Q from different knowledge sources (i) The specific questions in ′ are respectively subjected to complementary multi-source reasoning to obtain the answer set If Q (i) is an intermediate node, for Q (i) The answer set of the corresponding sub-questions After probability aggregation, select the k answers with the highest probability and the corresponding probability as Q (i) Candidate answers to the corresponding sub-questions and the probability distribution of candidate answers p (i) , then set i=i+1 and return to step 3; If Q (i) is a leaf node, for Q (i) The answer set of the corresponding sub-questions After the selection probability aggregation, the answer with the highest probability and the corresponding probability are taken as Q (i) Candidate answers to the corresponding sub-questions and the probability distribution of candidate answers p (i) , let i=i+1, and return to step 3; If Q (i) is the root node, for Q (i) The answer set of the corresponding sub-questions After probability aggregation, select the k answers with the highest probability and the corresponding probability as Q (i) Candidate answers to the corresponding sub-questions and the probability distribution of candidate answers p (i) , then execute step 6; Step 6: Output the candidate answer with the highest probability as the answer to Q, and end.

2. The large language model reasoning question answering enhancement method for multi-hop questions according to claim 1 is characterized in that: The multi-hop problem Q in step 1 is constructed as a problem tree, specifically: First, the multi-hop question Q and the prompt words for question decomposition are input into the large language model for question decomposition to obtain N sub-questions; The N sub-problems obtained by decomposition include sub-problems corresponding to leaf nodes, sub-problems corresponding to intermediate nodes, and sub-problems corresponding to root nodes; Then, a problem tree is constructed based on the N sub-problems to obtain a problem tree sequence.

3. The large language model reasoning question answering enhancement method for multi-hop questions according to claim 2 is characterized in that: The nodes in the problem tree are numbered in the order of post-order traversal; There is a placeholder in the parent node to replace the answer to the corresponding question of the child node; The placeholder is #i, where i represents the child node number.

4. The large language model reasoning question answering enhancement method for multi-hop questions according to claim 3 is characterized by: The problem tree is constructed based on N sub-problems to obtain a problem tree sequence, which is specifically: Q decomp ={Q (1) ,Q (2) ,Q (i) ,...,Q (N) } Q (i) ={q (i) ,a (i) ,p (i) } Among them, q (i) is the subproblem corresponding to the i-th node, a (i) ∈E k is the set of candidate answers to the sub-question corresponding to the i-th node, p (i) ∈R k is the probability set of candidate answers for the sub-question corresponding to the i-th node, k is the preset number of candidate answers, i is the node number in the question tree, i∈[1,N], Q (i) is the i-th node, Q (N) is the Nth node and Q (N) is the root node.

5. The large language model reasoning question answering enhancement method for multi-hop questions according to claim 4 is characterized in that: In step 4, the Q (i) Perform beam combination on the corresponding sub-problems and convert Q (i) Replace the placeholders in the corresponding sub-questions with the corresponding candidate answers, and then let Q (i) Corresponding sub-problem q (i) = placeholders are replaced with the corresponding candidate answers, and q (i) Store in set Q (i) ' and execute step 5, specifically: If Q (i) Corresponding sub-problem q (i) If the number of placeholders in is 1, then q (i) The placeholders in the loop are replaced with the k candidate answers with the highest probability for the sub-problems corresponding to the placeholders, and a total of k replaced sub-problems are obtained. (i) = Sub-questions after the placeholders are replaced with the corresponding candidate answers, and the k replaced sub-questions are stored in the set Q (i) ' and execute step 5; If Q (i) Corresponding sub-problem q (i) If the number of placeholders in is greater than 1, the Cartesian product of each candidate answer combination of the sub-questions corresponding to the placeholders is calculated, and the probability of the Cartesian product of each candidate answer combination is obtained. Finally, Q (i) Corresponding sub-problem q (i) Replace the placeholders in with the corresponding candidate answer combinations, and then loop to let q (i) = Sub-questions after the placeholders are replaced with the corresponding candidate answers, and all replaced sub-questions are stored in the set Q (i) ' and execute step five.

6. The large language model reasoning question answering enhancement method for multi-hop questions according to claim 5 is characterized in that: The probability of the Cartesian product of each candidate answer combination is obtained by the following formula: in, is the j'th candidate answer corresponding to question x1, is the j*th candidate answer to question xn, n is Q (i) The total number of placeholders in yes The corresponding probability is yes The corresponding probability, j, j'...j* is the number of the candidate answer.

7. The large language model reasoning question answering enhancement method for multi-hop questions according to claim 6 is characterized in that: In step 5, the LLM model is used to analyze Q from different knowledge sources. (i) The specific questions in ′ are respectively subjected to complementary multi-source reasoning to obtain the answer set Specifically: in, is the answer set, K s is the knowledge retrieved from different knowledge sources, Rp{implicit internal knowledge, explicit internal knowledge, Wikipedia knowledge, web search knowledge}, The k answers with the highest confidence for each specific sub-question reasoning are stored in .

8. The large language model reasoning question answering enhancement method for multi-hop questions according to claim 7 is characterized in that: In step 5, the Q (i) The answer set of the corresponding sub-questions After probability aggregation, select the k answers with the highest probability and the corresponding probability as Q (i) Candidate answers to the corresponding sub-questions and the probability distribution of candidate answers p (i) , specifically: First, get The probability distribution of the answer in j ; Then, according to p j choose The k most probable answers are Q (i) Candidate answers to the corresponding sub-questions At the same time, obtain the probability distribution p of the candidate answers (i) .

9. The large language model reasoning question answering enhancement method for multi-hop questions according to claim 8, characterized in that: The acquisition The probability distribution of the answer in j , specifically: Among them, n,j∈[1,k], j,n are candidate answer labels, f j is the frequency of the jth candidate answer in set L, which is the set of k candidate answers with the highest confidence among all answers to a specific question, g n is the frequency of the nth candidate answer in the set L, τ is the temperature, p j is the probability distribution of the jth candidate answer.

10. The large language model reasoning question answering enhancement method for multi-hop questions according to claim 9, characterized in that: According to p j choose The k most probable answers are Q (i) Candidate answers to the corresponding sub-questions At the same time, obtain the probability distribution p of the candidate answers (i) , specifically: First, according to p j get The probability that one of the answers is y: Where y is One of the candidate answers, Q (i) ′ is Q (i) The specific question set obtained after the placeholders in the corresponding sub-questions are replaced with the corresponding candidate answers, P(m (i) ) is a specific question (i) The corresponding probability, P(y|x=m (i) ) is the specific problem m (i) The probability distribution of the answer being y when p(y) is Q (i) The probability that the answer to the corresponding sub-question is y; Then, yes Each candidate answer y in the equation is probability aggregated and sorted according to the probability of the answer from large to small. Arrange the answers in the formula and get the first k answers and their corresponding probabilities as Q (i) Candidate answers to the corresponding sub-questions and the corresponding probability p (i) .

Images