Recursive retrieval enhancement generation method and device based on multi-dimensional quality control

Abstract

The invention relates to the technical field of recursive retrieval, in particular to a recursive retrieval enhancement generation method and device based on multi-dimensional quality control, and the method comprises the steps: generating candidate thinking steps according to a user query problem, and generating a corresponding action scheme for each thinking step to construct a plurality of thinking-action candidate pairs; performing logical coherence evaluation on the plurality of thinking-action candidate pairs to obtain an optimal thinking-action candidate pair; obtaining a plurality of candidate documents according to the optimal thinking-action candidate pair, and performing information correlation evaluation on the plurality of candidate documents to select an optimal document; generating a plurality of candidate answers according to the selected optimal document, and performing fact support evaluation on the plurality of candidate answers to select an observation result; and updating the reasoning history according to the recursive retrieval process, and iteratively executing the process until a preset termination condition is met, and outputting a final answer. Therefore, the problems of error accumulation, insufficient quality control and the like in related methods are solved.

Description

Technical Field

[0001] This invention relates to the field of recursive retrieval technology, and in particular to a recursive retrieval enhancement generation method and apparatus based on multi-dimensional quality control. Background Technology

[0002] In recent years, Large Language Models (LLMs) have demonstrated revolutionary breakthroughs in the field of natural language processing. Intelligent question-answering systems, exemplified by ChatGPT, have not only garnered widespread attention in academia but have also profoundly impacted how human society acquires information and interacts with knowledge. However, in practical applications, LLMs still face significant limitations: their pre-trained knowledge has temporal boundaries, making them prone to generating illusionary content, and they lack adaptability to private domain knowledge in specific contexts. These problems are particularly prominent in knowledge-intensive application scenarios, hindering the practicality and reliability of LLMs. To address these challenges, Retrieval-Augmented Generation (RAG) technology has emerged. RAG integrates external knowledge sources to assist the content generation process of large language models, effectively mitigating knowledge boundary limitations and the problem of illusion generation. Traditional RAG systems typically consist of three core stages: index building, similarity retrieval, and answer generation. In the indexing stage, the system preprocesses, segments, and vectorizes external documents; in the retrieval stage, relevant text fragments are retrieved from a vector database based on query similarity; and in the generation stage, the retrieval results are combined with the user query to guide the large language model to generate the final answer. This technology excels in knowledge-intensive tasks such as intelligent question answering, document summarization, and content creation, significantly improving the accuracy and interpretability of large language models in handling complex problems. In industry, RAG-based intelligent question answering platforms such as RAGFlow, Dify, and FastGPT have become important carriers of artificial intelligence applications.

[0003] For complex question-answering tasks requiring multi-step reasoning, single-round retrieval often falls short of obtaining sufficient information. Recursive retrieval enhancement techniques address this issue by iteratively optimizing the retrieval process. Representative methods include: IRCoT, which uses thought chains to guide the retrieval process and optimizes the reasoning chain based on retrieval results; ToC, which constructs clarification trees to systematically handle ambiguous information in queries; and ReAct, which achieves dynamic information acquisition through the alternating execution of reasoning and actions. These methods gradually converge to the most relevant information through feedback loop mechanisms, demonstrating significant advantages in handling specialized and complex queries.

[0004] While recursive retrieval methods offer flexible retrieval direction adjustments, they lack precise quality control mechanisms within their execution chain. For example, incorrect sub-problem decomposition can mislead subsequent retrieval directions; irrelevant documents can introduce noise; and inaccurate sub-solutions can accumulate errors during iteration. These problems not only interfere with the decision quality of subsequent steps but also significantly increase the model's context processing burden, ultimately limiting the overall system's problem-solving capabilities. Summary of the Invention

[0005] This invention provides a recursive retrieval enhancement generation method and apparatus based on multi-dimensional quality control to solve problems such as error accumulation and insufficient quality control in related methods.

[0006] A first aspect of this invention provides a recursive retrieval enhancement generation method based on multi-dimensional quality control, comprising the following steps: Based on the user's query question, candidate thinking steps are generated, and corresponding action plans are generated for each thinking step to construct multiple thinking-action candidate pairs; The logical coherence of the multiple thought-action candidate pairs is evaluated to obtain the optimal thought-action candidate pair; Multiple candidate documents are obtained based on the best thinking-action candidate pair, and the information relevance of the multiple candidate documents is evaluated in order to select the best document from the multiple candidate documents; Based on the selection of the best document, multiple candidate answers are generated, and the factual support of the multiple candidate answers is evaluated to select the observation from the multiple candidate answers; The reasoning history is updated based on the aforementioned action plan generation process, logical coherence evaluation process, information relevance evaluation process, and fact support evaluation process. The action plan generation process, logical coherence evaluation process, information relevance evaluation process, and fact support evaluation process are iteratively executed until the preset termination condition is met, and the final answer to the user query question is output.

[0007] Optionally, the step of evaluating the logical coherence of the plurality of thought-action candidate pairs to obtain the optimal thought-action candidate pair includes: Receive the reasoning chain of the previous process in the reasoning history and the multiple thought-action candidate pairs; The reasoning chain and the current thought-action candidate pair are input into a pre-built heuristic rule template to guide the large model. The reasoning steps of each thought-action candidate pair are evaluated for coherence in order to select the best thought-action candidate pair from the multiple thought-action candidate pairs.

[0008] Optionally, the step of obtaining multiple candidate documents based on the best think-action candidate pair and evaluating the information relevance of the multiple candidate documents to select the best document from the multiple candidate documents includes: Multiple candidate documents are obtained based on the best thought-action candidate pairs; Receive a subquery from the previous process in the reasoning history; The subquery and the multiple candidate documents are input into a pre-built heuristic rule template to guide a large model. The relevance of the subquery for each candidate document is evaluated to select the best document from the multiple candidate documents.

[0009] Optionally, the step of generating multiple candidate answers based on the selected best document and performing a factual support evaluation on the multiple candidate answers to select the observation from the multiple candidate answers includes: Based on the selection of the best document, multiple candidate answers are generated; Receive the subquery and subdocument from the previous process in the reasoning history; The subquery, the subdocument, and the multiple candidate answers are input into a pre-built heuristic rule template to guide a large model. The factual support of each candidate answer is evaluated, and the observation is selected from the multiple candidate answers.

[0010] A second aspect of the present invention provides a recursive retrieval enhancement generation device based on multi-dimensional quality control, comprising: The initialization module is used to generate candidate thinking steps based on the user's query question, and generate corresponding action plans for each thinking step to build multiple thinking-action candidate pairs; The system comprises the following modules: a logical coherence evaluation module, used to evaluate the logical coherence of the multiple thought-action candidate pairs to obtain the optimal thought-action candidate pair; an information relevance evaluation module, used to obtain multiple candidate documents based on the optimal thought-action candidate pair and evaluate the information relevance of the multiple candidate documents to select the best document; a fact support evaluation module, used to generate multiple candidate answers based on the selected best document and evaluate the fact support of the multiple candidate answers to select the observation result; and an update and iterative execution module, used to update the reasoning history based on the aforementioned action plan generation process, logical coherence evaluation process, information relevance evaluation process, and fact support evaluation process, and iteratively execute the action plan generation process, the logical coherence evaluation process, the information relevance evaluation process, and the fact support evaluation process until a preset termination condition is met, and output the final answer to the user query question.

[0011] Optionally, the logical coherence evaluation module includes: The first receiving unit is used to receive the reasoning chain of the previous process in the reasoning history and the multiple thought-action candidate pairs; the coherence evaluation unit is used to input the reasoning chain and the current thought-action candidate pair into a pre-built heuristic rule template to guide the large model, and to evaluate the coherence of the reasoning steps of each thought-action candidate pair in order to select the best thought-action candidate pair from the multiple thought-action candidate pairs.

[0012] Optionally, the information relevance assessment module includes: The first acquisition unit is used to acquire multiple candidate documents based on the best thought-action candidate pair; the second receiving unit is used to receive the sub-query of the previous process in the reasoning history; the relevance evaluation unit is used to input the sub-query and the multiple candidate documents into a pre-built heuristic rule template to guide the large model, and to evaluate the relevance of the sub-query of each candidate document in order to select the best document from the multiple candidate documents.

[0013] Optionally, the fact support assessment module includes: The second acquisition unit is used to generate multiple candidate answers based on the selection of the best document; the third receiving unit is used to receive the subquery and sub-document of the previous process in the reasoning history; the fact support evaluation unit is used to input the subquery, the sub-document and the multiple candidate answers into a pre-built heuristic rule template to guide the large model, perform fact support evaluation on each candidate answer, and filter out the observation results from the multiple candidate answers.

[0014] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the recursive retrieval enhancement generation method based on multi-dimensional quality control as described in the above embodiments.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described recursive retrieval enhancement generation method based on multi-dimensional quality control.

[0016] The recursive retrieval enhancement generation method and apparatus based on multi-dimensional quality control proposed in this invention, based on the ReAct framework, systematically introduces a multi-dimensional quality assessment module into the recursive retrieval chain to construct a multi-dimensional quality assessment system. Three specialized assessment dimensions—logical coherence, information relevance, and factual support—are designed for different execution stages of recursive retrieval. Compared to the single assessment standard of relevant methods, this achieves more refined and targeted quality control of the execution process, significantly improving overall system performance. Furthermore, the relevance judgment capability of the retrieval re-ranking model is transferred to other dimensions of evaluation in recursive retrieval scenarios, achieving efficient quality assessment functionality through heuristic rules and template design.

[0017] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0018] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 A flowchart illustrating a recursive retrieval enhancement generation method based on multi-dimensional quality control according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a quality control process in recursive retrieval reasoning according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the mechanism of a quality assessment model provided according to an embodiment of the present invention; Figure 4 This is a block diagram of a recursive retrieval enhancement generation device based on multi-dimensional quality control according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0019] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0020] The following describes, with reference to the accompanying drawings, a recursive retrieval enhancement generation method and apparatus based on multi-dimensional quality control according to embodiments of the present invention. Addressing the issues mentioned in the background section, existing methods lack precise monitoring of each stage of the execution chain, leading to the accumulation and amplification of erroneous reasoning steps in subsequent executions, causing the final answer to deviate from the correct path. Furthermore, retrieved documents often exhibit inconsistent quality, lacking an effective relevance filtering mechanism, resulting in answer generation based on low-quality information sources. Additionally, large language models are prone to illusions during generation, with answers lacking factual basis misleading users' questions. The present invention provides a recursive retrieval enhancement generation method based on multi-dimensional quality control. This method introduces an evaluation mechanism at the three nodes of recursive retrieval reasoning: "reasoning for the next action, retrieving relevant documents, and generating subquery answers." A comprehensive evaluation is conducted from three dimensions: action logic coherence, document information relevance, and answer factual support. Combined with a "multi-sampling + evaluation filtering" strategy, a high-quality execution sequence is gradually constructed. Furthermore, a capability transfer mechanism based on heuristic rule templates is proposed, designing evaluation templates for different execution stages and utilizing the instruction-following characteristics of large language models to achieve quality evaluation capability transfer, thereby controlling the quality of the execution stages. This solved the problems of error accumulation and insufficient quality control in the relevant methods.

[0021] Specifically, Figure 1 This is a flowchart illustrating a recursive retrieval enhancement generation method based on multi-dimensional quality control, provided in an embodiment of the present invention.

[0022] like Figure 1 As shown, the recursive retrieval enhancement generation method based on multi-dimensional quality control includes the following steps: In step S101, candidate thinking steps are generated based on the user's query question, and corresponding action plans are generated for each thinking step to construct multiple thinking-action candidate pairs.

[0023] In step S102, the logical coherence of multiple think-action candidate pairs is evaluated to obtain the best think-action candidate pair.

[0024] In some embodiments, a logical coherence evaluation is performed on multiple think-action candidate pairs to obtain the optimal think-action candidate pair, including: Receive the reasoning chain of the previous process in the reasoning history and multiple think-action candidate pairs; The reasoning chain and the current thought-action candidate pair are input into a pre-built heuristic rule template to guide the large model. The reasoning steps of each thought-action candidate pair are evaluated for coherence in order to select the best thought-action candidate pair from multiple thought-action candidate pairs.

[0025] In actual execution, user query questions are obtained and input into the target large language model to generate candidate thinking steps. For each thinking step, a corresponding action plan is generated to construct multiple thinking-action candidate pairs.

[0026] Furthermore, to ensure the logical consistency of reasoning steps and prevent logical breaks in the thought-action chain, this embodiment of the invention utilizes historical reasoning context to evaluate the coherence of the current candidate steps. Specifically, it receives the reasoning chain of the previous process and multiple thought-action candidate pairs from the reasoning history, and inputs the reasoning chain and the current thought-action candidate pair into a pre-constructed heuristic rule template to guide a large model. The coherence of the reasoning steps for each thought-action candidate pair is evaluated to select the best thought-action candidate pair from among the multiple thought-action candidate pairs.

[0027] Furthermore, as shown in Table 1 below, the evaluation criteria for the coherence assessment process follow five dimensions of the logical task: (1) the thinking steps need to be based on reasonable reasoning from the observation results; (2) the actions must be logically derived from the thinking; (3) the search action should seek the required information; (4) the closing action should provide a complete answer; and (5) the alternating format of thinking-action-observation must be strictly followed. Through this multi-dimensional evaluation mechanism, the most logically coherent steps can be selected from multiple candidates to continue execution, effectively avoiding logical jumps or contradictions in the reasoning chain and providing a reliable reasoning basis for subsequent steps.

[0028] Table 1 Quality Assessment Rule Template

[0029] Among them, the pre-built heuristic rule template guides the capability transfer mechanism of the large model to effectively transfer the evaluation capability of the re-ranking model to the quality control task of recursive retrieval.

[0030] Specifically, such as Figure 2 and 3 As shown, the core mechanism of the Qwen3 re-ranking model used in related technologies is to evaluate the relevance of the query to the document through a "yes / no" binary judgment. This re-ranking model formats the input using a specific prompt template, then calculates the probability distribution of "yes" and "no" terms, and selects the judgment result with the higher probability. Therefore, this embodiment of the invention constructs a capability transfer mechanism for heuristic rule templates based on the Qwen3 re-ranking model, which can extend the binary judgment mechanism to the three quality control nodes of recursive retrieval. The implementation elements of the capability transfer mechanism for heuristic rule templates include the following: (1) Template formatting strategy: Input format based on Qwen3 reordering model <instruct> <query> <document>In this embodiment of the invention, a dedicated prompt template is designed for each quality control node. For example, logical coherence assessment uses the format <reasoning history><candidate steps><coherence instructions>, while information relevance assessment uses the structure <subquery><candidate document><relevance instructions>.

[0031] (2) Binary Judgment Refinement: Based on the Qwen3 reordering model, the simple relevance judgment is refined into a multi-level quality assessment. By embedding specific assessment criteria (such as the five criteria of Logic Task, Relevance Task and Factual Task mentioned above) in the prompt instructions, the large model is guided to make fine-grained judgments according to the predefined quality dimensions.

[0032] (3) Probability Calculation and Ranking: Based on the calculation logic of the Qwen3 reordering model, the quality assessment probability score is obtained through torch.nn.functional.log_softmax and exp() operations. For multiple candidate options, the quality score of each candidate option is calculated, and then the option with the highest score is selected as the optimal choice.

[0033] (4) Multi-task adaptation mechanism: By adjusting the instruction part in the prompt template, the same framework can be adapted to different quality control tasks. For example, the task parameter can be adjusted from "retrieving relevant documents" to "evaluating logical coherence" or "verifying factual support", so that one mechanism can support multiple evaluation needs.

[0034] This allows the final heuristic rule template-guided large model to avoid the cost of retraining a specialized model while utilizing the judgment capabilities of a mature reordering model. Through the combination of template engineering and heuristic rules, refined quality control of the recursive retrieval execution process is achieved.

[0035] In step S103, multiple candidate documents are obtained based on the best thought-action candidate pair, and the information relevance of the multiple candidate documents is evaluated in order to select the best document from the multiple candidate documents.

[0036] In some embodiments, multiple candidate documents are obtained based on best-think-action candidate pairs, and the information relevance of the multiple candidate documents is evaluated to select the best document from the multiple candidate documents, including: Obtain multiple candidate documents based on the best thinking-action candidate pairs; Receives a subquery from the previous process in the reasoning history; The subquery and multiple candidate documents are fed into a pre-built heuristic rule template to guide the large model. The relevance of the subquery for each candidate document is evaluated to select the best document from the multiple candidate documents.

[0037] In actual implementation, to ensure that subsequent answer generation is based on a high-quality information foundation, this embodiment of the invention selects the information source most relevant to the subquery from multiple retrieved documents. Specifically, multiple candidate documents are obtained based on the best thought-action candidate pair, the subquery from the previous process in the reasoning history is received, and a large model is guided to perform relevance evaluation through heuristic rule templates. The relevance of the subquery in each candidate document is evaluated to select the best document from multiple candidate documents.

[0038] Furthermore, as shown in Table 1 above, the relevance assessment process can follow five key criteria of the relevance task: (1) directly responding to the specific query content; (2) containing relevant and useful information; (3) accurate and timely content; (4) providing sufficient detailed information; and (5) closely focusing on the query topic. By integrating these assessment dimensions, the most valuable information sources can be accurately identified, providing high-quality factual evidence for answer generation, significantly improving retrieval accuracy and subsequent generation results.

[0039] In step S104, multiple candidate answers are generated based on the selection of the best document, and the factual support of the multiple candidate answers is evaluated to select the observation from the multiple candidate answers.

[0040] In some embodiments, multiple candidate answers are generated based on the selection of the best document, and factual support is evaluated on the multiple candidate answers to select the observation from the multiple candidate answers, including: Multiple candidate answers are generated based on the selection of the best document; Receives subqueries and subdocuments from the previous process in the reasoning history; Subqueries, subdocuments, and multiple candidate answers are input into a pre-built heuristic rule template to guide a large model, which evaluates the factual support of each candidate answer and filters out the observations from the multiple candidate answers.

[0041] In practical implementation, to ensure that the generated sub-answers have sufficient factual basis and effectively avoid the illusion generation problem of large language models, this embodiment of the invention filters out the results most relevant to the subquery and sub-document from multiple candidate answers. Specifically, multiple candidate answers are generated based on the selection of the best document. The subquery and sub-document from the previous process in the reasoning history are received. The subquery, sub-document, and multiple candidate answers are input into a pre-built heuristic rule template to guide the large model. The factual support of each candidate answer is evaluated, and the observation result is filtered out from the multiple candidate answers.

[0042] Furthermore, as shown in Table 1 above, the factual support assessment process can adopt a five-level assessment strategy for factual tasks: (1) directly answering sub-questions; (2) all statements are documented; (3) avoiding illusions or unfounded inferences; (4) ensuring accuracy of details and facts; and (5) strictly adhering to the documented scope. Through this rigorous factual support assessment, the system can effectively filter out accurate and complete answers, ensuring that each observation has a reliable factual basis, thereby improving the credibility and accuracy of the entire question-and-answer system.

[0043] In step S105, the reasoning history is updated according to the aforementioned action plan generation process, logical coherence evaluation process, information relevance evaluation process, and fact support evaluation process, and the action plan generation process, logical coherence evaluation process, information relevance evaluation process, and fact support evaluation process are iteratively executed until the preset termination condition is met, and the final answer to the user's query question is output.

[0044] In actual execution, the reasoning history is updated based on the aforementioned action plan generation process, logical coherence assessment process, information relevance assessment process, and fact support assessment process to prepare for the next iteration. After the update is completed, the action plan generation process, logical coherence assessment process, information relevance assessment process, and fact support assessment process are executed iteratively until the action plan generation process outputs the final answer instruction or reaches the preset maximum iteration step limit, and outputs the final answer to the user's query question.

[0045] It should be noted that the quality control in the embodiments of the present invention can be implemented using different model architectures. For example, a specially trained classifier model can be used to train logical coherence, information relevance, and factual support as independent binary or multi-class classification models; an evaluator based on reinforcement learning can be used to train the quality evaluation agent through reward signals; the quality evaluation criteria can be designed in various ways, for example, for logical coherence, formal verification based on logical rules can be used, or a coherence measure based on semantic similarity can be used, or a comprehensive scoring function can be constructed by combining multiple evaluation dimensions; the underlying retrieval and generation components can be configured in various ways, for example, different combinations of retrieval tools can be used, such as BM25 combined with dense retrieval, multi-path recall fusion ranking, etc.; different generation models can be used, such as the GPT series, Claude series, open source large models, etc.; an end-to-end training method of retrieval tool-generator can be used to jointly optimize retrieval and generation performance; a multi-stage generation strategy can also be used, first generating a draft and then refining it, or using the integrated output of multiple generation models.

[0046] The following specific embodiment further illustrates the recursive retrieval enhancement generation method based on multi-dimensional quality control proposed in this invention.

[0047] Existing research has explored quality control of the generation process from several perspectives. For example, Self-RAG trains a model to automatically decide whether retrieval is needed during the generation process and learns to use special labels to achieve end-to-end control of "retrieval-judgment-rewriting," effectively improving factual consistency, but requires a large amount of labeled data for model training. Self-Consistency employs a strategy of sampling multiple reasoning paths and selecting the final answer through a voting mechanism, improving the accuracy of mathematical and common-sense reasoning tasks without additional training, but its evaluation criteria are relatively simple, mainly relying on result consistency judgment. Tree of Thoughts extends linear reasoning to a tree-like search structure, supporting branching exploration, quality assessment, and backtracking mechanisms, performing well in complex tasks requiring multiple steps of trial and error, but its evaluation mechanism is mainly based on heuristic rules and lacks systematic quality control standards. The paper "Decomposition Enhances Reasoning via Self-Evaluation Guided Decoding" proposes a "decomposition-self-evaluation-bundle search" decoding strategy, which allows the model to self-evaluate its reasoning steps and perform bundle search accordingly. This strategy has achieved significant improvements in mathematical reasoning tasks, but its evaluation dimensions are relatively singular, mainly focusing on confidence scores.

[0048] Therefore, it can be seen that the recursive retrieval enhancement generation method based on multi-dimensional quality control proposed in this invention, in terms of evaluation granularity, compared with the single evaluation standard or simple confidence score adopted by related methods, designs three dedicated evaluation dimensions for different execution stages of recursive retrieval: logical coherence, information relevance, and factual support, achieving more refined quality control. In terms of implementation, compared with the Self-RAG method which requires training special labels or the Tree of Thoughts method which relies on simple heuristic rules, this invention achieves evaluation by combining heuristic rule templates with large model capability transfer, which avoids additional model training costs and provides more accurate evaluation capabilities than simple rules. In terms of control scope, compared with focusing on the optimization of a single link, such as Self-Consistency focusing on the final answer selection and Tree of Thoughts focusing on reasoning path exploration, this invention constructs a quality control mechanism covering the entire "reasoning-retrieval-generation" link, realizing comprehensive monitoring and optimization of the recursive retrieval execution process. This systematic multi-dimensional evaluation framework can more effectively prevent error accumulation and improve the overall performance of multi-hop question answering tasks.

[0049] In summary, the recursive retrieval enhancement generation method based on multi-dimensional quality control proposed in this embodiment of the invention has the following beneficial effects: (1) In terms of optimizing search results, the information relevance evaluation mechanism can accurately identify the most valuable information sources from multiple candidate documents, avoid interference from irrelevant information in answer generation, and improve search accuracy and subsequent generation results; the search F1 score is 5% higher than that of ReAct technology; (2) In terms of improving answer accuracy, through a multi-dimensional quality assessment mechanism, the system can select the optimal choice at each key node, effectively preventing the cumulative propagation of errors in the execution chain and significantly improving the accuracy of multi-hop question answering tasks; experimental results show that, compared with related methods, the accuracy on complex reasoning tasks is improved by 15-18% compared with ReAct technology; (3) In terms of improving reasoning efficiency, through the precise screening mechanism of three quality control nodes, the system can quickly eliminate low-quality candidate steps, irrelevant documents and inaccurate answers, avoid exploring invalid reasoning paths, and significantly reduce redundant calculations in the reasoning chain; statistics show that the average number of reasoning steps is reduced by 20% compared with ReAct technology.

[0050] Next, referring to the accompanying drawings, a recursive retrieval enhancement generation device based on multi-dimensional quality control according to an embodiment of the present invention is described.

[0051] Figure 4 This is a block diagram of a recursive retrieval enhancement generation device based on multi-dimensional quality control, provided in an embodiment of the present invention.

[0052] like Figure 4 As shown, the recursive retrieval enhancement generation device 40 based on multi-dimensional quality control provided in this embodiment of the invention includes: an initialization module 401, a logical coherence evaluation module 402, an information relevance evaluation module 403, a fact support evaluation module 404, and an update and iteration execution module 405.

[0053] The initialization module 401 generates candidate thinking steps based on the user's query and generates corresponding action plans for each thinking step to construct multiple thought-action candidate pairs. The logical coherence evaluation module 402 evaluates the logical coherence of the multiple thought-action candidate pairs to obtain the optimal pair. The information relevance evaluation module 403 obtains multiple candidate documents based on the optimal thought-action candidate pair and evaluates their information relevance to select the best document. The factual support evaluation module 404 generates multiple candidate answers based on the selected best document and evaluates their factual support to select the observation result. The update and iterative execution module 405 updates the reasoning history based on the aforementioned action plan generation, logical coherence evaluation, information relevance evaluation, and factual support evaluation processes, and iteratively executes these processes until a preset termination condition is met, outputting the final answer to the user's query.

[0054] In some embodiments, the logical coherence assessment module 402 includes: The first receiving unit is used to receive the reasoning chain of the previous process in the reasoning history and multiple thought-action candidate pairs; The coherence evaluation unit is used to input the reasoning chain and the current think-action candidate pair into a pre-built heuristic rule template to guide the large model, and to evaluate the coherence of the reasoning steps of each think-action candidate pair in order to select the best think-action candidate pair from multiple think-action candidate pairs.

[0055] In some embodiments, the information relevance assessment module 403 includes: The first acquisition unit is used to acquire multiple candidate documents based on the best think-action candidate pairs; The second receiving unit is used to receive subqueries from the previous process in the reasoning history; The relevance evaluation unit is used to input subqueries and multiple candidate documents into a pre-built heuristic rule template to guide the large model, evaluate the relevance of the subqueries for each candidate document, and select the best document from multiple candidate documents.

[0056] In some embodiments, the fact support assessment module 404 includes: The second acquisition unit is used to generate multiple candidate answers based on the selection of the best document; The third receiving unit is used to receive subqueries and subdocuments from the previous process in the reasoning history; The fact support evaluation unit is used to input subqueries, subdocuments, and multiple candidate answers into a pre-built heuristic rule template to guide the large model, evaluate the fact support of each candidate answer, and filter out the observations from multiple candidate answers.

[0057] It should be noted that the foregoing explanation of the recursive retrieval enhancement generation method based on multi-dimensional quality control also applies to the recursive retrieval enhancement generation device based on multi-dimensional quality control in this embodiment, and will not be repeated here.

[0058] The recursive retrieval enhancement generation device based on multi-dimensional quality control proposed in the embodiments of the present invention has the following beneficial effects: (1) In terms of optimizing search results, the information relevance evaluation mechanism can accurately identify the most valuable information sources from multiple candidate documents, avoid interference from irrelevant information in answer generation, and improve search accuracy and subsequent generation results; the search F1 score is 5% higher than that of ReAct technology; (2) In terms of improving answer accuracy, through a multi-dimensional quality assessment mechanism, the system can select the optimal choice at each key node, effectively preventing the cumulative propagation of errors in the execution chain and significantly improving the accuracy of multi-hop question answering tasks; experimental results show that, compared with related methods, the accuracy on complex reasoning tasks is improved by 15-18% compared with ReAct technology; (3) In terms of improving reasoning efficiency, through the precise screening mechanism of three quality control nodes, the system can quickly eliminate low-quality candidate steps, irrelevant documents and inaccurate answers, avoid exploring invalid reasoning paths, and significantly reduce redundant calculations in the reasoning chain; statistics show that the average number of reasoning steps is reduced by 20% compared with ReAct technology.

[0059] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0060] The electronic device may include: a memory 501, a processor 502, and a computer program stored on the memory 501 and capable of running on the processor 502.

[0061] When the processor 502 executes the program, it implements the recursive retrieval enhancement generation method based on multi-dimensional quality control provided in the above embodiments.

[0062] Furthermore, electronic devices also include: Communication interface 503 is used for communication between memory 501 and processor 502.

[0063] The memory 501 is used to store computer programs that can run on the processor 502.

[0064] Memory 501 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0065] If the memory 501, processor 502, and communication interface 503 are implemented independently, then the communication interface 503, memory 501, and processor 502 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0066] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.

[0067] Processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0068] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described recursive retrieval enhancement generation method based on multi-dimensional quality control.

[0069] In the description of this specification, the references to "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0070] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0071] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0072] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0073] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0074] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0075] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0076] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.< / document> < / query> < / instruct>

Claims

1. A recursive retrieval enhancement generation method based on multi-dimensional quality control, characterized in that, Includes the following steps: Based on the user's query question, candidate thinking steps are generated, and corresponding action plans are generated for each thinking step to construct multiple thinking-action candidate pairs; The logical coherence of the multiple thought-action candidate pairs is evaluated to obtain the optimal thought-action candidate pair; Multiple candidate documents are obtained based on the best thinking-action candidate pair, and the information relevance of the multiple candidate documents is evaluated in order to select the best document from the multiple candidate documents; Based on the selection of the best document, multiple candidate answers are generated, and the factual support of the multiple candidate answers is evaluated to select the observation from the multiple candidate answers; The reasoning history is updated based on the aforementioned action plan generation process, logical coherence evaluation process, information relevance evaluation process, and fact support evaluation process. The action plan generation process, logical coherence evaluation process, information relevance evaluation process, and fact support evaluation process are iteratively executed until the preset termination condition is met, and the final answer to the user query question is output.

2. The recursive retrieval enhancement generation method based on multi-dimensional quality control according to claim 1, characterized in that, The step of evaluating the logical coherence of the plurality of thought-action candidate pairs to obtain the optimal thought-action candidate pair includes: Receive the reasoning chain of the previous process in the reasoning history and the multiple thought-action candidate pairs; The reasoning chain and the current thought-action candidate pair are input into a pre-built heuristic rule template to guide the large model. The reasoning steps of each thought-action candidate pair are evaluated for coherence in order to select the best thought-action candidate pair from the multiple thought-action candidate pairs.

3. The recursive retrieval enhancement generation method based on multi-dimensional quality control according to claim 1, characterized in that, The step of obtaining multiple candidate documents based on the best thought-action candidate pair and evaluating the information relevance of the multiple candidate documents to select the best document from the multiple candidate documents includes: Multiple candidate documents are obtained based on the best thought-action candidate pairs; Receive a subquery from the previous process in the reasoning history; The subquery and the multiple candidate documents are input into a pre-built heuristic rule template to guide a large model. The relevance of the subquery for each candidate document is evaluated to select the best document from the multiple candidate documents.

4. The recursive retrieval enhancement generation method based on multi-dimensional quality control according to claim 1, characterized in that, The process of generating multiple candidate answers based on the selected best document, and evaluating the factual support of the multiple candidate answers to select the observation from the multiple candidate answers, includes: Based on the selection of the best document, multiple candidate answers are generated; Receive the subquery and subdocument from the previous process in the reasoning history; The subquery, the subdocument, and the multiple candidate answers are input into a pre-built heuristic rule template to guide a large model. The factual support of each candidate answer is evaluated, and the observation is selected from the multiple candidate answers.

5. A recursive retrieval enhancement generation device based on multi-dimensional quality control, characterized in that, include: The initialization module is used to generate candidate thinking steps based on the user's query question, and generate corresponding action plans for each thinking step to build multiple thinking-action candidate pairs; The logical coherence evaluation module is used to evaluate the logical coherence of the multiple thought-action candidate pairs in order to obtain the best thought-action candidate pair. The information relevance assessment module is used to obtain multiple candidate documents based on the best thinking-action candidate pair, and to perform information relevance assessment on the multiple candidate documents in order to select the best document from the multiple candidate documents. The fact support assessment module is used to generate multiple candidate answers based on the selection of the best document, and to perform fact support assessment on the multiple candidate answers in order to select the observation result from the multiple candidate answers; The update and iteration execution module is used to update the reasoning history based on the aforementioned action plan generation process, logical coherence evaluation process, information relevance evaluation process, and fact support evaluation process, and to iteratively execute the action plan generation process, the logical coherence evaluation process, the information relevance evaluation process, and the fact support evaluation process until a preset termination condition is met, and output the final answer to the user query question.

6. The recursive retrieval enhancement generation device based on multi-dimensional quality control according to claim 5, characterized in that, The logical coherence evaluation module includes: The first receiving unit is used to receive the reasoning chain of the previous process in the reasoning history and the multiple thought-action candidate pairs; The coherence evaluation unit is used to input the reasoning chain and the current thought-action candidate pair into a pre-built heuristic rule template to guide the large model, and to evaluate the coherence of the reasoning steps of each thought-action candidate pair in order to select the best thought-action candidate pair from the multiple thought-action candidate pairs.

7. The recursive retrieval enhancement generation device based on multi-dimensional quality control according to claim 5, characterized in that, The information relevance assessment module includes: The first acquisition unit is used to acquire multiple candidate documents based on the best thought-action candidate pair; The second receiving unit is used to receive the sub-query of the previous process in the reasoning history; The relevance evaluation unit is used to input the subquery and the multiple candidate documents into a pre-built heuristic rule template-guided large model, and to evaluate the relevance of the subquery of each candidate document in order to select the best document from the multiple candidate documents.

8. The recursive retrieval enhancement generation device based on multi-dimensional quality control according to claim 5, characterized in that, The fact-supporting assessment module includes: The second acquisition unit is used to generate multiple candidate answers based on the selection of the best document; The third receiving unit is used to receive the sub-queries and sub-documents of the previous process in the reasoning history; The fact support evaluation unit is used to input the subquery, the subdocument, and the multiple candidate answers into a pre-built heuristic rule template to guide the large model, evaluate the fact support of each candidate answer, and filter out the observation results from the multiple candidate answers.

9. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, the processor executing the program to implement the recursive retrieval enhancement generation method based on multi-dimensional quality control as described in any one of claims 1-4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the recursive retrieval enhancement generation method based on multi-dimensional quality control as described in any one of claims 1-4.

Images