Reverse rollback type retrieval enhancement generation method and device, equipment and storage medium

Abstract

The invention relates to the technical field of retrieval enhancement generation, and discloses an reflection rollback type retrieval enhancement generation method and device, equipment and a storage medium. The method comprises the steps of generating a rewritten query statement based on a question input by a user and an answer candidate set; executing retrieval based on the rewritten query statement, and adding a retrieval result into an answer candidate set; the state value of the latest retrieval result is calculated through a state evaluation function fusing the confusion degree and the confidence degree; judging whether the state value of the latest retrieval result is lower than the state value of the previous retrieval result of the answer candidate set, and if so, triggering a rollback mechanism; and summarizing each retrieval result in the answer candidate set through a large language model, and outputting a final answer. According to the reflection rollback type retrieval enhancement generation method provided by the invention, a self-reflection and rollback mechanism is introduced, so that the problem of performance reduction caused by low-quality external retrieval information in a large language model is effectively solved.

Description

Technical Field

[0001] The present invention relates to the technical field of retrieval enhancement generation, and in particular to a reflective rollback retrieval enhancement generation method, device, equipment and storage medium. Background Art

[0002] Large language models have achieved remarkable results in the field of natural language processing and are widely used in tasks such as question answering, dialogue generation, and summary extraction. These advances stem from the model's powerful language understanding capabilities and advanced generation techniques. However, despite the excellent generation capabilities of large language models, they still face the "hallucination problem" when interacting with humans, that is, large language models may generate overly certain statements when answering unknown or uncertain content, resulting in inaccurate information and even harmful consequences.

[0003] To solve this problem, the Retrieval-Augmented Generation (RAG) technology introduces external knowledge retrieval and designs a waiver mechanism to help the large language model obtain enough information before generating answers. This method effectively improves the performance of the large language model and expands the knowledge scope through external tools, significantly improving the quality and accuracy of the generated content. However, the retrieval-augmented generation technology also brings new challenges. For example, excessive retrieval may cause the large language model to be confused about familiar questions, thereby affecting the generation quality.

[0004] Therefore, how to use enhanced knowledge retrieval to improve the accuracy and security of answers remains a key issue that needs to be solved urgently. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a reflective rollback retrieval enhancement generation method, device, equipment and storage medium, which balances the accuracy and security of generated content by integrating external knowledge retrieval and dynamic candidate set optimization.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a reflective rollback retrieval enhancement generation method, comprising the following steps: S1, generates a rewritten query statement based on the question input by the user and the answer candidate set; S2, the large language model determines whether it is necessary to call an external knowledge retrieval tool for information retrieval based on the rewritten query statement: if so, the retrieval is performed and the retrieval results are added to the answer candidate set; S3, calculates the state value of the latest retrieval result by integrating the state evaluation function of perplexity and confidence; the perplexity is calculated based on the logarithmic loss of the probability distribution of the retrieval result, and the confidence is quantified by the entropy of the probability distribution of the retrieval result; S4. Determine whether the status value of the latest search result is lower than that of the previous search result in the answer candidate set. If so, trigger the rollback mechanism and remove the latest search result from the answer candidate set. S5. Repeat steps S1 to S4. If the status value of the answer candidate set reaches the set threshold or the number of iterations exceeds the limit, terminate the search and summarize each search result in the answer candidate set through a large language model and output the final answer.

[0007] In one embodiment, generating the rewritten query statement based on the user input question and the answer candidate set specifically includes: Input the user input question and the answer memory set into the query rewriting module to generate the rewritten query statement ; The query rewriting module is implemented based on a fine-tuned large language model.

[0008] In one embodiment, calculating the status value of the latest search result through the status evaluation function that fuses perplexity and confidence specifically includes: Perplexity ; is the search result, is the user input question, represents the total number of entries, represents the th entry, represents all the entries before the th entry; represents the probability that the th entry in the search result appears given the previous i - 1 entries and the user input question ; Confidence ; Status evaluation function ; where and are hyperparameters.

[0009] In one embodiment, the answer candidate set is managed according to the first-in, first-out principle. Each search result is added to the end of the answer candidate set as an observation value, and the search result at the end of the answer candidate set is removed during the rollback operation.

[0010] In a second aspect, the present invention provides a reflection rollback type retrieval enhancement generation device, including: A query rewriting module that generates a rewritten query statement based on the user input question and the answer candidate set; Retrieval module. Based on the rewritten query statement, the large language model determines whether to call an external knowledge retrieval tool for information retrieval. If so, it performs the retrieval and adds the retrieval results to the answer candidate set. State evaluation module. It calculates the state value of the latest retrieval result through a state evaluation function that fuses perplexity and confidence. The perplexity is calculated based on the logarithmic loss of the probability distribution of the retrieval result, and the confidence is quantified through the entropy of the probability distribution of the retrieval result. State comparison module. It determines whether the state value of the latest retrieval result is lower than the state value of the previous retrieval result in the answer candidate set. If so, it triggers a rollback mechanism to remove the latest retrieval result from the answer candidate set. Summary module. If the state value of the answer candidate set reaches the set threshold or the number of iterations exceeds the limit, it terminates the retrieval and summarizes the various retrieval results in the answer candidate set through the large language model and outputs the final answer.

[0011] In one embodiment, generating the rewritten query statement based on the user input question and the answer candidate set specifically includes: Inputting the user input question and the answer memory set into the query rewriting module to generate the rewritten query statement ; The query rewriting module is implemented based on a fine-tuned large language model.

[0012] In one embodiment, calculating the state value of the latest retrieval result through the state evaluation function that fuses perplexity and confidence specifically includes: Perplexity ; is the retrieval result, is the user input question, represents the total number of entries, represents the th entry, represents all entries before the th entry; represents the probability that the th entry in the retrieval result appears given the previous i - 1 entries and the user input question ; Confidence ; State evaluation function ; where and are hyperparameters.

[0013] In one embodiment, the answer candidate set is managed according to the first-in, first-out principle. Each retrieval result is added to the end of the answer candidate set as an observation value, and the retrieval result at the end of the answer candidate set is removed during the rollback operation.

[0014] In a third aspect, the present invention provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the method according to any one of the embodiments of the first aspect are implemented.

[0015] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method according to any one of the embodiments of the first aspect are implemented.

[0016] Compared with the prior art, the beneficial technical effects of the present invention are: Inspired by chain-of-thought reasoning and memory stack management, the present invention proposes a self-reflective multi-source rollback RAG method, aiming to balance the security and quality of answers. Specifically, for a question input by a user, a fine-tuned large language model can be used to evaluate whether the question is answerable and whether it involves dangerous topics. If the question is non-compliant, the large language model refuses to answer; if it is compliant, it enters the reasoning stage. During the reasoning process, the present invention optimizes the question expression through a question rewriting module and converts it into a high-quality form. Then, the large language model determines whether external retrieval is required and selects a suitable tool for information retrieval, organizes the results and adds them to the answer candidate set. In the generation stage, the large language model scores the retrieval results and adjusts the answer candidate set according to the score change. If the current score drops compared with the previous stage, the corresponding answer is removed, and the retrieval and optimization continue. This process is iterated until the score of the answer candidate set reaches a preset threshold, and finally the answer with the highest score is output, improving the security and accuracy of the retrieval method. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flowchart of the method in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] A preferred embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

[0019] The present invention designs a chained retrieval strategy with a rollback mechanism. If the retrieval result has a negative impact on the generation of the final answer, it is removed from the candidate set. In addition, the present invention introduces a query rewriting module to optimize the user input query and proposes a state evaluation function to evaluate the quality of the generated content by combining context perplexity and output confidence.

[0020] Problem definition: Given a question input by a user , a document knowledge base , and a set of answer candidates , denotes the i-th entry, is the total number of entries in the document knowledge base, the elements in are the search strategies the retrieval results returned during each retrieval process , that is , initially set to empty. denotes the tools available during the retrieval process (e.g., knowledge graph, web search, etc.).

[0021] The core of the method of the present invention is the search strategy , which is defined as: ; Send into the set of answer candidates . The first field of is the action flag. When 's action flag is marked as "Summarization", the question input by the user is sorted and merged with all the retrieval results in , and the final answer is output: .

[0022] As Figure 1 shown, the present invention proposes a reflection rollback retrieval enhanced generation method, including the following steps: S1, based on the question input by the user and the set of answer candidates, generate a rewritten query statement; S2, the large language model determines whether to call an external knowledge retrieval tool for information retrieval based on the rewritten query statement: if so, perform the retrieval and add the retrieval results to the set of answer candidates; S3, calculate the status value of the latest retrieval results through a state evaluation function that fuses perplexity and confidence; the perplexity is calculated based on the logarithmic loss of the probability distribution of the retrieval results, and the confidence is quantified through the entropy of the probability distribution of the retrieval results; S4, determine whether the status value of the latest retrieval results is lower than the status value of the previous retrieval results in the set of answer candidates. If so, trigger the rollback mechanism and remove the latest retrieval results from the set of answer candidates; S5, repeat steps S1 to S4. If the status value of the set of answer candidates reaches the set threshold or the number of iterations exceeds the limit, terminate the retrieval and summarize each retrieval result in the set of answer candidates through the large language model and output the final answer.

[0023] In one embodiment, generating a rewritten query statement based on the user input question and the answer candidate set in step S1 specifically includes: Input the question entered by the user and the answer memory set into the query rewriting module to generate a rewritten query statement ; The query rewriting module is implemented based on a fine-tuned large language model.

[0024] Specifically, in the multi-step reasoning process, the question in each step must be determined based on the known and unknown knowledge of the large language model, which will affect the next search strategy and the final result of the large language model. However, repeatedly inputting the same question may lead to meaningless redundant searches and may introduce unnecessary content. For the given question entered by the user , the present invention hopes that the large language model can gradually optimize its answer. Therefore, the present invention proposes a query rewriting module that will reconsider the question and optimize the search strategy at each step of the reasoning.

[0025] The query rewriting module accepts the question entered by the user and the answer memory set , combines them into a new query , is the nth retrieval result. The goal is to integrate the question entered by the user with the answer memory set so that the large language model can clearly understand what it knows and what it doesn't know, thereby supporting the next query decision. Finally, the query rewriting module outputs the rewritten query statement .

[0026] In the implementation process, the present invention first uses an existing large language model to generate an annotated data set, which contains about 1300 user questions. By designing a prompt template, the large language model is guided to generate a rewritten query statement. Then, the best-performing rewritten query statements will be manually reviewed and selected for fine-tuning the large language model. Then, the present invention applies the Low-Rank Adaptation (LoRA) fine-tuning technique to adjust the large language model parameters and learn the query rewriting method. Finally, the rewriting results are applied to the reasoning process so that the large language model can integrate the current memory and optimize the subsequent reasoning steps.

[0027] In one embodiment, calculating the status value of the latest retrieval result through a status evaluation function that fuses perplexity and confidence specifically includes: Perplexity ; is the retrieval result, For the problem input by the user, represents the total number of entries, represents the th entry, represents all the entries before the th entry; represents the probability that the th entry appears in the retrieval result in the case of given the first i - 1 entries and the problem input by the user ; Confidence ; State evaluation function ; where and are hyperparameters used to control the and relative importance in the final metric. In a preferred embodiment, and are both set to 0.5.

[0028] Specifically, if the retrieval result helps answer the question, the reasoning process continues; if the result is unhelpful, the current retrieval result is removed from the candidate set , and the system reverts to the previous state for the next round of retrieval. The present invention evaluates the retrieval result through the state evaluation function.

[0029] To more comprehensively evaluate the performance of the large - language model in different tasks, the present invention introduces a weighted state evaluation function (perplexity - confidence fusion) for evaluating the large - language model in multi - task scenarios. This method not only considers the prediction ability of the large - language model for the given context but also the uncertainty in the large - language model's prediction, providing a more accurate basis for selecting the large - language model in complex tasks.

[0030] Confidence is used to measure the prediction confidence of the large - language model in the given input and task environment. Specifically, it quantifies the uncertainty by calculating the entropy of the probability distribution output by the large - language model. The higher the entropy value, the more uncertain the large - language model's prediction for the current task; the lower the entropy value, the more reliable the large - language model's prediction and the higher the confidence.

[0031] Rollback: Compare the current state value with the previous state value . If the current state value deteriorates, it indicates that is not helpful for answering the question. At this time, is removed from the answer candidate set , and the state value is restored to If the status value remains stable or improves, proceed to the next step of reasoning.

[0032] When the status value reaches the preset threshold terminate the reasoning process, summarize and output the answers in the answer candidate set . If the number of reasoning iterations exceeds the preset value , it indicates that the large language model (LLM) cannot provide a satisfactory answer within the number of iterations, and thus an abstention strategy is executed. This operation is to prevent the large language model from giving wrong answers when it has insufficient understanding of the question.

[0033] In one of the embodiments, the answer candidate set is managed according to the first-in, first-out principle. Each retrieval result is added to the end of the answer candidate set as an observation value, and the retrieval result at the end of the answer candidate set is removed during the rollback operation.

[0034] In memory management technology, memory follows the "first-in, first-out" (FIFO) principle. Drawing on this method, the memory designed for large language model retrieval in the present invention, namely the answer candidate set , also follows this principle. Specifically, each new retrieval result added to the answer candidate set is derived by reasoning based on the previous retrieval results. After the current retrieval result enters the answer candidate set, if its status value decreases compared to the previous state, it indicates that it is not helpful for answering the question. In this case, a rollback operation is performed to remove from the answer candidate set and return to the previous stage for the next round of retrieval. If the status value remains stable or improves, continue with the next step of reasoning until the final answer is obtained.

[0035] The training of the large language model of the present invention can be achieved through the following process: The input is the user input question and a document knowledge base . The large language model is trained for 100 epochs, and an early stopping mechanism is set at the 40th epoch; the Adam optimizer is used with a learning rate of 0.01, and the batch size is set to 32.

[0036] In each training epoch, run according to the following steps: Feed the user input question into the large language model for question analysis and retrieve and return the retrieval result ; Call the status evaluation function for the retrieval result Perform an evaluation to obtain the current status value ; Compare the current status value and , and determine whether it meets . If so, retain the in the answer candidate set. If not, discard the , and set the to .

[0037] When the number of retrieval rounds of the large language model is greater than the preset value , or reaches the preset threshold , end this round of training; After the large language model training is completed, use the test set to evaluate the large language model.

[0038] The reflection rollback retrieval enhanced generation method proposed by the present invention effectively solves the problem of performance degradation caused by low-quality external retrieval information in large language models by introducing self-reflection and rollback mechanisms. The present invention proposes a state evaluation function that combines context perplexity and confidence, ensuring an accurate evaluation of the answer generation quality. In addition, the query rewriting module enhances the relevance and accuracy of the generated answers. Experimental results show that the present invention is superior to existing methods in terms of question answering performance and security, demonstrating its advantages in knowledge-intensive tasks.

[0039] It should be understood that although the steps in the flowchart of the accompanying drawings of the specification are shown in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings of the specification may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same moment, but can be executed at different moments, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0040] Based on the description of the above method embodiments, the present invention also provides an apparatus. The apparatus may be a system (including a distributed system), software (application), module, component, server, client, etc. that uses the method described in the embodiments of this specification and combines the necessary implementation hardware. Based on the same innovative concept, the apparatus in one or more embodiments provided by the embodiments of the present disclosure is as described in the following embodiments. Since the implementation solutions for the apparatus to solve problems are similar to those of the method, the implementation of the specific apparatus in the embodiments of this specification may refer to the implementation of the foregoing method, and repeated parts will not be elaborated. As used hereinafter, the term "module" or "modular unit" is a combination of software and / or hardware that can implement a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0041] Specifically, the present invention proposes a reflective rollback retrieval enhanced generation apparatus, including the following steps: A query rewriting module that generates a rewritten query statement based on the question input by the user and the answer candidate set; A retrieval module, where the large language model determines whether to call an external knowledge retrieval tool for information retrieval based on the rewritten query statement: if so, perform the retrieval and add the retrieval result to the answer candidate set; A state evaluation module that calculates the state value of the latest retrieval result through a state evaluation function that fuses perplexity and confidence; the perplexity is calculated based on the logarithmic loss of the probability distribution of the retrieval result, and the confidence is quantified through the entropy of the probability distribution of the retrieval result; A state comparison module that determines whether the state value of the latest retrieval result is lower than the state value of the previous retrieval result in the answer candidate set. If so, trigger the rollback mechanism to remove the latest retrieval result from the answer candidate set; A summary module that terminates the retrieval if the state value of the answer candidate set reaches a set threshold or the number of iterations exceeds the limit, and summarizes each retrieval result in the answer candidate set through the large language model and outputs the final answer.

[0042] In one embodiment, the present invention also provides a computer-readable storage medium including instructions, such as a memory including instructions, and the above instructions can be executed by a processor to complete the above method. The storage medium may be a computer-readable storage medium. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0043] In one of the embodiments, the present invention further provides a computer device, which may be a server. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the data used in the above method. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0044] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0045] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention, and any reference signs in the claims should not be regarded as limiting the claims involved.

[0046] In addition, it should be understood that although this specification is described according to embodiments, not every embodiment only contains an independent technical solution. This narrative way of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A reflective rollback retrieval enhancement generation method, characterized in that: The following steps are involved: S1, generates a rewritten query statement based on the question input by the user and the answer candidate set; S2, the large language model determines whether it is necessary to call an external knowledge retrieval tool for information retrieval based on the rewritten query statement: if so, the retrieval is performed and the retrieval results are added to the answer candidate set; S3, calculates the state value of the latest retrieval result by integrating the state evaluation function of perplexity and confidence; the perplexity is calculated based on the logarithmic loss of the probability distribution of the retrieval result, and the confidence is quantified by the entropy of the probability distribution of the retrieval result; S4, determining whether the status value of the latest search result is lower than the status value of the previous search result in the answer candidate set. If so, triggering a rollback mechanism to remove the latest search result in the answer candidate set; S5, repeat steps S1 to S4. If the status value of the answer candidate set reaches the set threshold or the number of iterations exceeds the limit, the search is terminated and the search results in the answer candidate set are summarized through the large language model and the final answer is output.

2. According to claim 1, a reflective rollback retrieval enhancement generation method is characterized in that: The generating of the rewritten query statement based on the question input by the user and the answer candidate set specifically includes: Questions entered by the user and answer memory set , input to the query rewrite module , generate the rewritten query statement ; The query rewriting module Large language model implementation based on fine-tuning.

3. The reflective rollback retrieval enhancement generation method according to claim 1, characterized in that: The state value of the latest search result is calculated by the state evaluation function integrating the perplexity and the confidence, specifically including: Perplexity ; To retrieve the results, For user input questions, Indicates the total number of entries. Indicates entries, Indicates All entries before the entry; Indicates the question given the first i-1 terms and the user input In the case of Middle The probability of a term appearing; Confidence ; State evaluation function ;in, and is a hyperparameter.

4. The reflective rollback retrieval enhancement generation method according to claim 1, characterized in that: The answer candidate set is managed using the first-in-first-out principle, and each retrieval result is added to the end of the answer candidate set as an observation value. The retrieval result at the end of the answer candidate set is removed during a rollback operation.

5. A reflective rollback retrieval enhancement generation device, characterized in that: include: The query rewriting module generates a rewritten query statement based on the question entered by the user and the candidate answer set; Retrieval module, the large language model determines whether it is necessary to call an external knowledge retrieval tool for information retrieval based on the rewritten query statement: if so, the retrieval is performed and the retrieval results are added to the answer candidate set; The state evaluation module calculates the state value of the latest search result by integrating the state evaluation function of perplexity and confidence. The perplexity is calculated based on the logarithmic loss of the probability distribution of the search result, and the confidence is quantified by the entropy of the probability distribution of the search result. The status comparison module determines whether the status value of the latest search result is lower than the status value of the previous search result in the answer candidate set. If so, the rollback mechanism is triggered to remove the latest search result in the answer candidate set. The summary module terminates the search and summarizes the search results in the answer candidate set through the large language model and outputs the final answer if the status value of the answer candidate set reaches the set threshold or the number of iterations exceeds the limit.

6. The reflective rollback retrieval enhancement generation device according to claim 5, characterized in that: The generating of the rewritten query statement based on the question input by the user and the answer candidate set specifically includes: Questions entered by the user and answer memory set , input to the query rewrite module , generate the rewritten query statement ; The query rewriting module Large language model implementation based on fine-tuning.

7. The reflective rollback retrieval enhancement generation device according to claim 5, characterized in that: The state value of the latest search result is calculated by the state evaluation function integrating the perplexity and the confidence, specifically including: Perplexity ; To retrieve the results, For user input questions, Indicates the total number of entries. Indicates entries, Indicates All entries before the entry; Indicates the question given the first i-1 terms and the user input In the case of Middle The probability of a term appearing; Confidence ; State evaluation function ;in, and is a hyperparameter.

8. The reflective rollback retrieval enhancement generation device according to claim 5, characterized in that: The answer candidate set is managed using the first-in-first-out principle, and each retrieval result is added to the end of the answer candidate set as an observation value. The retrieval result at the end of the answer candidate set is removed during a rollback operation.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images