Reflective rollback search enhancement generation method, device, equipment and storage medium

Abstract

The present invention relates to the technical field of retrieval enhancement generation, and discloses a reflective rollback retrieval enhancement generation method, apparatus, device and storage medium. The method comprises: generating a rewritten query statement based on a question input by a user and a set of answer candidates; performing a retrieval based on the rewritten query statement, and adding the retrieval results to the answer candidate set; calculating the state value of the latest retrieval result by a state evaluation function that integrates perplexity and confidence; judging whether the state value of the latest retrieval result is lower than the state value of the previous retrieval result in the answer candidate set, and if so, triggering a rollback mechanism; summarizing each retrieval result in the answer candidate set through a large language model and outputting the final answer. The reflective rollback retrieval enhancement generation method proposed by the present invention effectively solves the problem of performance degradation caused by low-quality external retrieval information in a large language model by introducing self-reflection and rollback mechanisms.

Description

Technical Field

[0001] The present invention relates to the technical field of retrieval enhancement generation, and in particular to a reflective rollback retrieval enhancement generation method, device, equipment and storage medium. Background Art

[0002] Large language models have achieved remarkable results in natural language processing and are widely used in tasks such as question answering, dialogue generation, and summary extraction. These advances stem from their powerful language understanding capabilities and advanced generative techniques. However, despite their impressive generative capabilities, large language models still face the "hallucination problem" when interacting with humans. This problem involves generating overly certain statements when answering unknown or uncertain questions, leading to inaccurate information and even harmful consequences.

[0003] To address this issue, Retrieval-Augmented Generation (RAG) technology uses external knowledge retrieval and a design abstention mechanism to help large language models acquire sufficient information before generating answers. This approach effectively improves the performance of large language models and, by expanding their knowledge base through external tools, significantly enhances the quality and accuracy of generated content. However, RAG also introduces new challenges. For example, excessive retrieval can cause large language models to become confused about familiar questions, thus affecting generation quality.

[0004] Therefore, how to use enhanced knowledge retrieval to improve the accuracy and security of answers remains a key issue that needs to be solved urgently. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention provides a reflective rollback retrieval enhancement generation method, device, equipment and storage medium, which balances the accuracy and security of generated content by integrating external knowledge retrieval and dynamic candidate set optimization.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a reflective rollback retrieval enhancement generation method, comprising the following steps:

[0008] S1, based on the question and answer candidate set input by the user, generates a rewritten query statement;

[0009] S2, based on the rewritten query statement, the large language model determines whether it is necessary to call an external knowledge retrieval tool for information retrieval: if so, the retrieval is performed and the retrieval results are added to the answer candidate set;

[0010] S3, calculates the state value of the latest retrieval result by integrating the state evaluation function of perplexity and confidence; perplexity is calculated based on the logarithmic loss of the probability distribution of the retrieval result, and confidence is quantified by the entropy of the probability distribution of the retrieval result;

[0011] S4, determine whether the status value of the latest search result is lower than the status value of the previous search result in the answer candidate set. If so, trigger the rollback mechanism and remove the latest search result in the answer candidate set;

[0012] S5, repeat steps S1 to S4. If the status value of the answer candidate set reaches the set threshold or the number of iterations exceeds the limit, the search is terminated and the search results in the answer candidate set are summarized through the large language model and the final answer is output.

[0013] In one embodiment, generating a rewritten query statement based on the question input by the user and the set of candidate answers specifically includes:

[0014] Questions entered by the user and answer memory set , input to the query rewrite module , generate the rewritten query statement ; The query rewriting module Implementation of large language models based on fine-tuning.

[0015] In one embodiment, the calculating of the state value of the latest search result by a state evaluation function integrating perplexity and confidence specifically includes:

[0016] Perplexity ; To retrieve the results, Questions entered by the user, Indicates the total number of entries. Indicates the entries, Indicates the All entries before the entry; Indicates the question given the first i-1 terms and user input In the case of Middle The probability of a term appearing;

[0017] Confidence ;

[0018] State evaluation function ;in, and is a hyperparameter.

[0019] In one embodiment, the answer candidate set is managed using a first-in-first-out principle, and each retrieval result is added to the end of the answer candidate set as an observation value. The retrieval result at the end of the answer candidate set is removed during a rollback operation.

[0020] In a second aspect, the present invention provides a reflective rollback retrieval enhancement generation device, comprising:

[0021] The query rewriting module generates rewritten query statements based on the user-entered question and answer candidate set;

[0022] In the retrieval module, the large language model determines whether it is necessary to call an external knowledge retrieval tool for information retrieval based on the rewritten query statement. If so, the retrieval is performed and the retrieval results are added to the answer candidate set.

[0023] The state evaluation module calculates the state value of the latest retrieval result by integrating the state evaluation function of perplexity and confidence. The perplexity is calculated based on the logarithmic loss of the probability distribution of the retrieval result, and the confidence is quantified by the entropy of the probability distribution of the retrieval result.

[0024] The status comparison module determines whether the status value of the latest search result is lower than the status value of the previous search result in the answer candidate set. If so, the rollback mechanism is triggered to remove the latest search result in the answer candidate set.

[0025] The summary module terminates the search if the status value of the answer candidate set reaches the set threshold or the number of iterations exceeds the limit, summarizes the search results in the answer candidate set through the large language model, and outputs the final answer.

[0026] In one embodiment, generating a rewritten query statement based on the question input by the user and the set of candidate answers specifically includes:

[0027] Questions entered by the user and answer memory set , input to the query rewrite module , generate the rewritten query statement ; The query rewriting module Implementation of large language models based on fine-tuning.

[0028] In one embodiment, the calculating of the state value of the latest search result by a state evaluation function integrating perplexity and confidence specifically includes:

[0029] Perplexity ; To retrieve the results, Questions entered by the user, Indicates the total number of entries. Indicates the entries, Indicates the All entries before the entry; Indicates the question given the first i-1 terms and user input In the case of Middle The probability of a term appearing;

[0030] Confidence ;

[0031] State evaluation function ;in, and is a hyperparameter.

[0032] In one embodiment, the answer candidate set is managed using a first-in-first-out principle, and each retrieval result is added to the end of the answer candidate set as an observation value. The retrieval result at the end of the answer candidate set is removed during a rollback operation.

[0033] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method of any one embodiment of the first aspect when executing the computer program.

[0034] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any one of the embodiments of the first aspect.

[0035] Compared with the prior art, the beneficial technical effects of the present invention are:

[0036] Inspired by thought chain reasoning and memory stack management, this paper proposes a self-reflective multi-source fallback RAG method, which aims to balance the security and quality of answers. Specifically, for questions input by users, a fine-tuned large language model can be used to evaluate whether the question is answerable and whether it involves dangerous topics. If the question does not meet the requirements, the large language model refuses to answer; if it meets the requirements, it enters the reasoning phase. During the reasoning process, the present invention optimizes the question statement through a question rewriting module, converting it into a high-quality form. Next, the large language model determines whether external search is required and selects appropriate tools for information search, organizes the results, and adds them to the answer candidate set. In the generation phase, the large language model scores the search results and adjusts the answer candidate set based on the score changes. If the current score decreases compared to the previous stage, the corresponding answer is removed, and the search and optimization process continues. This process is iterated until the score of the answer candidate set reaches the preset threshold, and the highest-scoring answer is ultimately output, improving the security and accuracy of the retrieval method. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Flowchart of a method in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] A preferred embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

[0039] This paper designs a chained retrieval strategy with a rollback mechanism. If a retrieval result has a negative impact on the final answer generation, it will be removed from the candidate set. In addition, this paper introduces a query rewriting module to optimize user input queries and proposes a state evaluation function that evaluates the quality of generated content by combining context perplexity and output confidence.

[0040] Problem definition: Given a problem input by a user , a document knowledge base , and a candidate set of answers , represents the i-th term, is the total number of terms in the document knowledge base, The elements in are search strategies The search results returned during each search ,Right now , The initial setting is empty. Represents the tools available in the retrieval process (e.g., knowledge graphs, web searches, etc.).

[0041] The core of the method of the present invention is the search strategy , which is defined as:

[0042] ;

[0043] Will Input answer candidate set . The first field is the action flag. When the action flag is marked as "Summarization", the user input question and All search results in the , output the final answer:

[0044] .

[0045] like Figure 1 As shown, the present invention proposes a reflective rollback retrieval enhancement generation method, comprising the following steps:

[0046] S1, based on the question and answer candidate set input by the user, generates a rewritten query statement;

[0047] S2, based on the rewritten query statement, the large language model determines whether it is necessary to call an external knowledge retrieval tool for information retrieval: if so, the retrieval is performed and the retrieval results are added to the answer candidate set;

[0048] S3, calculates the state value of the latest retrieval result by integrating the state evaluation function of perplexity and confidence; perplexity is calculated based on the logarithmic loss of the probability distribution of the retrieval result, and confidence is quantified by the entropy of the probability distribution of the retrieval result;

[0049] S4, determine whether the status value of the latest search result is lower than the status value of the previous search result in the answer candidate set. If so, trigger the rollback mechanism and remove the latest search result in the answer candidate set;

[0050] S5, repeat steps S1 to S4. If the status value of the answer candidate set reaches the set threshold or the number of iterations exceeds the limit, the search is terminated and the search results in the answer candidate set are summarized through the large language model and the final answer is output.

[0051] In one embodiment, generating a rewritten query statement based on the question input by the user and the set of candidate answers in step S1 specifically includes:

[0052] Questions entered by the user and answer memory set , input to the query rewrite module , generate the rewritten query statement ; The query rewriting module Implementation of large language models based on fine-tuning.

[0053] Specifically, in the multi-step reasoning process, the question of each step must be determined based on the known and unknown knowledge of the large language model, which will affect the next search strategy and final result of the large language model. However, repeatedly entering the same question may lead to meaningless redundant search and may introduce unnecessary content. For a given user-entered question , the present invention hopes that the large language model can gradually optimize its answer. Therefore, the present invention proposes a query rewriting module that reconsiders the question and optimizes the search strategy at each step of reasoning.

[0054] Query Rewrite Module , which accepts user input for questions and answer memory set , combining them into new queries , is the nth search result. The goal is to convert the question entered by the user into Memory set with answers Integration enables the large language model to clearly understand what it knows and what it doesn’t know, thereby supporting the next query decision. Finally, the query rewrite module outputs the rewritten query statement .

[0055] During implementation, the present invention first uses an existing large language model to generate an annotated dataset containing approximately 1,300 user questions. A prompt template is designed to guide the large language model in generating rewritten query statements. The rewritten queries are then manually reviewed and selected for fine-tuning the large language model. The present invention then applies the Low Rank Adaptation (LoRA) fine-tuning technique to adjust the large language model parameters and learn a query rewriting method. Finally, the rewritten results are applied to the inference process, enabling the large language model to integrate current memory and optimize subsequent inference steps.

[0056] In one embodiment, the calculating of the state value of the latest search result by a state evaluation function integrating perplexity and confidence specifically includes:

[0057] Perplexity ; To retrieve the results, Questions entered by the user, Indicates the total number of entries. Indicates the entries, Indicates the All entries before the entry; Indicates the question given the first i-1 terms and user input In the case of Middle The probability of a term appearing;

[0058] Confidence ;

[0059] State evaluation function ;in, and is a hyperparameter used to control and The relative importance of the final indicator. In a preferred embodiment, and The values of are all set to 0.5.

[0060] Specifically, if the retrieval result helps answer the question, the reasoning process continues; if the result is not helpful, the current retrieval result is removed from the candidate set. The system is removed and restored to its previous state for the next round of retrieval. The present invention evaluates the retrieval results through a state evaluation function.

[0061] To more comprehensively evaluate the performance of large language models across different tasks, this paper introduces a weighted state evaluation function (perplexity-confidence fusion) for evaluating large language models in multi-task scenarios. This method not only considers the large language model's predictive power for a given context but also accounts for the uncertainty in its predictions, providing a more precise basis for selecting large language models for complex tasks.

[0062] Confidence measures the confidence of a large language model's predictions given a given input and task environment. Specifically, it quantifies uncertainty by calculating the entropy of the probability distribution of the large language model's output. Higher entropy values indicate less certainty in the large language model's predictions for the current task; lower entropy values indicate more reliable predictions and higher confidence.

[0063] Rollback: Change the current state value With the previous state value If the current status value deteriorates, it means It is not helpful to answer the question. From the answer candidate set Remove it and restore the status value to If the state value remains stable or improves, proceed to the next step of reasoning.

[0064] When the status value reaches the preset threshold When , terminate the reasoning process and sort the answer candidate set If the number of inference iterations exceeds the preset value , it means that the large language model (LLM) If a satisfactory answer cannot be provided within 10 iterations, the abstention strategy is executed. This is done to prevent the large language model from giving incorrect answers when it does not understand the question well enough.

[0065] In one embodiment, the answer candidate set is managed using a first-in-first-out principle, and each retrieval result is added to the end of the answer candidate set as an observation value. The retrieval result at the end of the answer candidate set is removed during a rollback operation.

[0066] In memory management technology, memory follows the "first in, first out" (FIFO) principle. Drawing on this method, the memory designed by the present invention for large language model retrieval, i.e., the answer candidate set , also follows this principle. Specifically, each new retrieval result added to the answer candidate set All based on The current search results are derived by reasoning based on the previous search results. After entering the answer candidate set, if its state value decreases compared to the previous state, it indicates that This does not help answer the question. In this case, a rollback will be performed. Remove it from the answer candidate set and return to the previous stage for the next round of retrieval. If the state value remains stable or improves, continue to the next step of reasoning until the final answer is obtained.

[0067] The large language model training of the present invention can be achieved through the following process:

[0068] Input is a user input question and a document repository The large language model is trained for 100 epochs, with an early stopping mechanism set at 40 epochs; the Adam optimizer is used, the learning rate is 0.01, and the batch size (batch_size) is set to 32.

[0069] In each training epoch, the following steps are run:

[0070] Enter the user's question Input the large language model to analyze the question and retrieve the results. ;

[0071] Call the status evaluation function to retrieve the results Evaluate the current state value ;

[0072] Compare current state values and , to determine whether If yes, then keep the answer candidate set If not, discard , and Set to .

[0073] When the large language model retrieval round is greater than the preset value ,or Reaching the preset threshold , end this round of training;

[0074] After the large language model training is completed, the test set is used to evaluate the large language model.

[0075] The reflective rollback retrieval-enhanced generation method proposed in this paper effectively addresses the performance degradation caused by low-quality external retrieval information in large language models by introducing self-reflection and rollback mechanisms. A state evaluation function combining contextual perplexity and confidence is proposed to ensure accurate assessment of the quality of generated answers. Furthermore, a query rewriting module enhances the relevance and accuracy of generated answers. Experimental results demonstrate that this method outperforms existing methods in both question-answering performance and security, demonstrating its advantages in knowledge-intensive tasks.

[0076] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times. The order of execution of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.

[0077] Based on the description of the above method embodiments, the present invention also provides a device. The device may be a system (including a distributed system), software (application), module, component, server, client, etc. that uses the method described in the embodiments of this specification and is combined with the necessary implementation hardware. Based on the same innovative concept, the device in one or more embodiments provided in the embodiments of the present disclosure is as described in the following embodiments. Since the implementation scheme of the device to solve the problem is similar to the method, the implementation of the specific device in the embodiments of this specification can refer to the implementation of the aforementioned method, and the repetitions will not be repeated. As used below, the term "module" or "module group" refers to a combination of software and / or hardware that can implement a predetermined function. Although the device described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and conceived.

[0078] Specifically, the present invention proposes a reflective rollback search enhancement generation device, comprising the following steps:

[0079] The query rewriting module generates rewritten query statements based on the user-entered question and answer candidate set;

[0080] In the retrieval module, the large language model determines whether it is necessary to call an external knowledge retrieval tool for information retrieval based on the rewritten query statement. If so, the retrieval is performed and the retrieval results are added to the answer candidate set.

[0081] The state evaluation module calculates the state value of the latest retrieval result by integrating the state evaluation function of perplexity and confidence. The perplexity is calculated based on the logarithmic loss of the probability distribution of the retrieval result, and the confidence is quantified by the entropy of the probability distribution of the retrieval result.

[0082] The status comparison module determines whether the status value of the latest search result is lower than the status value of the previous search result in the answer candidate set. If so, the rollback mechanism is triggered to remove the latest search result in the answer candidate set.

[0083] The summary module terminates the search if the status value of the answer candidate set reaches the set threshold or the number of iterations exceeds the limit, summarizes the search results in the answer candidate set through the large language model, and outputs the final answer.

[0084] In one embodiment, the present invention further provides a computer-readable storage medium, such as a memory, comprising instructions, wherein the instructions are executable by a processor to perform the above method. The storage medium may be a computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0085] In one embodiment, the present invention further provides a computer device, which may be a server. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data used in the above method. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the above method is implemented.

[0086] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0087] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. It is intended that all variations within the meaning and range of equivalents of the claims be embraced herein, and any reference signs in the claims should not be construed as limiting the claims to which they relate.

[0088] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A reflective rollback retrieval enhancement generation method, characterized in that: The following steps are involved: S1, based on the question and answer candidate set input by the user, generates a rewritten query statement; S2, based on the rewritten query statement, the large language model determines whether it is necessary to call an external knowledge retrieval tool for information retrieval: if so, the retrieval is performed and the retrieval results are added to the answer candidate set; S3, calculates the state value of the latest retrieval result by integrating the state evaluation function of perplexity and confidence: perplexity ; To retrieve the results, Questions entered by the user, Indicates the total number of entries. Indicates the entries, Indicates the All entries before the entry; Indicates that given the first i-1 terms and In the case of Middle The probability of a term appearing; Confidence ; State evaluation function ; and is a hyperparameter; The perplexity is calculated based on the logarithmic loss of the probability distribution of the retrieval results, and the confidence is quantified by the entropy of the probability distribution of the retrieval results; S4, determine whether the status value of the latest search result is lower than the status value of the previous search result in the answer candidate set. If so, trigger the rollback mechanism and remove the latest search result in the answer candidate set; S5, repeating steps S1 to S4. If the status value of the answer candidate set reaches the set threshold or the number of iterations exceeds the limit, the search is terminated and the search results in the answer candidate set are summarized through the large language model and the final answer is output; The answer candidate set is managed using the first-in-first-out principle. Each retrieval result is added to the end of the answer candidate set as an observation value, and the retrieval result at the end of the answer candidate set is removed during the rollback operation.

2. A reflective rollback retrieval enhancement generation method according to claim 1, characterized in that: The generating of a rewritten query statement based on the question input by the user and the candidate answer set specifically includes: Questions entered by the user and answer memory set , input to the query rewrite module , generate the rewritten query statement ; The query rewriting module Implementation of large language models based on fine-tuning.

3. A reflective rollback search enhancement generation device, characterized in that: include: The query rewriting module generates rewritten query statements based on the user-entered question and answer candidate set; In the retrieval module, the large language model determines whether it is necessary to call an external knowledge retrieval tool for information retrieval based on the rewritten query statement. If so, the retrieval is performed and the retrieval results are added to the answer candidate set. The state evaluation module calculates the state value of the latest retrieval result by integrating the state evaluation function of perplexity and confidence: perplexity ; To retrieve the results, Questions entered by the user, Indicates the total number of entries. Indicates the entries, Indicates the All entries before the entry; Indicates that given the first i-1 terms and In the case of Middle The probability of a term appearing; Confidence ; State evaluation function ; and is a hyperparameter; The perplexity is calculated based on the logarithmic loss of the probability distribution of the retrieval results, and the confidence is quantified by the entropy of the probability distribution of the retrieval results; The status comparison module determines whether the status value of the latest search result is lower than the status value of the previous search result in the answer candidate set. If so, the rollback mechanism is triggered to remove the latest search result in the answer candidate set. The summary module terminates the search if the status value of the answer candidate set reaches the set threshold or the number of iterations exceeds the limit, summarizes the search results in the answer candidate set through the large language model, and outputs the final answer; The answer candidate set is managed using the first-in-first-out principle. Each retrieval result is added to the end of the answer candidate set as an observation value, and the retrieval result at the end of the answer candidate set is removed during the rollback operation.

4. The reflective rollback search enhancement generation device according to claim 3, characterized in that: The generating of a rewritten query statement based on the question input by the user and the candidate answer set specifically includes: Questions entered by the user and answer memory set , input to the query rewrite module , generate the rewritten query statement ; The query rewriting module Implementation of large language models based on fine-tuning.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 2 are implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 2 are implemented.

Images