Self-error-correction retrieval enhancement agent system

Abstract

The invention provides a self-error-correction retrieval enhancement agent system. Firstly, error detection is carried out on RAG generation content through a trained retrieval enhancement error correction model, then error features are input into an agent model to carry out automatic planning of a solution, and finally a closed-loop process of error correction is completed through an execution engine. In the aspect of the implementation method, the scheme constructs a three-stage data mining normal form including error response sampling, key feature labeling and error type labeling, and the error recognition precision of the error correction model is effectively improved. Furthermore, a'generation-detection-correction-evaluation 'closed-loop optimization loop is formed, and the effect that the RAG capability is autonomously enhanced by utilizing the evaluation-guided proxy workflow under the condition that the RAG system does not need manual intervention is achieved.

Description

Technical Field

[0001] The present disclosure relates to the field of computers, and more particularly, to a self-correcting retrieval-augmented agent system. Background Art

[0002] Recent advancements in large language models (LLMs) have shown remarkable performance in a wide range of downstream tasks. However, LLMs are still prone to hallucinations and factual inconsistencies, which undermine the reliability of the generated responses. Retrieval-augmented generation (RAG) emerged as a promising approach to enhance the output of LLMs by integrating information from retrieval. In practical RAG applications, the lack of gold responses requires an automatic evaluation strategy. Basic research utilizes large language models as judgment tools to automatically evaluate model outputs and generate important feedback. However, due to the complexity and knowledge intensity of RAG tasks, errors in retrieved and generated content are often more subtle than in other tasks (e.g., factual inaccuracies in specific details). Therefore, relying solely on one LLM for evaluation usually cannot provide precise and reliable judgments. To address this issue, self-RAG and MetaRAG introduced dynamic self-evaluation, enabling RAG systems to continuously monitor and optimize the retrieval and generation processes. The corrective RAG framework enhances the system's robustness by evaluating and improving retrieved content. For detailed diagnosis, RAGChecker provides fine-grained evaluation at the question level, focusing on the quality of retrieval and generation. Recently, RAG-RewardBench established a comprehensive reward model benchmark to evaluate multi-hop reasoning and citation accuracy.

[0003] Although these RAG error correction methods have shown certain effectiveness, there are still several limitations:

[0004] Insufficient generalization ability: Predefined error categories and correction strategies often fail to adapt to the diversity of RAG tasks and a wide range of error patterns.

[0005] Lack of fine-grainedness and high cost: Manually designed error taxonomies are difficult to capture subtle errors in RAG outputs, and their implementation requires a large amount of computational and manual annotation costs.

[0006] Therefore, automatically constructing a well-structured and comprehensive error classification system remains a fundamental challenge for the practical deployment of RAG systems. It is worth noting that developing such a system relies on diverse error responses in different domains and styles. However, there is a lack of high-quality error annotation datasets in this field, which hinders the creation of a general error-aware model that can identify and mitigate RAG errors. Summary of the Invention

[0007] The objective of the embodiments of the present disclosure is to provide a self-correcting retrieval-augmented agent system, which solves the problem that there is a lack of high-quality error-annotated datasets in the current field, and thus there is no general error perception model that can identify and mitigate RAG errors.

[0008] In a general aspect, a self-correcting retrieval-augmented agent system is provided. For the content text generated by retrieval augmentation, we use a trained retrieval-augmented error correction model to identify errors. Then, we hand over the error identification information to an agent model for automated solution planning. Then, through a planning module, we plan the program of the solution used by the agent in reasoning. Finally, we adopt an execution module to run the solution program to achieve self-correction of the generated content. The content running detection method includes three steps:

[0009] The first step is error response sampling. We obtain a mixed dataset from the training set of a knowledge-intensive open-source dataset, set a model pool including multiple different open-source large models, and use the same sampling hyperparameters to enable each model to sample responses from the mixed RAG dataset. We use a strong supervision model as an evaluation tool to filter out error samples and provide detailed error analysis to obtain an error response set.

[0010] The second step is key annotation and marking. We construct an error reply dataset, and then obtain atomic error labels through open-set annotation for open labeling. Furthermore, we capture the intention of the atomic-level error annotation process through label normalization technology.

[0011] The third step is error label summarization. We obtain a dual summary of error labels through error clustering and large model classification. Then, through the RAG error system, we use the mapping relationship between high, medium, and low-level labels to map and obtain the RAG error dataset. The specific method of the mapping is as follows:

[0012]

[0013] Where each x contains the RAG input (q, D q ) and the model prediction p, y represents the key feedback, including 3-layer error labels and a binary error judgment label e, and k represents the number of samples.

[0014] The specific method of the open-set annotation is as follows: Design a key prompt to guide Qwen2.5-72B to analyze the error reasoning of the dataset and generate a set of parsable JSON format labels.

[0015] The specific method of the label normalization technology is as follows: Two normalization strategies are adopted: 1) Remove long-tail labels with a frequency lower than the specified threshold α, and delete atomic labels with a length exceeding 25 tokens; 2) Filter out empty responses that do not conform to the JSON format.

[0016] The specific implementation method of the planning module is as follows: Given a test set (q, D q ), first use the RAG generator to generate a prediction result p, use the aligned RAG error evaluation model to generate an error evaluation feedback y, and use the same input test set (q, D q ), evaluation feedback y, predefined mapping table T, and action function F. Under the guidance of the evaluation signal, the planning model autonomously selects and sorts the necessary solution actions, generates a planning program for each function, and skips this process if the judgment of the evaluation signal is correct.

[0017] The alignment method of the aligned RAG error evaluation model is as follows:

[0018] Design two step-by-step alignment training objectives: supervised fine-tuning step and coarse-to-fine DPO alignment:

[0019] The supervised fine-tuning step first constructs an SFT dataset and applies the standard supervised fine-tuning objective to the base model:

[0020]

[0021] where L(θ) represents the loss function, D train represents the training set, (x i , y i ) represents the input and output, and the final output will follow the JSON format, including a binary error judgment and 3-layer error labels;

[0022] The coarse-to-fine DPO alignment step randomly selects k response samples from the correct and wrong pools as negative examples respectively. The first group helps the large model learn the rough difference between the correct response and the wrong response, and the second group captures the subtle differences between various wrong responses. Finally, these two groups of negative samples are merged to form a pairwise preference set, and then the coarse-to-fine alignment is achieved according to direct preference optimization:

[0023]

[0024] where the reference model π ref is initially set to π θ and remains unchanged. The hyperparameters β and the sigmoid function are used by σ, and L SDPO aims to maximize the log probability of preference y + relative to the non-preferred y - .

[0025] The technical effects to be achieved by the embodiments of the present invention are as follows:

[0026] The present invention proposes a novel framework that utilizes an evaluation-guided proxy workflow to autonomously enhance the RAG (Retrieval-Augmented Generation) ability. First, a data-driven error mining pipeline is designed to establish a hierarchical RAG error system. Using this system, an error evaluation model with coarse-to-fine training objectives is gradually aligned to achieve the automation of fine-grained error feedback. Then, an evaluation-guided proxy workflow is introduced, which promotes error-driven correction by autonomously customizing the executor-based solution process according to the error feedback. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and other objects and features of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0028] Figure 1 is a schematic diagram showing the architecture of a self-correcting retrieval-augmented agent system according to an embodiment of the present disclosure;

[0029] Figure 2 is a schematic diagram showing an overview of the workflow of a self-correcting RAG agent according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] The following detailed description is provided to assist the reader in obtaining a comprehensive understanding of the methods, devices, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, devices, and / or systems described herein will be apparent after understanding the disclosure of the present application. For example, the order of operations described herein is merely illustrative and is not limited to those set forth herein, but may be changed as will be apparent after understanding the disclosure of the present application, except for operations that must occur in a specific order. In addition, descriptions of features known in the art may be omitted for greater clarity and conciseness.

[0031] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Instead, the examples described herein are provided only to illustrate some of the many possible ways of implementing the methods, devices, and / or systems described herein, which will be apparent after understanding the disclosure of the present application.

[0032] As used herein, the term "and / or" includes any one of the associated listed items and any combination of any two or more thereof.

[0033] Although terms such as "first", "second", and "third" may be used herein to describe various components, elements, regions, layers, or sections, these components, elements, regions, layers, or sections should not be limited by these terms. Instead, these terms are only used to distinguish one component, element, region, layer, or section from another. Thus, the first component, first element, first region, first layer, or first section referred to in the examples described herein may also be referred to as the second component, second element, second region, second layer, or second section without departing from the teachings of the examples.

[0034] In the specification, when an element (such as a layer, region, or substrate) is described as "on" another element, "connected to" or "coupled to" another element, the element may be directly "on" the other element, directly "connected to" or "coupled to" the other element, or there may be one or more other elements intervening therebetween. In contrast, when an element is described as "directly on" another element, "directly connected to" or "directly coupled to" another element, there may be no other elements intervening therebetween.

[0035] The terms used herein are for the purpose of describing various examples only and are not intended to limit the disclosure. Unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. The terms "comprising", "including", and "having" specify the presence of the stated features, quantities, operations, components, elements, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0036] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains after understanding this disclosure. Unless explicitly defined as such herein, terms (such as those defined in a general dictionary) should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and should not be interpreted in an idealized or overly formal manner.

[0037] Furthermore, in the description of the examples, when a detailed description of related structures or functions that are considered to be well-known would cause an ambiguous interpretation of the disclosure, such detailed descriptions will be omitted.

[0038] Figure 1 is a schematic diagram showing a self-correcting retrieval-augmented agent system according to an embodiment of the present disclosure.

[0039] The present invention realizes content monitoring through a self-correcting retrieval-augmented agent (RAG-Critic) framework, enhancing the general RAG ability of LLMs through critical feedback.

[0040] Build RAG-Critic from three aspects: 1) Design a three-step process for error response mining and annotation, and establish a hierarchical RAG error classification system. 2) Utilize a high-quality error system to further align the RAG error evaluation model through a coarse-to-fine training objective. 3) Introduce a self-correcting agent framework to promote the error-driven correction process by autonomously customizing and designing an executor-based solution process according to error feedback.

[0041] Step 1: Error response sampling:

[0042] To achieve general RAG error recognition, the present invention first needs to construct a comprehensive and diverse error response set. Therefore, the present invention considers two aspects: data source and model sampling.

[0043] Data source: To balance data scale and diversity while maintaining accessibility, the present invention obtains a mixed dataset from the training sets of 9 knowledge-intensive open-source datasets. As shown in Table 1, the datasets of the present invention cover 6 task paradigms. Subsequently, the present invention retrieves relevant external knowledge for the datasets. Given a task query, the present invention uses a dense retriever to retrieve the top-K relevant paragraphs from the Wikipedia corpus.

[0044]

[0045] Table 1 Nine knowledge-intensive open-source datasets

[0046] Model sampling: To mitigate the inherent bias in responses from the same LLM, the present invention selects a diverse pool of 15 open-source models from 9 series, with parameter sizes ranging from 3B to 70B. As shown in Table 1=2, the model pool of the present invention uses the same sampling hyperparameters, enabling each model to sample responses from the mixed RAG dataset.

[0047]

[0048] Table 2 Sampling model pool display (3B - 70B)

[0049] Considering that rule-based metrics cannot evaluate the correctness of responses at a high-level semantics, the present invention uses a strong supervision model Qwen2.5 - 72B as an evaluation tool to filter out error samples and provide detailed error analysis. Then, the present invention obtains a comprehensive and diverse error response set, establishing a solid foundation for subsequent RAG error alignment.

[0050] Step 2: Key annotation and labels

[0051] In actual RAG scenarios, response errors often exhibit multifaceted and meticulous characteristics. Therefore, accurately distinguishing the subtle differences between error responses requires capturing the intent of the atomic-level error annotation process. Inspired by data selection work, the present invention utilizes open-set annotation and normalization techniques to address this challenge.

[0052] Open-set annotation: The goal of the present invention is to generate diverse and meaningful labels for identified errors. Different from previous evaluation-based RAG work, the present invention does not provide predefined labels during the annotation process but adopts open-set annotation. This choice allows for more flexible coverage of the diverse errors existing in open-domain RAG tasks. Specifically, the present invention designs a key prompt to guide Qwen2.5-72B to analyze the error reasoning of the dataset and generate a set of parsable JSON-format labels. It is worth noting that each question is assigned multiple open labels, ultimately resulting in more than 20,000 atomic error labels.

[0053] Label normalization: Importantly, the present invention observes significant noise in the original atomic labels, mainly due to the long-tail effect and instruction-following problems. To ensure its high quality and relevance, the present invention implements two normalization strategies: 1) removing long-tail labels with frequencies below the specified threshold α and deleting atomic labels with lengths exceeding 25 tokens; 2) filtering out empty responses that do not conform to the JSON format. During this denoising process, the present invention successfully obtains 4,000 atomic labels, providing the underlying error classification for the RAG error system of the present invention.

[0054] The third step: Error label aggregation

[0055] To hierarchically classify the atomic error label set, the present invention adopts a data-driven automated annotation mechanism as the basis and supplements it with high-level manual aggregation, thus achieving an efficient annotation process while minimizing human intervention.

[0056] Clustering and LLM classification: Given the atomic error label set, the present invention applies hierarchical clustering to obtain 20 class centers. Then, the present invention traces the sample set covered by each cluster and randomly selects 50 labels. Using GPT-4o as the supervised model, the present invention summarizes the central error types of each cluster, ultimately obtaining 20 secondary error types.

[0057] Manual aggregation: Relying solely on the automated aggregation of LLM may lead to mechanical text and biases. For this reason, the present invention hires three well-educated annotators to be responsible for classifying the 20 secondary labels and summarizing the top-level error types. Subsequently, the annotators conduct cross-validation and discussion, ultimately forming a comprehensive hierarchical error system, including 7 top-level labels, 19 secondary labels, and more than 4,000 tertiary labels, thereby enhancing the fine-grainedness of error classification. AsFigure 1 As shown, there is a mapping relationship between high-level and low-level tags in the hierarchical error system of the present invention. Finally, the present invention reverse-maps the three-layer tags according to the error system to annotate the error dataset and synthesize the first fine-grained error recognition. In order to establish a QA dataset in the RAG field The present invention is defined as follows:

[0058]

[0059] where each x contains the RAG input (q, D q ) and the model prediction p. y represents the key feedback, including three-layer error tags and a binary error judgment tag e. k represents the number of samples.

[0060] RAG error evaluation alignment

[0061] After thoroughly analyzing the RAG error system, the direct goal of the present invention is to refine its knowledge into an evaluation model for automatic error annotation. It should be noted that the present invention naturally generates many positive and negative response samples during the error evaluation process in the second stage, enabling the present invention to design two gradually aligned training objectives:

[0062] Supervised fine-tuning (SFT): To maintain a balance between error responses and correct responses, the present invention first randomly selects the same number of correct samples as in the error response set to construct the SFT dataset. The present invention applies the standard supervised fine-tuning objective to the base model:

[0063]

[0064] where L(θ) represents the loss function, D train represents the training set, and (x i , y i ) represents the input and output.

[0065] To simulate the real-world RAG scenario, the input of the present invention does not contain the golden answer. Finally, the final output of the error evaluation model of the present invention will follow the JSON format, including a binary error judgment and three-layer error tags.

[0066] Coarse-to-Fine DPO Alignment: To achieve excellent error alignment, an ideal RAG error evaluation model should possess two key capabilities: 1) being able to roughly distinguish correct and incorrect responses, and 2) being able to meticulously label three layers of error tags from the error system. To unleash the potential of the LLM, the present invention randomly selects k response samples from both the correct and error pools as negative examples. The first group helps the LLM learn the rough difference between correct and incorrect responses, while the second group captures the subtle differences among various incorrect responses. Finally, the present invention combines these two groups of negative samples to form a pairwise preference set. After that, the present invention realizes coarse-to-fine alignment according to Direct Preference Optimization (DPO):

[0067]

[0068] where the reference model π ref is initially set to π θ and remains unchanged. The hyperparameters β and the sigmoid function are used by σ.

[0069] L SDPO is aimed at maximizing the log probability of preference y + with respect to the non-preferred y - .

[0070] Self-Correcting RAG Agent Framework:

[0071] As Figure 2 , the ultimate goal of the present invention is to improve the RAG performance of the LLM by leveraging the feedback of the error evaluation model. To this end, the present invention proposes "error-action mapping" and "evaluation-guided agent workflow", integrating offline and online strategies for error-driven correction.

[0072] Error-Action Mapping:

[0073] Based on the error system, it is crucial to formulate corresponding solutions for each error type. The present invention uses GPT-4o to summarize the offline solutions for layer-one and layer-two errors, and then performs manual optimization to create an error-action mapping table. This table serves as a guide for policy planning.

[0074]

[0075] Table 3 Error-Action Mapping Table.

[0076] Self-Correcting Agent Workflow:

[0077] As Figure 2 , the present invention designs a "generate-evaluate-plan-execute" workflow to facilitate automated problem-solving.

[0078] Action Design: Guided by evaluation feedback and error-action mapping, the present invention aims to automate the planning and execution of solutions. Drawing on the work of code execution, the present invention decomposes each solution path into several sub-actions, each sub-action corresponding to a detailed problem-solving strategy (e.g., re-retrieval, specific document deletion). As shown in Table 4, the present invention further defines a set of action functions with 5 different functions, implementing more than 15 fine-grained sub-actions according to different inputs.

[0079]

[0080] Table 4 Error-Action Mapping Table.

[0081] Evaluation-Guided Planning: In real-world scenarios, relying solely on predefined offline solutions often fails to flexibly address the diverse challenges of RAG tasks. To this end, the present invention further introduces a planning agent for automated solution planning in reasoning. Specifically, given a test set (q, D q ), the present invention first uses the RAG generator to generate a prediction result p. In addition, the present invention uses an aligned RAG error evaluation model to generate error evaluation feedback y. Using the same input (q, D q ), evaluation feedback y, predefined mapping table T, and action function F, the planning model autonomously selects and orders the necessary solution actions under the guidance of the evaluation signal to generate a planning program for each function. This process can be expressed as:

[0082]

[0083] It should be noted that if the judgment of the evaluation signal is correct, the present invention will skip this process.

[0084] Execution-Based Correction: Once the planning program for correction is generated, the present invention uses the Python execution environment to sequentially execute these action functions to generate a corrected answer. In particular, the model optimization and reasoning processes required by the program will utilize the original RAG model to achieve automated self-correction.

[0085] Although some embodiments of the present disclosure have been shown and described, those skilled in the art should understand that these embodiments can be modified without departing from the principles and spirit of the present disclosure as defined by the claims and their equivalents.

Claims

1. A self-correcting retrieval-augmented agent system, characterized in that, For the content text of retrieval-augmented generation, we use the trained retrieval-augmented error correction model to identify errors. Then, we hand over the error identification information to the proxy model for automated solution planning. After that, we plan the solution program used by the proxy in reasoning through the planning module. Finally, we adopt the execution module to run the solution program to achieve self-correction of the generated content. Specifically, the system implementation includes three steps: The first step is error response sampling. We obtain a mixed dataset from the training set of the knowledge-intensive open-source dataset, set a model pool including multiple different open-source large models, and use the same sampling hyperparameters to enable each model to sample responses from the mixed RAG dataset. We use a strong supervision model as an evaluation tool to filter out error samples and provide detailed error analysis to obtain an error response set. The second step is key annotation and marking. We construct an error reply dataset, and then obtain atomic error labels through open-set annotation for open annotation. Furthermore, we capture the intention of the atomic-level error annotation process through label normalization technology. Step 3 is the summary of error labels. Through error clustering and large model classification, a dual summary of error labels is obtained. Then, through the RAG error system, a RAG error dataset is obtained by mapping using the mapping relationship between high, medium, and low-level labels. The specific method of the mapping is as follows: where each x contains the RAG input (q, D q ) and the model prediction p, y represents the key feedback, including three layers of error labels and a binary error judgment label e, k represents the number of samples.

2. The self-correcting retrieval augmented agent system according to claim 1, wherein The specific method of the open-set annotation is as follows: Design a key prompt to guide Qwen2.5-72B to analyze the error reasoning of the dataset and generate a set of parsable JSON-format labels.

3. A self-correcting retrieval-enhanced intelligent agent system according to claim 2, characterized in that, The specific method of the label normalization technology is as follows: Adopt two normalization strategies: 1) Remove long-tail labels with frequencies lower than the specified threshold α, and delete atomic labels with lengths exceeding 25 tokens; 2) Filter out empty responses that do not conform to the JSON format.

4. A self-correcting retrieval-enhanced intelligent agent system according to claim 3, characterized in that, The specific implementation method of the planning module is as follows: Given a test set (q, D q ), first, use the RAG generator to generate a prediction result p, use the aligned RAG error evaluation model to generate an error evaluation feedback y, and use the same input test set (q, D q ), evaluation feedback y, predefined mapping table T, and action function F. Under the guidance of the evaluation signal, the planning model autonomously selects and ranks necessary solution actions, generates a planning program for each function, and skips this process if the judgment of the evaluation signal is correct.

5. The self-correcting retrieval augmented agent system according to claim 4, wherein The alignment method of the aligned RAG error evaluation model is as follows: Design two gradually aligned training objectives: supervised fine-tuning step and coarse-to-fine DPO alignment. The supervised fine-tuning step first constructs an SFT dataset and applies the standard supervised fine-tuning objective to the base model. Among them, L(θ) represents the loss function, and D train represents the training set, (x i , y i ) represents the input and output. The final output will follow the JSON format, including a binary error judgment and three-layer error labels; The coarse-to-fine DPO alignment step randomly selects k response samples from the correct and error pools as negative examples respectively. The first group helps the large model learn the rough difference between the correct response and the error response, and the second group captures the subtle differences between various error responses. Finally, these two groups of negative samples are merged to form a pairwise preference set, and then the coarse-to-fine alignment is achieved according to direct preference optimization. where the reference model π ref is initially set to π θ and remains unchanged, the hyperparameters β and the sigmoid function are used by σ, L SDPO aims to maximize the log probability of preference y + with respect to non-preference y - .

Images