Rag reference verification method and system based on leave-one-out ablation and large language model

Abstract

This invention discloses a method and system for RAG citation verification based on Leave-One-Out ablation and a large language model. The method includes: generating initial answers for all N retrieved documents using a large language model and annotating their sources; performing ablation testing based on Leave-One-Out to obtain N ablation answers; quantifying the impact of each missing document on the answer content through multiple ablation generation processes; comparing the initial answers with each ablation answer to calculate the content difference; receiving the difference analysis results and using another large language model or different instances of the same large language model to perform causal analysis and judgment on each removed document; filtering and marking citations in the initial answers and generating a final credible answer. This method effectively solves the technical problems of low citation verification accuracy and poor interpretability in existing RAG systems, providing a reliable technical means for building a highly credible retrieval enhancement generation system, and has significant practical application value.

Description

Technical Field

[0001] This invention relates to the field of retrieval enhancement generation evaluation technology, and in particular to a method and system for RAG citation verification based on leave-one-out ablation and large language models. Background Technology

[0002] With the rapid development of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) technology has become a mainstream method for improving the accuracy and factuality of LLM responses. However, in practical applications, RAG systems commonly suffer from citation hallucination, where the cited sources in the model-generated answers do not match the actual content of the retrieved documents, or the model fabricates citations without real support.

[0003] In existing technologies, the following methods are mainly used for reference verification: The first type is based on prompt engineering, which guides the model to generate citation-based answers by requiring the model to annotate source references (e.g., [Source 1], [Source 2]) in the prompts. This method is simple to implement, but it heavily relies on the model's ability to follow instructions and is prone to "false annotations," where the generated content does not actually originate from the annotated reference documents.

[0004] The second category is similarity-based matching methods, including fuzzy matching and vector semantic similarity calculation. These methods verify the authenticity of citations by calculating the textual similarity or cosine similarity of embedded vectors between the generated text and the retrieved document blocks. However, due to the powerful rewriting and generation capabilities of large language models, even the illusory content generated by the model itself often maintains a high semantic similarity to the original document, leading to a high false positive rate and an inability to effectively distinguish between "real citations" and "seemingly similar illusory citations."

[0005] The third type is based on self-consistency checks. This method generates answers by sampling the same question multiple times, and takes the consistent content from the multiple generation as the reliable part. This method can reduce illusions to some extent, but it mainly targets the overall consistency of the answer. It has limited ability to verify the authenticity of individual references and has high computational cost.

[0006] The common drawback of existing technologies is the lack of accurate means to assess the true contribution of the retrieved documents. It is difficult to effectively determine whether each cited document truly supports the specific facts in the generated answer, resulting in insufficient reliability of the final output citations and easy misleading of users. This poses a significant application risk, especially in fields such as law, finance, government reports, and medicine, where the accuracy of facts is extremely important.

[0007] Therefore, there is an urgent need for a technical solution that can accurately assess the contribution of retrieved documents to the generated answers and effectively distinguish between genuine and illusory citations, so as to improve the credibility and practical value of the RAG system. Summary of the Invention

[0008] To address the aforementioned technical problems of severe RAG citation illusion and difficulty in accurately verifying citation authenticity, this invention provides a RAG citation verification method and system based on leave-one-out ablation and a large language model.

[0009] This invention is achieved using the following technical solution: The first aspect is the RAG citation verification system based on leave-one-out ablation and a large language model, which includes the following modules: The retrieval module retrieves relevant documents from the knowledge base based on the user's query and outputs the Top-N documents as candidates. Top-N means that from all candidate results, the top N most relevant documents are selected according to similarity, relevance and score from high to low. Initial answer generation module: Based on a structured Prompt containing citation annotation requirements, it uses a large language model to generate initial answers for all N retrieved documents, and requires the model to annotate the sources of key facts; wherein, the key facts include all objective evidence from the documents in the initial answer; Leave-One-Out ablation module: Perform the Leave-One-Out operation on all retrieved documents in sequence, removing one document at a time, and regenerate the answer based on the remaining N-1 documents, generating a total of N ablation answers; Difference Analysis Module: Compares the ablation differences between the initial answer and each ablation answer to quantify the contribution of each document to the answer; The LLM Judge verification module introduces another large language model or different instances of the same large language model as the Judge, performs semantic causal analysis on the ablation differences of each document, determines the reason for the decrease in answer quality after removing the document, and verifies the authenticity and contribution of RAG citations. Reference filtering and output module: Based on the verification results of the LLM Judge verification module, the RAG references in the initial answer are filtered, marked or corrected, and the final output is an answer with high reference reliability.

[0010] Specifically, the retrieval module is built on the principle of hybrid retrieval, configured with multiple retrieval methods, and selects the Top-N candidate document blocks with the highest relevance to the user's query from the knowledge base for output, providing basic data for subsequent answer generation.

[0011] Specifically, the retrieval methods include keyword matching, vector semantic similarity calculation, and rule filtering.

[0012] Specifically, the difference analysis module is based on multi-dimensional text comparison analysis. It compares the initial answer with the ablation answer generated after each removal of a single document, and analyzes and quantifies the difference features at the sentence level, key fact level and semantic level, respectively, to provide structured difference data support for the causal analysis of the LLM Judge verification module.

[0013] Specifically, the differences include missing facts, logical changes, and differences in expression.

[0014] On the other hand, the RAG reference verification method based on leave-one-out ablation and large language models includes the following steps: Step S1: After the user initiates a query, the top-N candidate document blocks are retrieved. The large language model is then called to generate an initial answer for all N retrieved documents and to annotate the source. Step S2: Perform ablation testing based on the Leave-One-Out method, removing one document at a time and regenerating N ablation answers using the remaining documents; quantify the impact of missing documents on the answer content through multiple ablation generation processes; Step S3: Compare the initial answer with each ablation answer and calculate the content difference. Step S4: Receive the difference analysis results, call another large language model or different instances of the same large language model to perform causal analysis and judgment on each removed document; Step S5: Verify the conclusion, filter and mark the references in the initial answer, and generate the final credible answer.

[0015] Specifically, step S4, causal analysis and judgment, includes: determining which key facts in the answer are lost or significantly degraded in quality after the document is removed, and whether the document supports the corresponding citations in the initial answer.

[0016] The beneficial effects of this invention are as follows: (1) Significantly improved verification accuracy: By combining Leave-One-Out ablation with LLM Judge, the accuracy of the present invention in verifying the authenticity of references is significantly better than the existing methods based on vector similarity matching and Prompt engineering, which effectively reduces the false references generated by the model and improves the credibility of the RAG system output.

[0017] (2) High interpretability: This solution can not only determine whether a reference is reliable, but also clearly give the "specific reasons for the decline in answer quality after removing the document" through LLM Judge, providing users with clear reference basis and traceable path, which significantly improves the credibility and transparency of the answer.

[0018] (3) Wide applicability: This solution is well adapted to search document sets of different sizes, and is especially suitable for professional fields such as government reports, legal documents, medical consultations, and financial analysis that require high reliability of citation.

[0019] (4) Balanced performance and accuracy: The two-stage verification strategy of "ablation first and judge later" avoids the high computational cost of simply generating multiple times and has higher verification reliability than single Prompt + similarity matching, achieving a good balance between accuracy and efficiency.

[0020] (5) The system is robust: Even when there is noise in the retrieved documents or some documents are highly similar in content, this scheme can still effectively distinguish the true contribution of each document and reduce the probability of misjudgment caused by semantic confusion.

[0021] In summary, this solution effectively solves the technical problems of low citation verification accuracy and poor interpretability in existing RAG systems, providing a reliable technical means for building a highly trustworthy retrieval enhancement generation system, and has significant practical application value. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0023] Figure 1 This is a flowchart of the RAG reference verification method based on leave-one-out ablation and large language model in an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0025] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0026] The following is in conjunction with the appendix Figure 1 The following describes some embodiments of the present invention in detail. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0027] This invention proposes a RAG citation verification method and system based on leave-one-out ablation and large language models. In a preferred embodiment, the system includes: The retrieval module retrieves relevant documents from the knowledge base based on the user's query and outputs a Top-N list of documents (Top-N represents the N most relevant documents selected from all candidate results, ranked from highest to lowest based on similarity, relevance, and score). This module is built on a hybrid retrieval principle, configuring multiple retrieval methods to efficiently filter the Top-N candidate documents with the highest relevance to the user's query from the knowledge base, providing foundational data for subsequent answer generation. Retrieval methods include keyword matching, vector semantic similarity calculation, and rule filtering. Initial answer generation module: Based on the structured Prompt containing citation annotation requirements, it uses a large language model to generate initial answers for all N retrieved documents, and requires the model to annotate the sources of key facts. Key facts include all objective evidence from the documents in the initial answer. Leave-One-Out ablation module: For each retrieved document, the Leave-One-Out operation is performed sequentially, removing one document at a time. Based on the remaining N-1 documents, the answer is regenerated, resulting in a total of N ablation answers. The difference analysis module compares the ablation differences between the initial answer and each ablation answer, quantifying the contribution of each document to the answer. Specifically, based on multi-dimensional text comparison analysis, this module compares the initial answer with each ablation answer generated after removing a single document, analyzing and quantifying difference features at the sentence, key fact, and semantic levels. This provides structured difference data support for the causal analysis in the LLM Judge validation module. Difference features include missing facts, logical changes, and expression differences. The LLM Judge verification module introduces another large language model or different instances of the same large language model as the Judge, performs semantic causal analysis on the ablation differences of each document, determines the reason for the decrease in answer quality after removing the document, and verifies the authenticity and contribution of RAG citations. Reference filtering and output module: Based on the verification results of the LLM Judge verification module, the RAG references in the initial answer are filtered, marked or corrected, and the final output is an answer with high reference reliability.

[0028] In one specific embodiment, the principles of each module of the system are as follows: The retrieval module is built on the principle of hybrid retrieval. This module integrates multiple retrieval technologies such as keyword matching, vector semantic similarity calculation, and rule filtering to efficiently filter the Top-N candidate document blocks with the highest relevance to the user's query from the knowledge base, providing basic data for subsequent answer generation.

[0029] The initial answer generation module is built based on prompt engineering and structured generation principles. By designing a structured prompt that includes citation annotation requirements, it guides the large language model to generate initial answers using all N retrieved documents, and forces the model to cite key facts (such as [Source 1], [Source 2]), providing a benchmark for subsequent validation.

[0030] Leave-One-Out ablation module: This module systematically observes the impact of missing documents on the answer by sequentially removing individual search documents and regenerating the answer based on the remaining N-1 documents, resulting in a total of N ablation answers.

[0031] The difference analysis module is built on the principle of multi-dimensional text comparison analysis. This module compares the initial answer with the ablation answer generated after removing a single document one by one, analyzing the differences at the sentence level, key fact level, and semantic level, including factual omissions, logical changes, and expression differences, and performing preliminary quantification to provide structured difference data support for subsequent causal analysis in LLMJudge.

[0032] The LLM Judge verification module (core innovation module) is built upon the principles of LLM-as-Judge (large language model as judge) and causal reasoning. This module utilizes another large language model (distinct from the model that generates the answer) as the judge to perform deep causal analysis at the semantic level on ablation discrepancies, determining "the specific reasons for the decrease in answer quality after removing the document," thereby accurately evaluating the true contribution and citation validity of each document to the answer. This is the key innovation that distinguishes this invention from existing technologies.

[0033] The citation filtering and output module is built on a rule-driven filtering mechanism and a confidence threshold decision principle. Based on the validation results of the LLM Judge, this module filters, marks (e.g., "verified" or "unverified"), or corrects citations in the initial answer, ultimately outputting an answer with high credibility and interpretability.

[0034] This solution also proposes a RAG reference verification method based on leave-one-out ablation and a large language model, including the following steps: Step S1: After the user initiates a query, the top-N candidate document blocks are retrieved. The large language model is then called to generate an initial answer for all N retrieved documents and to annotate the source. Step S2: Perform ablation testing based on the Leave-One-Out method, removing one document at a time and regenerating N ablation answers using the remaining documents; quantify the impact of missing documents on the answer content through multiple ablation generation processes; Step S3: Compare the initial answer with each ablation answer and calculate the content difference. Step S4: Receive the difference analysis results, call another large language model or different instances of the same large language model to perform causal analysis and judgment on each removed document; causal analysis and judgment include: judging which key facts in the answer are lost or significantly degraded in quality after the document is removed, and whether the document truly supports the corresponding citation in the initial answer; Step S5: Verify the conclusion, filter and mark the references in the initial answer, and generate the final credible answer.

[0035] In one specific embodiment, the core processing flow of this solution is as follows: Figure 1 As shown, it includes: Step 1 (Initial Generation): After the retrieval module obtains N relevant documents, the initial answer generation module calls the large language model to generate an initial answer containing citation annotations. This step completes the functions of fact extraction and preliminary citation annotation, providing a baseline answer for subsequent verification.

[0036] Step 2 (Leave-One-Out Ablation): The Leave-One-Out ablation module removes one document at a time and regenerates the answer using the remaining documents. Through multiple ablation iterations, the impact of each missing document on the answer content is quantified. This step aims to initially identify the contribution sensitivity of each document, providing a data foundation for subsequent fine-grained validation.

[0037] Step 3 (Difference Analysis): The difference analysis module compares the initial answer with each ablation answer and calculates the content difference (which may include indicators such as missing facts, logical changes, and expression differences). This step provides the LLM Judge with quantifiable difference characteristics.

[0038] Step 4 (LLM Judge Verification): The LLM Judge verification module receives the discrepancy analysis results and performs causal judgment on each removed document. Specifically, it determines "which key facts in the answer are lost or significantly degraded in quality after removing the document, and whether the document truly supports the corresponding citation in the initial answer." This module is the core innovation of this invention, effectively distinguishing between "true citations" and "pseudo-citations that appear similar but are actually model illusions," significantly improving the accuracy and reliability of citation verification.

[0039] Step 5 (Result Filtering and Output): The citation filtering and output module filters and marks (e.g., marked as "verified" or "unverified") citations in the initial answer based on the verification conclusion of the LLM Judge, and generates the final credible answer. This step ensures that the output answer has high interpretability and factual reliability.

[0040] Through the modules and processing flow described above, this solution achieves a complete closed loop from "preliminary citation annotation" to "accurate contribution assessment" and then to "reliable citation output," effectively solving the technical problems of difficulty in detecting citation illusion and low accuracy in verifying citation authenticity in existing technologies.

[0041] In this embodiment, the difference between the initial answer and each ablation answer is compared to quantify the contribution of each document to the answer. This is achieved by combining Leave-One-Out ablation, quantitative difference analysis, and LLM Judge qualitative verification, including: First, generate an initial answer using all retrieved documents; For each retrieved document i (i=1, ..., N), remove the document and regenerate the ablation answer using only the remaining N-1 documents; Compare the initial answer with each ablation answer in terms of content, structure, and factual aspects; The contribution of each document is calculated using multi-dimensional quantitative indicators; Finally, the quantification results are input into the LLM Judge module for qualitative analysis at the causal level to determine whether the document truly supports the corresponding reference in the initial answer.

[0042] This part of the technical design uses a combination of quantitative and qualitative methods to achieve an accurate assessment of the contribution of each document.

[0043] In this embodiment, the calculation of "content difference" adopts an LLM Judge-driven causal difference analysis method, which uses a large language model for semantic understanding and causal reasoning capabilities to achieve dynamic evaluation. The difference analysis module first performs a preliminary comparison between the initial answer and each ablation answer, extracting differences at the content level (including missing key facts, logical changes, and differences in expression). Subsequently, this difference information, along with the initial answer and the ablation answer, is input into the LLMJudge verification module.

[0044] The LLM Judge module uses a structured Prompt for analysis and judgment. Based on this Prompt, the LLM Judge outputs the contribution level (high / medium / low), confidence level, and a description of the reasons for the discrepancies for each document. This approach leverages the causal analysis capabilities of LLM to achieve intelligent and dynamic evaluation of content discrepancies. This method can more accurately distinguish between "semantically similar but actually unsupported illusory content" and "facts truly supported by the retrieved documents," significantly improving the reliability and interpretability of citation verification.

[0045] Here is an example of a dedicated prompt for the LLM Judge: PythonLLM_JUDGE_PROMPT="""You are a rigorous citation authenticity assessment expert (LLMJudge).

[0046] Initial answer (generated using all documents): {full_answer} The ablation solution after removing the {doc_index}th document: {leave_one_answer} Please compare the two answers above and complete the following judgment task: 1. What specific changes occurred in the answer after removing this document? 2. Were these changes caused by missing key facts, logical breaks, or loss of important information? 3. To what extent does this document truly contribute to the initial answer? Please strictly adhere to the following JSON format when outputting; do not add any explanations: { "contribution_level":"high / medium / low", "confidence":85, "key_changes":["List the main changes, as specifically as possible"], "is_real_support":true / false, "reason": "A clear, concise explanation of whether the document truly supports the corresponding citation in the initial answer, and the basis for that judgment." } """ Example of use (after filling): Example of input given to Judge: full_answer: Initial complete answer leave_one_answer: The answer after removing the third document. doc_index: 3.

[0047] Existing retrieval augmentation (RAG) systems suffer from severe citation illusion and low accuracy in citation authenticity verification, including: While the suggestion engineering approach can require the model to label the source of the references, the model is prone to "false labeling," that is, the references labeled in the generated answer do not match the actual content of the searched document, which can mislead users. Verification methods based on text similarity or vector semantic similarity are prone to errors because large language models have powerful rewriting capabilities. Even the illusory content generated by the model itself is often highly similar to the original document in semantics, making it difficult to effectively distinguish between real and fake citations, resulting in a high false judgment rate. Existing self-consistency checking methods mainly focus on the overall consistency of the answer, lacking a precise assessment of the true contribution of individual citations, and are unable to determine from a causal perspective whether a searched document truly supports the specific facts in the answer; The methods described above generally suffer from low verification reliability and poor interpretability. Their application carries significant risks in fields such as law, finance, government reporting, and healthcare, where the accuracy and traceability of facts are extremely important.

[0048] This invention proposes a reference verification method based on Leave-One-Out ablation and LLM Judge, which effectively overcomes the shortcomings of the prior art. It accurately evaluates the true contribution of each retrieved document to the generated answer from the perspective of causal relationship, and significantly improves the reliability and interpretability of references in the RAG system.

[0049] For the foregoing embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0050] The above embodiments describe the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Modifications and variations made by those skilled in the art without departing from the spirit and scope of the invention should be within the protection scope of the appended claims.

Claims

1. A RAG citation verification system based on leave-one-out ablation and large language model, characterized in that, Includes the following modules: The retrieval module retrieves relevant documents from the knowledge base based on the user's query and outputs the Top-N documents as candidates. Top-N means that from all candidate results, the top N most relevant documents are selected according to similarity, relevance and score from high to low. Initial answer generation module: Based on a structured Prompt containing citation annotation requirements, it uses a large language model to generate initial answers for all N retrieved documents, and requires the model to annotate the sources of key facts; wherein, the key facts include all objective evidence from the documents in the initial answer; Leave-One-Out ablation module: Perform the Leave-One-Out operation on all retrieved documents in sequence, removing one document at a time, and regenerate the answer based on the remaining N-1 documents, generating a total of N ablation answers; Difference Analysis Module: Compares the ablation differences between the initial answer and each ablation answer to quantify the contribution of each document to the answer; The LLM Judge verification module introduces another large language model or different instances of the same large language model as the Judge, performs semantic causal analysis on the ablation differences of each document, determines the reason for the decrease in answer quality after removing the document, and verifies the authenticity and contribution of RAG citations. Reference filtering and output module: Based on the verification results of the LLM Judge verification module, the module filters, marks, or corrects the RAG references in the initial answer and outputs the answer.

2. The RAG citation verification system based on leave-one-out ablation and large language model as described in claim 1, characterized in that, The retrieval module is built on the principle of hybrid retrieval, configured with multiple retrieval methods, and selects the Top-N candidate document blocks with the highest relevance to the user's query from the knowledge base for output.

3. The RAG citation verification system based on leave-one-out ablation and large language model as described in claim 2, characterized in that, The retrieval methods include keyword matching, vector semantic similarity calculation, and rule filtering.

4. The RAG citation verification system based on leave-one-out ablation and large language model as described in claim 1, characterized in that, The difference analysis module is based on multi-dimensional text comparison analysis. It compares the initial answer with the ablation answer generated after each removal of a single document. It analyzes and quantifies the difference features at the sentence level, key fact level, and semantic level, respectively, providing structured difference data support for the causal analysis of the LLM Judge verification module.

5. The RAG citation verification system based on leave-one-out ablation and large language model as described in claim 4, characterized in that, The differences include missing facts, logical changes, and differences in expression.

6. A RAG citation verification method based on leave-one-out ablation and large language model, implemented based on the RAG citation verification system based on leave-one-out ablation and large language model as described in any one of claims 1 to 5, characterized in that, Includes the following steps: Step S1: After the user initiates a query, the top-N candidate document blocks are retrieved. The large language model is then called to generate an initial answer for all N retrieved documents and to annotate the source. Step S2: Perform ablation testing based on Leave-One-Out, removing one document at a time and regenerating N ablation answers using the remaining documents; quantify the impact of missing documents on the answer content through multiple ablation generation processes; Step S3: Compare the initial answer with each ablation answer and calculate the content difference. Step S4: Receive the difference analysis results, call another large language model or different instances of the same large language model to perform causal analysis and judgment on each removed document; Step S5: Verify the conclusion, filter and mark the references in the initial answer, and generate the final credible answer.

7. The RAG reference verification method based on leave-one-out ablation and large language model as described in claim 6, characterized in that, The causal analysis and judgment step S4 includes: determining which key facts in the answer are lost or degraded in quality after the document is removed, and whether the document supports the corresponding citations in the initial answer.

Images