An adversarial collaborative retrieval enhancement method

Abstract

An adversarial collaborative retrieval enhancement method, an Adversarial Collaborative Retrieval Enhancement Framework (AC-RAG), is proposed to improve the performance of large language models (LLMs) in domain-specific natural language processing (NLP) tasks. In the pre-check phase, the detector makes a preliminary determination of whether the task requires retrieval. In the question analysis phase, the detector decomposes the query into subtasks, providing retrieval clues for the solver. In the retrieval and integration phase, the solver retrieves and integrates information based on the subtasks. In the post-check phase, the detector assesses information completeness and repeats the search when necessary. This framework introduces a neutral moderator to optimize collaboration between agents, significantly reducing retrieval hallucinations and semantic discrepancies, and improving retrieval accuracy and system efficiency. Furthermore, the model optimization phase utilizes inference path data to fine-tune the detector and solver, enabling continuous model evolution. Experimental results demonstrate that the proposed AC-RAG outperforms existing methods in multiple vertical domains, demonstrating superior performance for complex queries.

Description

Technical Field

[0001] The present invention relates to artificial intelligence large language models (LLMs) technology, and in particular to an adversarial collaborative retrieval enhancement method. Background Art

[0002] In the application of large language models (LLMs), particularly in the field of retrieval-augmented generation (RAG), existing technologies face two major challenges: retrieval hallucination and semantic discrepancy. Retrieval hallucination refers to the problem of a model being overconfident in solving a domain-specific task, omitting necessary retrieval steps or accepting irrelevant results. Semantic discrepancy refers to the situation where the relationship between a query and a document is more expressed as "relevance" than "similarity." However, existing search engines are primarily based on semantic similarity, and this mismatch can reduce the accuracy of retrieved content. Due to these issues, existing methods perform poorly on domain-specific problems.

[0003] It should be noted that the information disclosed in the above background technology section is only used to understand the background of this application, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0004] The main purpose of the present invention is to overcome the defects existing in the above-mentioned background technology and provide an adversarial collaborative retrieval enhancement method.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] An adversarial collaborative retrieval enhancement method, comprising:

[0007] S1, Pre-check stage: The detector makes a preliminary judgment on the task input into the large language model to determine whether the task can be solved directly without entering the subsequent retrieval process; if the task can be solved directly, the solver provides the answer;

[0008] S2, Problem Analysis Phase: If the pre-check phase determines that the task requires further retrieval, the detector decomposes the input query into domain-specific subtasks. These subtasks serve as challenges and clues for the solver to perform retrieval, thereby generating more precise and semantically relevant queries.

[0009] S3, retrieval and integration stage: The solver searches for relevant documents based on the subtasks obtained in the problem analysis stage, and converts the retrieved long and noisy documents into concise and informative summaries;

[0010] S4, post-check phase: The detector evaluates the summary information obtained in the retrieval and integration phase. If the information is insufficient to explain the professional concept, the retrieval and integration phase is repeated until a satisfactory answer is found or the preset maximum number of interaction rounds is reached;

[0011] S5, model optimization stage: Based on the complete interaction sequence verified in the post-inspection stage, generate structured reasoning path data; use the reasoning path data to fine-tune the detector and solver to enhance the model's reasoning capabilities.

[0012] Furthermore, the pre-check stage specifically includes:

[0013] Perform preliminary evaluation on the input task using a detector that is not trained on a specific domain;

[0014] The detector evaluates whether there is any misunderstanding of professional terms or concepts in the task;

[0015] If the detector determines that there is no comprehension bias in the task, the answerer directly provides the answer without further retrieval;

[0016] If the detector identifies a deviation in understanding, it triggers the subsequent problem analysis phase.

[0017] Furthermore, the problem analysis stage specifically includes:

[0018] Use detectors to identify technical terms in queries that may cause misunderstandings and break them down into manageable sub-questions;

[0019] Gradually delve deeper and solve problems through multiple rounds of interaction, each round building on the accumulated memory of the previous round;

[0020] In the first round of interaction, the detector directly asks questions about the problem itself; starting from the second round, the detector combines the current question with the memory of previous interactions to ask new questions;

[0021] Two large-scale autoregressive language models are used to perform detection and resolution functions, one for identifying key terms and the other for providing explanations of these terms.

[0022] Cumulative memory represents the accumulated information after the interaction ends, which is used to assist in problem analysis and answering in subsequent rounds;

[0023] In the subsequent rounds of question analysis, the detector analyzes the current question and previous interactions to identify new sub-questions and provides answers to these sub-questions through the solver.

[0024] Furthermore, the search and integration stage specifically includes:

[0025] Relying on the solver's response to the question, rather than the initial query, for retrieval;

[0026] The maximum marginal relevance (MMR) algorithm is used to retrieve documents from the vector database that are semantically related to the original answer of the solver but not duplicated, ensuring the relevance and diversity of the retrieval results while reducing information redundancy.

[0027] The retrieved documents are integrated and processed, irrelevant information and noise are filtered out, and converted into concise and information-rich summaries.

[0028] Furthermore, the post-inspection stage specifically includes:

[0029] Verify the completeness and accuracy of current information through detectors to ensure that the professional concepts introduced by retrieval enhancement will not cause knowledge gaps or misunderstandings;

[0030] If missing information is detected, the system initiates additional interaction rounds to supplement and improve the required knowledge until the result is satisfactory or the preset maximum number of interaction rounds is reached;

[0031] At the end of each interaction, the system's memory module is updated to ensure that all relevant information is remembered by the system and used for subsequent processing;

[0032] Once the system determines that the current information is comprehensive enough or the interaction limit is reached, the solver provides the final answer based on its latest memory state.

[0033] Furthermore, the method further comprises adaptive retrieval control, wherein:

[0034] In the pre-check stage, the confidence score of the detector for the task is calculated to determine whether further retrieval is needed. The confidence score is obtained by monitoring the generation probability of positive tokens in the detector's response. The probability reflects the confidence of the large language model in its response. If the confidence score is higher than the set threshold, it is considered that the task can be solved directly without retrieval.

[0035] In the post-check stage, the detector's answer is also evaluated by calculating the confidence score. If the confidence score is higher than the set threshold, the detector's answer is regarded as a positive answer, indicating that sufficient information has been retrieved and the retrieval process ends. If the confidence score is lower than the threshold, the retrieval and integration stages are repeated to supplement and improve the required knowledge until the results are satisfactory or the preset maximum number of interaction rounds is reached.

[0036] Furthermore, the model optimization stage specifically includes:

[0037] Reasoning trajectory generation: Captures the complete interaction sequence of verification information in the post-inspection phase and generates structured reasoning path data including question decomposition, response process, and memory state evolution;

[0038] Fine-tuning training execution: Using the structured reasoning path data, supervised fine-tuning training is performed on the detector and the solver to enhance the model's reasoning capabilities in complex problem decomposition, information integration, and error identification.

[0039] An adversarial collaborative retrieval enhancement method, comprising:

[0040] S1, Pre-check stage: The detector makes a preliminary judgment on the task input into the large language model to determine whether the task can be solved directly without entering the subsequent retrieval process; if the task can be solved directly, the solver provides the answer;

[0041] S2, Problem Analysis Phase: If the pre-check phase determines that the task requires further retrieval, the detector decomposes the input query into domain-specific subtasks. These subtasks serve as challenges and clues for the solver to perform retrieval, thereby generating more precise and semantically relevant queries.

[0042] S3, retrieval and integration stage: The solver searches for relevant documents based on the subtasks obtained in the problem analysis stage, and converts the retrieved long and noisy documents into concise and informative summaries;

[0043] S4, post-check phase: The detector evaluates the summary information obtained in the retrieval and integration phase. If the information is not sufficient to explain the professional concept, the retrieval and integration phases are repeated until a satisfactory answer is found or the preset maximum number of interaction rounds is reached.

[0044] An adversarial collaborative retrieval enhancement method, comprising:

[0045] S1, Pre-check stage: The neutral moderator asks the detector whether there is any understanding bias in the task; if the detector feedback is no bias, the neutral moderator instructs the solver to provide the answer directly; if there is a bias, it enters the problem analysis stage;

[0046] S2, Question Analysis Phase: The neutral moderator instructs the detector to decompose the query into domain-specific subtasks and instructs the solver to explain the professional terms in the subtasks;

[0047] S3, Retrieval and Integration Stage: The neutral moderator instructs the resolver to integrate the retrieved information and summarize the key points; the resolver retrieves documents according to the instructions and generates information summaries;

[0048] S4, post-check stage: The neutral moderator asks the detector whether the current information is complete; if the detector feedback is complete, the solver is instructed to generate the final answer based on the context; if the information is incomplete, the retrieval and integration stages are repeated until a satisfactory answer is obtained or the maximum number of interaction rounds is reached.

[0049] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the adversarial collaborative retrieval enhancement method.

[0050] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the adversarial collaborative retrieval enhancement method is implemented.

[0051] The present invention has the following beneficial effects:

[0052] This paper proposes an innovative retrieval-augmented generation (RAG) method to address the problem of misleading information (referred to as "retrieval hallucinations") in information retrieval using large language models (LLMs) for specific domain-specific natural language processing (NLP) tasks. This paper designs an adversarial collaborative retrieval-augmented generation (AC-RAG) framework that enhances retrieval performance through two collaborating agents: a problem detection agent (Detector), responsible for identifying problems in the retrieval content; and a problem resolution agent (Resolver), responsible for resolving these problems. This adversarial collaboration not only innovates technically but also improves operational methods by introducing a neutral moderator to guide the two agents in an in-depth dialogue, thereby acquiring more accurate knowledge information.

[0053] To improve the performance of Retrieval Augmentation Generation (RAG), the adversarial collaborative retrieval-augmented generation framework designed in this paper employs two collaborative components for RAG: a detector and a resolver. The detector is responsible for identifying subtasks within a task, while the resolver is responsible for solving the subtasks identified by the detector. This collaborative approach, with a clear division of labor, enables deeper semantic mining and precise knowledge retrieval for each task. This innovative improvement overcomes the shortcomings of existing retrieval-augmented generation (RAG) systems, particularly addressing the problems of retrieval hallucinations and semantic discrepancies, thereby improving the accuracy and efficiency of large language models (LLMs) in specific domain tasks.

[0054] Experiments in multiple vertical fields show that the AC-RAG framework significantly improves the accuracy of RAG retrieval, surpassing many existing popular methods.

[0055] The main innovative contributions of this invention include:

[0056] 1. Introduction of Adversarial Collaborative Components: This paper proposes an innovative technical framework consisting of two main components: a detector and a resolver. These two components use different models based on their specific characteristics, thereby optimizing their collaborative relationship. This setup significantly reduces the problem of hallucinations during retrieval and improves overall retrieval and generation efficiency by clearly dividing the responsibilities of detection and resolution.

[0057] 2. Innovative Use of a Neutral Moderator: This invention further proposes a novel collaborative approach involving a neutral moderator. This moderator not only guides the adversarial interaction between two agents but also manages task allocation and information exchange, thereby defining a workflow that optimizes agent collaboration. This innovative aspect of the proposed technical solution ensures efficient information processing and precise system responses.

[0058] 3. Adaptive learning mechanism for generating reasoning trajectories based on large models: This invention innovatively treats the verified multi-round interaction process between the detector and the solver as a large language model to generate high-quality reasoning trajectories. By capturing, structuring, and utilizing these successful interactive reasoning path data, which include complex problem decomposition and collaborative solution steps, the reasoning capabilities of the detector and solver are fine-tuned. This mechanism enables the AC-RAG framework to self-supervise learning and continuously evolve from its own successful practices, significantly improving the system's deep reasoning capabilities in complex problem decomposition, semantic understanding, information integration, and error recognition. This allows for continuous performance optimization and ensures the accuracy and comprehensiveness of the output without the need for large amounts of external manually annotated data, further reducing the risk of retrieval hallucinations and semantic discrepancies.

[0059] The method of the present invention not only improves the performance of the model in specific domain tasks, but also reduces unnecessary retrieval operations through refined retrieval control, thereby optimizing the performance and response speed of the entire system.

[0060] Other beneficial effects of the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is an overall flow chart of the adversarial collaborative retrieval enhancement method according to an embodiment of the present invention.

[0062] Figure 2 This is an overall flow chart of an adversarial collaborative retrieval enhancement method according to another embodiment of the present invention.

[0063] Figure 3 This is an overall flow chart of an adversarial collaborative retrieval enhancement method according to another embodiment of the present invention.

[0064] Figure 4 Schematic diagram of the Adversarial Collaborative Retrieval Enhancement Generation Framework (AC-RAG) according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope of the present invention and its application.

[0066] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0067] This paper proposes an innovative retrieval-augmented generation (RAG) method for domain-specific natural language processing (NLP) tasks using large language models (LLMs). This method effectively addresses the problem of misleading information (referred to in this paper as "retrieval hallucination") in information retrieval. The paper designs an adversarial collaborative retrieval augmentation framework (AC-RAG) that enhances retrieval performance through two collaborating agents: a problem detection agent (Detector), responsible for identifying problems in the retrieval content; and a problem resolution agent (Resolver), responsible for resolving these problems. This adversarial collaboration is not only technologically innovative but also operationally advanced, introducing a neutral moderator to guide the two agents in in-depth dialogue, thereby acquiring more accurate knowledge information.

[0068] See Figure 1 , an embodiment of the present invention provides an adversarial collaborative retrieval enhancement method, comprising the following steps:

[0069] S1. Pre-check stage: The detector makes a preliminary judgment on the task input into the large language model to determine whether the task can be solved directly without entering the subsequent retrieval process; if the task can be solved directly, the solver provides the answer.

[0070] In a preferred embodiment, the pre-check stage specifically includes: using a detector that has not been trained in a specific field to perform a preliminary evaluation of the input task to ensure the thoroughness and rigor of the processing process; the detector evaluates whether there are any deviations in professional terms or concepts in the task, which may cause difficulties in understanding; if the detector determines that there is no understanding deviation in the task, the answerer directly provides the answer without further retrieval; if the detector identifies an understanding deviation, it indicates that the task needs to be solved through retrieval enhancement, thereby triggering the subsequent problem analysis stage.

[0071] S2, Problem Analysis Phase: If the pre-check phase determines that the task requires further retrieval, the detector decomposes the input query into domain-specific subtasks. These subtasks serve as challenges and clues for the solver to perform retrieval, thereby generating more precise and semantically relevant queries.

[0072] In a preferred embodiment, the problem analysis stage specifically includes: using a detector to identify professional terms in the query that may cause misunderstandings, and breaking them down into manageable sub-problems; gradually digging deeper and solving the problem through multiple rounds of interaction, each round of interaction is based on the accumulated memory of the previous round; in the first round of interaction, the detector directly questions the problem itself; starting from the second round, the detector combines the current problem and the memory of previous interactions to raise new questions; using two large-scale autoregressive language models to perform detection and solution functions respectively, one for identifying key terms and the other for providing explanations of these terms; cumulative memory represents the accumulated information after the interaction ends, which is used to assist problem analysis and answering in subsequent rounds; in the problem analysis stage of subsequent rounds, the detector analyzes the current problem and previous interaction content to identify new sub-problems, and provides answers to these sub-problems through the solver.

[0073] S3. Retrieval and integration stage: The solver searches for relevant documents based on the subtasks obtained in the problem analysis stage, and converts the retrieved long and noisy documents into concise and informative summaries.

[0074] In a preferred embodiment, the retrieval and integration stage specifically includes: relying on the solver's response to the question rather than the initial query question for retrieval; using the maximum marginal relevance (MMR) algorithm to retrieve documents from the vector database that are semantically related but non-duplicate to the solver's original answer to ensure the relevance and diversity of the retrieval results while reducing information redundancy; integrating and processing the retrieved documents, filtering out irrelevant information and noise, and converting them into a refined and information-rich summary to reflect a deep understanding and accurate answer to the question posed by the detector.

[0075] S4, post-check stage: The detector evaluates the summary information obtained in the retrieval and integration stage. If the information is not sufficient to explain the professional concept, the retrieval and integration stage is repeated until a satisfactory answer is found or the preset maximum number of interaction rounds is reached.

[0076] In a preferred embodiment, the post-check stage specifically includes: verifying the completeness and accuracy of the current information through a detector to ensure that the professional concepts introduced by the retrieval enhancement will not cause knowledge gaps or misunderstandings; if information omissions are detected, the system initiates additional rounds of interaction to supplement and improve the required knowledge until the results are satisfactory or the preset maximum number of interaction rounds is reached; at the end of each interaction, the system's memory module is updated to ensure that all relevant information is remembered by the system and used for subsequent processing; once the system determines that the current information is comprehensive enough or the interaction number limit is reached, the solver will provide a final answer based on its latest memory status to ensure the high quality and accuracy of the final output.

[0077] In a preferred embodiment, the adversarial collaborative retrieval enhancement method of the present invention further includes adaptive retrieval control, wherein:

[0078] In the pre-check stage, the confidence score of the detector for the task is calculated to determine whether further retrieval is needed. The confidence score is obtained by monitoring the generation probability of positive tokens in the detector's response. The probability reflects the confidence of the large language model in its response. If the confidence score is higher than the set threshold, it is considered that the task can be solved directly without retrieval.

[0079] In the post-check stage, the detector's answer is also evaluated by calculating the confidence score. If the confidence score is higher than the set threshold, the detector's answer is regarded as a positive answer, indicating that sufficient information has been retrieved and the retrieval process ends. If the confidence score is lower than the threshold, the retrieval and integration stages are repeated to supplement and improve the required knowledge until the results are satisfactory or the preset maximum number of interaction rounds is reached.

[0080] like Figure 2 As shown, in another embodiment, the adversarial collaborative retrieval enhancement method further includes the following steps:

[0081] S5, model optimization stage: Based on the complete interaction sequence verified in the post-inspection stage, generate structured reasoning path data; use the reasoning path data to fine-tune the detector and solver to enhance the model's reasoning capabilities.

[0082] In a preferred embodiment, the model optimization stage specifically includes:

[0083] Reasoning trajectory generation: Captures the complete interaction sequence of verification information in the post-inspection phase and generates structured reasoning path data including question decomposition, response process, and memory state evolution;

[0084] Fine-tuning training execution: Using the structured reasoning path data, supervised fine-tuning training is performed on the detector and the solver to enhance the model's reasoning capabilities in complex problem decomposition, information integration, and error identification.

[0085] like Figure 3 As shown, the embodiment of the present invention also provides another adversarial collaborative retrieval enhancement method, including the following steps:

[0086] S1, Pre-check stage: The neutral moderator asks the detector whether there is any understanding bias in the task; if the detector feedback is no bias, the neutral moderator instructs the solver to provide the answer directly; if there is a bias, it enters the problem analysis stage;

[0087] S2, Question Analysis Phase: The neutral moderator instructs the detector to decompose the query into domain-specific subtasks and instructs the solver to explain the professional terms in the subtasks;

[0088] S3, Retrieval and Integration Stage: The neutral moderator instructs the resolver to integrate the retrieved information and summarize the key points; the resolver retrieves documents according to the instructions and generates information summaries;

[0089] S4, post-check stage: The neutral moderator asks the detector whether the current information is complete; if the detector feedback is complete, the solver is instructed to generate the final answer based on the context; if the information is incomplete, the retrieval and integration stages are repeated until a satisfactory answer is obtained or the maximum number of interaction rounds is reached.

[0090] This method proposes an Adversarial Collaborative Retrieval Augmentation (AC-RAG) framework. By introducing two collaborative agents, a detector and a resolver, it significantly improves the accuracy and efficiency of large language models (LLMs) in domain-specific natural language processing (NLP) tasks. Through adversarial collaboration, this framework effectively addresses two major challenges in the field of Retrieval Augmentation Generation (RAG): retrieval hallucination and semantic discrepancy. The retrieval hallucination problem is addressed by the detector, which identifies subtasks within a task and identifies problems in the retrieval content, while the resolver resolves these problems, ensuring accurate and relevant retrieval. Semantic discrepancy is ameliorated through refined retrieval control. The AC-RAG framework not only deepens the model's understanding of domain-specific tasks but also optimizes the collaborative process between agents through the innovative use of a neutral moderator, ensuring efficient information processing and accurate system responses. Furthermore, this method demonstrates excellent performance in experiments across multiple vertical domains, significantly improving RAG retrieval accuracy and outperforming many existing popular methods. It also reduces unnecessary retrieval operations and optimizes system performance and responsiveness.

[0091] The following further describes specific embodiments of the present invention and experimental verification.

[0092] This paper presents an adversarial collaborative retrieval enhancement method and proposes an Adversarial Collaborative Retrieval Enhancement Generation (AC-RAG) framework. This framework uses two agents with clearly defined roles to jointly optimize the effectiveness of retrieval enhancement generation (RAG). These two agents, a detector and a resolver, collaborate adversarially under the guidance of a neutral moderator. Within this framework, retrieval enhancement is achieved through four steps:

[0093] 1. Pre-check phase: The detector first determines whether the task can be solved directly without going through the retrieval process. If the task is determined to be solvable directly, the solver will immediately provide the answer.

[0094] 2. Challenge Dissection: The detector decomposes the query into domain-specific subtasks. These subtasks then serve as retrieval challenges and clues for the solver, helping it generate more precise and semantically relevant queries.

[0095] 3. Retrieval & Integration: The solver retrieves relevant documents based on semantically consistent queries and converts long and noisy documents into concise and informative summaries.

[0096] 4. Post-check phase: If the current information is not sufficient for the detector to be confident that the specialized concept has been fully explained, the process will be repeated, continuing to align semantics and retrieve relevant documents until a satisfactory answer is found.

[0097] This approach not only improves the performance of the model in specific domain tasks, but also reduces unnecessary retrieval operations through refined retrieval control, thereby optimizing the performance and response speed of the entire system.

[0098] In our Adversarial Collaborative Retrieval-Enhanced Generation (AC-RAG) framework, we designed two agents with clearly defined roles: a detector and a resolver. This design draws inspiration from human interaction patterns: the questioner may not possess the in-depth knowledge required to provide an answer, while the responder should possess relevant expertise. Based on this philosophy, the detector is not trained on a specific domain, maintaining its original large-scale language model state so that it can identify and pose questions from a neutral perspective. Meanwhile, the resolver is deeply customized and optimized for a specific domain to ensure it can effectively resolve the questions posed by the detector. This division of labor ensures the system's efficiency and accuracy in processing expert knowledge.

[0099] The core of the AC-RAG framework is to optimize the overall performance of Retrieval-Augmented Generation (RAG) through the effective collaboration of these two agents. First, the detector is responsible for performing a preliminary analysis of the input task to determine whether there are any gaps in understanding caused by specialized terminology or complex concepts. This stage, known as pre-checking, is a key step in determining whether further search is necessary. After confirming that in-depth analysis is required, the detector breaks the task down into more specific sub-questions, a process known as challenge dissection. These sub-questions are designed to explore specialized concepts within a specific domain, providing clear solution directions for the solver. The solver then takes over these sub-questions, searching for solutions through in-depth analysis and applying its expertise. After the solver processes and answers these sub-questions, the system enters the retrieval and integration phase. In this phase, the solver's answers serve as the basis for further searches to obtain relevant information and literature support, enhancing the accuracy and reliability of the answers. Finally, in the post-checking phase, the detector re-evaluates the information provided by the solver to ensure that all questions have been properly addressed and the entire query has been thoroughly answered.

[0100] This multi-agent interactive workflow not only improves problem-solving accuracy but also optimizes processing speed, ensuring the system's efficiency and practicality in real-world applications. Through this innovative adversarial collaborative model, the AC-RAG framework can provide more accurate and comprehensive solutions in complex information environments, significantly improving the performance of the entire system.

[0101] Pre-inspection stage

[0102] Previous research has shown that blindly applying retrieval-augmented generation (RAG) can sometimes produce adverse effects. Our method takes these findings into account and proposes an innovative strategy. Unlike traditional methods, our method utilizes an untrained detector to precisely perform the task, ensuring a thorough and rigorous process.

[0103] The AC-RAG framework's workflow begins with a pre-check phase. During this phase, the detector's primary task is to assess whether there are any gaps in specialized terminology or concepts that could lead to difficulties in understanding. If the detector determines that no such gaps exist, the task is straightforward, and the answerer can directly provide an answer. Conversely, if such gaps exist, it indicates that retrieval augmentation is needed to fill these gaps and more accurately answer the question.

[0104] Problem analysis stage

[0105] To effectively address complex queries, it's crucial to mimic the way humans solve problems: identifying the core challenge and breaking it down into simpler, more manageable subproblems. In our adversarial collaborative search enhancement framework, this task is performed by detectors. The detectors identify technical terms that could lead to biased understanding and prompt the solver to provide detailed explanations. This framework employs a multi-round interactive process to ensure progressively deeper exploration and resolution.

[0106] In the first round of interaction, the detector asks questions directly about the question itself. Starting from the second round, the detector not only considers the current question, but also refers to the memory of previous interactions to ask new questions. and Two large-scale autoregressive language models that perform detection and resolution functions, Representative The accumulated memory after the round of interaction, where , .here, Indicates the The key terms identified in the round It is the summary information obtained from the corresponding retrieval.

[0107] In the In the question parsing phase of the round, the detector analyzes the current question and previous interactions to identify new, worthy sub-questions. The solver then provides answers to these sub-questions. This process can be expressed as follows:

[0108]

[0109] in is the task query to be solved, Yes In this way, the system of the present invention not only enhances the depth of understanding of the question, but also improves the accuracy and relevance of the answer.

[0110] Search and integration stage

[0111] In the adversarial collaborative retrieval enhancement generation framework of the present invention, retrieval is not directly based on the initial query question, but relies on the solver's response to the question in each round. Specifically, the Maximum Marginal Relevance (MMR) algorithm is used to retrieve the original answer of the solver from a special vector database. Semantically related but non-duplicate meta-documents. The reason for choosing MMR is to take into account the relevance and diversity of documents and effectively reduce information redundancy.

[0112] After retrieving relevant documents, the solver integrates and processes them, filtering out irrelevant information and noise, and transforming them into a concise and information-rich summary that reflects the solver's deep understanding and accurate answer to the question posed by the detector in the current interaction round. The relevant workflow can be described by the following mathematical representation:

[0113] ,

[0114] in represents a retrieval system for retrieval enhancement, Represents the retrieved document content, and The solver is used to process and integrate the retrieved information.

[0115] Post-inspection stage

[0116] To ensure that the new professional concepts introduced by search enhancement do not cause knowledge gaps or misunderstandings, the present invention specifically incorporates a post-check step into the process. In this step, the checker verifies the completeness and accuracy of the current information. If any information is detected to be missing, the system will initiate additional rounds of interaction to supplement and improve the required knowledge until the results are satisfactory or the preset maximum number of rounds is reached. .

[0117] At the end of each interaction, the system's memory module is updated to ensure that all relevant information is remembered by the system and used for subsequent processing:

[0118]

[0119] Here, Indicates the The memory state after the round interaction, and are the question identified by the detector and the solution generated by the answerer, respectively.

[0120] Once the system determines that the current information is comprehensive enough, or the set interaction limit has been reached, the solver will use its latest memory status to solve the problem. Provide a final answer :

[0121]

[0122] in This process ensures the high quality and accuracy of the final output, improves the performance of the retrieval enhancement system in complex queries, and minimizes the risk of misunderstanding due to information bias.

[0123] Adaptive retrieval

[0124] In the pre-check and post-check stages, in a preferred embodiment, the adversarial collaborative retrieval enhancement generation framework implements fine-grained adaptive retrieval control by monitoring the generation probabilities of specific tokens in the detector's response. These probabilities reflect the confidence of the large language model in its response. In both stages, the interaction with the detector is mainly to guide it to give a "yes" or "no" answer. Among them, the positive token "yes" indicates that further retrieval is needed or that enough information has been retrieved. Despite the existence of this guidance mechanism, the model's response is not always as accurate as expected. Therefore, the set of positive tokens, marked as , including "yes", "YES", and "Sure" and other expressions. The confidence score of the post-round inspection is calculated using the following formula:

[0125]

[0126] here, represents the output length generated by the large language model, is in position The token at is the sequence of tokens preceding it. In particular, when When , we only focus on the first token generated by the large language model. , the pre-check phase is considered a special case, when in Two thresholds are set. and , is used to evaluate whether the detector's answer is positive. Specifically, if the pre-check Or post-inspection , then the detector's answer is regarded as a positive answer in the corresponding stage.

[0127] Generation of high-quality reasoning trajectories and fine-tuning of reasoning capabilities

[0128] In the above-mentioned adversarial collaborative retrieval enhancement framework, the present invention further makes full use of the unique and iterative multi-round interaction mechanism between the detector and the solver. This mechanism is not only a key process for solving complex queries, but also a valuable process for generating rich implicit reasoning paths. In each successful task processing - especially those interaction sequences that have been rigorously evaluated and verified as complete and accurate in the "post-check stage", the detector's question decomposition, guidance and evaluation, and the solver's information retrieval, summarization and question answering together constitute a complete and traceable reasoning trajectory from the original query to the final answer. The reasoning trajectory essentially reflects the Chain-of-Thought (CoT) process of how large language models approach the optimal solution through decomposition, exploration, verification and iteration when faced with complex or uncertain problems. Specifically, each successful interaction (for example, a sequence of interactions in which the detector's confidence score in the post-check phase is higher than the set threshold, indicating that the task has been successfully solved) includes the detector's questioning, decomposition, and evaluation steps for the task or subtask; the solver's retrieval query, processing of relevant documents, and summary generation based on the detector's feedback; and, in each round of interaction, how the system combines the current question and accumulated memory to evolve the context and make decisions.

[0129] All of the structured intermediate steps and decision rationales generated during successful interactions are efficiently captured and stored in a structured manner, forming high-quality interactive reasoning path data. This data not only includes traditional input-output pairs but also the deep reasoning steps and strategies demonstrated by LLMs when solving problems, thus becoming a unique form of self-supervised policy learning data generated by large models.

[0130] Fine-tuning reasoning capabilities

[0131] To further improve the performance and generalization of LLMs in the AC-RAG framework, this paper introduces a fine-tuning mechanism for reasoning capabilities based on the aforementioned high-quality interactive reasoning path data. The specific steps are as follows.

[0132] 1. Data Construction and Screening: The system continuously monitors and collects successful, multi-round interactive sessions during actual operation, which undergo rigorous post-verification by the detector to verify that the final answer is accurate, complete, and requires no further search. These sessions are considered "gold standard" examples of valid reasoning strategies. By extracting the entire interaction sequence from these successful sessions (including the detector's questions, the solver's responses, the retrieved content, and the evolution of memory states), a supervised thought chain dataset for training is constructed.

[0133] 2. Model Fine-tuning: Utilizing this carefully selected, high-quality thought chain supervision data, we fine-tune the large language model that performs the detector and answerer functions (Supervise Fine-tuning (SFT)). This fine-tuning is not simply a parameter update; rather, it aims to enable the model to better learn and internalize these effective multi-round interaction strategies, problem decomposition patterns, and information integration logic. By simulating and replicating successful interaction processes, the model significantly enhances its deep reasoning capabilities in the following key areas:

[0134] Decomposition of complex problems: The model can more intelligently and meticulously split highly complex query tasks into a series of logically related and manageable subtasks.

[0135] Critical questioning and verification: When evaluating information, detectors can more accurately identify potential knowledge gaps, conceptual biases, or reasoning fallacies, and proactively guide further exploration.

[0136] Efficient information integration and refinement: When processing multi-source search results, the solver can more effectively extract core information, filter out redundancy and noise, and generate more refined, accurate, and semantically coherent summaries.

[0137] Adaptive learning and generalization: This enables the AC-RAG framework to continuously learn and optimize from its own experience, especially when faced with unprecedented, highly specialized, or ambiguous queries, demonstrating greater robustness and adaptability, effectively reducing reliance on external manually labeled data.

[0138] By fine-tuning the thought chain based on the inference trajectory data generated by the interaction of the large model itself, the AC-RAG framework of the present invention not only realizes intelligent adversarial collaboration in design, but also realizes adaptive evolution in learning mechanism, thereby demonstrating excellent performance, higher accuracy and continuous optimization potential when handling natural language processing tasks in complex, dynamic and professional fields.

[0139] Examples

[0140] like Figure 3 and Figure 4 As shown in Figure 2, an example process of the adversarial collaborative retrieval enhancement method is shown, which includes the following four stages:

[0141] S1: Pre-inspection stage.

[0142] The neutral moderator asked the detector, “Are there any medical terms in the question that you don’t understand?”

[0143] If the detector determines that there is no question, it instructs the answerer to "answer the question directly".

[0144] S2: Problem analysis stage.

[0145] If the detector determines that there is a problem, the neutral moderator instructs the detector: "Please list the technical terms that need further explanation."

[0146] Next, the neutral moderator instructed the solver based on the listed technical terms: "Please explain the definition of the technical terms."

[0147] S3: Retrieval and integration stage.

[0148] The neutral moderator instructed the solver: “Read through and understand the context, then summarize the key points.”

[0149] S4: Post-inspection stage.

[0150] The neutral moderator asked the detector, “Do you think the context provided is sufficient to answer the question?”

[0151] If the detector deems the context sufficient, the neutral moderator instructs the answerer to “answer the question based on contextual knowledge.”

[0152] To evaluate the effectiveness of the AC-RAG framework for complex, concept-intensive tasks, comprehensive experiments were conducted across multiple vertical domains. The experimental results demonstrate that the proposed approach significantly improves the accuracy of retrieval enhancement and the quality of generated content. These experiments not only demonstrate the practical potential of AC-RAG but also validate its stability and reliability across a variety of application scenarios.

[0153] The main innovative contributions of this invention include:

[0154] 1. Introduction of Adversarial Collaborative Components: This paper proposes an innovative technical framework consisting of two main components: a detector and a resolver. These two components use different models based on their specific characteristics, thereby optimizing their collaborative relationship. This setup significantly reduces the problem of hallucinations during retrieval and improves overall retrieval and generation efficiency by clearly dividing the responsibilities of detection and resolution.

[0155] 2. Innovative Use of a Neutral Moderator: This invention further proposes a novel collaborative approach involving a neutral moderator. This moderator not only guides the adversarial interaction between two agents but also manages task allocation and information exchange, thereby defining a workflow that optimizes agent collaboration. This innovative aspect of the proposed technical solution ensures efficient information processing and precise system responses.

[0156] An embodiment of the present invention further provides a storage medium for storing a computer program, which at least performs the above method when executed.

[0157] An embodiment of the present invention further provides a control device, comprising a processor and a storage medium for storing a computer program; wherein the processor is configured to execute at least the method described above when executing the computer program.

[0158] An embodiment of the present invention further provides a processor, which executes a computer program and at least performs the method described above.

[0159] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a magnetic disk memory or a magnetic tape memory. The storage medium described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0160] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0161] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0162] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0163] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0164] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0165] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0166] The features disclosed in several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0167] The features disclosed in several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0168] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. Those skilled in the art will recognize that, without departing from the scope of the present invention, several equivalent substitutions or obvious variations can be made, and the performance or use of the same should be considered to fall within the scope of protection of the present invention.

Claims

1. A method for enhancing adversarial collaborative retrieval, characterized in that: include: S1, pre-check stage: The detector makes a preliminary judgment on the task input into the large language model to determine whether the task can be solved directly without entering the subsequent retrieval process; If the task is directly solvable, the solver provides the answer; S2, Problem Analysis Phase: If the pre-check phase determines that the task requires further retrieval, the detector decomposes the input query into domain-specific subtasks. These subtasks serve as challenges and clues for the solver to perform retrieval, thereby generating more precise and semantically relevant queries. S3, retrieval and integration stage: The solver searches for relevant documents based on the subtasks obtained in the problem analysis stage, and converts the retrieved long and noisy documents into concise and informative summaries; S4, post-check phase: The detector evaluates the summary information obtained in the retrieval and integration phase. If the information is insufficient to explain the professional concept, the retrieval and integration phase is repeated until a satisfactory answer is found or the preset maximum number of interaction rounds is reached; S5, model optimization phase: based on the complete interaction sequence verified in the post-inspection phase, generate structured reasoning path data; Fine-tune the detector and the solver using the inference path data to enhance the model's inference capability; The problem analysis phase specifically includes: Use detectors to identify technical terms in queries that may cause misunderstandings and break them down into manageable sub-questions; Gradually delve deeper and solve problems through multiple rounds of interaction, each round building on the accumulated memory of the previous round; In the first round of interaction, the detector directly asks questions about the problem itself; starting from the second round, the detector combines the current question with the memory of previous interactions to ask new questions; Two large-scale autoregressive language models are used to perform detection and resolution functions, one for identifying key terms and the other for providing explanations of these terms. Cumulative memory represents the accumulated information after the interaction ends, which is used to assist in problem analysis and answering in subsequent rounds; In the subsequent rounds of question analysis, the detector analyzes the current question and previous interactions to identify new sub-questions and provides answers to these sub-questions through the solver; The post-inspection stage specifically includes: Verify the completeness and accuracy of current information through detectors to ensure that the professional concepts introduced by retrieval enhancement will not cause knowledge gaps or misunderstandings; If missing information is detected, the system initiates additional interaction rounds to supplement and improve the required knowledge until the result is satisfactory or the preset maximum number of interaction rounds is reached; At the end of each interaction, the system's memory module is updated to ensure that all relevant information is remembered by the system and used for subsequent processing; Once the system determines that the current information is comprehensive enough or the interaction limit is reached, the solver provides the final answer based on its latest memory state.

2. The adversarial collaborative retrieval enhancement method according to claim 1, characterized in that: The pre-check stage specifically includes: Perform preliminary evaluation on the input task using a detector that is not trained on a specific domain; The detector evaluates whether there is any misunderstanding of professional terms or concepts in the task; If the detector determines that there is no comprehension bias in the task, the answerer directly provides the answer without further retrieval; If the detector identifies a deviation in understanding, it triggers the subsequent problem analysis phase.

3. The adversarial collaborative retrieval enhancement method according to claim 1 or 2, wherein: The search and integration stage specifically includes: Relying on the solver's response to the question, rather than the initial query, for retrieval; The maximum marginal relevance (MMR) algorithm is used to retrieve documents from the vector database that are semantically related to the original answer of the solver but not duplicated, ensuring the relevance and diversity of the retrieval results while reducing information redundancy. The retrieved documents are integrated and processed, irrelevant information and noise are filtered out, and converted into concise and information-rich summaries.

4. The adversarial collaborative retrieval enhancement method according to claim 1 or 2, wherein: Also included are adaptive retrieval controls, which: In the pre-check stage, the confidence score of the detector for the task is calculated to determine whether further retrieval is needed. The confidence score is obtained by monitoring the generation probability of positive tokens in the detector's response. The probability reflects the confidence of the large language model in its response. If the confidence score is higher than the set threshold, it is considered that the task can be solved directly without retrieval. In the post-check stage, the detector's answer is also evaluated by calculating the confidence score. If the confidence score is higher than the set threshold, the detector's answer is regarded as a positive answer, indicating that sufficient information has been retrieved and the retrieval process ends. If the confidence score is lower than the threshold, the retrieval and integration stages are repeated to supplement and improve the required knowledge until the results are satisfactory or the preset maximum number of interaction rounds is reached.

5. The adversarial collaborative retrieval enhancement method according to claim 1 or 2, wherein: The model optimization stage specifically includes: Reasoning trajectory generation: Captures the complete interaction sequence of verification information in the post-inspection phase and generates structured reasoning path data including question decomposition, response process, and memory state evolution; Fine-tuning training execution: Using the structured reasoning path data, supervised fine-tuning training is performed on the detector and the solver to enhance the model's reasoning capabilities in complex problem decomposition, information integration, and error identification.

6. A method for enhancing adversarial collaborative retrieval, characterized in that: include: S1, pre-check stage: The detector makes a preliminary judgment on the task input into the large language model to determine whether the task can be solved directly without entering the subsequent retrieval process; If the task is directly solvable, the solver provides the answer; S2, Problem Analysis Phase: If the pre-check phase determines that the task requires further retrieval, the detector decomposes the input query into domain-specific subtasks. These subtasks serve as challenges and clues for the solver to perform retrieval, thereby generating more precise and semantically relevant queries. S3, retrieval and integration stage: The solver searches for relevant documents based on the subtasks obtained in the problem analysis stage, and converts the retrieved long and noisy documents into concise and informative summaries; S4, post-check phase: The detector evaluates the summary information obtained in the retrieval and integration phase. If the information is insufficient to explain the professional concept, the retrieval and integration phase is repeated until a satisfactory answer is found or the preset maximum number of interaction rounds is reached; The problem analysis phase specifically includes: Use detectors to identify technical terms in queries that may cause misunderstandings and break them down into manageable sub-questions; Gradually delve deeper and solve problems through multiple rounds of interaction, each round building on the accumulated memory of the previous round; In the first round of interaction, the detector directly asks questions about the problem itself; starting from the second round, the detector combines the current question with the memory of previous interactions to ask new questions; Two large-scale autoregressive language models are used to perform detection and resolution functions, one for identifying key terms and the other for providing explanations of these terms. Cumulative memory represents the accumulated information after the interaction ends, which is used to assist in problem analysis and answering in subsequent rounds; In the subsequent rounds of question analysis, the detector analyzes the current question and previous interactions to identify new sub-questions and provides answers to these sub-questions through the solver; The post-inspection stage specifically includes: Verify the completeness and accuracy of current information through detectors to ensure that the professional concepts introduced by retrieval enhancement will not cause knowledge gaps or misunderstandings; If missing information is detected, the system initiates additional interaction rounds to supplement and improve the required knowledge until the result is satisfactory or the preset maximum number of interaction rounds is reached; At the end of each interaction, the system's memory module is updated to ensure that all relevant information is remembered by the system and used for subsequent processing; Once the system determines that the current information is comprehensive enough or the interaction limit is reached, the solver provides the final answer based on its latest memory state.

7. A method for enhancing adversarial collaborative retrieval, characterized in that: include: S1, pre-check phase: a neutral moderator asks whether there is comprehension bias in the detector task; If the detector feedback is unbiased, the neutral regulator instructs the solver to provide the answer directly; If there is a deviation, the problem analysis phase will begin; S2, Question Analysis Phase: The neutral moderator instructs the detector to decompose the query into domain-specific subtasks and instructs the solver to explain the professional terms in the subtasks; S3, Retrieval and Integration Stage: The neutral moderator instructs the resolver to integrate the retrieved information and summarize the key points; the resolver retrieves documents according to the instructions and generates information summaries; S4, post-check stage: the neutral moderator asks the detector whether the current information is complete; If the detector feedback is complete, the solver is instructed to generate the final answer based on the context; If the information is incomplete, the retrieval and integration phases are repeated until a satisfactory answer is obtained or the maximum number of interaction rounds is reached; The problem analysis phase specifically includes: Use detectors to identify technical terms in queries that may cause misunderstandings and break them down into manageable sub-questions; Gradually delve deeper and solve problems through multiple rounds of interaction, each round building on the accumulated memory of the previous round; In the first round of interaction, the detector directly asks questions about the problem itself; starting from the second round, the detector combines the current question with the memory of previous interactions to ask new questions; Two large-scale autoregressive language models are used to perform detection and resolution functions, one for identifying key terms and the other for providing explanations of these terms. Cumulative memory represents the accumulated information after the interaction ends, which is used to assist in problem analysis and answering in subsequent rounds; In the subsequent rounds of question analysis, the detector analyzes the current question and previous interactions to identify new sub-questions and provides answers to these sub-questions through the solver; The post-inspection stage specifically includes: Verify the completeness and accuracy of current information through detectors to ensure that the professional concepts introduced by retrieval enhancement will not cause knowledge gaps or misunderstandings; If missing information is detected, the system initiates additional interaction rounds to supplement and improve the required knowledge until the result is satisfactory or the preset maximum number of interaction rounds is reached; At the end of each interaction, the system's memory module is updated to ensure that all relevant information is remembered by the system and used for subsequent processing; Once the system determines that the current information is comprehensive enough or the interaction limit is reached, the solver provides the final answer based on its latest memory state.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the adversarial collaborative retrieval enhancement method according to any one of claims 1 to 7 is implemented.

Images