A question-answering method based on long document retrieval and multi-agent debate

Abstract

This invention belongs to the field of natural language processing technology, specifically relating to a question-answering method based on long document retrieval and multi-agent debate. Addressing the problems of large retrieval space, evidence ranking dependent on relevance, and shared input in multi-agent debate in long document question answering, this invention: organizes long documents into long retrieval units, retrieving candidate evidence based on the maximum similarity of sub-blocks; generates pseudo-evidence that depends solely on the question as fallback candidates; uses the "reduction in answer generation cost after evidence addition" as a utility label, and trains a preference selector to filter evidence through prior correction; finally, in a multi-agent debate where evidence input information is asymmetrical—supporters access evidence, challengers initially only access the question, and the judge makes a ruling—the final answer is output through multiple rounds of cross-questioning. This invention improves the effectiveness and robustness of evidence selection under a fixed retrieval budget, reduces factual illusions, and can be applied to knowledge base question answering, scientific literature retrieval, and legal and financial analysis.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, specifically relating to a question-answering method based on long document retrieval and multi-agent debate. Background Technology

[0002] Long document retrieval and question answering is a task that addresses lengthy, complex, and informationally dispersed document collections by generating answers through retrieval, reading, and reasoning. With the rapid development of large-scale models, this task typically uses retrieval-enhanced generation as its basic framework, further extending to long-context retrieval-enhanced generation. Because it simultaneously involves information retrieval, long text modeling, and evidence constraint generation, long document question answering has become an important research direction in natural language processing, and has high application value in scenarios such as enterprise knowledge base question answering, scientific literature retrieval, and legal and financial auxiliary analysis.

[0003] Compared to short text question answering, the challenge of long document question answering lies in the fact that evidence is often scattered across multiple paragraphs or even multiple documents. The system not only needs to recall relevant text but also needs to identify the relationships between entities, events, and attributes within a long context and perform multi-hop inference accordingly. However, existing research still has several shortcomings in long document question answering scenarios:

[0004] First, traditional retrieval enhancement methods often use short paragraphs as retrieval units, resulting in an excessively large retrieval space and high recall overhead. Although dense retrieval and reordering methods improve semantic matching capabilities, they still use relatively short paragraphs as basic retrieval units, which easily fragments the complete chain of evidence into a large number of local fragments, leading to problems such as retrieval space expansion, noise accumulation, and increased computational overhead.

[0005] Second, the ranking of candidate evidence usually relies mainly on relevance or similarity signals, making it difficult to further distinguish between evidence that is "semantically relevant" and evidence that "truly contributes to the correct answer." When long units contain both key facts and irrelevant details, noise may still be fed into the reader model and amplified by the generative model, creating a factual illusion.

[0006] Third, iterative or dynamic retrieval methods improve evidence supplementation by repeatedly triggering retrieval during the generation process. However, these methods often involve querying and updating around short segments, resulting in high inference costs. In long document scenarios, they may not be able to stably maintain the integrity of cross-segment relationship chains.

[0007] Fourth, evidence utilization methods based on self-reflection, answer constraints, or retrieval summary compression have driven the evolution of evidence selection from "relevance" to "usefulness." However, most of these methods are geared towards paragraph-level evidence organization or post-generation verification, and mainly rely on the self-verification capabilities within a single model. When the model forms a shared bias against noisy evidence, its error correction effect is often limited.

[0008] Fifth, existing multi-agent debate methods mainly focus on collaboration between candidate answers and reasoning processes, and are often geared towards closed-book or weakly evidence-constrained scenarios. Roles often share the same input or similar priors. In long document question answering, if multiple agents build upon the same batch of retrieved evidence, the debate process may still revolve around the same erroneous premise, making it difficult to truly question and adjudicate the reliability of the evidence chain.

[0009] Therefore, how to preserve the key relationship structure under a fixed retrieval budget and context window, identify evidence that truly helps in answering the question, and effectively verify the final reasoning results remains a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0010] The purpose of this invention is to provide a question-answering method based on long document retrieval and multi-agent debate, so as to improve the effectiveness and robustness of evidence selection under a fixed retrieval budget, and reduce factual illusions and reasoning biases induced by noisy evidence.

[0011] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0012] A question-answering method based on long document retrieval and multi-agent debate includes the following steps:

[0013] Step S1: Organize the long document corpus into long retrieval units, segment the long retrieval units into sub-blocks and establish vector indexes, and retrieve the candidate evidence set based on the vector similarity between the question and the sub-blocks.

[0014] Step S2: The generative model generates false evidence based solely on the question, and adds the false evidence to the candidate evidence set to form an enhanced candidate set.

[0015] Step S3: Define the evidence utility labels of candidate evidence based on the answer generation cost, train a preference selector that aligns evidence utility, score the enhanced candidate set by the preference selector and select the highest-scoring evidence as the final evidence set;

[0016] Step S4: The final set of evidence is input into the multi-agent debate audience, and the supporters, challengers and judges generate the final answer through multiple rounds of cross-examination and adjudication.

[0017] Furthermore, the specific steps of step S1 are as follows:

[0018] Step S101: Organize the long document corpus into long retrieval units according to documents or document groups. This significantly reduces the number of search units, shifting the search from "finding the needle in a massive amount of paragraphs" to "finding the relevant area in a smaller number of long units";

[0019] Step S102, each long search unit Divide into m sub-blocks Calculate the vector representation of each sub-block and establish a vector index;

[0020] Step S103, based on question q and sub-block The problem of calculating the inner product of vectors and matching scores for long search units. (1) After performing a nearest neighbor search on all sub-blocks, backtrack to the corresponding long search unit and take the first one. Each long search unit serves as a candidate evidence set. ,in and These represent the query encoder and the text encoder, respectively. Represents the vector dot product. This indicates that the overall correlation of a long cell is approximated by the sub-block with the highest score within that long cell.

[0021] Furthermore, the specific steps of step S2 are as follows:

[0022] Step S201, construct based solely on the problem Generation prompts ;

[0023] Step S202, from the generative model According to the prompt Generate an encyclopedic factual summary text as false evidence. (2);

[0024] Step S203, remove false evidence Add to candidate evidence set To obtain the enhanced candidate set (3). to For long searcher returned This provides several candidate pieces of evidence, thus preventing performance degradation when all external evidence is unfavorable.

[0025] Furthermore, the specific steps of step S3 are as follows:

[0026] Step S301, provide the standard answer. Represented as a token sequence Define the average negative log-likelihood of the reader model generating the standard answer without incorporating external evidence as the answer generation cost. (4); where, This represents the conditional generation probability of the reader model. Indicates the prefix of the answer.

[0027] Step S302, define the candidate evidence to be added The cost of generating a standard answer later (5);

[0028] Step S303: If a certain piece of evidence can effectively support a correct answer, then the cost of generating the standard answer after adding this evidence should decrease. Based on this, the incremental evidence utility of the candidate evidence is defined. (6); where, This indicates that candidate evidence reduces the cost of generating the correct answer and makes a positive contribution to answering. This indicates that the evidence has limited help in generating the correct answer and may even introduce interference.

[0029] Step S304: To reduce the bias caused by the reader model relying solely on parameter memory for answers, a priori correction strength coefficient is introduced. Construct the generated distribution after prior correction: (7), where, This represents the logits output by the reader model. For the priori correction strength coefficient; when When this formula degenerates into a common "problem + evidence" generation distribution; when At this time, the influence of prior knowledge within the model is partially offset, thus highlighting the incremental contribution of external evidence to answer generation; based on this corrected distribution, the cost of evidence generation after prior correction is: (8); Accordingly, the final definition of the evidentiary utility label is: (9); where, This represents the evidence utility signal after prior correction. The larger the value, the more the candidate evidence can promote the generation of the standard answer after eliminating interference from the model's internal memory.

[0030] Step S305, during the training phase, for each training sample The long searcher returns a candidate set. and false evidence Add them together to obtain the enhanced candidate set ;right Calculate the utility label for each candidate piece of evidence to form the target utility vector. : (10);

[0031] Step S306, the preference selector is denoted as Its input is the question and candidate evidence, and its output is the predicted utility score of that evidence; for the same candidate set, the selector outputs a prediction vector. : (11);

[0032] Since evidence selection focuses more on the relative merits of different candidates for the same problem, the target utility vector and the predicted utility vector are normalized to preference distributions respectively: (12); (13); where, The temperature coefficient is used to adjust the smoothness of preference differences among candidate evidence; the selector works by minimizing the target preference distribution. With the predicted preference distribution The KL divergence between them is used for training: (14);

[0033] In step S307, during the reasoning phase, the system first returns a set of candidate evidence using a long retrieval unit. And obtained by the false evidence generation module The trained preference selector then enhances the candidate set. All candidates are scored, and the candidate with the highest score is selected. Each piece of evidence constitutes the final set of evidence. : (15) When high-quality evidence exists in the external search results, the selector tends to retain true long units that can support a complete chain of reasoning; when the overall quality of the external search is poor, false evidence is preferred. It may also be selected This provides robust backoff based on knowledge within the model.

[0034] Furthermore, the specific steps of step S4 are as follows:

[0035] Step S401: Set up three types of intelligent agents with information asymmetry: supporters, challengers, and judges. Supporters have access to the final evidence set. The challenger is responsible for constructing the answer and argument based on evidence; the challenger only has access to the question in the initial round. The judges propose answers based on their internal knowledge and challenge their supporters; after multiple rounds of interaction, the judges read the final statements from both sides and output the final answer.

[0036] Step S402, let the number of debate rounds be... In the first round, supporters and challengers each generated their first-round statements. , (16); where, , and These represent the generator functions for supporters, challengers, and judges, respectively. and They represent the first The output of supporters and challengers;

[0037] Step S403, the In this round, both sides update their views based on the other's previous round. , (17);

[0038] Step S404, after After rounds of interaction, the referee will output the final answer. (18); where, and This is the first round of presentations. and For the final round of presentations, This indicates an input problem. This represents the final set of evidence. This indicates the final answer output by the referee.

[0039] Furthermore, in step S4, supporters, challengers, and referees are all instantiated from the same large language model using different role prompt templates, i.e. , , ,in , , These represent the supporter template, challenger template, and referee template, respectively.

[0040] Furthermore, the supporter template requires the output of "answer candidate - key evidence - reasoning chain", the challenger template requires the output of "main objection - missing evidence / conflicting evidence - revised opinion", and the judge template requires the output of "final answer - reasons for acceptance - evidence pointing to". After generation, the system checks whether the output contains necessary fields and evidence pointing to. If fields are missing, evidence references are empty, or the output exceeds the preset length, the system will prompt the model to complete or truncate it again.

[0041] Compared with the prior art, the present invention has the following advantages:

[0042] (1) The present invention organizes long document corpora into long retrieval units and performs retrieval by approximating the long unit score with the maximum similarity of sub-blocks, which significantly reduces the number of retrieval units involved in candidate ranking and subsequent processing: the vector index is still established with sub-blocks of length 256 tokens as the granularity, and the matching accuracy is not lost due to the length of the retrieval unit, while the candidate evidence presented externally is in the form of long retrieval units; under the condition that the same number of candidate entries are returned in a single retrieval, the text range covered is expanded to m times that when the sub-blocks are candidate entries, which is equivalent to the number of candidate entries required to cover the same text range being reduced to 1 / m, where m is the number of sub-blocks contained in a single long retrieval unit, while the scale of entries for downstream evidence utility scoring, preference ranking and context splicing remains at the level of k long retrieval units and does not expand with the increase of document length (see Table 7). Meanwhile, since long retrieval units are based on documents or document groups, cross-segment relationship chains within a document are not segmented within a single retrieval unit. Cross-document relationship chains are covered by the first k long retrieval units, thus preserving the integrity of the cross-segment relationship structure and providing a more complete semantic context for subsequent long context reading (see Table 11).

[0043] (2) This invention introduces a false evidence generation mechanism. Before retrieval, the generation model generates a factual summary based solely on the question as internal knowledge candidate evidence. This summary is then entered into the preference selector for unified ranking along with the retrieval candidates. This mechanism avoids performance degradation when the overall quality of external retrieval is poor and provides a robust fallback path: when the evidence utility label of each real long retrieval unit in the candidate set approaches zero or is negative, the false evidence is selected into the final evidence set because its utility label is still positive. This ensures that the system still has usable evidence input when external retrieval fails. Tables 4 and 5 of Example 1 provide specific examples of this mechanism. The evidence utility label of false evidence g0 is 0.55. It ranks 3rd in the enhanced candidate set and is selected into the final evidence set, indicating that false evidence can compete with real retrieval evidence in the unified ranking without being systematically excluded. In Table 8, the EM of the Naive Gen without any external evidence is 16.80, which constitutes a lower bound reference for performance when there is no valid external evidence. This invention uses false evidence to take over the evidence input when the overall quality of external retrieval is poor, preventing degradation to this level.

[0044] (3) This invention proposes a preference selector for evidence utility alignment, which uses the "reduction in the cost of generating the correct answer after adding evidence" as the evidence utility label. After prior correction to eliminate interference from the model's internal memory, the evidence ranking objective is expanded from simple semantic relevance to generation utility oriented towards correct answering. This makes the evidence sent to the reader not only "relevant" but also "helpful to answer correctly", improving the effective information density of the input context under a fixed retrieval budget. As shown in Table 10, under the condition that the retrieval budget (k=12 candidate long retrieval units, n=8 final evidence units) and the context budget (approximately 30K tokens) remain unchanged, adding the preference selector increases EM from 42.21 to 45.31 (an increase of 3.10 points) and F1 from 55.62 to 56.88 (an increase of 1.26 points). That is, within the same number of evidence items and context length, evidence that "helps to answer correctly" replaces evidence that is "only semantically relevant".

[0045] (4) The present invention designs a multi-agent debate reader with asymmetric information. Supporters, challengers, and judges have different access rights to evidence, which makes the questioning and adjudication process directly focused on the integrity and credibility of the retrieved evidence chain. It can cross-examine residual noise and conflicting evidence, reduce the risk of shared bias spreading along a single path, and thus suppress factual illusions and reasoning biases induced by noisy evidence. As shown in Tables 8 and 10, in the experiments described in the effect verification section, the error rate of the complete scheme of the present invention was 53.73%, which is 4.06 percentage points lower than the basic framework LR-RAG (57.79%) without debate readers, and 10.47 percentage points lower than the standard retrieval enhancement generation method Naive RAG (64.20%), which is the lowest among all comparison methods and ablation variants. Table 11 further gives specific examples of errors induced by noisy evidence: On the same question where the comparison method with short fragments as retrieval units was induced to output incorrect answers by city names in irrelevant fragments, the present invention outputs the correct answer after challenger questioning and judge adjudication. Its process is isomorphic to the example shown in Table 6 of Example 1.

[0046] (5) The method proposed in this invention can provide high-tech products and services for scenarios such as knowledge base question answering, scientific literature retrieval, and legal and financial auxiliary analysis. It achieves more robust evidence selection and more reliable fact consistency generation under lower retrieval budget conditions, generating direct practical value and economic benefits. In the experiment described in the effect verification section, the retrieval budget is: a single retrieval returns k=12 candidate long retrieval units, which are then rearranged by a preference selector and retained as the final evidence set n=8. The overall context budget is about 30K tokens, the number of debate rounds is r=2, and no additional retrieval is triggered during the generation process. As shown in Tables 8 and 9, under this budget condition, the EM and F1 of this invention are higher than all iterative and dynamic methods that require repeated retrieval during the generation process, and are 5.47 points and 6.67 points ahead of the strongest performing DRAGIN, respectively. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is an overall flowchart of a question-answering method based on long document retrieval and multi-agent debate;

[0049] Figure 2 This is a flowchart of the long search unit construction and search process in step S1;

[0050] Figure 3 This is the flowchart for step S2, which generates false evidence.

[0051] Figure 4 This is a flowchart of the preference selection process for evidence utility alignment in step S3;

[0052] Figure 5 This is the flowchart of step S4, the multi-agent debate reader process;

[0053] Figure 6 This is a block diagram of the overall framework of the invention and the multi-agent debate reader structure;

[0054] Figure 7 This is a schematic diagram illustrating the multi-round interactions between supporters, challengers, and judges in this invention. Detailed Implementation

[0055] To gain a deeper understanding of this invention, we will provide a comprehensive and detailed description. However, this invention has various implementations and is not limited to the specific examples listed herein. These examples are presented to enhance a full understanding of the disclosure of this invention.

[0056] Example 1 This embodiment uses multi-hop question-and-answer as an application scenario. The corpus consists of encyclopedia entries and is organized into the basic units of the search index according to the long search unit strategy. (The question is to be answered.) Taking the question "Who directed the film adaptation of the Hugo Award-winning Chinese science fiction novel *The Wandering Earth*?" as an example, a correct answer requires connecting multiple relationships, including "Liu Cixin—Hugo Award winner," "Liu Cixin—author—novel *The Wandering Earth*," "The film *The Wandering Earth*—adapted from—novel *The Wandering Earth*," and "The film *The Wandering Earth*—director—Guo Fan." The standard answer... =“Guo Fan”.

[0057] like Figure 1 As shown, the question-answering method of the present invention based on long document retrieval and multi-agent debate includes the following steps: Step S1: Organize the long document corpus into long retrieval units and retrieve the candidate evidence set; Step S2: The generative model generates false evidence based solely on the question, forming an augmentation candidate set; Step S3: Define evidence utility labels based on the answer generation cost, train a preference selector, and select the final evidence set; Step S4: Input the final set of evidence into the multi-agent debate reader to generate the final answer.

[0058] like Figure 2 As shown, the specific steps for constructing and retrieving long search units in step S1 are as follows: S101, encyclopedia entries are organized into long search units according to documents or document groups, significantly reducing the number of search units. Long search units related to the question in this embodiment are shown in Table 1: Table 1 Examples of Long Search Units

[0059] S102, each long search unit Divide into sub-blocks of length 256 tokens Calculate the vector representation of each sub-block and establish the FAISS inner product index; S103, based on the question with child blocks The problem of calculating the inner product of vectors and matching scores for long search units. After performing a nearest neighbor search on all sub-blocks, backtrack to the corresponding long search unit and take the first neighbor. Each long search unit serves as a candidate evidence set. This embodiment takes The matching scores of each candidate long unit are shown in Table 2:

[0060] As shown in Table 2, g3 received the highest matching score because it contained "the movie The Wandering Earth - Director - Guo Fan". Although g4 contained superficial words such as "The Wandering Earth", it was actually actor information and still received a certain score, constituting noise.

[0061] like Figure 3 As shown, the specific steps for generating false evidence in step S2 are as follows: S201, Construction based solely on the problem Generation prompts The suggestion does not incorporate any external search evidence; S202, from the generative model According to the prompt Generate an encyclopedic factual summary text as false evidence. The false evidence generated in this embodiment As shown in Table 3: Table 3 Examples of False Evidence Generation

[0062] S203, false evidence Add to candidate evidence set To obtain the enhanced candidate set .

[0063] like Figure 4 As shown, the specific steps for preference selection in evidence utility alignment in step S3 are as follows: S301, the standard answer =“Guo Fan” represents the token sequence Define the average negative log-likelihood of the reader model generating the standard answer without incorporating external evidence as the answer generation cost. As shown in formula (4); S302, defining the inclusion of candidate evidence The cost of generating a standard answer later As shown in formula (5); S303, Defining Candidate Evidence On the issue Incremental Evidence Utility As shown in formula (6). For example, evidence g3 contains the answer "Guo Fan" and its relational chain; its addition significantly reduces the generation cost. The value is relatively large and positive; evidence g4 is actor noise information, and its inclusion may induce the model to generate incorrect answers along the "Wu Jing" clue. It is a negative value; S304, Introducing a priori correction strength coefficient In this embodiment, The prior-corrected generation distribution is constructed as shown in Equation (7), and the prior-corrected evidence generation cost is obtained. As shown in formula (8), the final evidence utility label As shown in formula (9). The utility labels for each candidate piece of evidence are shown in Table 4: Table 4 Examples of Candidate Evidence Utility Labels

[0064] As shown in Table 4, g3 has the highest utility label because it directly contains the answer and supports a complete relationship chain; g1 has positive utility because it provides a relationship chain of "Liu Cixin - Hugo Award winner - author of 'The Wandering Earth'"; g4 is noisy evidence and has a negative utility label; false evidence g0 also has some positive utility and can be used as a backtracking candidate.

[0065] S305, on enhancing the candidate set Calculate the utility label for each candidate piece of evidence to form the target utility vector. As shown in formula (10); S306, with preference selector Output predicted utility vector As shown in formula (11), the target utility vector and the predicted utility vector are normalized to preference distributions respectively. P and Q As shown in formulas (12) and (13), this embodiment takes the temperature coefficient. By minimizing P and Q The preference selector is trained using the KL divergence between the KL divergence and the training loss is shown in formula (14). In this embodiment, the preference selector is trained using the AdamW optimizer with a learning rate of 100%. The batch size was set to 16, the number of training rounds to 3, and the warm-up ratio to 0.1. After training, the prediction scores and rankings of the preference selector for the augmentation candidate set are shown in Table 5. Table 5 Examples of Preference Selector Prediction Scores and Rankings

[0066] S307, During the inference phase, the preference selector, after training, enhances the candidate set. Scoring is performed, and the n evidence units with the highest scores are selected as the final evidence set. As shown in formula (15). In this embodiment, n=8, and the overall context token budget is controlled at approximately 30K. Based on this, the relationship between the granularity of the retrieval unit and the text range covered by a single retrieval can be obtained, as detailed in Table 7. In this example That is, the selector prioritizes the retention of evidence units containing the closed relationship chain of "Liu Cixin - Hugo Award winner - author of 'The Wandering Earth' - film adaptation - director Guo Fan", and suppresses noise fragments g4 that are only superficially related to "The Wandering Earth" but do not directly contribute to the correct answer.

[0067] like Figure 5 As shown, the specific steps of the multi-agent debate with the readers in step S4 are as follows: S401, three types of intelligent agents with asymmetric information are set up: supporters, challengers, and judges. In this embodiment, all three types of intelligent agents are instantiated from the same underlying large language model Qwen2.5-14B-Instruct through different role prompt templates, as shown in formula (19). Among them, supporters can access the final evidence set. Challengers only access the question in the initial round. q After multiple rounds of interaction, the referee reads the final statements from both sides. S402, Round 1: Supporters generate answers and arguments based on evidence. Challengers generate answers and questions based on their internal knowledge. As shown in formula (16); S403, p. In each round, both sides update their arguments based on the other's previous round's viewpoints, as shown in formula (17). This embodiment uses the number of debate rounds. The maximum output length for supporters and challengers in each round is set to 256 tokens; S404, via r After rounds of interaction, the referee will output the final answer. As shown in formula (18), the maximum output length of the judge is set to 64 tokens. Examples of the debate outputs of the three-party agents in this embodiment are shown in Table 6: Table 6 Examples of Multi-Agent Debate Outputs

[0068] As shown in Table 6, the supporter proposed the answer "Guo Fan" based on the chain of evidence. The challenger focused on checking for erroneous associations induced by noise fragments such as "starring Wu Jing." After comparing the statements of both sides, the judge output the short answer "Guo Fan," pointing out that its basis came from a complete chain of evidence: "author—work—film—director." The final answer was given in tangible form and verified by evidence constraints, thus stably outputting the correct answer.

[0069] Through the above steps, this invention fully demonstrates, on a single question instance, how the four stages of long retrieval unit organization, false evidence generation, evidence utility alignment selection, and information asymmetry debate synergistically preserve the closed relationship chain of "Liu Cixin—Winning the Hugo Award—Creating 'The Wandering Earth'—Adapting into a Film—Directing Guo Fan," while suppressing the interference of noise fragment g4, which is only superficially related to "The Wandering Earth" and does not directly contribute to the correct answer, ultimately outputting a stable correct answer. However, a single question instance can only illustrate the feasible mechanism of this invention and is insufficient to confirm its technical effect on the overall dataset. Therefore, the effect verification section below verifies each of the beneficial effects described in the invention on a publicly available multi-hop question-answering benchmark dataset through retrieval unit granularity analysis, comparative experiments, ablation experiments, and example analysis, and records the experimental methods used and the corresponding experimental results.

[0070] Effect verification

[0071] This section statistically verifies the technical effects of the present invention on a publicly available multi-hop question-answering benchmark dataset, and records the experimental methods used and the corresponding experimental results to confirm each of the beneficial effects described in the invention.

[0072] (I) Experimental Setup

[0073] 1. Dataset

[0074] The experiment uses the publicly available multi-hop question-answering benchmark dataset HotpotQA. This dataset uses Wikipedia as its knowledge source, and questions typically require integrating multiple pieces of evidence across documents to complete the reasoning, exhibiting significant multi-hop characteristics and evidence dependence, making it suitable for testing the effectiveness of long document retrieval and evidence-driven generation. The corpus consists of Wikipedia entries and is organized into basic units of the retrieval index according to the long retrieval unit strategy described in step S1 of this invention. The experiment uses publicly available partitions of this dataset, where the training partition is used to train a preference selector aligned with evidence utility, and the validation and test partitions are used to report the final performance.

[0075] 2. Evaluation Indicators

[0076] Suppose the test set contains N samples, and the problem of the i-th sample is q. i The standard answer is a. i The model predicts the answer as âi. The following three indicators are used.

[0077] (1) Exact Match (EM): This measures whether the predicted answer and the standard answer are completely identical. After normalizing the case, punctuation, and articles of the predicted and standard answers respectively, if they are completely identical, it is recorded as 1; otherwise, it is recorded as 0. The average is then calculated over all samples. EM is the main indicator in this section.

[0078] (2) Word-level F1 score (F1): measures the degree of partial matching between the predicted answer and the standard answer at the word level. For the i-th sample, first calculate the precision and recall between the predicted answer and the standard answer token set, then take the harmonic mean of the two, and average them over all samples.

[0079] (3) Error Rate (ER): Defined as ER = 100 − EM, which is the percentage of samples whose system output is inconsistent with the standard answer. ER is derived from the identity of EM and does not involve additional measurement. It is used to measure the overall level of end-to-end answer errors. The specific form of errors caused by erroneous entity associations induced by noisy evidence is illustrated by the example analysis described in Part (V) of this section.

[0080] 3. Selection of comparison methods

[0081] The selection of the comparison method corresponds to the deficiencies described in points one through five of the background art, in order to verify the effectiveness of the present invention in addressing these five deficiencies one by one:

[0082] To address the shortcomings mentioned in the first point of the background technology (using short paragraphs as retrieval units leads to an excessively large retrieval space and fragmented evidence chains) and the second point (candidate ranking mainly relies on relevance, with noise being delivered to the reader along with key facts), the standard retrieval enhancement generation method Naive RAG, which uses short segments as retrieval units and ranks by relevance, and SuRe, which compresses and reorganizes retrieval information in a summary manner, are selected. To address the shortcomings mentioned in the third point of the background technology (iterative and dynamic retrieval repeatedly triggers query updates around short segments, resulting in high inference costs and unstable cross-segment relationship chains), IRCoT, Iter-RetGen, FLARE, and DRAGIN are selected. To address the shortcomings mentioned in the fourth point of the background technology (evidence utilization methods mainly rely on the self-verification capabilities within a single model), Self-RAG is selected. To address the shortcomings mentioned in the fifth point of the background technology (existing multi-agent debate methods share the same input or similar priors among roles, often oriented towards closed-book or weak evidence constraint scenarios), MAD is selected. Naive Gen is also set as a lower bound reference for performance without using any external evidence.

[0083] The comparison methods are explained below:

[0084] (1) Naive Gen: The answer is directly generated by the generative model based on the question, without using external retrieval;

[0085] (2) MAD: Improves closed-book answering ability through multi-agent interaction without using retrieval;

[0086] (3) Naive RAG: Standard retrieval enhancement generative framework, which retrieves top-k evidence and then concatenates it into the input to generate a model to directly answer the question;

[0087] (4) IRCot: Iterative retrieval and response are guided by chain thinking, and retrieval is triggered step by step through intermediate reasoning;

[0088] (5) Iter-RetGen: The retrieval and generation alternate iteratively, gradually approaching the answer through a cycle of "generation - re-retrieval - re-generation";

[0089] (6) FLARE: Utilizes uncertainties or feedback mechanisms in the generation process to trigger retrieval and correction;

[0090] (7) SuRe: Compresses and reorganizes search information by means of summarization or rewriting to reduce noise and improve usability;

[0091] (8) Self-RAG: Combines retrieval, generation and self-reflection to improve the quality of the final answer through self-assessment and revision;

[0092] (9) DRAGIN: Introduces dynamic retrieval and multi-stage interaction mechanism, and performs multiple rounds of reasoning and verification during the generation process;

[0093] (10) The method of the present invention: that is, the complete solution described in the claims of the present invention, which includes all the technical features of steps S1 to S4;

[0094] 4. Implementation parameters

[0095] The generation model uniformly adopts Qwen2.5-14B-Instruct to ensure consistent generation capabilities when comparing different methods and avoid confusion caused by differences in reader models. All methods use the same decoding strategy during the generation phase: temperature is set to 0.2, top-p is set to 0.9, and the maximum generation length is set to 128 tokens. In the setting including multi-agent debate, the maximum output length per round for supporters and challengers is set to 256 tokens, the maximum output length for judges is set to 64 tokens, and the number of debate rounds r is set to 2. Supporters, challengers, and judges are all instantiated from the same base model using different role prompt templates to reduce the bias introduced by additional models and maintain reproducibility.

[0096] The retrieval parameters are consistent with those in Example 1: long retrieval units are divided into sub-blocks of length 256 tokens and a FAISS inner product vector index is established; the first k=12 candidate long retrieval units returned by the retrieval constitute the candidate evidence set; after adding false evidence g0, an enhancement candidate set is formed, which is then rearranged by the preference selector, and the first n=8 are selected as the final evidence set; the overall context token budget is controlled at approximately 30K; the maximum generation length of false evidence g0 is set to 256 tokens; the prior correction strength coefficient λ is set to 0.5, and the temperature coefficient τ is set to 1.0. The preference selector is trained using the AdamW optimizer, with a learning rate of 0.00002, a batch size of 16, 3 training epochs, and a warm-up ratio of 0.1. Except for the target module, all other settings remain consistent in the ablation experiment.

[0097] (II) Analysis of retrieval unit granularity and text coverage

[0098] This invention uses documents or document groups as long retrieval units, sub-blocks of length 256 tokens as the vector index granularity, and approximates the overall relevance of the long unit by using the sub-block with the highest score within the long unit. Let m be the number of sub-blocks contained in a single long retrieval unit. The impact of the retrieval unit granularity on the number of retrieval units and the text range covered by a single retrieval is shown in Table 7.

[0099] Table 7 Comparison of retrieval unit granularity and number of retrieval units

[0100] As shown in Table 7, the vector index is built with sub-blocks of length 256 tokens in both granularities, so the matching accuracy of the retrieval is not lost due to the increase in the length of the retrieval unit; the difference between the two lies in the candidate evidence items presented externally. Under the condition of returning the same number of candidate items k=12, the text range covered by a single retrieval of the present invention is m times that when sub-blocks are used as candidate items; equivalently, under the condition of covering the same text range, the number of candidate items that the present invention needs to return is reduced to 1 / m when sub-blocks are used as candidate items, and the total number of retrieval units in the corpus that need to participate in candidate sorting and backtracking is also reduced from the total number of sub-blocks to 1 / m of the total number of sub-blocks. Since the long retrieval unit is based on the document or document group as the granularity, and the sub-block length is fixed at 256 tokens, for documents with a length exceeding 256 tokens, m>1 is always true, and m increases with the increase of document length.

[0101] This is precisely the quantitative manifestation of step S101's shift from "finding the needle in a massive number of paragraphs" to "finding the relevant region in a smaller number of long units": the scale of candidate entries participating in scoring, ranking, and backtracking remains at the level of k long retrieval units, while the text range that a single retrieval can cover expands by m times, and the overhead of downstream evidence utility scoring, preference ranking, and context splicing does not increase with the length of the document. At the same time, since the long retrieval unit is at the granularity of document or document group, the cross-segment relationship chain within a document is not split within a single retrieval unit, and the cross-document relationship chain is covered by the first k long retrieval units. This is the source of the integrity of the cross-segment relationship structure described in the beneficial effect (1), and its qualitative verification can be found in the example analysis described in section (V) of this part.

[0102] (III) Comparison of experimental results and analysis

[0103] Table 8 shows the comparison results of the complete solution of this invention with nine existing technical methods on HotpotQA.

[0104] Table 8. Results of HotpotQA Comparison Experiment

[0105] The following conclusions can be drawn from Table 8:

[0106] 1. Optimal overall performance. The method of this invention achieves the best results among all compared methods in both the EM and F1 scores, with an EM score of 46.27 and an F1 score of 58.41. The error rate is reduced to 53.73%, the lowest among all compared methods.

[0107] 2. The effectiveness in addressing the shortcomings described in points one and two of the background technology. Compared with the standard search enhancement generation method Naive RAG, which uses short fragments as retrieval units and sorts results by relevance, the method of this invention improves EM by 10.47 points, F1 by 12.61 points, and reduces the error rate from 64.20% to 53.73%. Compared with SuRe (29.00 / 42.39), which compresses information using abstracts, EM is improved by 17.27 points and F1 by 16.02 points.

[0108] 3. The effect of solving the shortcomings mentioned in point 3 of the background technology. Compared with iterative and dynamic methods that require repeated triggering of retrieval during the generation process, the present invention achieves better results by performing only one retrieval, as shown in Table 9.

[0109] Table 9 Comparison between the present invention and iterative retrieval methods

[0110] The four methods mentioned above all require multiple rounds of retrieval and generation during the reasoning process, resulting in higher retrieval call counts and generation costs compared to this invention. This invention outperforms all other methods under the conditions of k=12 candidates, n=8 final evidence, a context budget of approximately 30K tokens, and a single retrieval. This demonstrates that aligning the evidence utility ranking with the answer generation cost as a label makes evidence selection within a fixed and lower retrieval budget more robust, eliminating the need to rely on repeated retrievals to compensate for insufficient evidence in a single retrieval.

[0111] 4. The effect of addressing the shortcomings mentioned in point four of the background technology. Compared with Self-RAG (24.80 / 38.81), which relies on self-verification within a single model, the method of this invention improves EM by 21.47 points and F1 by 19.60 points. Self-RAG's self-evaluation and revision are both completed within the same model, and its error correction ability is limited when the model forms a shared bias against noisy evidence. This invention overcomes the limitations of single-model self-verification by setting information asymmetry among supporters, challengers, and judges, ensuring that the challenger does not share the same batch of retrieved evidence as the supporter.

[0112] 5. The effectiveness of addressing the shortcomings mentioned in point 5 of the background technology. The EM values ​​of both retrieval-free methods are significantly lower than all methods incorporating external retrieval, indicating that external evidence remains crucial for multi-hop question answering tasks. Specifically, the EM of Naive Gen is 16.80, forming a lower bound for performance when no external evidence is used and no multi-agent verification is performed; the EM of MAD is 23.00, indicating that under the same condition of not using external evidence, multi-agent interaction itself can bring a gain of 6.20 points.

[0113] The false evidence g0 of this invention is generated solely by the generative model based on the question, without introducing any external retrieval evidence in the generation prompts. Its information source is the same as the internal knowledge of the model. When the overall quality of external retrieval is poor and the evidence utility label of each real long retrieval unit in the candidate set approaches zero or is negative, the false evidence is selected into the final evidence set by the preference selector according to formula (15) because its utility label is still positive. The system changes from being driven by external evidence to being generated by internal knowledge through debate verification, thereby avoiding the situation of degenerating to the point of having no valid evidence input. Tables 4 and 5 of Example 1 give specific examples of this mechanism: the false evidence g0 obtains a utility label of 0.55, ranks 3rd in the enhancement candidate set, and is selected into the final evidence set.

[0114] (iv) Ablation Experiment Results and Analysis

[0115] To analyze the contribution of each component module to system performance, ablation experiments were conducted on HotpotQA. All variants used the same generative model (Qwen2.5-14B-Instruct) and a unified inference configuration, keeping the retrieval budget (k=12, n=8) and context budget (approximately 30K tokens) unchanged. Comparisons were made only in two aspects: whether an evidence utility-aligned preference selector was enabled on the retrieval side, and whether a multi-agent debate reader was enabled on the generation side. Specifically: LR-RAG represents the basic framework retaining only long retrieval units and long context readers; LS-RAG represents adding a preference selector to LR-RAG but not using a debate reader; LD-RAG represents adding a multi-agent debate reader to LR-RAG but not using a preference selector; LDM-RAG represents including both of the above enhancements, i.e., the complete scheme of this invention. The results are shown in Table 10.

[0116] Table 10 Results of HotpotQA Ablation Experiment

[0117]

[0118] The following conclusions can be drawn from Table 10:

[0119] 1. Contribution of the Preference Selector. Compared to LR-RAG, LS-RAG improved EM by 3.10 points (42.21→45.31), F1 by 1.26 points (55.62→56.88), and the incorrect answer rate decreased from 57.79% to 54.69%. Both LS-RAG and LR-RAG have identical retrieval budgets, context budgets, generation models, and decoding strategies. The final evidence set for both is 8 long retrieval units, and the context is approximately 30K tokens. Therefore, the improvements are entirely due to the alignment and ranking of evidence utility labeled by the cost of answer generation: without increasing the number of evidence items or the length of the context, the selector replaced "semantically relevant" evidence with evidence that "helps to answer correctly," thereby increasing the effective information density of the input context under a fixed retrieval budget.

[0120] 2. Contributions from debate readers. Compared to LR-RAG, LD-RAG improved EM by 0.99 points (42.21→43.20) and F1 by 1.20 points (55.62→56.82), while the incorrect answer rate decreased from 57.79% to 56.80%. Both methods used the same sources of evidence and retrieval budgets; the only difference lay in whether the generation side included cross-examination of information asymmetry between supporters, challengers, and judges. Therefore, the improvements stemmed from the debate mechanism's cautious acceptance of evidence and adversarial verification of reasoning paths.

[0121] 3. Two types of enhanced composability. By further adding a debate reader to the LS-RAG scheme with the preference selector enabled, the complete LDM-RAG scheme of this invention is obtained. The EM score is improved from 45.31 to 46.27 (an improvement of 0.96 points), the F1 score from 56.88 to 58.41 (an improvement of 1.53 points), and the incorrect answer rate is reduced from 54.69% to 53.73%. This indicates that the preference selector and the debate reader respectively act in the screening stage before the search results enter the reader's field and the verification stage when the reader generates the answer. They are not interchangeable, and their benefits are cumulative.

[0122] 4. Overall Gains of the Complete Solution. Compared to the basic framework LR-RAG, LDM-RAG improves EM by 4.06 points, F1 by 2.79 points, and reduces the error rate by 4.06 percentage points. EM and F1 are the highest among the four variants. The fact that EM and F1 improve in the same direction and that the complete solution outperforms any single-module variant demonstrates that this invention improves both the two key steps of what evidence the reader sees and how the reader uses that evidence through the synergy of long-unit organization and evidence utility alignment at the retrieval end, and information asymmetry in multi-agent debate at the generation end.

[0123] (V) Example Analysis

[0124] To further illustrate the preservation of cross-segment relationship structure and suppression of noise-induced errors in complex multi-hop problems, a real-world example from the HotpotQA validation set is selected for qualitative analysis. The question is "The Oberoi family is part of a hotel company that has a head office in what city?", with the standard answer being Delhi. A correct answer requires at least two cascaded pieces of evidence: first, the Oberoi family has a subordinate relationship with The Oberoi Group; second, The Oberoi Group's headquarters are located in Delhi. The output comparisons of various methods on this example are shown in Table 11.

[0125] Table 11 Comparison of outputs of different methods on the Oberoi sample.

[0126] The difficulty of this example lies in the fact that the corpus not only contains entity information related to the Oberoi family, but also easily recalls interfering fragments containing surface words such as hotel and city. Furthermore, some unrelated hotel descriptions also contain explicit city names (such as Jakarta), which can easily be mistaken for the final answer when searching for short fragments or generating data directly.

[0127] As shown in Table 11, the Naive RAG, which uses short fragments as retrieval units, breaks the complete chain of evidence into local fragments, failing to recover the cross-segment relationship chain of "family-hotel group-headquarters city". Furthermore, it is misled by the city name in irrelevant hotel fragments, resulting in the incorrect answer "Jakarta". This also reflects the shortcomings described in points one, two, and four of the background technology. The LR-RAG and LS-RAG, which use long retrieval units, as well as the complete solution of this invention, all retain all links of the relationship chain within a single retrieval unit, thus outputting the correct answer "Delhi". This directly confirms that the integrity of the cross-segment relationship structure is guaranteed by the granularity of the long retrieval units.

[0128] Furthermore, the advantages of the complete solution of this invention in this example are reflected in two stages: on the retrieval side, the preference selector tends to retain evidence units containing closed relational chains and suppresses noise fragments that are only superficially related to "hotel" or "city" and do not directly contribute to the correct answer; on the generation side, the supporter proposes the answer "Delhi" based on the evidence chain, the challenger focuses on checking whether there are erroneous mappings introduced by information about other hotel locations, and the judge finally makes a ruling based on the completeness of the evidence and cross-document consistency. This process is isomorphic to the example shown in Table 6 of Example 1: in Example 1, the challenger questions the noise fragment g4, and the judge rejects the interference based on the closed relational chain formed by g1 and g3 and outputs the correct answer "Guo Fan".

[0129] The experimental results above show that the present invention improves what evidence readers see by organizing long search units and aligning with the utility of evidence at the retrieval end, and improves how readers use evidence by employing multi-agent debate with information asymmetry at the generation end. The benefits of the two stages are cumulative. Under the conditions of a single retrieval and a fixed and low retrieval budget and context budget, the EM and F1 scores of the present invention are superior to all nine prior art methods, and the error answer rate is the lowest among all comparison methods and ablation variants, thus confirming the beneficial effects described in the invention.

[0130] Contents not described in detail in this specification are prior art known to those skilled in the art. Although illustrative specific embodiments of the invention have been described above to facilitate understanding by those skilled in the art, it should be understood that the invention is not limited to the scope of the specific embodiments. Various modifications are readily apparent to those skilled in the art as long as they fall within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of this invention are protected.

Claims

1. A question-answering method based on long document retrieval and multi-agent debate, characterized in that, Includes the following steps: Step S1: Organize the long document corpus into long retrieval units, segment the long retrieval units into sub-blocks and establish vector indexes, and retrieve the candidate evidence set based on the vector similarity between the question and the sub-blocks. Step S2: The generative model generates false evidence based solely on the question, and adds the false evidence to the candidate evidence set to form an enhanced candidate set. Step S3: Define the evidence utility labels of candidate evidence based on the answer generation cost, train a preference selector that aligns evidence utility, score the enhanced candidate set by the preference selector and select the highest-scoring evidence as the final evidence set; Step S4: The final set of evidence is input into the multi-agent debate audience, and the supporters, challengers and judges generate the final answer through multiple rounds of cross-examination and adjudication.

2. The question-answering method based on long document retrieval and multi-agent debate according to claim 1, characterized in that, The specific steps of step S1 are as follows: Step S101: Organize the long document corpus into long retrieval units according to documents or document groups. ; Step S102, each long search unit Cut into Sub-block Calculate the vector representation of each sub-block and establish a vector index; Step S103, based on the problem with child blocks The problem of calculating the inner product of vectors and matching scores for long search units. After performing a nearest neighbor search on all sub-blocks, backtrack to the corresponding long search unit and take the first neighbor. Each long search unit serves as a candidate evidence set. ,in and These represent the query encoder and the text encoder, respectively.

3. The question-answering method based on long document retrieval and multi-agent debate according to claim 1, characterized in that, The specific steps of step S2 are as follows: Step S201, construct based solely on the problem Generation prompts ; Step S202, from the generative model According to the prompt Generate an encyclopedic factual summary text as false evidence. ; Step S203, remove false evidence Add to candidate evidence set To obtain the enhanced candidate set .

4. The question-answering method based on long document retrieval and multi-agent debate according to claim 1, characterized in that, The specific steps of step S3 are as follows: Step S301, provide the standard answer. Represented as a token sequence Define the average negative log-likelihood of the reader model generating the standard answer without incorporating external evidence as the answer generation cost. ; Step S302, define the candidate evidence to be added The cost of generating a standard answer later ; Step S303: Define the incremental evidentiary utility of candidate evidence. ; Step S304, introduce the prior correction strength coefficient Construct the prior-corrected generation distribution and the prior-corrected evidence utility label. ; Step S305, enhance the candidate set Calculate the utility label for each candidate piece of evidence to form the target utility vector. ; Step S306, using the preference selector Output the predicted utility vector, and normalize both the target utility vector and the predicted utility vector into preference distributions. and By minimizing and KL divergence between the two is used to train the preference selector; Step S307, during the inference phase, the trained preference selector enhances the candidate set. Score and select the highest score. The evidence serves as the final set of evidence. .

5. The question-answering method based on long document retrieval and multi-agent debate according to claim 4, characterized in that, The cost of answer generation in step S301 The calculation formula is: In the formula, This represents the conditional generation probability of the reader model. Indicates the prefix of the answer.

6. The question-answering method based on long document retrieval and multi-agent debate according to claim 4, characterized in that, The prior-corrected generated distribution in step S304 is: In the formula This represents the logits output by the reader model. For the priori correction strength coefficient, when It degenerates into a normal "problem + evidence" generation distribution when The influence of priors within the model is partially offset; the cost of evidence generation after prior correction is... A priori corrected evidentiary utility label .

7. The question-answering method based on long document retrieval and multi-agent debate according to claim 4, characterized in that, In step S306, the target preference distribution P and the predicted preference distribution Q are respectively: ; In the formula, The temperature coefficient is ; the training loss is .

8. The question-answering method based on long document retrieval and multi-agent debate according to claim 1, characterized in that, The specific steps of step S4 are as follows: Step S401: Set up three types of intelligent agents with information asymmetry: supporters, challengers, and judges. Supporters have access to the final evidence set. Challengers only access the question in the initial round. After multiple rounds of interaction, the referee reads the final statements from both sides. Step S402, let the number of debate rounds be... In the first round, supporters and challengers each generated their first-round statements. , ; Step S403, the In this round, both sides update their views based on the other's previous round. , ; Step S404, after After rounds of interaction, the referee will output the final answer. ,in , , These represent the generator functions for supporters, challengers, and judges, respectively.

9. A question-answering method based on long document retrieval and multi-agent debate according to claim 8, characterized in that, In step S4, supporters, challengers, and referees are all instantiated from the same large language model using different role prompt templates. , , ,in , , These represent the supporter template, challenger template, and referee template, respectively.

10. A question-answering method based on long document retrieval and multi-agent debate according to claim 9, characterized in that, The supporter template requires the output of "answer candidate - key evidence - reasoning chain", the challenger template requires the output of "main objection - missing evidence / conflicting evidence - revised opinion", and the judge template requires the output of "final answer - reasons for acceptance - evidence pointing to". After generation, the system checks whether the output contains necessary fields and evidence pointing to. If fields are missing, evidence references are empty, or the output exceeds the preset length, the system will prompt the model to complete or truncate it again.

Images