A method and system for generating search enhancements for resource adaptive allocation search

Abstract

This invention discloses a retrieval enhancement generation method and system with adaptive allocation of retrieval resources, belonging to the field of artificial intelligence and information retrieval technology. To address the technical problems of existing retrieval enhancement generation systems, such as static and fixed retrieval configurations, difficulty in jointly selecting retrievers, information granularity, and context quantity based on question differences, and the tendency for cost optimization to sacrifice answer quality, this invention constructs a unified multi-granularity candidate pool at the document, paragraph, and sentence levels. It determines quality-gated rewards based on training questions and trains a policy agent. The input question is state-encoded, and the target policy agent outputs a target composite action including retriever type, information granularity, and context quantity, thereby retrieving the context and generating the answer. This invention can adaptively allocate retrieval resources according to question characteristics, reducing retrieval and generation overhead while maintaining answer quality.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and information retrieval technology, and particularly relates to a retrieval enhancement generation method and system with adaptive allocation of retrieval resources. Background Technology

[0002] Retrieval-Augmented Generation (RAG) mitigates model illusion, supplements real-time knowledge, and improves factual reliability in complex question-answering tasks by introducing external knowledge base retrieval results before generating answers from a large language model. A typical RAG system generally includes steps such as document segmentation, index building, query retrieval, context concatenation, and answer generation. In existing solutions, the system typically pre-fixes a retrieval tool, such as BM25 based on keyword matching, DPR based on vector semantic matching, or a hybrid retrieval tool; it also typically fixes a text segmentation granularity, such as fixed-length text blocks, paragraphs, or document-level fragments; and it also fixes the number of fragments retrieved during inference, such as Top-3 or Top-5.

[0003] The aforementioned fixed process has the advantages of simple implementation and convenient engineering deployment, but it assumes that all user questions require the same retrieval intensity. For questions that can be answered with just one fact, the fixed retrieval of a large context will introduce a lot of irrelevant text, increasing the computational cost of retrieval, reordering, and generative models in processing the context. For questions that require multi-hop reasoning, cross-entity comparison, or global semantic understanding, the fixed small-granularity segmentation may disrupt the semantic coherence of the original text, preventing the generative model from obtaining sufficient evidence.

[0004] In recent years, some adaptive RAG methods have begun to attempt to determine whether to trigger a retrieval based on model confidence or to route to different retrieval processes based on preset question complexity. However, most of these solutions focus on "when to retrieve" or "whether to retrieve." Once a retrieval is decided, the retrieval type, information granularity, and context size are still pre-set, making it difficult to achieve a fine-grained balance between retrieval cost and answer quality for each specific question.

[0005] Existing retrieval enhancement generation systems have at least the following technical defects: (1) The retrieval configuration is static and fixed, making it difficult to adapt to the differences in information granularity, context depth and semantic coverage of different questions; (2) Retrieving too much context for simple factual questions leads to computational redundancy, increased generation delay, and may reduce the accuracy of the answer due to interference from irrelevant context; (3) Retrieving too fine granularity or insufficient context for complex reasoning questions can easily cause semantic fragmentation, lack of evidence and generation illusion; (4) Existing adaptive retrieval strategies focus on whether or when to retrieve, lacking joint decision-making on "how to retrieve", and cannot simultaneously select the retrieval instrument, text granularity and context scale; (5) If delay or cost is used as the optimization objective, the strategy model may collapse into an overly lightweight retrieval scheme, thereby sacrificing the quality and factuality of the answer. Summary of the Invention

[0006] The purpose of this invention is to propose a retrieval enhancement generation method and system with adaptive allocation of retrieval resources, in order to solve the technical problems of existing retrieval enhancement generation systems, such as static and fixed retrieval configuration, difficulty in jointly selecting retrieval tools, information granularity and context quantity according to question differences, and the tendency to sacrifice answer quality for cost optimization. The invention aims to achieve the effect of adaptively allocating retrieval resources according to the input question and reducing retrieval and generation costs while ensuring answer quality.

[0007] To achieve the above objectives, the present invention adopts the following technical solution.

[0008] A retrieval enhancement generation method with adaptive allocation of retrieval resources includes the following steps: The original documents in the knowledge base are organized hierarchically according to document level, paragraph level and sentence level to obtain a unified multi-granularity candidate pool; Training answers are obtained based on the training question, the unified multi-granularity candidate pool, and the composite action space. A quality gating reward is determined based on the answer quality, retrieval relevance, and computational cost of the composite action. The composite action space includes the retrieval type, information granularity, and number of contexts. The quality gating reward is triggered when the answer quality reaches a preset quality threshold. The policy agent is preheated and trained based on the composite action that meets the preset quality threshold and has the lowest computational cost, and then fine-tuned by reinforcement learning based on the quality-gated reward to obtain the target policy agent. The input problem is state-encoded to obtain a problem state representation; The problem state representation is input into the target policy agent to obtain the target composite action, which includes the retrieval type, information granularity, and number of contexts. Based on the target composite action, the retrieval index corresponding to the retrieval type and information granularity in the target composite action is called from the unified multi-granularity candidate pool, and the context is retrieved according to the number of contexts in the target composite action. The context and the input question are then input into the large language model to obtain the generated answer.

[0009] Furthermore, the original documents in the knowledge base are organized hierarchically at the document, paragraph, and sentence levels to obtain a unified multi-granularity candidate pool, including: The original document is retained as a document-level candidate; The original document is segmented according to natural paragraphs to obtain paragraph-level candidates; The original document is segmented into sentences to obtain sentence-level candidates; The document-level candidates, paragraph-level candidates, and sentence-level candidates are hierarchically organized to obtain a unified multi-granularity candidate pool.

[0010] Furthermore, the document-level candidates, paragraph-level candidates, and sentence-level candidates are hierarchically organized to obtain a unified multi-granularity candidate pool, including: Establish a hierarchical relationship among the document-level candidates, the paragraph-level candidates, and the sentence-level candidates; Based on the hierarchical relationship, granularity identifiers are configured for the document-level candidates, paragraph-level candidates, and sentence-level candidates to obtain a unified multi-granularity candidate pool.

[0011] Furthermore, training answers are obtained based on the training question, the unified multi-granularity candidate pool, and the composite action space. A quality-gated reward is determined based on the answer quality, retrieval relevance, and computational cost of the composite action, including: The training problem is state-encoded to obtain the training problem state representation; Based on the training problem state representation and the compound action space, determine the training compound actions; Based on the training compound action, the training context is retrieved from the unified multi-granularity candidate pool, and the training context and the training question are input into the large language model to obtain the training answer; The quality gating reward is determined based on the quality of the training answer, the retrieval relevance of the training context, and the computational cost of the training compound action.

[0012] Furthermore, based on the training problem state representation and the compound action space, the training compound actions are determined, including: The training problem state representation is input into the policy agent to obtain the training action selection result; Based on the training action selection results, the retrieval type, information granularity, and number of contexts are determined from the composite action space; The retriever type, the information granularity, and the number of contexts are combined into a training compound action.

[0013] Further, based on the quality of the training answer, the retrieval relevance of the training context, and the computational cost of the training compound action, a quality gating reward is determined, including: The quality of the answers is determined based on the training answers and reference answers. Determine retrieval relevance based on the training context and the training question; The computational cost is determined based on the training compound action; The efficiency reward is triggered when the answer quality reaches a preset quality threshold. The quality gating reward is determined based on the answer quality, the retrieval relevance, and the computational cost.

[0014] Furthermore, the policy agent is pre-trained based on a composite action that satisfies the preset quality threshold and has the lowest computational cost, and then fine-tuned through reinforcement learning based on the quality-gated reward to obtain the target policy agent, including: Based on the training problem and the compound action space, the approximate optimal compound action that meets the preset quality threshold and has the lowest computational cost is determined. Based on the near-optimal composite action, the policy agent is subjected to supervised preheating training to obtain a preheated policy agent. The preheating strategy agent is fine-tuned using reinforcement learning based on quality-gated rewards to obtain the target strategy agent.

[0015] Furthermore, based on the training problem and the compound action space, the approximate optimal compound action that satisfies the preset quality threshold and has the lowest computational cost is determined, including: Based on the training problem, the retrieval type, information granularity and context number in the composite action space are combined to obtain candidate composite actions; Based on the candidate composite action, candidate contexts are obtained from a unified multi-granularity candidate pool, and candidate answers are generated based on the candidate contexts. Based on the answer quality of the candidate answers, determine the available composite actions that meet the preset quality threshold from the candidate composite actions; Based on the computational cost of the available composite actions, an approximately optimal composite action is determined from the available composite actions.

[0016] Furthermore, the preheating strategy agent is fine-tuned using reinforcement learning based on quality-gated rewards to obtain the target strategy agent, including: The training problem is state-encoded to obtain a training problem state representation, and the training problem state representation is input into the warm-up strategy agent to obtain a fine-tuning compound action; Based on the aforementioned fine-tuning compound action, retrieval enhancement generation is performed to obtain the fine-tuned answer and fine-tuning context; The quality-gated reward is determined based on the fine-tuning answer, the fine-tuning context, and the fine-tuning compound action. The preheating policy agent is updated based on the quality gating reward to obtain the target policy agent.

[0017] Further, the preheating policy agent is updated based on the quality gating reward to obtain the target policy agent, including: Based on the preheating strategy agent and the updated strategy agent, determine the probability ratio between the old and new strategies; Based on the probability ratio of the old and new strategies and the quality-gated reward, the constraints of the updated policy agent are updated. The policy agent that completes the constraint update is identified as the target policy agent.

[0018] Furthermore, the input problem is state-encoded to obtain a problem state representation, including: The input question is semantically encoded to obtain a semantic representation of the question. Based on the semantic representation of the problem, the problem intent features, keyword salience features, entity salience features, semantic complexity features, and evidence requirement features are determined. The problem state representation is obtained by fusing the problem intent features, the keyword saliency features, the entity saliency features, the semantic complexity features, and the evidence requirement features.

[0019] Further, the problem state representation is input into the target policy agent to obtain the target composite action, including: The problem state representation is input into the target policy agent to obtain the composite action selection result; The retriever type, information granularity, and number of contexts are determined based on the results of the composite action selection. The retriever type, the information granularity, and the number of contexts are combined into a target composite action.

[0020] Further, the retriever type, information granularity, and number of contexts are determined based on the results of the composite action selection, including: Based on the results of the composite action selection, the searcher type is determined from sparse search, dense search, and hybrid search; Based on the results of the composite action selection, the information granularity is determined from the document level, paragraph level, and sentence level; Based on the result of the composite action selection, the number of contexts is determined from a preset set of numbers.

[0021] Further, based on the target composite action, a retrieval index corresponding to the retrieval type and information granularity in the target composite action is retrieved from the unified multi-granularity candidate pool, and the context is retrieved according to the number of contexts in the target composite action. The context and the input question are then input into a large language model to obtain a generated answer, including: Based on the information granularity in the target composite action, the target candidate range is determined from the unified multi-granularity candidate pool; Based on the retrieval type in the target composite action, the retrieval index corresponding to the target candidate range is invoked to perform a retrieval and obtain the candidate context; The context is determined from the candidate contexts based on the number of contexts in the target compound action; The context and the input question are input into a large language model to obtain the generated answer.

[0022] Further, based on the retrieval type in the target composite action, the retrieval index corresponding to the target candidate range is invoked to perform a retrieval, obtaining the candidate context, including: Based on the type of retrieval device in the target composite action, determine the sparse retrieval channel, dense retrieval channel, or mixed retrieval channel; Based on the sparse retrieval channel, the dense retrieval channel, or the hybrid retrieval channel, the target candidate range is retrieved to obtain the candidate context.

[0023] Furthermore, the context and the input question are input into a large language model to obtain a generated answer, including: The context and the input question are concatenated to obtain a generated prompt; The generated prompts are input into a large language model to obtain the generated answer.

[0024] A retrieval enhancement generation system with adaptive allocation of retrieval resources, comprising: The multi-granularity corpus construction module is used to hierarchically organize the original documents in the knowledge base according to the document level, paragraph level and sentence level to obtain a unified multi-granularity candidate pool; The quality gating reward module is used to obtain training answers based on the training question, the unified multi-granularity candidate pool, and the composite action space, and to determine the quality gating reward based on the answer quality, retrieval relevance, and computational cost of the composite action. The composite action space includes the retrieval type, information granularity, and number of contexts. The quality gating reward is triggered when the answer quality reaches a preset quality threshold. A two-stage training module is used to perform preheating training on the policy agent based on the composite action that meets the preset quality threshold and has the lowest computational cost, and to perform reinforcement learning fine-tuning on the policy agent based on the quality-gated reward to obtain the target policy agent. The problem status encoding module is used to encode the status of the input problem to obtain a problem status representation. An adaptive policy decision module is used to input the problem state representation into the target policy agent to obtain a target composite action, wherein the target composite action includes retrieval type, information granularity and context quantity; The retrieval and generation module is used to retrieve the retrieval index corresponding to the retrieval type and information granularity in the target composite action from the unified multi-granularity candidate pool, retrieve the context according to the number of contexts in the target composite action, input the context and the input question into the large language model, and obtain the generated answer.

[0025] The key point of this invention is: 1. This invention models the retrieval configuration selection process in retrieval enhancement generation as a query-aware reinforcement learning decision problem or a context-aware multi-armed slot machine decision problem, enabling the retrieval configuration to be adaptively selected based on the state representation of the input question.

[0026] 2. This invention determines the searcher type, information granularity, and context quantity simultaneously through compound actions, thereby achieving joint decision-making on search method, search scope, and context depth.

[0027] 3. This invention constructs a unified multi-granularity candidate pool including document level, paragraph level and sentence level, so that the retrieval scope can be dynamically switched between different information granularities according to the complexity of the input question.

[0028] 4. This invention proposes a quality-gated reward mechanism, which uses the answer quality reaching a preset quality threshold as the trigger condition for efficiency rewards, thereby preventing the policy agent from favoring low-quality, lightweight retrieval configurations during training.

[0029] 5. This invention employs a training process that combines Oracle-guided warm-up with near-end policy optimization reinforcement learning fine-tuning, enabling the policy agent to continue updating parameters based on quality-gated rewards on the basis of the warm-up policy.

[0030] 6. In the inference stage, this invention determines the target composite action in real time based on the problem state representation of the input question, and performs retrieval and answer generation based on the target composite action, thereby reducing retrieval and generation delays while maintaining the factuality and accuracy of the answer.

[0031] The advantages of this invention compared to the closest prior art are: 1. Compared to fixed-method retrieval enhancement generation schemes, this invention does not use the same retrieval tool, information granularity, and context size for all questions. Instead, it dynamically determines the target composite action based on the input question. Therefore, this invention can reduce redundant retrieval for simple questions and provide a matching context range for complex questions.

[0032] 2. Compared with dynamic search enhancement generation schemes that only determine whether or when to search, the present invention further makes joint decisions on searcher type, information granularity and context quantity, so that the search configuration selection covers search method, text granularity and context scale.

[0033] 3. Compared with problem routing methods based on manual rules, the policy agent of this invention is trained and can learn the rules of retrieval resource allocation according to different training questions and their corresponding feedback. For example, for questions about dates, locations, and entity types, the policy agent can choose sparse retrieval and a smaller number of contexts; for questions about multi-hop reasoning, causal explanations, and complex descriptions, the policy agent can choose dense retrieval or mixed retrieval and a larger number of contexts.

[0034] 4. The quality-gated reward mechanism of this invention uses the achievement of a preset quality threshold as the trigger condition for efficiency rewards, ensuring that efficiency optimization is based on meeting the answer quality standard, thus avoiding over-compression of the search scope when solely targeting latency or cost. Experimental results show that this invention achieves a good balance between quality and efficiency on multiple question-answering benchmarks, achieving approximately 20.2 times speedup on NewsQA and improving answer quality on SQuAD, indicating that this invention has engineering application value. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the framework of a retrieval enhancement generation method with adaptive allocation of retrieval resources according to the present invention; Figure 2 This is a schematic diagram comparing the static RAG with the adaptive retrieval process of this invention; Figure 3 A schematic diagram illustrating the construction of a unified multi-granularity candidate pool; Figure 4 This is a schematic diagram of the quality gating reward function logic; Figure 5 This is a schematic diagram of the two-stage strategy training process; Figure 6 This is a schematic diagram of the adaptive retrieval execution process during the inference phase; Figure 7 This diagram illustrates the quality and efficiency effects on different datasets. Detailed Implementation

[0036] To make the various technical features, advantages, or effects of the present invention more apparent and understandable, detailed descriptions are provided below through embodiments.

[0037] This invention provides a retrieval augmentation generation method (QARP) with adaptive allocation of retrieval resources. Before running this method, the knowledge base includes original documents, the training phase includes training questions and reference answers, and the composite action space pre-includes the retrieval type, information granularity, and number of contexts, and pre-sets an encoding model for state encoding, a policy agent for action selection, and a large language model for answer generation.

[0038] This method models the retrieval configuration selection process in retrieval enhancement generation as a contextual multi-armed slot machine or reinforcement learning decision problem. For the problem... The coding model generates a problem state representation. The policy agent is based on the problem state representation Output compound action ,in, Indicates the searcher type; Indicates information granularity; Indicates the number of contexts. Figure 1 This illustrates the processing relationships between policy agents, multi-granularity retrieval, generative models (large language models), and quality-gated reward mechanisms. Figure 2 This demonstrates the difference between this method and existing static RAGs, which lies in dynamically selecting the processing relationship of information granularity and context scope according to the needs of the problem.

[0039] Step S1: Organize the original documents in the knowledge base hierarchically according to document level, paragraph level and sentence level to obtain a unified multi-granularity candidate pool.

[0040] In an optional embodiment of the present invention, step S1 may include: Step S11: Retain the original document as a document-level candidate.

[0041] Specifically, document-level candidates are used to represent the complete text content of the original document, denoted as... .

[0042] Step S12: The original document is segmented according to natural paragraphs to obtain paragraph-level candidates.

[0043] Specifically, paragraph-level candidates are used to represent the content of natural paragraphs in the original document, denoted as... .

[0044] Step S13: Segment the original document according to sentences to obtain sentence-level candidates.

[0045] Specifically, sentence-level candidates are used to represent sentence content in the original document, denoted as... .

[0046] Step S14: The document-level candidates, paragraph-level candidates, and sentence-level candidates are hierarchically organized to obtain a unified multi-granularity candidate pool.

[0047] Specifically, the unified multi-granularity candidate pool is denoted as... And satisfy . Figure 3 This demonstrates a hierarchical structure for document-level, paragraph-level, and sentence-level candidates.

[0048] In an optional embodiment of the present invention, step S14 may include: Step S141: Establish the hierarchical relationship between the document-level candidates, the paragraph-level candidates, and the sentence-level candidates.

[0049] Step S142: Configure granularity identifiers for the document-level candidates, paragraph-level candidates, and sentence-level candidates based on the hierarchical relationship to obtain a unified multi-granularity candidate pool.

[0050] Step S2: Obtain training answers based on the training question, the unified multi-granularity candidate pool, and the composite action space. Determine quality gating rewards based on the answer quality, retrieval relevance, and computational cost of the composite actions. The composite action space includes retrieval type, information granularity, and number of contexts. The quality gating rewards are triggered when the answer quality reaches a preset quality threshold.

[0051] Specifically, the searcher types include sparse search, dense search, and hybrid search; the information granularity includes document level, paragraph level, and sentence level; the context number is used to represent the number of candidates retrieved, and can be set to 1, 3, 5, or other preset values ​​according to the context window and efficiency requirements.

[0052] In an optional embodiment of the present invention, step S2 may include: Step S21: Encode the state of the training problem to obtain the state representation of the training problem.

[0053] Step S22: Based on the training problem state representation and the compound action space, determine the training compound action.

[0054] Step S23: Based on the training compound action, retrieve the training context from the unified multi-granularity candidate pool, and input the training context and the training question into the large language model to obtain the training answer.

[0055] Step S24: Determine the quality gating reward based on the answer quality of the training answer, the retrieval relevance of the training context, and the computational cost of the training compound action.

[0056] In an optional embodiment of the present invention, step S22 may include: Step S221: Input the training problem state representation into the policy agent to obtain the training action selection result.

[0057] Step S222: Based on the training action selection results, determine the retrieval type, information granularity, and number of contexts from the composite action space.

[0058] Step S223: Combine the retrieval type, the information granularity, and the number of contexts into a training compound action.

[0059] In an optional embodiment of the present invention, step S24 may include: Step S241: Determine the answer quality based on the training answer and the reference answer.

[0060] Step S242: Determine the retrieval relevance based on the training context and the training question.

[0061] Step S243: Determine the computational cost based on the training compound action.

[0062] Step S244: The efficiency reward is triggered by the answer quality reaching a preset quality threshold. The quality gating reward is determined based on the answer quality, the retrieval relevance, and the computational cost.

[0063] Specifically, in one implementation, the quality gating reward satisfies .in, Indicates quality gated rewards; Indicates the quality of the answer; Indicates search relevance; Indicates a compound action The computational cost; , and Indicates the weighting coefficient; Indicates the preset quality threshold; Indicates an indicator function.

[0064] Specifically, when the answer quality does not reach the preset quality threshold, the quality gating reward does not include the efficiency reward; when the answer quality reaches the preset quality threshold, the quality gating reward includes the efficiency reward corresponding to the computation cost. Figure 4 The logic of how the efficiency penalty coefficient affects answer quality and inference delay is shown.

[0065] Step S3: Perform preheating training on the policy agent based on the composite action that satisfies the preset quality threshold and has the lowest computational cost, and perform reinforcement learning fine-tuning on the policy agent based on the quality-gated reward to obtain the target policy agent.

[0066] Specifically, the warm-up training is Oracle-guided warm-up, and the reinforcement learning fine-tuning is reinforcement learning fine-tuning based on proximal policy optimization. Figure 5 This demonstrates how Oracle boot warm-up is integrated with near-end policy optimization reinforcement learning fine-tuning.

[0067] In an optional embodiment of the present invention, step S3 may include: Step S31: Based on the training problem and the compound action space, determine the approximate optimal compound action that satisfies the preset quality threshold and has the lowest computational cost.

[0068] Step S32: Based on the near-optimal composite action, supervised preheating training is performed on the policy agent to obtain the preheated policy agent.

[0069] Specifically, supervised warm-up training can use cross-entropy loss to train the policy agent.

[0070] Step S33: Based on quality-gated rewards, perform reinforcement learning fine-tuning on the preheating strategy agent to obtain the target strategy agent.

[0071] In an optional embodiment of the present invention, step S31 may include: Step S311: Based on the training problem, the retrieval type, information granularity and context quantity in the composite action space are combined to obtain candidate composite actions.

[0072] Step S312: Obtain candidate context from the unified multi-granularity candidate pool based on the candidate composite action, and generate candidate answer based on the candidate context.

[0073] Step S313: Based on the answer quality of the candidate answers, determine the available composite actions that meet the preset quality threshold from the candidate composite actions.

[0074] Step S314: Based on the computational cost of the available composite actions, determine the approximately optimal composite action from the available composite actions.

[0075] In an optional embodiment of the present invention, step S33 may include: Step S331: Encode the state of the training problem to obtain the state representation of the training problem, and input the state representation of the training problem into the warm-up strategy agent to obtain the fine-tuning compound action.

[0076] Step S332: Based on the fine-tuning compound action, perform retrieval enhancement generation to obtain the fine-tuned answer and fine-tuning context.

[0077] Step S333: Determine the quality gating reward based on the fine-tuning answer, the fine-tuning context, and the fine-tuning compound action.

[0078] Step S334: Update the preheating strategy agent based on the quality gating reward to obtain the target strategy agent.

[0079] In an optional embodiment of the present invention, step S334 may include: Step S3341: Determine the probability ratio of the old and new strategies based on the preheating strategy agent and the updated strategy agent.

[0080] Step S3342: Based on the probability ratio of the new and old strategies and the quality gating reward, update the constraints of the updated policy agent.

[0081] Step S3343: The policy agent that has completed the constraint update is identified as the target policy agent.

[0082] Step S4: Encode the input problem into a state to obtain a problem state representation.

[0083] Specifically, state encoding can be accomplished by an encoding model that supports multi-granularity semantic representation and hybrid retrieval features. The question state representation is used to characterize the high-level intent of the input question, keyword saliency, entity saliency, semantic complexity, and potential evidence requirements.

[0084] In an optional embodiment of the present invention, step S4 may include: Step S41: Semantically encode the input question to obtain a semantic representation of the question.

[0085] Step S42: Determine the problem intent features, keyword salience features, entity salience features, semantic complexity features, and evidence requirement features based on the problem semantic representation.

[0086] Step S43: The problem intent feature, the keyword saliency feature, the entity saliency feature, the semantic complexity feature, and the evidence requirement feature are fused to obtain the problem state representation.

[0087] Step S5: Input the problem state representation into the target policy agent to obtain the target composite action, which includes the retrieval type, information granularity, and number of contexts.

[0088] Specifically, the target compound action is denoted as ,in, Indicates the searcher type; Indicates information granularity; Indicates the number of contexts.

[0089] In an optional embodiment of the present invention, step S5 may include: Step S51: Input the problem state representation into the target policy agent to obtain the composite action selection result.

[0090] Step S52: Determine the retrieval type, information granularity, and number of contexts based on the results of the composite action selection.

[0091] Step S53: Combine the retriever type, the information granularity, and the number of contexts into a target composite action.

[0092] In an optional embodiment of the present invention, step S52 may include: Step S521: Based on the results of the composite action selection, determine the searcher type from sparse search, dense search, and hybrid search.

[0093] Step S522: Based on the result of the composite action selection, determine the information granularity from the document level, paragraph level, and sentence level.

[0094] Step S523: Determine the number of contexts from a preset number set based on the result of the composite action selection.

[0095] Specifically, for questions involving entities, dates, and locations, the target composite action can include sparse retrieval, sentence-level or document-level retrieval, and Top-1; for semantically rich questions that require causal explanation or multi-hop evidence aggregation, the target composite action can include dense retrieval, mixed retrieval, paragraph-level or document-level retrieval, and Top-3 or Top-5.

[0096] Step S6: Based on the target composite action, retrieve the retrieval index corresponding to the retrieval type and information granularity in the target composite action from the unified multi-granularity candidate pool, retrieve the context according to the number of contexts in the target composite action, input the context and the input question into the large language model to obtain the generated answer.

[0097] Specifically, Figure 6 The complete process from inputting a question to selecting a target compound action, generating a retrieval, and recording feedback is illustrated. In one implementation, answer quality, retrieval relevance, and delayed feedback can be recorded for subsequent online or offline strategy updates.

[0098] In an optional embodiment of the present invention, step S6 may include: Step S61: Determine the target candidate range from the unified multi-granularity candidate pool based on the information granularity in the target composite action.

[0099] Step S62: Based on the retrieval type in the target composite action, call the retrieval index corresponding to the target candidate range to perform a retrieval and obtain the candidate context.

[0100] Step S63: Determine the context from the candidate contexts based on the number of contexts in the target compound action.

[0101] Step S64: Input the context and the input question into the large language model to obtain the generated answer.

[0102] In an optional embodiment of the present invention, step S62 may include: Step S621: Determine the sparse retrieval channel, dense retrieval channel, or mixed retrieval channel based on the retrieval type in the target composite action.

[0103] Step S622: Based on the sparse retrieval channel, the dense retrieval channel, or the hybrid retrieval channel, the target candidate range is retrieved to obtain the candidate context.

[0104] In an optional embodiment of the present invention, step S64 may include: Step S641: Concatenate the context and the input question to obtain a generated prompt.

[0105] Step S642: Input the generated prompt into the large language model to obtain the generated answer.

[0106] This invention also provides a retrieval enhancement generation system with adaptive allocation of retrieval resources, comprising: The multi-granularity corpus construction module is used to hierarchically organize the original documents in the knowledge base according to the document level, paragraph level and sentence level to obtain a unified multi-granularity candidate pool; The quality gating reward module is used to obtain training answers based on the training question, the unified multi-granularity candidate pool, and the composite action space, and to determine the quality gating reward based on the answer quality, retrieval relevance, and computational cost of the composite action. The composite action space includes the retrieval type, information granularity, and number of contexts. The quality gating reward is triggered when the answer quality reaches a preset quality threshold. A two-stage training module is used to perform preheating training on the policy agent based on the composite action that meets the preset quality threshold and has the lowest computational cost, and to perform reinforcement learning fine-tuning on the policy agent based on the quality-gated reward to obtain the target policy agent. The problem status encoding module is used to encode the status of the input problem to obtain a problem status representation. An adaptive policy decision module is used to input the problem state representation into the target policy agent to obtain a target composite action, wherein the target composite action includes retrieval type, information granularity and context quantity; The retrieval and generation module is used to retrieve the retrieval index corresponding to the retrieval type and information granularity in the target composite action from the unified multi-granularity candidate pool, retrieve the context according to the number of contexts in the target composite action, input the context and the input question into the large language model, and obtain the generated answer.

[0107] Method performance testing: This invention was validated on question-answering benchmarks such as SQuAD, NewsQA, HotpotQA, and TriviaQA. Experiments used Qwen2-7B-Instruct as the generative model, BGE-M3 as the state encoder, and a three-layer multilayer perceptron as the policy network, trained through proximal policy optimization. Evaluation metrics included answer quality, retrieval accuracy, and average latency. Answer quality was measured using BERTScore, retrieval accuracy using NDCG@5, and efficiency was measured using average latency and relative speedup. Figure 7 This diagram illustrates the quality and efficiency performance of this application on different datasets, demonstrating the efficiency gains of the adaptive strategy in factual and complex reasoning tasks. Table 1 provides the main experimental results.

[0108] Table 1 Main Experiment Results As shown in Table 1, this invention achieves a better balance between quality and efficiency on all four datasets. On SQuAD, the quality of the proposed solution reaches 0.4702, an improvement of approximately 11.4% compared to the Best Static solution, while latency is reduced to 12.17 ms. On NewsQA, the quality of the proposed solution reaches 0.6330, with an average latency of only 0.68 ms, representing a 20.2x speedup compared to the Common configuration. On complex tasks such as HotpotQA and TriviaQA, this invention maintains competitive answer quality while significantly reducing latency, demonstrating that the adaptive retrieval strategy can avoid resource mismatch in different question types caused by static configuration.

[0109] Table 2. Results of dynamic search baseline comparison Table 2 shows a comparison of the present invention with existing dynamic retrieval frameworks on TriviaQA. Self-RAG improves answer quality through reflective generation, but the additional generation overhead results in a latency of 185.4 ms; Adaptive-RAG reduces some latency through question routing, but still uses a fixed retrieval configuration after the path is selected. QARP directly optimizes the retriever, granularity, and number of contexts, reducing latency to 20.6 ms, demonstrating the efficiency advantage of structured retrieval decisions.

[0110] Table 3. Results of Strategy Action Distribution Table 3 illustrates that the actions learned by this invention have clear interpretability. In NewsQA, many questions can be answered through sparse retrieval and Top-1 context, thus the strategy converges to a lightweight configuration; SQuAD is more biased towards dense retrieval and document-level coverage; HotpotQA and TriviaQA, due to the need for higher recall due to multi-hop reasoning and complex entity descriptions, tend to have a strategy that favors paragraph-level Top-5 configuration.

[0111] Table 4. Sensitivity Experiment of Efficiency Penalty Coefficient Table 4 verifies the role of the efficiency penalty coefficient in the quality gating reward. When the efficiency penalty coefficient is 0, the system tends to over-recall, with an average latency as high as 561.52 ms; when the penalty coefficient is too large, the system excessively pursues speed, and the answer quality decreases significantly. The optimal balance is achieved when the penalty coefficient is 0.001, maintaining the highest average answer quality while reducing the average latency to 24.90 ms.

[0112] Table 5 Typical Question Types and Adaptive Retrieval Actions Table 5 presents the strategic behaviors for typical questions. For time-related factual questions, the strategy tends to use BM25, sentence-level, and Top-1 to quickly locate the answer by utilizing word overlap. For bridging reasoning questions, the strategy tends to use dense retrieval, paragraph-level, and Top-5 to retain a sufficient chain of evidence. For numerical comparison questions, the strategy can choose a combination of retrieval and sentence-level context to achieve a balance between entity recall and fact localization.

[0113] In summary, the experiment demonstrates from five perspectives—main results, dynamic baseline comparison, strategy action distribution, reward coefficient sensitivity, and typical cases—that this invention can adaptively allocate retrieval resources in question-answering tasks of varying complexity. This avoids redundant computation in static RAGs and reduces the risk of missing evidence in complex questions, thereby achieving a better overall effect in terms of answer quality, retrieval accuracy, and inference latency.

[0114] Alternative implementation methods: The retrieval device in this invention is not limited to BM25, DPR, or hybrid retrieval, and can also be replaced by other sparse retrieval, vector retrieval, graph retrieval, multimodal retrieval, or rearrangement models; the state encoder is not limited to BGE-M3, and can also use other sentence vector models, large language model encoders, or encoders that integrate query statistical features; the generation model is not limited to Qwen2-7B-Instruct, and can also be replaced by other closed-source or open-source large language models.

[0115] The action space in this invention can be further expanded, including whether to perform query rewriting, whether to perform multi-hop sub-question decomposition, whether to call the reorderer, select the context compression ratio, select the number of retrieval rounds, or select different knowledge sources, etc. The answer quality indicators in the reward function can also use exact matching, F1, BERTS score, human feedback score, fact consistency test score, or a combination of multiple indicators. The above substitutions and expansions do not affect the core idea of ​​this invention: dynamically selecting retrieval configuration based on query status and achieving quality-first efficiency optimization through quality-gated rewards.

[0116] Although the present invention has been disclosed above with reference to embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent substitutions made by those skilled in the art to the technical solutions of the present invention should be covered within the protection scope of the present invention, which is defined by the claims.

Claims

1. A retrieval enhancement generation method with adaptive allocation of retrieval resources, characterized in that, Includes the following steps: The original documents in the knowledge base are organized hierarchically according to document level, paragraph level and sentence level to obtain a unified multi-granularity candidate pool; Training answers are obtained based on the training question, the unified multi-granularity candidate pool, and the composite action space. A quality gating reward is determined based on the answer quality, retrieval relevance, and computational cost of the composite action. The composite action space includes the retrieval type, information granularity, and number of contexts. The quality gating reward is triggered when the answer quality reaches a preset quality threshold. The policy agent is preheated and trained based on the composite action that meets the preset quality threshold and has the lowest computational cost, and then fine-tuned by reinforcement learning based on the quality-gated reward to obtain the target policy agent. The input problem is state-encoded to obtain a problem state representation; The problem state representation is input into the target policy agent to obtain the target composite action, which includes the retrieval type, information granularity, and number of contexts. Based on the target composite action, the retrieval index corresponding to the retrieval type and information granularity in the target composite action is called from the unified multi-granularity candidate pool, and the context is retrieved according to the number of contexts in the target composite action. The context and the input question are then input into the large language model to obtain the generated answer.

2. The method as described in claim 1, characterized in that, The original documents in the knowledge base are hierarchically organized at the document, paragraph, and sentence levels to obtain a unified multi-granularity candidate pool, including: The original document is retained as a document-level candidate; The original document is segmented according to natural paragraphs to obtain paragraph-level candidates; The original document is segmented into sentences to obtain sentence-level candidates; The document-level candidates, paragraph-level candidates, and sentence-level candidates are hierarchically organized to obtain a unified multi-granularity candidate pool.

3. The method as described in claim 1, characterized in that, Training answers are obtained based on the training question, the unified multi-granularity candidate pool, and the composite action space. A quality-gated reward is determined based on the answer quality, retrieval relevance, and computational cost of the composite action, including: The training problem is state-encoded to obtain the training problem state representation; Based on the training problem state representation and the compound action space, determine the training compound actions; Based on the training compound action, the training context is retrieved from the unified multi-granularity candidate pool, and the training context and the training question are input into the large language model to obtain the training answer; The quality gating reward is determined based on the quality of the training answer, the retrieval relevance of the training context, and the computational cost of the training compound action.

4. The method as described in claim 3, characterized in that, A quality-gated reward is determined based on the quality of the training answer, the retrieval relevance of the training context, and the computational cost of the training compound action, including: The quality of the answers is determined based on the training answers and reference answers. Determine retrieval relevance based on the training context and the training question; The computational cost is determined based on the training compound action; The efficiency reward is triggered when the answer quality reaches a preset quality threshold. The quality gating reward is determined based on the answer quality, the retrieval relevance, and the computational cost.

5. The method as described in claim 1, characterized in that, The policy agent is pre-trained based on a composite action that satisfies the preset quality threshold and has the lowest computational cost, and then fine-tuned through reinforcement learning based on the quality-gated reward to obtain the target policy agent, including: Based on the training problem and the compound action space, the approximate optimal compound action that meets the preset quality threshold and has the lowest computational cost is determined. Based on the near-optimal composite action, the policy agent is subjected to supervised preheating training to obtain a preheated policy agent. The preheating strategy agent is fine-tuned using reinforcement learning based on quality-gated rewards to obtain the target strategy agent.

6. The method as described in claim 5, characterized in that, Based on the training problem and the compound action space, the approximate optimal compound actions that satisfy a preset quality threshold and have the lowest computational cost are determined, including: Based on the training problem, the retrieval type, information granularity and context number in the composite action space are combined to obtain candidate composite actions; Based on the candidate composite action, candidate contexts are obtained from a unified multi-granularity candidate pool, and candidate answers are generated based on the candidate contexts. Based on the answer quality of the candidate answers, determine the available composite actions that meet the preset quality threshold from the candidate composite actions; Based on the computational cost of the available composite actions, an approximately optimal composite action is determined from the available composite actions.

7. The method as described in claim 5, characterized in that, The preheating policy agent is fine-tuned using reinforcement learning based on quality-gated rewards to obtain the target policy agent, including: The training problem is state-encoded to obtain a training problem state representation, and the training problem state representation is input into the warm-up strategy agent to obtain a fine-tuning compound action; Based on the aforementioned fine-tuning compound action, retrieval enhancement generation is performed to obtain the fine-tuned answer and fine-tuning context; The quality-gated reward is determined based on the fine-tuning answer, the fine-tuning context, and the fine-tuning compound action. The preheating policy agent is updated based on the quality gating reward to obtain the target policy agent.

8. The method as described in claim 7, characterized in that, The preheating policy agent is updated based on the quality-gated reward to obtain the target policy agent, including: Based on the preheating strategy agent and the updated strategy agent, determine the probability ratio between the old and new strategies; Based on the probability ratio of the old and new strategies and the quality-gated reward, the constraints of the updated policy agent are updated. The policy agent that completes the constraint update is identified as the target policy agent.

9. The method as described in claim 1, characterized in that, Based on the target composite action, a retrieval index corresponding to the retrieval type and information granularity in the target composite action is retrieved from the unified multi-granularity candidate pool. The context is then retrieved according to the number of contexts in the target composite action. The context and the input question are input into a large language model to generate an answer, including: Based on the information granularity in the target composite action, the target candidate range is determined from the unified multi-granularity candidate pool; Based on the retrieval type in the target composite action, the retrieval index corresponding to the target candidate range is invoked to perform a retrieval and obtain the candidate context; The context is determined from the candidate contexts based on the number of contexts in the target compound action; The context and the input question are input into a large language model to obtain the generated answer.

10. A retrieval enhancement generation system with adaptive allocation of retrieval resources, characterized in that, include: The multi-granularity corpus construction module is used to hierarchically organize the original documents in the knowledge base according to the document level, paragraph level and sentence level to obtain a unified multi-granularity candidate pool; The quality gating reward module is used to obtain training answers based on the training question, the unified multi-granularity candidate pool, and the composite action space, and to determine the quality gating reward based on the answer quality, retrieval relevance, and computational cost of the composite action. The composite action space includes the retrieval type, information granularity, and number of contexts. The quality gating reward is triggered when the answer quality reaches a preset quality threshold. A two-stage training module is used to perform preheating training on the policy agent based on the composite action that meets the preset quality threshold and has the lowest computational cost, and to perform reinforcement learning fine-tuning on the policy agent based on the quality-gated reward to obtain the target policy agent. The problem status encoding module is used to encode the status of the input problem to obtain a problem status representation. An adaptive policy decision module is used to input the problem state representation into the target policy agent to obtain a target composite action, wherein the target composite action includes retrieval type, information granularity and context quantity; The retrieval and generation module is used to retrieve the retrieval index corresponding to the retrieval type and information granularity in the target composite action from the unified multi-granularity candidate pool, retrieve the context according to the number of contexts in the target composite action, input the context and the input question into the large language model, and obtain the generated answer.

Images