Dynamic retrieval strategy adaptive adjustment method and system based on reinforcement learning

Abstract

The invention relates to the technical field of artificial intelligence and information retrieval, in particular to a dynamic retrieval strategy self-adaptive adjustment method and system based on reinforcement learning. According to the method, a retrieval process is modeled into a Lukov decision process, and a strategy model is trained to realize autonomous optimization and dynamic adjustment of a retrieval strategy. The method comprises the following steps: defining a state space comprising a query type, context correlation, a historical retrieval effect and knowledge base characteristics; defining an action space including vector retrieval, keyword retrieval and mixed retrieval; constructing a dual-target reward function fusing the retrieval income and the retrieval cost; training a strategy model by utilizing a historical retrieval track; calling the strategy model in real time to dynamically adjust the retrieval mode when the retrieval enhancement generation system runs; and the model is iteratively optimized through continuous feedback. According to the method, the problems of low recall rate and low calculation efficiency of a traditional fixed retrieval strategy under complex query are solved, dynamic balance between retrieval precision and resource consumption is realized, and the method is suitable for intelligent retrieval scenes of multi-field and multi-modal knowledge bases.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and information retrieval technology, specifically relating to a dynamic retrieval strategy adaptive adjustment method and system based on reinforcement learning, which is particularly suitable for improving the intelligence and adaptive capability of the retrieval module in a retrieval enhancement generation system. Background Technology

[0002] Retrieval-enhanced generation (RAG) systems, which enhance the generation quality of large language models by retrieving external knowledge bases, have become the mainstream architecture for knowledge-intensive question answering. Traditional RAG systems typically employ fixed retrieval strategies, such as semantic retrieval based on vector similarity or text retrieval based on keyword matching, lacking dynamic awareness of query complexity, contextual relevance, and historical retrieval performance. Existing technologies include rule-based or supervised learning-based retrieval strategy selectors, but they still suffer from the following drawbacks: first, they rely on manually preset rules, making it difficult to adapt to diverse query patterns and knowledge base structures; second, they cannot achieve a dynamic balance between retrieval accuracy and computational efficiency; and third, they lack cross-domain adaptive capabilities, leading to decreased retrieval performance on new domain knowledge bases. Therefore, there is an urgent need for a retrieval strategy adjustment method that can learn autonomously, optimize dynamically, and possess strong generalization capabilities. Summary of the Invention

[0003] The technical problem to be solved by this invention is: how to break through the limitations of traditional fixed retrieval strategies, realize the autonomous and dynamic adjustment of retrieval strategies, and optimize the efficiency of computing resource utilization while ensuring retrieval quality.

[0004] To address the aforementioned technical problems, this invention provides a dynamic retrieval strategy adaptive adjustment method based on reinforcement learning. The technical solution includes: modeling the retrieval process as a Wolf-Knowledge decision process; training the strategy model through reinforcement learning, enabling it to autonomously select the optimal retrieval method based on real-time query features and contextual environment; designing a dual-objective reward function to simultaneously optimize retrieval revenue and cost; and utilizing historical retrieval trajectories for model training and iterative optimization, supporting cross-domain transfer and online learning. This invention also provides a corresponding system, including a state-aware module, a strategy decision-making module, a retrieval execution module, and a feedback learning module. These modules work collaboratively to achieve dynamic adjustment and continuous optimization of the retrieval strategy.

[0005] The beneficial effects of this invention include: (1) Enables autonomous and dynamic selection of retrieval strategies, significantly improving recall and precision under complex queries; (2) By optimizing the dual objectives, the consumption of computing resources can be effectively reduced while ensuring the quality of retrieval; (3) It has cross-domain adaptive capabilities and can be applied to text, structured data and multimodal knowledge bases; (4) Supports continuous learning and iterative optimization to adapt to the dynamic changes in knowledge base content and query patterns. Attached Figure Description

[0006] Figure 1 This is a flowchart of a dynamic retrieval strategy adaptive adjustment method based on reinforcement learning provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the system module structure provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a Morkov decision process modeling provided in an embodiment of the present invention. Detailed Implementation The present invention will be further described in detail below with reference to the accompanying drawings.

[0007] like Figure 1 As shown, the dynamic retrieval strategy adaptive adjustment method of the present invention includes the following steps: Step S101: State Awareness. Receive user query and current context information, extract semantic features through a pre-trained language model, and construct a state vector by combining historical retrieval performance metrics and knowledge base type identifiers.

[0008] Step S102: Policy Decision. Input the state vector into the policy model trained by reinforcement learning. The model outputs the probability distribution of each retrieval action (vector retrieval, keyword retrieval, and hybrid retrieval). Select the current retrieval strategy based on probability sampling.

[0009] Step S103: Search Execution. Based on the selected strategy, the corresponding search engine is invoked to perform the search operation and obtain relevant document fragments.

[0010] Step S104: Performance Evaluation. Calculate recall, precision, and retrieval time based on the search results, and calculate the immediate reward using a bi-objective reward function.

[0011] Step S105: Model Update. Store the current state, action, reward, and next state in the experience replay buffer, and periodically sample data to train and update the policy model.

[0012] like Figure 2 As shown, the system of the present invention includes the following modules: The state awareness module 201 is used to extract query and context features in real time, integrate historical retrieval data, and output a state representation.

[0013] The strategy decision module 202 has a built-in strategy network, which receives state representations and outputs action probabilities to realize dynamic selection of retrieval strategies.

[0014] The retrieval execution module 203 includes a vector retrieval unit, a keyword retrieval unit, and a hybrid retrieval unit, and calls the corresponding retrieval device according to the instructions of the strategy decision module.

[0015] The feedback learning module 204 collects retrieval performance data, calculates reward values, and updates the policy network parameters based on the reinforcement learning algorithm.

[0016] like Figure 3 As shown, the Morkov decision process modeling includes: state space S (query features, context relevance, historical effects, knowledge base type), action space A (vector retrieval, keyword retrieval, hybrid retrieval), reward function R (integrating retrieval benefits and costs), and state transition function P (describing the impact of retrieval actions on the system state).

[0017] In a preferred embodiment, the policy model is trained using a proximal policy optimization algorithm, and the reward function is designed as follows: ( R = \alpha \cdot \text{Recall} + \beta \cdot \text{Precision} - \gamma \cdot \text{TimeCost} ) where (\alpha), (\beta), and (\gamma) are weight coefficients, and TimeCost is the normalized retrieval time.

[0018] This system can be integrated into the LangChain framework, achieving seamless integration with existing RAG pipelines through a custom Retriever class. Experiments show that on a hybrid knowledge base of financial and technical documents, this invention improves recall by approximately 28% and reduces retrieval time by approximately 35% compared to a fixed retrieval strategy.

[0019] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A dynamic retrieval strategy self-adaptive adjustment method based on reinforcement learning, characterized in that, The method comprises the following steps: (1) model the retrieval process as a Markov decision process, define state space, action space and reward function; (2) train a policy model using historical retrieval trajectories, the policy model learns to select the optimal retrieval strategy under different states; (3) call the policy model in real time to dynamically select the retrieval method during the running of the retrieval enhancement generation system; (4) continuously optimize the policy model according to the retrieval effect feedback. The method of claim 1, wherein The state space includes at least one of the semantic features of the current query, the context correlation score, the historical retrieval success rate and the knowledge base type identifier; the action space includes three retrieval methods of vector retrieval, keyword retrieval and hybrid retrieval; the reward function is a double-objective optimization function, which optimizes the retrieval revenue index and the retrieval cost index at the same time; the retrieval revenue index includes at least one of recall rate, precision and answer relevance; the retrieval cost index includes at least one of retrieval response time and computing resource occupancy rate; the policy model is trained using proximal policy optimization or deep Q network algorithm, and supports transfer learning of cross-domain knowledge base; The historical retrieval trajectory is collected by LangSmith tool, including state sequence, action sequence and corresponding reward value. A system for implementing the method of any one of claims 1 to 7, characterized in that It comprises: a state perception module for extracting query and context features and constructing a state vector; a strategy decision module containing a policy model trained by reinforcement learning, for outputting retrieval action probability distribution; a retrieval execution module supporting dynamic calling of three retrieval methods of vector retrieval, keyword retrieval and hybrid retrieval; a feedback learning module for collecting retrieval effect data and updating policy model parameters. The system of claim 8, wherein The system is integrated in LangChain or LlamaIndex framework, and is seamlessly connected with retrieval enhancement generation pipeline. The system according to claim 8, characterized in that The strategy decision module supports two optimization modes of online learning and offline batch updating.

Images