Joint optimization retrieval enhancement generation system and method thereof

Abstract

The invention discloses a joint optimization retrieval enhancement generation system and method, and relates to the technical field of artificial intelligence, and the system comprises a strategy network, a retriever and a reinforcement learning module; wherein the strategy network is used for determining whether retrieval is needed or not according to an input state, generating a retrieval query and generating a final answer; the retriever is used for returning related information according to a retrieval query generated by the strategy network; and the reinforcement learning module is used for performing joint optimization on the strategy network and the retriever through a reinforcement learning algorithm. The strategy network and the retriever are subjected to collaborative optimization through the reinforcement learning algorithm, so that the system can integrally optimize collaborative work among all the modules, the overall performance of the system is remarkably improved, retrieval efficiency, answer generation quality, response speed and the like are remarkably improved, and by dynamically adjusting behaviors of all the modules, the search efficiency of the system is improved. The system can be excellent in performance in different input states, and the requirement of a user for efficient and accurate knowledge acquisition is met.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically to a jointly optimized retrieval enhancement generation system and method. Background Technology

[0002] In the field of artificial intelligence, information retrieval and natural language generation are two important tasks. Information retrieval systems are used to quickly find content relevant to user queries from massive amounts of data, while natural language generation systems are used to generate natural, fluent, and accurate answers based on user input. With the explosive growth of information, users' needs for knowledge acquisition are becoming increasingly diverse, and users are placing higher demands on the combination of these two technologies. They expect the system to not only retrieve relevant information but also generate high-quality answers based on that information.

[0003] Traditional Retrieval Augmentation (RAG) systems typically consist of multiple modules, including a Retriever and a Generator. However, these modules are usually trained separately. Due to the lack of collaborative optimization between the modules, the overall system performance may be limited. Therefore, how to overcome the problem of neglecting information redundancy and the impact of interaction during the optimization process of traditional retrieval augmentation systems, and effectively jointly optimize the entire system, is the problem that this invention aims to solve. To this end, a jointly optimized retrieval augmentation system and its method are proposed. Summary of the Invention

[0004] The purpose of this invention is to provide a jointly optimized retrieval enhancement generation system and method to solve the problems mentioned in the background art.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: The first aspect is a jointly optimized retrieval enhancement generation system, which includes a policy network, a retrieval unit, and a reinforcement learning module. The policy network is used to determine whether a search is needed, generate a search query, and generate a final answer based on the input state. The policy network is the core of the system. The retrieval device is used to return relevant information based on the retrieval query generated by the strategy network, and to perform efficient and accurate information retrieval in a pre-organized knowledge base; The reinforcement learning module is used to jointly optimize the policy network and the retrieval system through reinforcement learning algorithms, so as to improve the overall performance of the system.

[0006] A further improvement of the technical solution of the present invention is that the policy network includes a decision-maker unit, a query rewriter unit, an answer generator unit, an external memory component, and an internal memory component; The decision-maker unit is used to assess whether the current amount of information is sufficient to answer the question based on the input status, which includes the input question and optional observation results, and to determine whether a retrieval is needed. It then issues a retrieval instruction and is responsible for receiving and processing the input status, which includes the input question raised by the user and optional observation results. The query rewriter unit is used to generate a query request adapted to the retrieval machine when the decision-maker module decides that a retrieval is required. Natural language processing technology is used to parse the input question, extract key information, and construct the most suitable query request for the retrieval machine based on this. The answer generator unit is used to generate the final answer based on the retrieved observations and the original question, and is the output of the policy network; The external memory component serves as the information repository of the policy network, used to receive and store various information obtained from the environment, including historical issues, user feedback, and external knowledge bases, so that the policy network can call them up at any time in subsequent processing; The internal memory component is used to store the agent's interpretation and understanding of itself, record the key information of the strategy adopted by the policy network when processing the problem, the generated query and the obtained answer, and the policy network's understanding and interpretation of its own processing process, including intermediate results, reasoning process, etc.

[0007] A further improvement to the technical solution of the present invention is that the decision-maker unit specifically includes: The input state consists of an input question and optional observations. The input question is the original question posed by the user, expressed in natural language, and is the main object of analysis for the decision-making unit. The optional observations are additional background information, historical interaction records, and contextual information provided by the user to supplement the semantic background of the input question and help the decision-making unit to more comprehensively understand the intent and needs of the question. Semantic analysis is performed on the input state to determine whether the current information is sufficient to answer the question. Natural Language Processing (NLP) technology is used to preprocess the input question and observation results to extract key information and semantic features. If the judgment result is yes, meaning that the current information is sufficient to answer the question, the decision-maker unit will directly activate the answer generator unit to generate the final answer by combining the input question and the optional observation results. If the judgment result is no, meaning that the current information is insufficient to answer the question, the decision-maker unit will generate a precise search instruction based on the results of semantic analysis, and then activate the searcher to perform the search.

[0008] A further improvement to the technical solution of the present invention is that the query rewriter unit specifically includes: It receives retrieval instructions from the decision-making unit and simultaneously provides input questions from the user; After receiving the input, the query rewriter unit performs semantic parsing on the input question and extracts key information from the input question, including the core intent of the question, the topics involved, and key concepts. Based on the extracted key information, the query rewriter unit generates a query request suitable for the retrieval machine to process.

[0009] A further improvement to the technical solution of the present invention is that the retrieval device specifically includes: Receive a query request generated by the policy network, wherein the query request includes keywords or phrases, semantic information and retrieval instructions; Based on the received query request, information retrieval is performed in the pre-organized knowledge base to search for information related to the query request; The retrieval system returns relevant information to the policy network. If no definite answer is found, it returns relevant results for further processing by the policy network. Relevant results include document fragments, document lists, and summaries or summaries. Document fragments contain text snippets related to the query keywords or semantics. Document lists list the most relevant documents or web page links. Summaries or summaries automatically summarize or extract key information from the retrieved documents to reduce the processing burden on the policy network.

[0010] A further improvement to the technical solution of the present invention is that the answer generator unit specifically includes: The answer generator unit receives the search results and the original question returned by the retrieval machine; Semantic integration is performed on the relevant information returned by the search engine to extract useful information related to the question. At the same time, useful information from multiple search results is merged to form a coherent and complete semantic representation. Based on the search results and the original question, combined with the extracted information, the final answer is generated and then output to the user. Depending on the type of question (factual question, descriptive question, comparative question, etc.), the generated answer is presented to the user in natural language form through text, voice or other interactive methods.

[0011] A further improvement to the technical solution of the present invention is that the reinforcement learning module specifically includes: The system initializes the environment and sets initial parameters, and defines a reward function to reward or punish based on the system's actions and results. This involves building a system environment that includes a policy network, a retrieval system, and an external knowledge base, ensuring that all components can interact normally, and setting initial parameters for the policy network and retrieval system, including the weights of the neural network and the parameters of the retrieval algorithm, as the starting point for reinforcement learning. Then, the reward function is defined as the core of reinforcement learning, incorporating answer accuracy, retrieval result relevance, and system response time as components of the reward function, and calculating the reward value by combining performance across multiple dimensions. The user's input questions and the system's observation results are fed into the policy network for decision-making. The network analyzes whether a search is needed or what kind of query request to generate. Based on the output of the policy network, the searcher executes the corresponding search action and returns the search results. Based on the output actions of the policy network and the search results of the retrieval machine, the accuracy of the answer, the relevance of the search results, and the system response time are comprehensively analyzed. The reward value of each stage is calculated in combination with the predefined reward function. If the search results are highly relevant to the question and the search efficiency is high, a higher reward is given; otherwise, a lower reward or penalty is given. By using a reinforcement learning algorithm based on the policy gradient method, the parameters of the policy network are updated using gradients based on the reward value calculated by the reward function, enabling the policy network to learn the optimal decision policy under different input states, thereby optimizing system performance.

[0012] A further improvement to the technical solution of the present invention is that the calculation process for the return value at each stage is as follows: Natural language processing technology is used to calculate the similarity score between the system-generated answer and the real answer, and the baseline similarity score set according to the system requirements is extracted. Combined with the set control parameters, the accuracy of the answer is calculated. For each result returned by the search engine, its relevance score to the query request is calculated. The TF-IDF algorithm is used to calculate the relevance score between each search result and the query request. The relevance scores of all search results are summed and the sum is calculated. The sum of the relevance scores is divided by the number of search results to obtain the average relevance score. Then, the square root of the average relevance score is taken to obtain the relevance of the search results. The actual response time is the total time required for the system to receive the user's input question, generate the final answer, and output it to the user. The baseline response time set according to the system requirements is extracted, and then the actual response time is divided by the baseline response time to obtain the system response time. The return value for each stage is calculated by comprehensively considering the accuracy of the answer, the relevance of the search results, and the system response time, and by combining the weight coefficients matched according to the system requirements.

[0013] A further improvement to the technical solution of the present invention lies in the application of the external memory component and the internal memory component, specifically including: The policy network reads relevant information from external and internal memory components, merges the two sets of information, and forms a comprehensive input state. Based on the fused relevant information, the policy network makes decisions, processes the input state, analyzes the semantics of the current problem, evaluates whether a retrieval operation is needed, and if so, what kind of query request to generate. Combining the contextual information in external memory and the reasoning experience in internal memory, it generates the optimal decision result, which includes whether to trigger the retrieval machine, the content of the generated query request, or to directly generate the answer. After a decision is executed, the policy network updates the contents of the external and internal memory components based on the decision results and subsequent execution feedback. For the external memory components, the policy network stores new interaction records, relevant information returned by the retrieval system, or user feedback on the generated answers. For the internal memory components, the policy network records the reasoning process of this decision, new understanding of the problem, and lessons learned from the execution of the decision. This allows the policy network to continuously learn and optimize its decision-making capabilities, thereby improving the overall performance and adaptability of the system.

[0014] Secondly, the jointly optimized retrieval enhancement generation method, implemented based on the aforementioned jointly optimized retrieval enhancement generation system, includes the following steps: S1. Construct a system environment that includes a strategy network, a retrieval system, and a knowledge base. Set the initial parameters for the strategy network and the retrieval system, and define a reward function that integrates the accuracy of the answer, the relevance of the retrieval results, and the system response time. S2. Input the user's input question and the system's observation results into the policy network. The policy network evaluates whether the information is sufficient to answer the question through the decision-maker unit and decides whether to trigger a retrieval. S3. If a retrieval is required, the query rewriter unit generates a query request suitable for the retrieval tool, and the retrieval tool retrieves relevant information from the knowledge base based on the query request and returns it. S4. The answer generator unit receives the search results and the original question, performs semantic integration on the search information, extracts useful information, combines the question to generate the final answer and outputs it. S5. Based on the output actions and retrieval results of the policy network, calculate the reward value for each stage in conjunction with the reward function, and evaluate the system's performance in various aspects. S6. Using reinforcement learning algorithms, the parameters of the policy network are updated based on the reward value to optimize the performance of the policy network and the retrieval machine, thereby improving the overall efficiency and accuracy of the system.

[0015] Due to the adoption of the above technical solution, the technical progress achieved by this invention compared to the prior art is as follows: This invention provides a jointly optimized retrieval enhancement generation system and method. By using reinforcement learning algorithms to collaboratively optimize the policy network and the retrieval system, it overcomes the performance bottleneck caused by the separate training of modules in traditional systems. This enables the system to optimize the collaborative work between various modules as a whole, thereby significantly improving the overall performance of the system, including retrieval efficiency, answer generation quality, and response speed. By dynamically adjusting the behavior of each module, the system can perform well under different input states, meeting users' needs for efficient and accurate knowledge acquisition.

[0016] This invention provides a jointly optimized retrieval enhancement generation system and method. It utilizes a query rewriter unit in a policy network to perform deep semantic parsing of user questions, generating query requests most suitable for the retrieval machine. This not only extracts the core intent and key information of the question, but also improves the accuracy of retrieval through semantic expansion and optimization. The retrieval machine can quickly return documents or fragments highly relevant to the question based on the optimized query request, thereby providing higher quality input to the answer generator unit and further improving the accuracy and relevance of the generated answers.

[0017] This invention provides a jointly optimized retrieval enhancement generation system and method. The decision-maker unit in the policy network can intelligently determine whether a retrieval is needed and when to directly generate an answer based on the input state (including user questions and optional observations). This dynamic decision-making mechanism enables the system to flexibly respond to different types of user questions, avoid unnecessary retrieval operations, and save system resources and time. At the same time, through continuous optimization by the reinforcement learning module, the decision-maker unit can continuously learn and adjust its decision-making strategy to adapt to constantly changing inputs and user needs, thereby improving the system's adaptability and robustness. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0019] Figure 1 This is a schematic diagram of the functional modules of the system of the present invention; Figure 2 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1, as Figure 1 As shown, the present invention provides a jointly optimized retrieval enhancement generation system, including a policy network, a retrieval unit, and a reinforcement learning module; The strategy network is the core of the system. It is used to determine whether retrieval is needed, generate retrieval queries, and generate the final answer based on the input state. The strategy network comprehensively evaluates the input state (which includes the user's question and optional observations) to determine whether information retrieval is necessary. When the strategy network determines that the current information is insufficient to answer the question, it generates a precise retrieval query and activates the retrieval device to perform information retrieval. In addition, the strategy network is responsible for generating the final answer by combining the retrieved relevant information with the original question. The strategy network includes a decision-maker unit, a query rewriter unit, an answer generator unit, an external memory component, and an internal memory component. The decision-maker unit is responsible for assessing whether the current information is sufficient to answer the question based on the input state, including the input question and optional observations. It determines whether a retrieval is needed and then issues a retrieval command. The unit receives and processes the input state, which includes the user's input question and optional observations. If the information is insufficient, it initiates the retrieval process; if the information is sufficient, it directly activates the answer generator unit to generate the final answer. The input state includes the received input question and optional observations. The input question, expressed in natural language, is the user's original question and is the primary object of analysis for the decision-maker unit. Optional observations include additional background information, historical interaction records, and contextual information provided by the user to supplement the semantic background of the input question, helping the decision-maker unit to more comprehensively understand the question's intent. The system performs semantic analysis on the input state to determine whether the current information is sufficient to answer the question. This involves preprocessing the input question and observations using Natural Language Processing (NLP) technology to extract key information and semantic features. Through semantic analysis, the decision-making unit understands the core intent of the input question, the key information requirements, and the relevance to the observations. By assessing the semantic complexity of the input question, the degree of missing key information, and the supplementary value of the observations, the system determines whether the current information is sufficient to answer the question. If the result is yes, meaning the current information is sufficient to answer the question, the decision-making unit directly activates the answer generator unit to generate the final answer by combining the input question and optional observations. If the result is no, meaning the current information is insufficient to answer the question, the decision-making unit generates precise retrieval instructions based on the semantic analysis results, thereby activating the retrieval system to perform the search. The query rewriter unit generates a query request adapted to the search engine when the decision-maker module determines that a retrieval is required. It utilizes natural language processing technology to parse the input question, extract key information, and construct the most suitable query request for the search engine. It receives the retrieval instruction from the decision-maker unit and simultaneously provides the user's input question. After receiving the input, the query rewriter unit performs semantic parsing on the input question, extracting key information including the core intent, related topics, and key concepts. It decomposes the input question into basic lexical units, labels the part of speech of each word, and identifies the grammatical relationships between words through dependency analysis, helping to understand the structure and semantics of the question, identifying semantic roles within the question, and further clarifying the question. Semantic structure, based on word segmentation and dependency relation analysis, extracts keywords from the question. Through semantic analysis, it identifies the topic or core concept of the question, ensuring that the generated query request accurately points to relevant knowledge. Combining semantic role labeling and contextual information, it understands the intent of the question. Based on the extracted key information, the query rewriter unit generates a query request suitable for the retrieval machine to process. In this process, the extracted keywords are reasonably combined to form a query expression that can be effectively processed by the retrieval machine. According to the semantics of the question, the keywords are semantically expanded to generate related synonyms or near-synonyms to improve the recall rate of the retrieval. Furthermore, the query request is optimized according to the characteristics of the retrieval machine and the performance requirements of the retrieval machine to ensure retrieval efficiency and accuracy. The answer generator unit generates the final answer based on the retrieved observations and the original question. It is the output of the policy network. By semantically parsing the relevant information returned by the retrieval system, it extracts useful information related to the question. Combining this extracted information with the semantics of the original question, it generates the final answer. The generated answer is further optimized to ensure its logic and coherence, avoiding redundancy or ambiguity. The answer generator unit receives the search results and the original question returned by the retrieval system. The search results include exact answers, relevant document fragments, document lists, or automatic summaries. The original question, expressed in natural language, is the user's initial question and serves as the primary basis for generating the answer. Semantic integration is performed on the relevant information returned by the retrieval system to extract useful information related to the question. Using natural language processing techniques such as word segmentation, part-of-speech tagging, dependency analysis, and semantic role labeling, the unit performs semantic parsing on the text fragments returned by the retrieval system to extract key information and semantic structure. The semantics of the search results are aligned with the semantics of the original question to ensure that the extracted information matches the core intent of the question. The system is highly relevant, extracting keywords, phrases, sentences, or paragraphs directly related to the question from the search results. Redundant information is removed to avoid generating lengthy or repetitive answers. Useful information from multiple search results is integrated to form a coherent and complete semantic representation. Based on the search results and the original question, the extracted information is combined to generate the final answer, which is then output to the user. Specifically, depending on the question type (factual, descriptive, comparative, etc.), an appropriate answer template is selected, and the extracted key information is filled into the template to generate a preliminary answer. Natural language generation technology ensures the generated answer is grammatically and semantically natural and fluent. A coherent answer is generated based on the semantics of the question and the logical relationship between the search results. The language style of the answer is adjusted according to the application scenario and user needs. Furthermore, the length of the answer is controlled based on the complexity of the question and user needs; concise answers are generated for simple questions, and more detailed answers are generated for complex questions. The generated answer is presented to the user in natural language form through text, voice, or other interactive methods. The external memory component serves as the information repository for the policy network, receiving and storing various information acquired from the environment, including historical questions, user feedback, and external knowledge bases. This allows the policy network to access these information at any time during subsequent processing and to quickly retrieve stored information according to the network's needs, providing relevant contextual support. Through continuous accumulation and learning, it provides rich data support for the policy network, further enhancing its decision-making and answer generation capabilities. The internal memory component is used to store the agent's interpretation and understanding of itself, record the key information of the strategy adopted by the policy network when processing the problem, the generated query and the obtained answer, the policy network's understanding and interpretation of its own processing process, including intermediate results, reasoning process, etc., and by storing the semantic parsing results of the input problem, it helps the policy network to better understand the intent and background of the problem. When dealing with complex problems, the internal memory component can provide reasoning support to help the policy network perform multi-step reasoning and decision-making. The retrieval unit returns relevant information based on the retrieval query generated by the policy network. It performs efficient and accurate information retrieval in a pre-organized knowledge base, ensuring the rapid return of the most relevant information to the query request. The retrieval unit receives query requests generated by the policy network, which include keywords or phrases, semantic information, and retrieval instructions. Keywords or phrases are used to locate documents or fragments related to the question in the knowledge base. Semantic information includes the topic, intent, or contextual information of the question to help the retrieval unit more accurately understand the query requirements. Retrieval instructions contain specific retrieval commands or parameters, such as the search scope and sorting method. Based on the received query request, the retrieval unit performs information retrieval in the pre-organized knowledge base, searching for information related to the query request. The retrieval unit returns the relevant information to the policy network. If no definitive answer is found, it returns relevant results for further processing by the policy network. Relevant results include document fragments, document lists, and summaries or summaries. Document fragments contain text fragments related to the query keywords or semantics. Document lists list the most relevant documents or webpage links. Summaries or summaries automatically summarize or extract key information from the retrieved documents to reduce the processing burden on the policy network. The policy network reads relevant information from external and internal memory components. The external memory component stores... Information obtained from the external environment (user input, search engine results, etc.) provides a rich context for the policy network. The internal memory component records the policy network's own reasoning process, understanding of the input question, and historical information of previous interactions. The policy network fuses the two parts of relevant information through a reading mechanism to form a comprehensive input state. Based on the fused relevant information, the policy network makes decisions, processes the input state, analyzes the semantics of the current question, assesses whether a search operation is needed, and if so, what kind of query request to generate. Combining the contextual information in the external memory and the reasoning experience in the internal memory, the optimal decision result is generated. The decision result includes whether to trigger the search engine, the content of the generated query request, or directly generate an answer. After the decision is executed, the policy network updates the contents of the external memory component and the internal memory component based on the decision result and subsequent execution feedback. For the external memory component, the policy network stores new interaction records, relevant information returned by the search engine, or user feedback on the generated answer. For the internal memory component, the policy network records the reasoning process of this decision, the new understanding of the question, and the lessons learned from the decision execution, enabling the policy network to continuously learn and optimize its decision-making ability, improving the overall performance and adaptability of the system. The reinforcement learning module is used to jointly optimize the policy network and the retrieval system through reinforcement learning algorithms to improve the overall system performance. By designing a reward function, corresponding rewards or penalties are given according to the system's actions and results to accurately reflect the system's goals and preferences, thereby guiding the system towards a better policy.

[0022] Example 2, as Figure 1 As shown, based on Embodiment 1, the present invention provides a technical solution: preferably, the reinforcement learning module specifically includes: The system initializes the environment and sets initial parameters, defining a reward function to reward or penalize based on system actions and results. This involves constructing a system environment including a policy network, a retrieval system, and an external knowledge base, ensuring proper interaction between components. Initial parameters are set for the policy network and retrieval system, including neural network weights and retrieval algorithm parameters, serving as the starting point for reinforcement learning. The reward function is then defined as the core of reinforcement learning, incorporating answer accuracy, retrieval result relevance, and system response time as components. The reward value is calculated by combining performance metrics from multiple dimensions. User input questions and system observations (historical interaction records, retrieval system returned information, etc.) are input into the policy network for decision-making. Analysis is... The system determines whether a retrieval or query request needs to be generated. Based on the output of the policy network (retrieval instructions, query requests, etc.), the retrieval machine executes the corresponding retrieval action and returns the retrieval results. Based on the output action of the policy network and the search results of the retrieval machine, the accuracy of the answer, the relevance of the retrieval results, and the system response time are comprehensively analyzed. The reward value of each stage is calculated in conjunction with a predefined reward function. If the retrieval result is highly relevant to the question and the retrieval efficiency is high, a higher reward is given; otherwise, a lower reward or penalty is given. Using the reinforcement learning algorithm of the policy gradient method, the parameters of the policy network are updated according to the reward value calculated by the reward function, so that the policy network learns the optimal decision policy under different input states to optimize system performance. The calculation process for the return value at each stage is as follows: Natural language processing (NLP) technology is used to calculate the similarity score between the system-generated answer and the actual answer. A baseline similarity score set according to system requirements is extracted, and combined with the set control parameters, the answer accuracy is calculated. For each result returned by the retrieval system, its relevance score to the query request is calculated. The TF-IDF algorithm is used to calculate the relevance score between each retrieval result and the query request. The relevance scores of all retrieval results are summed, and the sum of the relevance scores is divided by the number of retrieval results to obtain the average relevance score. The square root of the average relevance score is then taken to obtain the relevance of the retrieval result. The total time required for the system to receive the user's input question, generate the final answer, and output it to the user is obtained, which is the actual response time. A baseline response time set according to system requirements is extracted, and the actual response time is divided by the baseline response time to obtain the system response time. The answer accuracy, retrieval result relevance, and system response time are comprehensively calculated, and combined with the weight coefficients matched according to system requirements, the reward value for each stage is calculated. The formula for calculating the return value at each stage is as follows:

[0023] In the formula, The reward value for each stage, These are weighted parameters, corresponding to the contribution of answer accuracy, search result relevance, and system response time, respectively. They are adjusted according to actual needs. , For the accuracy of the answer, The score is based on the similarity between the actual answer and the correct answer. The baseline similarity score represents the minimum level of similarity that the answers must achieve. These are control parameters used to control the steepness of the function. To ensure the relevance of search results, For the first The relevance score of each search result to the query request. The number of search results returned. For system response time, The actual response time is [actual response time], while the baseline response time is [baseline response time]. This indicates the upper limit of the response time.

[0024] Example 3, as Figure 2 As shown, based on Examples 1-2, the present invention also provides a jointly optimized retrieval enhancement generation method, implemented based on a jointly optimized retrieval enhancement generation system, including the following steps: S1. Construct a system environment that includes a strategy network, a retrieval system, and a knowledge base. Set the initial parameters for the strategy network and the retrieval system, and define a reward function that integrates the accuracy of the answer, the relevance of the retrieval results, and the system response time. S2. Input the user's input question and the system's observation results into the policy network. The policy network evaluates whether the information is sufficient to answer the question through the decision-maker unit and decides whether to trigger a retrieval. S3. If a retrieval is required, the query rewriter unit generates a query request suitable for the retrieval tool, and the retrieval tool retrieves relevant information from the knowledge base based on the query request and returns it. S4. The answer generator unit receives the search results and the original question, performs semantic integration on the search information, extracts useful information, combines the question to generate the final answer and outputs it. S5. Based on the output actions and retrieval results of the policy network, calculate the reward value for each stage in conjunction with the reward function, and evaluate the system's performance in various aspects. S6. Using reinforcement learning algorithms, the parameters of the policy network are updated based on the reward value to optimize the performance of the policy network and the retrieval machine, thereby improving the overall efficiency and accuracy of the system.

[0025] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A jointly optimized retrieval enhancement generation system, characterized in that: This includes policy networks, retrieval systems, and reinforcement learning modules; The policy network is used to determine whether a search is needed, generate a search query, and generate a final answer based on the input state. The retriever is used to return relevant information based on the retrieval query generated by the policy network; The reinforcement learning module is used to jointly optimize the policy network and the retriever using reinforcement learning algorithms.

2. The jointly optimized retrieval enhancement generation system according to claim 1, characterized in that: The policy network includes a decision-maker unit, a query rewriter unit, an answer generator unit, an external memory component, and an internal memory component; The decision-maker unit is used to assess whether the current amount of information is sufficient to answer the question based on the input status, which includes the input question and optional observation results, and to determine whether a retrieval is needed, and then issue a retrieval instruction. The query rewriter unit is used to generate a query request adapted to the retrieval system when the decision-maker module determines that a retrieval is required. The answer generator unit is used to generate a final answer based on the retrieved observations and the original question; The external memory component serves as an information repository for the policy network, used to receive and store various information acquired from the environment; The internal memory component is used to store the agent's interpretation and understanding of itself, and to record key information such as the strategies adopted by the policy network when processing problems, the generated queries, and the answers obtained.

3. The jointly optimized retrieval enhancement generation system according to claim 2, characterized in that: The decision-making unit specifically includes: The input state is to receive an input question and optional observations. The input question is the original question posed by the user and expressed in natural language. The optional observations are additional background information, historical interaction records and contextual information provided by the user to supplement the semantic background of the input question. Semantic analysis is performed on the input state to determine whether the current information is sufficient to answer the question. Natural language processing technology is used to preprocess the input question and observation results to extract key information and semantic features. If the judgment result is yes, meaning that the current information is sufficient to answer the question, the decision-maker unit will directly activate the answer generator unit to generate the final answer by combining the input question and the optional observation results. If the judgment result is no, meaning that the current information is insufficient to answer the question, the decision-maker unit will generate a precise search instruction based on the results of semantic analysis, and then activate the searcher to perform the search.

4. The jointly optimized retrieval enhancement generation system according to claim 3, characterized in that: The query rewriter unit specifically includes: It receives retrieval instructions from the decision-making unit and simultaneously provides input questions from the user; After receiving the input, the query rewriter unit performs semantic parsing on the input question and extracts key information from the input question, including the core intent of the question, the topics involved, and key concepts. Based on the extracted key information, the query rewriter unit generates a query request suitable for the retrieval machine to process.

5. The jointly optimized retrieval enhancement generation system according to claim 2, characterized in that: The retrieval device specifically includes: Receive a query request generated by the policy network, wherein the query request includes keywords or phrases, semantic information and retrieval instructions; Based on the received query request, information retrieval is performed in the pre-organized knowledge base to search for information related to the query request; The retrieval system returns relevant information to the policy network. If no definite answer is found, it returns relevant results for further processing by the policy network. These results include document fragments, document lists, and summaries or outlines.

6. The jointly optimized retrieval enhancement generation system according to claim 5, characterized in that: The answer generator unit specifically includes: The answer generator unit receives the search results and the original question returned by the retrieval machine; Semantic integration is performed on the relevant information returned by the search engine to extract useful information related to the question. At the same time, useful information from multiple search results is merged to form a coherent and complete semantic representation. Based on the search results and the original question, combined with the extracted information, the final answer is generated and then output to the user.

7. The jointly optimized retrieval enhancement generation system according to claim 1, characterized in that: The reinforcement learning module specifically includes: Initialize the environment and set initial parameters, and define a reward function to give rewards or penalties based on the system's actions and results; The user's input questions and the system's observation results are fed into the policy network for decision-making. The network analyzes whether a search is needed or what kind of query request to generate. Based on the output of the policy network, the searcher executes the corresponding search action and returns the search results. Based on the output actions of the policy network and the search results of the retrieval machine, the accuracy of the answer, the relevance of the search results, and the system response time are comprehensively analyzed, and the reward value of each stage is calculated in combination with a predefined reward function. The reinforcement learning algorithm using the policy gradient method updates the parameters of the policy network using gradients based on the reward value calculated by the reward function.

8. The jointly optimized retrieval enhancement generation system according to claim 7, characterized in that: The calculation process for the return value of each stage is as follows: Natural language processing technology is used to calculate the similarity score between the system-generated answer and the real answer, and the baseline similarity score set according to the system requirements is extracted. Combined with the set control parameters, the accuracy of the answer is calculated. For each result returned by the search engine, its relevance score to the query request is calculated. The relevance scores of all search results are summed and their total is calculated. The total relevance score is divided by the number of search results to obtain the average relevance score. The square root of the average relevance score is then taken to obtain the relevance of the search results. The actual response time is the total time required for the system to receive the user's input question, generate the final answer, and output it to the user. The baseline response time set according to the system requirements is extracted, and then the actual response time is divided by the baseline response time to obtain the system response time. The return value for each stage is calculated by comprehensively considering the accuracy of the answer, the relevance of the search results, and the system response time, and by combining the weight coefficients matched according to the system requirements.

9. The jointly optimized retrieval enhancement generation system according to claim 8, characterized in that: The application of the external memory component and the internal memory component specifically includes: The policy network reads relevant information from external and internal memory components, merges the two sets of information, and forms a comprehensive input state. Based on the fused relevant information, the policy network makes decisions, processes the input state, analyzes the semantics of the current problem, evaluates whether a retrieval operation is needed, and if so, generates what kind of query request. Combining the contextual information in external memory and the reasoning experience in internal memory, it generates the optimal decision result. After a decision is executed, the policy network updates the contents of the external and internal memory components based on the decision results and subsequent execution feedback. For the external memory components, the policy network stores new interaction records, relevant information returned by the retrieval system, or user feedback on the generated answers. For the internal memory components, the policy network records the reasoning process of this decision, new understanding of the problem, and lessons learned from the execution of the decision.

10. A jointly optimized retrieval enhancement generation method, implemented based on the jointly optimized retrieval enhancement generation system according to any one of claims 1-9, characterized in that, Includes the following steps: S1. Construct a system environment that includes a strategy network, a retrieval system, and a knowledge base. Set the initial parameters for the strategy network and the retrieval system, and define a reward function that integrates the accuracy of the answer, the relevance of the retrieval results, and the system response time. S2. Input the user's input question and the system's observation results into the policy network. The policy network evaluates whether the information is sufficient to answer the question through the decision-maker unit and decides whether to trigger a retrieval. S3. If a retrieval is required, the query rewriter unit generates a query request suitable for the retrieval tool, and the retrieval tool retrieves relevant information from the knowledge base based on the query request and returns it. S4. The answer generator unit receives the search results and the original question, performs semantic integration on the search information, extracts useful information, combines the question to generate the final answer and outputs it. S5. Based on the output actions and retrieval results of the policy network, calculate the reward value for each stage in conjunction with the reward function, and evaluate the system's performance in various aspects. S6. Using reinforcement learning algorithms, update the parameters of the policy network based on the reward value to optimize the performance of the policy network and the retrieval system.

Images