A data flow method connecting multiple RAG systems

Abstract

The present invention relates to the field of artificial intelligence technology, and specifically discloses a data circulation method connecting multiple RAG systems, including: sending user questions to the RAG system so that the RAG system can determine the corresponding dynamic description content; reasoning based on the dynamic description content and historical conversations to determine whether it is necessary to call the RAG system to answer the user question; when it is necessary to call the RAG system, the user question is split into several sub-questions based on the user question and the dynamic description content, and the RAG systems corresponding to the several sub-questions are called in sequence to obtain corresponding answer results; when the user question can be replied to based on the answer results, the answer results are returned to the user end. The present invention can circulate data elements across departments, institutions, and fields without destroying the original single RAG system, and combine them with each other under the premise of protecting privacy and security to generate new value.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more particularly, to a data flow method for connecting multiple RAG systems. Background Art

[0002] like Figure 1 As shown in the figure, it is now very popular for big models to realize data element application by using a single retrieval-augmented generation technology (RAG) system. Through the process of "user question input → perform retrieval → combine the retrieved relevant knowledge with the user question to form an enhanced prompt → send it to the big model (LLM) to generate a response", combining information retrieval and big models, it can solve the problems of big model illusions, difficulties in knowledge update and maintenance, and potential risks of privacy data leakage to a certain extent.

[0003] For example, a company may have numerous different departments and subsidiaries, each with its own local RAG system housing a vast amount of information from various sources and types, some of which must be kept confidential, while others are permitted to circulate between departments or subsidiaries. Individual RAG systems have limitations. While a growing number of RAG systems are currently in use, most existing systems operate independently, with data elements unable to communicate with each other. This limits their potential for wider application and value. For example, individual RAG systems cannot share data resources or collaborate with other RAG systems, hindering their efficiency and effectiveness in handling complex tasks.

[0004] In addition, how to protect data privacy and security within the connected RAG system is also a major challenge; therefore, how to enable the circulation of local data elements in it while minimizing damage to the original system and protecting data privacy and security, and combine them with data elements in other RAG systems to generate more value in actual scenarios has become an urgent problem to be solved.

[0005] ReAct (Reasoning and Acting) intelligent agent is a framework that combines reasoning and action driven by a large model. It also introduces an observation phase. After each action, it first observes the current status before proceeding to the next step of reasoning. It aims to improve the performance and interpretability of large language models (LLMs) when solving complex tasks. Figure 2As shown in the figure, the operating process of the ReAct agent is as follows: the user question enters the Agent (intelligent agent) → enters the reasoning stage, combines the pre-written tool information in the system to analyze the input data, generate decisions and plans → execute specific actions according to the decisions, call the corresponding tools → get feedback from the called tools → the big model observes the decisions and plans generated in the reasoning stage and the feedback data obtained in the tool call → the big model adjusts the reasoning and actions based on this information to determine whether it can generate the answer directly or whether it needs to perform cyclic iterations to continuously optimize the results.

[0006] In the current ReAct intelligent agent framework, the information about the tools needed for reasoning is pre-set. This results in rigid and general descriptions of the tools, making it difficult for the intelligent agent to make decisions and plans based on this information, and unable to accurately determine the tools needed to solve the input problem. Summary of the Invention

[0007] In response to the above-mentioned drawbacks in the existing technology, the present invention provides a data circulation method that connects multiple RAG systems. Without destroying the original single RAG system, it enables data elements to circulate across departments, institutions, and fields, and combines them with each other under the premise of protecting privacy and security to generate new value.

[0008] As a first aspect of the present invention, a data flow method for connecting multiple RAG systems is provided, wherein the multiple RAG systems are connected to an intelligent agent, and the data flow method for connecting multiple RAG systems comprises the following steps:

[0009] Step S1: When the intelligent agent receives a user question input by a user terminal, it embeds the user question into a corresponding prompt word template to generate a first prompt word, and then sends the first prompt word to all RAG systems that the user can access under the permission, so that the large model in each RAG system can determine the dynamic description content corresponding to the RAG system based on the first prompt word, wherein the dynamic description content is the specific aspects of the RAG system that can answer the user question;

[0010] Step S2: returning corresponding dynamic description content to the intelligent agent through each RAG system;

[0011] Step S3: After the intelligent agent receives the dynamic description content returned by each RAG system, it embeds all the dynamic description content and historical conversations into the corresponding prompt word template to generate a second prompt word, and performs reasoning based on the second prompt word to determine whether it is necessary to call the corresponding RAG system to answer the user question;

[0012] Step S4: When the intelligent agent determines that a corresponding RAG system needs to be called to answer the user question, the intelligent agent embeds the user question and all dynamic description content into a corresponding prompt word template to generate a third prompt word, and then splits the user question into several sub-questions and their corresponding RAG systems based on the third prompt word;

[0013] Step S5: The intelligent agent sequentially calls the RAG systems corresponding to the sub-questions, and inputs the corresponding sub-questions into the corresponding RAG systems to obtain the answer results returned by the corresponding RAG systems;

[0014] Step S6: When the intelligent agent determines that it can reply to the user question based on the answer results returned by the RAG system, the answer results returned by each RAG system are formed into a final answer result and returned to the user terminal.

[0015] Furthermore, the method further comprises the following steps:

[0016] Use the role-based access rights management mechanism to assign one or more roles to each user. Different roles have different access rights to the RAG system.

[0017] Among them, for the RAG system where users can only access non-sensitive information, dynamic data masking technology is used to desensitize the dynamic description content through regular expressions combined with the reasoning and generation capabilities of the large model in the RAG system to obtain the desensitized dynamic description content;

[0018] Among them, for the RAG system where users can only access non-sensitive information, dynamic data masking technology is used to desensitize the answer results returned by the RAG system through regular expressions combined with the reasoning and generation capabilities of the large model in the RAG system to obtain the desensitized answer results.

[0019] Furthermore, the use of a role-based access rights management mechanism to assign one or more roles to each user, where different roles have different access rights to the RAG system, further includes:

[0020] Assign one or more roles to each user based on the actual organizational structure, so that the user can determine the role they belong to after logging in;

[0021] Set different roles to have different access rights, so that certain access rights are allowed under certain roles;

[0022] Whether resources can be accessed and whether operations can be performed is set as the RAG system access rights to ensure that the intelligent agent can only access the RAG system in the authority when handling the user problem.

[0023] Furthermore, the step S6 further includes the following steps:

[0024] When the intelligent agent is unable to reply to the user question according to the answer result returned by the RAG system, it returns to step S3 to perform reasoning again to update the answer result returned by the RAG system until it can reply to the user question according to the updated answer result returned by the RAG system.

[0025] The data circulation method for connecting multiple RAG systems provided by this invention has the following advantages: by introducing an intelligent agent based on large model capabilities, leveraging its decision-making and execution capabilities to connect multiple independent RAG systems, it enables the circulation and application of data elements across domains, institutions, and departments. It also incorporates role-based access control and dynamic data masking (DDM) technology to ensure effective data protection during circulation, preventing privacy leaks and unauthorized access. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present invention, but do not constitute a limitation of the present invention.

[0027] Figure 1 This is an operation flow chart of a single RAG system in the prior art.

[0028] Figure 2 This is a flowchart of the operation of the ReAct intelligent agent in the prior art.

[0029] Figure 3 This is a flow chart of the data flow method for connecting multiple RAG systems provided by the present invention.

[0030] Figure 4 This is a flowchart of a specific implementation of the data flow method for connecting multiple RAG systems provided by the present invention.

[0031] Figure 5 Schematic diagram of the connection between the ReAct intelligent agent provided by the present invention and multiple RAG systems.

[0032] Figure 6 This is a flowchart of the implementation of the RAG system provided by the present invention. DETAILED DESCRIPTION

[0033] To further illustrate the technical means and effectiveness of the present invention in achieving its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of the data flow method for connecting multiple RAG systems proposed in accordance with the present invention. It should be understood that the described embodiments are only a subset of the embodiments of the present invention, and not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0034] In this embodiment, a data flow method for connecting multiple RAG systems is provided, such as Figure 3-5 As shown, multiple RAG systems are connected to the intelligent agent, and the data flow method connecting multiple RAG systems includes the following steps:

[0035] Step S1: When the intelligent agent receives a user question input by a user terminal, it embeds the user question into a corresponding prompt word template to generate a first prompt word, and then sends the first prompt word to all RAG systems that the user can access under the permission, so that the large model in each RAG system can determine the dynamic description content corresponding to the RAG system based on the first prompt word, wherein the dynamic description content is the specific aspects of the RAG system that can answer the user question;

[0036] It's important to note that the large model within a single RAG system determines whether it can answer any aspect of the user's question, and if so, which aspect. This information is then returned to the intelligent agent as a dynamic description of each RAG system. In this scenario, the "large model" refers to the local large model within the department's RAG system. Specifically, this "large model" can be Llama3_70B_Instruct, Qwen_2_72B, or ChatGLM3_7B, depending on the local configuration.

[0037] Specifically, "dynamic description content" refers to a description of the RAG system tailored to the user's question, rather than a general summary. This description changes based on the user's question. For example, if a user asks a question about a person's name, but this name doesn't typically appear in the pre-written RAG system description (RAG system descriptions typically include "xxx Company Annual Report" or "Summary of Past Projects of Department A"), the large model in the intelligent agent may misjudge the situation and skip the relevant RAG system. Using dynamic description content, however, might result in a description like "providing information on projects in which person xxx has participated in Department A."

[0038] Example 1:

[0039] 1. Background: Group B has multiple subsidiaries, each of which has multiple departments. Each department is equipped with a local RAG system to manage and answer various types of information.

[0040] 2. User question: "Information about customer Zhang Xiaojuan" is sent to the RAG system in combination with appropriate prompt words.

[0041] 3. The dynamic description returned by the RAG system is as follows:

[0042] a)RAG_1: "I can answer questions about the utility bill details for Zhang Xiaojuan."

[0043] b) RAG_2: "I can answer the question about the registered address of Zhang Xiaojuan."

[0044] c) RAG_3: "I can answer questions about Zhang Xiaojuan's purchase history at store E in March."

[0045] Example 2:

[0046] 1. Background: Group B has multiple subsidiaries, each of which has multiple departments. Each department is equipped with a local RAG system to manage and answer various types of information.

[0047] 2. User question: "Project experience with large-scale federated learning" is sent to the RAG system along with appropriate prompt words.

[0048] 3. The dynamic description returned by the RAG system is as follows:

[0049] a)RAG_1: “I can answer questions about project experience in large-scale model deployment.”

[0050] b) RAG_2: “I can answer questions about project experience in federated learning.”

[0051] c) RAG_3: "I can answer questions about my experience with the large-scale federated learning project in interbank finance."

[0052] Step S2: returning corresponding dynamic description content to the intelligent agent through each RAG system;

[0053] Step S3: After the intelligent agent receives the dynamic description content returned by each RAG system, it embeds all the dynamic description content and historical conversations into the corresponding prompt word template to generate a second prompt word, and performs reasoning based on the second prompt word to determine whether it is necessary to call the corresponding RAG system to answer the user question; the purpose is to make a judgment on the progress of solving the current user problem: if the intelligent agent believes that it has obtained enough information, it does not need to call the RAG system to answer the user question; if the intelligent agent believes that it has not obtained enough information, it needs to call the corresponding RAG system to help itself answer the user question.

[0054] It should be noted that the conversation history refers to the content of past user questions and LLM interactions, which will appear in the "Current Conversation" section of the prompt word below. This includes the previous rounds of input from the intelligent agent in the current conversation, the reasoning generated in the large model's Thought, Action, and Action Input formats, and the tool return after calling the tool: Observation.

[0055] Step S4: When the intelligent agent determines that a corresponding RAG system needs to be called to answer the user question, the intelligent agent embeds the user question and all dynamic description content into a corresponding prompt word template to generate a third prompt word, and then splits the user question into several sub-questions and their corresponding RAG systems based on the third prompt word;

[0056] For example, the user problem is decomposed into:

[0057] Sub-question 1 = "Zhang Xiaojuan's utility bill payment details for the past month", the RAG system corresponding to sub-question 1 = "RAG_1";

[0058] Sub-question 2 = "Zhang Xiaojuan's address", the RAG system corresponding to sub-question 2 = "RAG_2".

[0059] Step S5: The intelligent agent sequentially calls the RAG systems corresponding to the sub-questions, and inputs the corresponding sub-questions into the corresponding RAG systems to obtain the answer results returned by the corresponding RAG systems;

[0060] Step S6: When the intelligent agent determines that it can reply to the user question based on the answer results returned by the RAG system, the answer results returned by each RAG system are formed into a final answer result and returned to the user terminal.

[0061] Preferably, the method further comprises the following steps:

[0062] The RAG system uses a role-based access control mechanism to assign one or more roles to each user. Different roles have different access rights to the RAG system.

[0063] For RAG systems where users are limited to accessing non-sensitive information, Dynamic Data Masking (DDM) technology is used to desensitize the dynamic description content through regular expressions combined with the reasoning and generation capabilities of large models in the RAG system. This results in desensitized dynamic description content. This improves the shortcomings of the previous ReAct Agent framework, which had rigid descriptions of callable tools and lacked detailed descriptions related to user issues. Furthermore, it prevents the leakage of RAG descriptions unrelated to the issue or sensitive information in the RAG system that the user is not authorized to view.

[0064] It should be noted that the prompt words are used to explicitly let the large model in the RAG system generate answers using the following workflow:

[0065] 1. Identify sensitive words in sentences, such as time-related words;

[0066] 2. Replace the recognized words with ***.

[0067] In the example above, "I can answer questions about Zhang Xiaojuan's purchase records in E store in March", the time is: March.

[0068] 1. Use the inference generation capabilities of large models to desensitize information: "Please identify all time information and replace it with ****";

[0069] 2. Then use regular expressions to fill in the gaps in the answers, replacing all fixed-format time / number content with ***;

[0070] 3. Dynamic description content after desensitization: I can answer questions about customer Zhang Xiaojuan’s purchase records at store E in month ***.

[0071] In the example above, "I can answer questions about my experience in the large-scale federated learning project on finance with Jiangsu Bank": partners include Jiangsu Bank and Finance.

[0072] 1. Use the inference generation capabilities of large models to desensitize information: "Please identify all entity names and replace them with ****";

[0073] 2. Then use regular expressions to fill in the gaps in the answers, replacing all fixed-format entity names with ***;

[0074] 3. Dynamic description content after desensitization: I can answer questions about my experience in the large-scale federated learning project in the financial field in cooperation with ***.

[0075] For RAG systems where users are restricted to accessing non-sensitive information, Dynamic Data Masking (DDM) is used to desensitize the responses returned by the RAG system using regular expressions combined with the inference generation capabilities of the RAG system's large model. This desensitized response is then generated. Sensitive information varies depending on the scenario, company, or department, but is generally consistent with the examples in the dynamic description. Desensitization is performed using the large model, followed by regular expressions to address any gaps.

[0076] Preferably, Figure 6 As shown, the role-based access rights management mechanism is used to assign one or more roles to each user. Different roles have different access rights to the RAG system, and also includes:

[0077] Assign one or more roles to each user based on the actual organizational structure, so that the user can determine the role they belong to after logging in;

[0078] Set different roles to have different access rights, so that certain access rights are allowed under certain roles;

[0079] Whether resources can be accessed and whether operations can be performed is set as the RAG system access rights to ensure that the intelligent agent can only access the RAG system in the authority when handling the user problem.

[0080] For example, user A has the role of "Administrator of Company C, Group B" and has full permissions to query the independent RAG systems (RAG_1 and RAG_2) of all departments of Company C. He also has permissions to query non-sensitive information in the RAG_3 system that belongs to Group B but not Company C.

[0081] Preferably, the step S6 further includes the following steps:

[0082] When the intelligent agent is unable to reply to the user question according to the answer result returned by the RAG system, it returns to step S3 to reason and think again to update the answer result returned by the RAG system until it can reply to the user question according to the updated answer result returned by the RAG system.

[0083] The data flow method for connecting multiple RAG systems provided by this invention can facilitate the circulation and application of data elements between RAG systems, circulate previously uncirculated data elements, and further create value. Furthermore, without significantly modifying the existing RAG systems, the privacy and security of the data element flow are guaranteed.

[0084] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment, it is not intended to limit the present invention. Any technician familiar with the present profession can make slight changes or modifications to equivalent embodiments using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A data flow method for connecting multiple RAG systems, characterized in that: Multiple RAG systems are connected to the intelligent agent. The data flow method connecting multiple RAG systems includes the following steps: Step S1: When the intelligent agent receives a user question input by a user terminal, it embeds the user question into a corresponding prompt word template to generate a first prompt word, and then sends the first prompt word to all RAG systems that the user can access under the permission, so that the large model in each RAG system can determine the dynamic description content corresponding to the RAG system based on the first prompt word, wherein the dynamic description content is the specific aspects of the RAG system that can answer the user question; Step S2: returning corresponding dynamic description content to the intelligent agent through each RAG system; Step S3: After the intelligent agent receives the dynamic description content returned by each RAG system, it embeds all the dynamic description content and historical conversations into the corresponding prompt word template to generate a second prompt word, and performs reasoning based on the second prompt word to determine whether it is necessary to call the corresponding RAG system to answer the user question; Step S4: When the intelligent agent determines that a corresponding RAG system needs to be called to answer the user question, the intelligent agent embeds the user question and all dynamic description content into a corresponding prompt word template to generate a third prompt word, and then splits the user question into several sub-questions and their corresponding RAG systems based on the third prompt word; Step S5: The intelligent agent sequentially calls the RAG systems corresponding to the sub-questions, and inputs the corresponding sub-questions into the corresponding RAG systems to obtain the answer results returned by the corresponding RAG systems; Step S6: When the intelligent agent determines that it can reply to the user question based on the answer results returned by the RAG system, the answer results returned by each RAG system are formed into a final answer result and returned to the user terminal.

2. The data flow method for connecting multiple RAG systems according to claim 1, characterized in that: The following steps are also included: Use the role-based access rights management mechanism to assign one or more roles to each user. Different roles have different access rights to the RAG system. Among them, for the RAG system where users can only access non-sensitive information, dynamic data masking technology is used to desensitize the dynamic description content through regular expressions combined with the reasoning and generation capabilities of the large model in the RAG system to obtain the desensitized dynamic description content; Among them, for the RAG system where users can only access non-sensitive information, dynamic data masking technology is used to desensitize the answer results returned by the RAG system through regular expressions combined with the reasoning and generation capabilities of the large model in the RAG system to obtain the desensitized answer results.

3. The data flow method for connecting multiple RAG systems according to claim 2, characterized in that: The role-based access rights management mechanism is used to assign one or more roles to each user. Different roles have different access rights to the RAG system, and also includes: Assign one or more roles to each user based on the actual organizational structure, so that the user can determine the role they belong to after logging in; Set different roles to have different access rights, so that certain access rights are allowed under certain roles; Whether resources can be accessed and whether operations can be performed is set as the RAG system access rights to ensure that the intelligent agent can only access the RAG system in the authority when handling the user problem.

4. The data flow method for connecting multiple RAG systems according to claim 1, characterized in that: The step S6 further includes the following steps: When the intelligent agent is unable to reply to the user question according to the answer result returned by the RAG system, it returns to step S3 to perform reasoning again to update the answer result returned by the RAG system until it can reply to the user question according to the updated answer result returned by the RAG system.

Images