Large model intelligent agent system based on thought clustering planning and domain knowledge retrieval
By introducing thinking clustering planning and domain knowledge retrieval into the large model intelligent agent system, the limitations of task planning and tool calling capabilities in complex tasks are solved, and more efficient and accurate decision-making and tool calling are achieved.
Patent Information
- Application Number
- CN202411344735.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-25
- Publication Date
- 2025-01-24
- Estimated Expiration
- 2044-09-25
AI Technical Summary
In practical business applications, highly complex problems often have long chains and require a large set of tools, which rely heavily on business knowledge, which has greatly limited the accuracy of task planning and the ability to call tools.
Provides a large-model intelligent agent system based on thinking clustering planning and domain knowledge retrieval, including user interaction module, domain knowledge retrieval module, thinking clustering planning module, base large language model, guidance optimization module, tool calling module, response The synthesis module and the model output module work together to improve the long-chain planning and complex task tool calling capabilities of large models in vertical fields.
It improves the decision-making accuracy and efficiency of large-model intelligent agent systems, enables it to build agents in business scenarios more efficiently and accurately, and solve tool calling and task planning problems in complex tasks.
Smart Images
Figure CN118861305B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a large-model intelligent agent system based on thought clustering planning and domain knowledge retrieval. Background Art
[0002] In traditional business processes, humans need to spend a lot of time and energy on repetitive operations and decision-making. With the development of artificial intelligence technology, machines have the ability to judge and execute tasks, which enables machines to assist humans in achieving specific goals in certain specific business scenarios. In the core system of artificial intelligence, these intelligent entities with judgment and execution capabilities are called AI Agents. The emergence of intelligent agents enables machines to undertake these tasks, freeing up the workload of humans and allowing humans to focus more on higher-level work such as creativity and decision-making. In short, AI intelligent agents are advanced software programs that can operate independently without human control. It can think, act and learn independently without constant human input. Some intelligent agents also have the ability to perceive environmental changes, collect and analyze environmental information in real time, and take reasonable actions based on this information to achieve set goals.
[0003] On the other hand, with the rapid development of large language models, the ability of machine decision-making intelligence has also been greatly improved. Generally speaking, large language models are very good at generating text, translating languages, writing different types of creative content, and planning tasks, but they are not suitable for executing the final task. In order to maximize the potential of large language models and enable agent agents to evolve into AI agents that make decisions and complete tasks independently, large language models are used to combine with traditional AI Agents models to produce a new and mainstream Agent paradigm: LLM as Agent. The introduction of this intelligent agent enables machines to respond to various business scenarios more flexibly and significantly changes the way people interact with machines. The core idea of the LLM agent is to use a language model inference engine to plan and select a series of actions to be taken. Then, through a series of chain rules, hard-coded actions are performed to execute the corresponding tasks based on the planned order. The main workflow mainly includes four key stages - task planning, tool selection, tool calling, and response generation.
[0004] Although the LLM as Agent paradigm has been widely accepted, and people in the research community have made great efforts to explore how to improve the tool learning capabilities of LLMs, and have achieved very effective results. However, in actual business applications, highly complex problems often have long chains, require more tool sets, and are extremely dependent on business knowledge, which greatly limits the accuracy of task planning and the ability to call tools. Improving the long-chain planning and complex task tool calling capabilities of large language models in vertical fields and building more efficient and accurate intelligent agents for business scenarios have become urgent issues to be solved. Summary of the invention
[0005] Based on this, it is necessary to provide a large-model intelligent agent system based on thinking clustering planning and domain knowledge retrieval to address the above technical problems.
[0006] A large model intelligent agent system based on thought clustering planning and domain knowledge retrieval, the system comprising:
[0007] The system includes a user interaction module, a domain knowledge retrieval module, a thought clustering planning module, a base large language model, a guidance optimization module, a tool calling module, a response synthesis module and a model output module;
[0008] The user interaction module is used to obtain user prompt input;
[0009] The domain knowledge retrieval module includes a domain knowledge base, a thought clustering method base and a retrieval unit; the retrieval unit includes a vector coding model and a vector database; the thought clustering method base includes a variety of clustering methods for thought clustering that are compatible with the domain knowledge;
[0010] The thought cluster planning module is used to call the domain knowledge retrieval module to perform thought cluster retrieval according to the user prompt input, and construct a thought chain example according to the retrieval result;
[0011] The base large language model is used to perform task planning based on task timing according to the user prompt input and the system prompt template to obtain a planning chain; the base large language model is a pre-trained large language model that matches the domain scenario;
[0012] The guiding optimization module is used to optimize and integrate the preset tool calling prompt template according to the planning chain, and output the optimized tool calling prompt template;
[0013] The tool calling module uses the tool calling large language model to obtain a structured tool calling template according to the planning chain and the optimization tool calling prompt template, and uses the structured tool calling template to call the tool to obtain the interaction result of the current plan; the tool calling module is a base large language model fine-tuned by a pre-built preference data set;
[0014] The response synthesis module is used to obtain the interaction results corresponding to the planning chain from the tool calling module and output a planning solution, and feed back the planning solution to the user interaction unit through the model output module.
[0015] In one embodiment, the system also includes a dual reflection module, which is respectively connected to the thinking cluster planning module and the tool calling module, and is used to determine the cause of the exception according to the current calling result and the expected result when an exception occurs during tool calling, obtain forward reflection results, and perform abnormal reverse reasoning to obtain the cause of the exception, obtain reverse reflection results, and feed back the forward reflection results and the reverse reflection results to the thinking cluster planning module.
[0016] In one of the embodiments, it is also used to extract the thinking chain entity in the user prompt input, input the thinking chain entity into the retrieval unit according to the thinking chain entity, vectorize the thinking chain entity through the vector encoding model, and obtain the thinking chain instruction; use the clustering method in the thinking clustering method library to perform clustering candidates and recall in the vector database according to the thinking chain instruction, and output the retrieval result; obtain the domain knowledge document corresponding to the retrieval result in the domain knowledge base, obtain the context information of the thinking chain entity according to the domain knowledge document, and construct a thinking chain example according to the context information and the domain scene prompt sequence.
[0017] In one of the embodiments, fine-tuning using a pre-constructed preference dataset includes: constructing a preference dataset; the preference dataset is obtained by acquiring a number of planning chains and corresponding calling tools from domain data and a knowledge base; and fine-tuning the tool-calling large language model using the preference dataset and a pre-set direct preference optimization loss function to obtain a fine-tuned tool-calling large language model.
[0018] In one embodiment, the direct preference optimization loss function is:
[0019] ;
[0020] in, Optimize the loss function for direct preference, is the input sample, and the preference label is , the non-preferred label is , is the scaling factor, is the sigmoid function, represents the policy model, represents the reference policy model, is the preference data set.
[0021] In one of the embodiments, it is also used to obtain a tool call prompt template; the tool call prompt template includes a system prompt; the system prompt is optimized according to the planning chain, and an adapted scene information prompt is generated, the tool call prompt template is optimized and integrated according to the optimized system prompt and scene information prompt, and an optimized tool call prompt template is output.
[0022] In one of the embodiments, the tool call module includes a template generator and a template executor; the template generator is driven by a tool call large language model, and is used to generate a structured tool call template according to the planning chain and the optimization tool call prompt template; the template executor is used to parse the structured tool call template generated by the template generator and call the target API interface or function to execute the current plan based on the executed plan and execution results.
[0023] In one of the embodiments, the template executor is further used to handle abnormal situations caused by abnormal reasons.
[0024] The above-mentioned large-model intelligent agent system based on thought clustering planning and domain knowledge retrieval obtains user prompt input through the user interaction module, calls the domain knowledge retrieval module according to the user prompt input through the thought clustering planning module to perform thought clustering retrieval, builds a thought chain example according to the retrieval result, and performs task planning based on task timing according to the user prompt input and the system prompt template through the base large language model to obtain a planning chain; the base large language model is a pre-trained large language model that matches the domain scenario, and is used to optimize and integrate the pre-set tool call prompt template according to the planning chain through the guidance optimization module, output the optimized tool call prompt template, and use the tool call module to use the tool to call the large language model, according to the planning chain and the optimized tool call prompt template, obtain a structured tool call template, use the structured tool call template to call the tool, and obtain the interaction result of the current plan, and use the response synthesis module to obtain the interaction result corresponding to the planning chain from the tool call module and output the planning scheme, and feed the planning scheme back to the user interaction unit through the model output module. The embodiment of the present invention can improve the decision accuracy and decision efficiency of the large-model intelligent agent system. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1A structural block diagram of a large model intelligent agent system based on thought clustering planning and domain knowledge retrieval in one embodiment;
[0026] Figure 2 A schematic diagram of agent retrieval and feedback caching in one embodiment;
[0027] Figure 3 A schematic diagram of constructing a preference data set in one embodiment. DETAILED DESCRIPTION
[0028] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0029] In one embodiment, Figure 1 As shown, a large model intelligent agent system based on thought clustering planning and domain knowledge retrieval is provided, including a user interaction module, a domain knowledge retrieval module, a thought clustering planning module, a base large language model, a guidance optimization module, a tool calling module, a response synthesis module and a model output module;
[0030] The user interaction module is used to obtain user prompt input;
[0031] The domain knowledge retrieval module includes a domain knowledge base, a mind clustering method base and a retrieval unit; the retrieval unit includes a vector coding model and a vector database; the mind clustering method base includes a variety of clustering methods for mind clustering that are compatible with the domain knowledge;
[0032] The thought clustering planning module is used to call the domain knowledge retrieval module to perform thought clustering retrieval according to the user prompt input, and to construct a thought chain example according to the retrieval results;
[0033] The base large language model is used to perform task planning based on task timing according to user prompt input and system prompt template to obtain the planning chain; the base large language model is a pre-trained large language model that matches the domain scenario;
[0034] The guided optimization module is used to optimize and integrate the pre-set tool call prompt templates according to the planning chain, and output the optimized tool call prompt templates;
[0035] The tool calling module uses the tool calling large language model to obtain a structured tool calling template according to the planning chain and the optimized tool calling prompt template, and uses the structured tool calling template to call the tool to obtain the interaction result of the current plan; the tool calling module is a base large language model that is fine-tuned through a pre-built preference data set;
[0036] The response synthesis module is used to obtain the interaction results corresponding to the planning chain from the tool call module and output the planning scheme, and feed the planning scheme back to the user interaction unit through the model output module.
[0037] In the above-mentioned large-model intelligent agent system based on thought cluster planning and domain knowledge retrieval, the user prompt input is obtained through the user interaction module, the domain knowledge retrieval module is called according to the user prompt input through the thought cluster planning module to perform thought cluster retrieval, a thought chain example is constructed according to the retrieval result, and the task planning based on the task sequence is performed according to the user prompt input and the system prompt template through the base large language model to obtain the planning chain; the base large language model is a pre-trained large language model matching the domain scene, and the guiding optimization module is used to optimize and integrate the pre-set tool call prompt template according to the planning chain, output the optimized tool call prompt template, and the tool call module uses the tool to call the large language model, according to the planning chain and the optimized tool call prompt template, a structured tool call template is obtained, and the tool call is performed using the structured tool call template to obtain the interaction result of the current plan, and the response synthesis module is used to obtain the interaction result corresponding to the planning chain from the tool call module and output the planning scheme, and the planning scheme is fed back to the user interaction unit through the model output module. The embodiment of the present invention can improve the decision accuracy and decision efficiency of the large-model intelligent agent system.
[0038] In one embodiment, Figure 2As shown, a schematic diagram of intelligent agent retrieval and feedback cache is provided. The domain knowledge retrieval module includes a domain knowledge base, a thinking clustering method library and a retrieval unit. The domain knowledge base is constructed according to the application scenario. The application scenario can be the industrial field, in which predictive maintenance, quality control, optimization of production processes and employee training are realized, or it can be the education and training field, providing personalized learning decisions, or it can be the field of autonomous driving, providing path planning decisions. LLM intelligent agents are widely used and will not be repeated here. The corresponding domain knowledge base is stored in the form of text, including domain concepts and terms in the field, basic theories, professional knowledge, technical methods, tools, software, databases and other resources used in the field, etc., and also includes specific cases and tool usage records. The vector encoding model corresponding to the adaptation of the retrieval system and the text knowledge base adopts an open source vector encoding model for vectorizing text encoding. The vector database can adopt a general vector database for vectorized text storage and index construction. Users can obtain matching text data through query prompts from the vector encoding model and the vector database index. The thought clustering method library contains various clustering methods for thought clustering, which are adapted to domain knowledge. According to user prompts, the thought chain used for query is parsed, and a domain thought candidate pool is constructed through the retrieval module and the clustering module. In this embodiment, Kmeans is used for clustering, and clusters related to user prompts are extracted and put into the candidate pool, which is then retrieved from the candidate pool through retrieval, and the corresponding domain knowledge is obtained through indexing to construct the thought chain. The goal of K-means is to minimize the square error, and its cost function (sum of square errors) can be expressed as:
[0039] ;
[0040] in, is the sum of squares of the errors between all samples in the cluster and their center point, It is The set of all samples of a cluster, It is The center point of the cluster, is a data point in the cluster.
[0041] Among them, vector retrieval is calculated using cosine similarity, and the cosine similarity calculation formula is:
[0042] ;
[0043] in, and Represents text and text The semantic feature vector of . and They refer to the first A quantity.
[0044] In one embodiment, fine-tuning using a pre-constructed preference dataset includes: constructing a preference dataset; the preference dataset is obtained by acquiring a number of planning chains and corresponding calling tools from domain data and a knowledge base; and fine-tuning the tool calling large language model using the preference dataset and a pre-set direct preference optimization loss function to obtain a fine-tuned tool calling large language model.
[0045] In this embodiment, if Figure 3 The diagram of preference dataset construction is shown in the figure. When constructing the preference dataset, the planning text is mapped to form a tool call template. The mapping input includes system prompts, user questions, and the plan that the model needs to execute. The output of the mapping is a structured template for the model's tool call, including private tool parameter names and other information. The constructed mapping format and labels are as follows: the input is system prompts and planning tasks, the preference label is a template in json format, referring to the idea of react, including current thinking and correctly selected tools and parameters, and the non-preference label is the wrongly selected tools and parameters, or abnormal template output. The preference dataset consists of several positive and negative sample pairs. Figure 3 The preferred output and non-preferred output of the expression constitute a positive and negative sample pair. In the result of a certain step planning: Search who is A[B search], it means "Search who is A", and the content in the brackets indicates that the tool is the B search engine. In the preferred output, Thought: I need to use B serach means "Idea: I need to use B search engine", Tool: B search means "Tool: B search engine", and Parameters:{query: who is A} means "Parameters:{query: who is A}". In the non-preferred output, Tool:search means "Tool: search engine", and Parameters: query: who is A means "Parameters: query: who is A".
[0046] In one embodiment, the direct preference optimization loss function is:
[0047] ;
[0048] in, Optimize the loss function for direct preference, is the input sample, the preference label is , the non-preferred label is , is the scaling factor, is the sigmoid function, represents the policy model, represents the reference policy model, is the preference data set.
[0049] In this embodiment, preference training is performed on a large language model based on samples of a preference data set to correct the template output preference of the large language model so as to generate a template that better conforms to the calling format and improve the accuracy of tool calling. Based on the preference data set, this embodiment uses the paradigm of direct preference optimization (DPO) for preference alignment.
[0050] In one embodiment, a domain knowledge retrieval module is called according to a user prompt input to perform a thought clustering retrieval, and a thought chain example is constructed based on the thought clustering retrieval process, including: extracting a thought chain entity from the user prompt input, inputting the thought chain entity into a retrieval unit according to the thought chain entity, vectorizing the thought chain entity through a vector coding model, and obtaining a thought chain instruction; clustering candidates and recalls are performed in a vector database according to the thought chain instruction using a clustering method in a thought clustering method library, and a retrieval result is output; the domain knowledge document corresponding to the retrieval result in the domain knowledge library is obtained, the context information of the thought chain entity is obtained according to the domain knowledge document, and a thought chain example is constructed according to the context information and the domain scene prompt sequence.
[0051] In this embodiment, after retrieving the relevant documents of the thought chain instruction, the output domain documents and the thought chain instruction are again demonstrated step by step to construct a context track example, that is, the original thought instruction is added with context background and domain scene prompts, and the instruction track is constructed sequentially, in chronological order, to supplement its previous or subsequent steps, to reduce or avoid the output of useless noise prompts. Finally, the final template is integrated and output through the large model to facilitate the improvement of the overall performance of the decision-making task. At the same time, in order to ensure the stability of the example, the large language model is then based on the integrated template of the system information and the retrieval results, as well as the historical cache pool, etc., and the large language model performs logical reasoning and outputs the planning chain. The historical call and cache information are integrated through a two-way reflection module.
[0052] In one embodiment, the pre-set tool call prompt template is optimized and integrated according to the planning chain, and the output of the optimized tool call prompt template includes: obtaining the tool call prompt template; the tool call prompt template includes a system prompt; optimizing the system prompt according to the planning chain, and generating an adaptive scene information prompt, optimizing and integrating the tool call prompt template according to the optimized system prompt and scene information prompt, and outputting the optimized tool call prompt template.
[0053] In this embodiment, the guidance optimization module is based on the output results of the planning module to optimize and integrate the prompt words used for the target task, which mainly includes two parts of optimization: on the one hand, based on the output results of the planning module, the system prompts representing the ability range and global goals of the intelligent agent are optimized, which can reduce the computing power and execution time required for the large language model call by reducing useless tool description templates and increasing the necessary tool description descriptions of important tools, and improve the accuracy of subsequent large language model tool selection. On the other hand, based on the existing plan and actual needs, and the pre-understanding of the large language model, the scene information prompts that adapt to the prompts of the domain intelligent agent are generated. Ensure the scene information provided by the stability of the generation, thereby avoiding the disorder of the tool call template. In this embodiment, the system uses the scene understanding ability of the large language model to eliminate the tool descriptions that are not related to the field, expand the scene-related information, and add the scene information to the system prompts.
[0054] In one embodiment, the tool call module includes a template generator and a template executor; the template generator is driven by a tool call large language model, and is used to generate a structured tool call template based on a planning chain and an optimized tool call prompt template; the template executor is used to parse the structured tool call template generated by the template generator and call the target API interface or function to execute the current plan based on the executed plan and execution results.
[0055] In this embodiment, the tool calling module includes a template generator and a template executor. The template generator is used to correspond to the system, scenario, user information and planning history information, and then generate a structured or formatted execution template. The template executor adopts chain tool calling logic, parses the template and calls the target API interface or function based on the executed plan and execution results, and handles abnormal situations. In this embodiment, the model output format adopts the Json format, referring to the idea of react, so that each round of output outputs its current thinking and the tools and parameters it selects. For example, the form of "Thought:...Tool:...parameter:..." is adopted. Based on the output result, the downstream template executor can directly call the registered tool after getting this result to perform the corresponding action.
[0056] In one embodiment, the template executor is further configured to handle abnormal situations caused by abnormal reasons.
[0057] In one embodiment, the system also includes a dual reflection module, which is respectively connected to the thinking cluster planning module and the tool calling module, and is used to determine the cause of the exception according to the current calling result and the expected result when an exception occurs during tool calling, obtain forward reflection results, and perform abnormal reverse reasoning to obtain the cause of the exception, obtain reverse reflection results, and feed back the forward reflection results and reverse reflection results to the thinking cluster planning module.
[0058] In this embodiment, if Figure 2 As shown, the two-way reflection module combines two feedback reasons for feedback, which is used to correct the template and call it repeatedly. Specifically, the forward reflection result and the reverse reflection result are put into the splicing guidance prompt template, and fed back to the thinking clustering planning module together. At the same time, the reflection module will also feedback and cache historical information, and cache it in the historical cache pool of the thinking clustering module. The historical information includes the cause of the exception and the error message for subsequent execution. Forward reflection determines the possible cause of the problem by comparing the expected result with the execution result. The reverse reflection module reversely infers the cause of the exception by executing the exception. The results of the two-way reflection are input into the base large language model in a guided manner, and are globally integrated and summarized for template error correction and planning correction.
[0059] In a specific embodiment, taking the field of autonomous driving as an example, the specific implementation process of the system of the present invention includes:
[0060] S1. Build a domain knowledge base retrieval module, where the domain knowledge base is mainly stored in the form of text. Select a vector database and a vector encoding model based on domain data to build a basic clustering retrieval system. At the same time, select a local large language model base based on the domain scenario, and copy the base large language model into two copies, one for task planning and other tasks, called the base large language model, and the other for tool calling, called the tool calling large language model.
[0061] For path planning, the domain knowledge base includes domain knowledge related to autonomous driving basics, computer vision, path planning and optimization, machine learning, and robotics, all of which are stored in text form.
[0062] S2. Based on domain data and knowledge base, we manually construct planning chains and function call samples to build preference datasets, which are used to train the calling preferences of the domain large language model called by the tool and to align the preferences of domain data and text structured outputs.
[0063] Domain data includes sensor data, specifically environmental information collected by cameras, radars, lidars, etc.
[0064] S3. Based on the domain preference dataset constructed in step S2, the tool is fine-tuned to call the local large language model, so that the large language model is aligned with the domain data preference and outputs a specific structured template of the domain agent. This large language model is mainly used for the tool calling module.
[0065] The tools include sensor data processing tools, maps, positioning systems, path planning algorithms and control systems. The sensor data processing tools are used to integrate data from multiple sensors to provide comprehensive environmental perception, obtain high-precision map data and real-time vehicle location through maps and positioning systems, and the path planning algorithm is used to perform specific path planning based on sensor data and location data. The path planning results are executed through the control system to control the vehicle's acceleration, deceleration, and steering.
[0066] S4. Based on the vector database and domain knowledge base selected in step S1, accept the user prompt input of the user input module, gradually retrieve the prompt related content through thought clustering, and build a thought chain example. And reconstruct the prompt template based on the example, the original system prompt, and the user question. The system prompt template and the user question are input into the large language model fine-tuned in step S3, and the base large language model plans based on the task sequence and outputs the planning chain.
[0067] S5. Input the planning chain obtained in step S4 into the guidance optimization module, optimize and integrate the tool call prompt template, integrate and partition the prompt template, remove the system prompts of useless tools, and guide the model to focus on necessary tools. At the same time, add scene information to improve the model's scene understanding ability and output a more accurate call template.
[0068] S6. Input the optimization tips of step S5 and the planning chain obtained in S4 into the tool calling module. The tool calling module implements tool structured output and tool calling, and decides whether to perform double reflection based on the results.
[0069] S7. If the calling tool is abnormal or cannot be executed, the calling result of step S6 will be input into the two-way reflection module, and after the information is integrated, it will be input into the tool calling chain again for re-output and sorting.
[0070] S8. If the calling tool is executed normally and outputs the result, the large language model in the response synthesis module will plan the steps and calling process, and the result will be reorganized and output. If it fails to execute multiple times, the model integration is abnormal and feedback is output.
[0071] In this embodiment, the large model intelligent agent system aims to use context clustering retrieval to introduce domain knowledge to enhance the decision-making and planning capabilities of the intelligent agent system, and at the same time guide the large model to correctly call business tools through domain template alignment training. The intelligent system of the present invention comprises the following modules: user interaction module, domain knowledge base retrieval module, thinking clustering planning module, guiding optimization module, tool calling module, two-way reflection module and response synthesis module. User interaction module, used for input and output of system and user interaction. Domain knowledge base retrieval module, including domain knowledge and business planning preference knowledge, knowledge base saved in text form, and clustering retrieval system based on knowledge base adaptation. Thinking clustering planning module combines the domain knowledge of clustering retrieval to construct trajectory examples, and combines historical cache information for planning to improve the overall performance of sequential decision tasks. Guiding optimization module, based on the output results of planning module, can effectively optimize and integrate tool prompt words. Tool calling module, with efficient tool calling and parameter selection capabilities, can accurately call tools and functions. Two-way reflection module is used to match the correct tool based on context positive and reverse error correction feedback. Response synthesis module, according to the call trajectory results, summarizes and outputs. In the multi-intelligent overall system of the present invention, multi-dimensional domain information is introduced in an overall manner of information retrieval clustering, optimized instance trajectories are constructed, and the model domain semantic understanding ability is optimized, thereby improving the planning and tool calling capabilities of large model intelligent agents in the business field.
[0072] Each module in the large model intelligent agent system based on thought clustering planning and domain knowledge retrieval can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.
[0073] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0074] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A large-model intelligent agent system based on thought clustering planning and domain knowledge retrieval, characterized in that: The system includes a user interaction module, a domain knowledge retrieval module, a thought clustering planning module, a base large language model, a guidance optimization module, a tool calling module, a response synthesis module and a model output module; The user interaction module is used to obtain user prompt input; The domain knowledge retrieval module includes a domain knowledge base, a thought clustering method base and a retrieval unit; the retrieval unit includes a vector coding model and a vector database; the thought clustering method base includes a variety of clustering methods for thought clustering that are compatible with the domain knowledge; The thought cluster planning module is used to call the domain knowledge retrieval module to perform thought cluster retrieval according to the user prompt input, build a thought chain example according to the retrieval result, and build a system prompt template according to the thought chain example, the original system prompt and the user question; The base large language model is used to perform task planning based on task timing according to the user prompt input and the system prompt template to obtain a planning chain; the base large language model is a pre-trained large language model that matches the domain scenario; The guiding optimization module is used to optimize and integrate the preset tool calling prompt template according to the planning chain, and output the optimized tool calling prompt template; The tool calling module uses the tool calling large language model to obtain a structured tool calling template according to the planning chain and the optimization tool calling prompt template, and uses the structured tool calling template to call the tool to obtain the interaction result of the current plan; the tool calling module is a base large language model fine-tuned by a pre-built preference data set; the response synthesis module is used to obtain the interaction result corresponding to the planning chain from the tool calling module and output the planning scheme, and feed the planning scheme back to the user interaction unit through the model output module; The system further comprises a dual reflection module, which is connected to the thought cluster planning module and the tool calling module respectively, and is used for judging the cause of the abnormality according to the current calling result and the expected result when an abnormality occurs in the tool calling module, obtaining a forward reflection result, performing an abnormal reverse reasoning to obtain the cause of the abnormality, obtaining a reverse reflection result, and feeding back the forward reflection result and the reverse reflection result to the thought cluster planning module; According to the user prompt input, the domain knowledge retrieval module is called to perform thought clustering retrieval, and examples of constructing a thought chain based on the thought clustering retrieval process include: Extracting a thought chain entity from the user prompt input, inputting the thought chain entity into the retrieval unit according to the thought chain entity, vectorizing the thought chain entity through a vector coding model, and obtaining a thought chain instruction; Using the clustering method in the thought clustering method library to cluster candidates and recall in the vector database according to the thought chain instruction, and outputting the search result; The domain knowledge document corresponding to the search result in the domain knowledge base is obtained, the context information of the thought chain entity is obtained according to the domain knowledge document, and a thought chain example is constructed according to the context information and the domain scenario prompt sequence.
2. The system according to claim 1, characterized in that Fine-tuning with pre-built preference datasets includes: Constructing a preference data set; the preference data set is obtained by acquiring several planning chains and corresponding calling tools from domain data and knowledge base; The tool call large language model is fine-tuned using the preference data set and a preset direct preference optimization loss function to obtain a fine-tuned tool call large language model.
3. The system according to claim 2, characterized in that The direct preference optimization loss function is: in, To directly optimize the loss function, x is the input sample and the preference label is y w , the non-preferred label is y l , β is the scaling factor, σ is the sigmoid function, π θ represents the policy model, π ref represents the reference strategy model, and D is the preference dataset.
4. The system according to claim 1, characterized in that According to the planning chain, the preset tool call prompt template is optimized and integrated, and the output optimized tool call prompt template includes: Obtain a tool call prompt template; the tool call prompt template includes a system prompt; optimize the system prompt according to the planning chain, and generate an adaptive scene information prompt, optimize and integrate the tool call prompt template according to the optimized system prompt and scene information prompt, and output an optimized tool call prompt template.
5. The system according to claim 1, characterized in that The tool calling module includes a template generator and a template executor; The template generator is driven by a tool call large language model and is used to generate a structured tool call template according to the planning chain and the optimization tool call prompt template; The template executor is used to parse the structured tool call template generated by the template generator and call the target API interface or function to execute the current plan according to the executed plan and execution results.
6. The system according to claim 5, characterized in that The template executor is also used to handle abnormal situations caused by abnormal reasons.