Complex task disassembling, classifying and optimizing method and system

Abstract

The invention provides a complex task disassembling, classifying and optimizing method, and belongs to the technical field of artificial intelligence. The method comprises the steps of obtaining learning data of a user, performing preprocessing, and performing training optimization on a large language model based on the preprocessed data; performing vectorization operation on knowledge which is not learned by the large language model and taking the knowledge as a plug-in knowledge base of the large language model; constructing an intelligent agent framework, and disassembling, classifying and optimizing an input complex task; the method comprises the following steps: analyzing a cue word of an input complex task; analyzing the analyzed cue word based on a large language model to generate a solution; calling a plug-in knowledge base to obtain information related to the current complex task; the method comprises the following steps: decomposing a complex task into a plurality of sub-tasks by utilizing Planning task planning, and formulating an effective strategy; and calling a related function based on an Action tool to realize the splitting, classification and optimization of the complex task. The intelligent agent can make a more reasonable and efficient decision in a complex environment, and an output result is prevented from deviating from expectation.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence, and particularly relates to a method and system for disassembling, classifying, and optimizing complex tasks. Background Art

[0002] The statements in this part merely provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of artificial intelligence technology, deep learning models have achieved remarkable results in various application scenarios. Especially large models, such as models based on the Transformer architecture, have shown excellent performance on large-scale data, including fields such as natural language processing, computer vision, and speech recognition. These large models can capture complex relationships and features in the data through a large number of parameters and deep network structures.

[0004] However, despite the high capabilities of large models, they still face some challenges when dealing with complex tasks. For example, for some complex tasks, these models may require a large amount of data and time for training, and it is difficult to optimize and adjust them. In addition, these models are usually designed for specific tasks or datasets and are difficult to adapt to other tasks or datasets.

[0005] To solve the above problems, it has been proposed to use an intelligent agent (AI Agent) to improve the performance and adaptability of large models when dealing with complex tasks. It allows a large language model to disassemble a complex task into several relatively simple tasks, each task can be solved by a specific tool (browser, calculator, etc.), and finally the large model summarizes the answers at each stage and finally outputs the result. Although the above method improves the ability of large models to handle complex tasks to a certain extent, there are still some limitations.

[0006] First, the generalization ability is insufficient. The training of the Agent usually depends on a specific dataset. When the service scenario changes or encounters unseen data, the performance of the Agent may drop severely, showing the problem of insufficient generalization ability. Second, for some complex tasks, the lack of rationality in the business disassembly of large models will lead to the final output result deviating from the expectation. Summary of the Invention

[0007] To overcome the deficiencies of the above prior art, the present invention provides a method and system for disassembling complex tasks based on a large model and an intelligent agent, which expands the knowledge of the large model by means of an external knowledge base. The behavior of the large model is aligned with human behavior through reinforcement learning to disassemble complex problems more reasonably.

[0008] To achieve the above object, one or more embodiments of the present invention provide the following technical solutions:

[0009] In the first aspect of the present invention, a method and system for disassembling, classifying, and optimizing complex tasks are provided;

[0010] A method for disassembling, classifying, and optimizing complex tasks includes:

[0011] Obtain the learning data of the user and perform preprocessing. Based on the preprocessed data, train and optimize the large language model;

[0012] Perform vectorization on the knowledge not learned by the large language model and use it as an external knowledge base for the large language model;

[0013] Construct an agent architecture to disassemble, classify, and optimize the input complex tasks;

[0014] Among them, the agent architecture parses the prompt words of the input complex tasks; based on the trained and optimized large language model, deeply analyzes the parsed prompt words, generates solutions for complex tasks; calls the external knowledge base to obtain information related to the current complex task to assist in decision-making and task execution; uses the Planning task planning to decompose the complex task into multiple subtasks and formulate effective strategies; based on the Action tool, call relevant functions to achieve complex task classification and optimization.

[0015] As a further technical solution, the preprocessing process includes cleaning, integrating, and standardizing the learning data of the user to form a unified data format, providing a data set for subsequent optimization of the large language model.

[0016] As a further technical solution, the process of training and optimizing the large language model includes:

[0017] Obtain the training parameters of the large language model and adjust them to accelerate model training and improve the performance of the large language model;

[0018] Use the PPO proximal policy optimization algorithm to evaluate the performance of the large language model.

[0019] As a further technical solution, the PPO proximal policy optimization algorithm introduces importance sampling weights, enabling the new policy to reuse the trajectories generated by the old policy multiple times; by constraining the range of the KL divergence, the range of policy differences is controlled; the optimization update objective of PPO is as follows:

[0020]

[0021] In the formula, θ represents the parameter vector of the policy network; represents from the old policy πθ oldPerform an expectation operation on the sampled trajectory τ; π θ (a t |s t ) represents the probability of taking action a t in state s t , which is determined by the current policy πθ; represents the probability of taking action a t in state s t , which is determined by the old policy πθ old ; represents the advantage estimate under the old policy, that is, the difference between the value of taking a specific action in this state and the average value; represents the Kullback-Leibler divergence between two probability distributions, which is used to measure the distance or similarity between them.

[0022] As a further technical solution, the process of using a vectorization model to vectorize knowledge that the large language model has not learned and using it as an external knowledge base for the large language model is as follows:

[0023] Vectorize the knowledge that the large language model has not learned, and convert the text data into a fixed-length digital form;

[0024] Store the text data converted into digital form in the knowledge base for enhanced generation of the large language model.

[0025] As a further technical solution, the enhanced generation of the large language model specifically is, based on vector similarity retrieval, retrieve relevant contexts in the knowledge base according to the semantic similarity between the user query and the embedding blocks, obtain knowledge blocks related to the input question, so as to enhance the generation ability of the large language model.

[0026] As a further technical solution, the process of implementing complex task splitting, classification, and optimization based on Action tool call is as follows:

[0027] Based on the function description to be called, request parameter description, and response parameter description, let the large language model appropriately select which function to call according to the user's input, while understanding the user's natural language and converting it into the request parameters for calling the function; based on the function name and parameters returned by the large language model, call the function and obtain the response.

[0028] The second aspect of the present invention provides a complex task splitting, classification, and optimization system.

[0029] A complex task splitting, classification, and optimization system includes:

[0030] A preprocessing module, configured to: obtain the learning data of the user and perform preprocessing;

[0031] The model training and tuning module is configured to: train and tune the large language model based on the preprocessed data;

[0032] The retrieval enhancement generation module is configured to: vectorize the knowledge that has not been learned by the large language model and use it as an external knowledge base of the large language model;

[0033] The task decomposition, classification and optimization module is configured to: build an intelligent agent architecture to decompose, classify and optimize the input complex tasks;

[0034] Among them, the intelligent agent architecture parses the prompt words of the input complex tasks; based on the trained and tuned large language model, the parsed prompt words are deeply analyzed to generate solutions for complex tasks; the knowledge base is called to obtain information related to the current complex task to assist decision-making and task execution; Planning task planning is used to decompose complex tasks into multiple subtasks and formulate effective strategies; based on the Action tool, related functions are called to realize complex task disassembly, classification and optimization.

[0035] The third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of a complex task decomposition, classification, and optimization method as described in the first aspect of the present invention.

[0036] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the complex task decomposition, classification, and optimization method described in the first aspect of the present invention are implemented.

[0037] One or more of the above technical solutions have the following beneficial effects:

[0038] The present invention uses the reward signal in reinforcement learning to guide the agent to make the next action choice. By accumulating rewards, the agent can learn which action should be taken in a given state to obtain the maximum long-term reward. This helps the agent make more reasonable and efficient decisions in complex environments.

[0039] At a higher level, the intelligent agent relies on large language models and retrieval-enhanced generation. The retrieval-enhanced generation technology provides the agent with more accurate and useful information support, combined with its own perception and decision-making capabilities, to perform specific tasks in various environments.

[0040] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings forming a part of this invention are used to provide a further understanding of the invention. The schematic embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0042] Figure 1 It is a flowchart of the method for the first embodiment.

[0043] Figure 2 It is a schematic diagram of function calls in the first embodiment.

[0044] Figure 3 It is a system structure diagram of the second embodiment. Detailed implementation manners

[0045] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this invention belongs.

[0046] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary embodiments according to this invention.

[0047] Without conflict, the embodiments in this invention and the features in the embodiments can be combined with each other.

[0048] Embodiment 1

[0049] This embodiment discloses a complex task decomposition method based on a large model and an intelligent agent;

[0050] As Figure 1 shown, a complex task decomposition method based on a large model and an intelligent agent includes:

[0051] Step S1, obtaining the learning data of the user and performing preprocessing;

[0052] By cleaning, integrating, and standardizing the learning data of the user, a unified data format is formed to provide a data set for subsequent model tuning.

[0053] Step S2, training and tuning the large language model based on the preprocessed data;

[0054] Model training is the core step of deep learning training. It involves using a large amount of data to train a model so that it can complete specific tasks. Model training includes processes such as forward propagation, backward propagation, and optimization algorithms. By continuously iteratively updating the parameters of the model, the performance of the model is gradually improved. Among them, forward propagation is used to input the features of the training data into the deep neural network, and the prediction result will be obtained according to the current parameters in the network; backward propagation is used to perform error analysis based on the prediction result of forward propagation and the calibration of the training data, so as to adjust the parameters of the neural network and make the prediction result of the network more and more accurate. Through the optimization algorithm, the loss function of the model parameters can be continuously reduced. Commonly used optimization algorithms include gradient descent method, quasi-Newton method, etc.

[0055] In step S2, a reinforcement learning scheme is adopted for tuning. The purpose of reinforcement learning is to "align" the behavior of the pre-trained large language model with that of humans, so that it can understand human instructions and give helpful answers to people, and correct incorrect and harmful knowledge. In this embodiment, human feedback reinforcement learning (RLHF) is adopted. RLHF is essentially to optimize the model through human feedback, guiding the output of the model to be as aligned with human thinking habits as possible, so that the decomposition of complex tasks is more in line with human habits.

[0056] During the training process of the large language model, appropriate training parameters of the large language model are obtained. Such as learning rate, discount factor, exploration rate, etc. The learning rate determines the speed at which the model updates parameters according to the feedback signal, the discount factor determines the importance of future rewards for the model, and the exploration rate determines the degree of exploration of the model during training. By adjusting these parameters, the model training can be accelerated and the performance of the model can be improved.

[0057] In addition, in order to accurately evaluate the performance of the deep reinforcement learning model, appropriate evaluation metrics are introduced in this embodiment. Such as cumulative reward, learning curve, convergence speed, etc. These evaluation metrics can quantitatively evaluate the model, so as to judge the performance and improvement space of the model.

[0058] Specifically, the reinforcement learning algorithm is the PPO (Proximal Policy Optimization) proximal policy optimization algorithm. In the traditional policy gradient algorithm, due to the update of the policy, the trajectories sampled by the old policy cannot be reused, otherwise there will be a biased estimate, resulting in very low sample efficiency. And for the above reasons, the update step is also very small.

[0059] In addition, in PPO, importance sampling weights are introduced, so that the new policy can reuse the trajectories generated by the old policy multiple times; and by constraining the range of KL divergence, the range of policy differences is controlled. The optimization update objective of PPO is as follows:

[0060]

[0061] where θ represents the parameter vector of the policy network; denotes the expectation operation over the trajectories τ sampled from the old policy πθ old ; π θ (a t |s t ) represents the probability of taking action a t in state s t , which is determined by the current policy πθ; denotes the probability of taking action a t in state s t , which is determined by the old policy πθ old ; denotes the advantage estimation under the old policy, i.e., the difference between the value of taking a specific action in this state and the average value; denotes the Kullback-Leibler divergence between two probability distributions (the new policy and the old policy), which is used to measure the distance or similarity between them.

[0062] Step S3, perform vectorization on the knowledge that the large language model has not learned and use it as an external knowledge base for the large language model;

[0063] Since it is difficult for the training data of the pre-trained large language model (LLM) to cover all fields of knowledge, the vectorization model vectorizes the knowledge that the large model has not learned and stores it in the knowledge base for retrieval during task decomposition, so as to improve the generalization ability of the large model and split complex tasks more reasonably. The specific steps include:

[0064] Step S31, vectorize the knowledge that the large language model has not learned, and convert the text data into a fixed-length digital form; so-called text vectorization is to numericalize the text, so as to facilitate modeling analysis. The text data faced by natural language processing is often unstructured and messy text data, while the data processed by machine learning algorithms is often fixed-length input and output. Therefore, machine learning cannot directly process the original text data and must convert the text data into a digital form.

[0065] Step S32, store the text data converted into digital form in the knowledge base for enhanced generation of the large language model. The enhanced generation of the large language model specifically means that based on vector similarity retrieval, relevant contexts are retrieved in the knowledge base according to the semantic similarity between the user query and the embedding block, and knowledge blocks related to the input question are obtained to enhance the generation ability of the large language model.

[0066] Step S4, construct an agent architecture to decompose, classify, and optimize the input complex tasks;

[0067] An agent is a computer program or entity that acts autonomously, perceives the environment, makes decisions, and interacts with other agents or humans in a specific environment. It has characteristics such as autonomy, reactivity, sociality, and adaptability, and can adjust its behavior according to environmental changes to achieve preset goals.

[0068] Specifically, in the process of building an agent architecture and decomposing, classifying, and optimizing complex tasks input, it includes:

[0069] Step S41, receiving and processing prompt words; Prompt words are the initial inputs received by the agent, describing the tasks the agent needs to complete or problems to be solved. The agent needs to parse and understand the prompt words to provide guidance for subsequent task planning and action execution. The writing of prompt words should clearly express requirements, unify pronouns, and avoid industry jargon to ensure that the agent correctly understands and executes tasks.

[0070] Step S42, large language model understanding, extraction, recognition, and selection;

[0071] The large language model is an important tool for the agent to perform task planning and knowledge reasoning. Through learning a large amount of text data, the large language model has powerful language processing capabilities and knowledge reasoning capabilities. The agent can use the large language model to deeply analyze the prompt words, generate possible solutions, and select and optimize them.

[0072] Step S43, calling, matching, and retrieving external knowledge bases;

[0073] When the agent executes tasks, it needs to retrieve and match information from an external knowledge base. The external knowledge base includes sensory memory, short-term memory, and long-term memory. Among them, long-term memory is further divided into text field content, files, and web page information. The agent retrieves relevant information from the external knowledge base to assist in decision-making and task execution.

[0074] Step S44, The main task of Planning task planning is to help the Agent decompose complex tasks into more manageable subtasks and formulate effective strategies. It is mainly divided into two types, one is a plan that does not rely on feedback, and the other is a plan based on feedback.

[0075] Feedback-independent planning does not refer to the feedback after task execution during the planning process and has several common strategies. For example, single-path reasoning generates a plan step by step in a cascading manner. Additionally, there is multi-path reasoning, which generates multiple alternative plan paths, forming a tree-like or graph-like structure. Of course, an external planner can also be used for rapid search to find the optimal plan. Feedback-based planning adjusts the plan according to the feedback after task execution, which is more suitable for situations that require long-term planning. The source of feedback may come from the objective feedback of the task execution result, the feedback given based on human subjective judgment, or even the feedback provided by an auxiliary model.

[0076] Step S45, the use and execution of the Action tool; the responsibility is to transform the abstract decision into specific actions and connect the internal and external environments of the Agent using the Action tool. When performing a task, the goals of the actions, the generation methods, the application scope, and the possible impacts need to be considered.

[0077] Ideal actions should be purposeful, such as completing a specific task, communicating with other agents, or exploring the environment. The generation of actions can rely on querying past memory experiences or following a preset plan. The scope of actions can not only be extended by using external tools such as APIs and knowledge bases but also by leveraging the inherent capabilities of large language models, such as planning, dialogue, and understanding common sense.

[0078] In this embodiment, the Action tool selects Function Calling, which is a mechanism for connecting large language models to external tools. Figure 2 When calling a large language model through an API, the caller can describe the function, including the function description, request parameter description, and response parameter description, allowing the large language model to appropriately select which function to call based on the user's input, while understanding the user's natural language and converting it into the request parameters for calling the function (returned in JSON format). The caller uses the function name and parameters returned by the large language model to call the function and obtain a response. Finally, if required, the response of the function is passed to the large language model, which organizes it into natural language to reply to the user. Complex task classification and optimization are achieved based on the relevant functions called.

[0079] Embodiment Two

[0080] Combined with Figure 3 , this embodiment discloses a complex task decomposition system based on a large model and an intelligent agent;

[0081] A complex task decomposition system based on a large model and an intelligent agent, comprising:

[0082] A preprocessing module, configured to: obtain the learning data of the user and perform preprocessing;

[0083] A model training and tuning module, configured to: train and tune a large language model based on the preprocessed data;

[0084] A knowledge vectorization module, configured to: perform vectorization operations on knowledge not learned by the large language model and use it as an external knowledge base of the large language model;

[0085] A task decomposition, classification, and optimization module, configured to: construct an agent architecture to decompose, classify, and optimize the input complex tasks;

[0086] Among them, the agent architecture parses the prompt words of the input complex tasks; based on the trained and tuned large language model, it deeply analyzes the parsed prompt words to generate solutions for complex tasks; it calls the knowledge base to obtain information related to the current complex task to assist in decision-making and task execution; it uses the Planning task planning to decompose the complex task into multiple subtasks and formulate effective strategies; based on the Action tool, it calls relevant functions to achieve the decomposition, classification, and optimization of complex tasks.

[0087] Embodiment III

[0088] The purpose of this embodiment is to provide a computer-readable storage medium.

[0089] A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the steps in a method for decomposing complex tasks based on a large model and an agent as described in Embodiment 1 of the present disclosure.

[0090] Embodiment IV

[0091] The purpose of this embodiment is to provide an electronic device.

[0092] An electronic device, including a memory, a processor, and a program stored on the memory and executable on the processor, and when the processor executes the program, it implements the steps in a method for decomposing complex tasks based on a large model and an agent as described in Embodiment 1 of the present disclosure.

[0093] The steps involved in the devices in the above Embodiments II, III, and IV correspond to those in Method Embodiment 1, and the specific implementation manners can refer to the relevant description part of Embodiment 1. The term "computer-readable storage medium" should be understood to include a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode, or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0094] Those skilled in the art should understand that the various modules or steps of the present invention described above can be implemented by a general-purpose computer device. Optionally, they can be implemented by program codes executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0095] Although the specific implementation manners of the present invention have been described above in conjunction with the accompanying drawings, it is not a limitation to the protection scope of the present invention. Those skilled in the art should understand that, based on the technical solutions of the present invention, various modifications or deformations that can be made without creative efforts by those skilled in the art are still within the protection scope of the present invention.

Claims

1. A complex task decomposition, classification and optimization method, characterized in that: include: Obtain the user's learning data and preprocess it, and then train and optimize the large language model based on the preprocessed data; Vectorize the knowledge that has not been learned by the large language model and use it as an external knowledge base for the large language model; Build an intelligent agent architecture to decompose, classify and optimize complex input tasks; Among them, the intelligent agent architecture parses the prompt words of the input complex tasks; based on the trained and tuned large language model, the parsed prompt words are deeply analyzed to generate solutions for complex tasks; the plug-in knowledge base is called to obtain information related to the current complex task to assist decision-making and task execution; the Planning task planning is used to decompose the complex task into multiple subtasks and formulate effective strategies; based on the Action tool, relevant functions are called to implement complex task splitting and optimization.

2. A complex task decomposition, classification and optimization method as claimed in claim 1, characterized in that: The preprocessing process includes cleaning, integrating and standardizing the user's learning data to form a unified data format, providing a data set for subsequent large language model tuning.

3. A complex task decomposition, classification and optimization method as claimed in claim 1, characterized in that: The process of training and tuning a large language model includes: Obtain the training parameters of the large language model and adjust them to accelerate model training and improve the performance of the large language model; The performance of large language models is evaluated using the PPO proximal policy optimization algorithm.

4. A complex task decomposition, classification and optimization method as claimed in claim 3, characterized in that: The PPO proximal strategy optimization algorithm introduces importance sampling weights so that the new strategy can use the trajectories generated by the old strategy multiple times; by constraining the range of KL divergence, the difference range of the strategy is controlled; the optimization update target of PPO is as follows: In the formula, θ represents the parameter vector of the policy network; Represents the change from the old policy πθ old The sampled trajectory τ performs the expected operation; π θ (a t |s t ) means in state s t Take action a t The probability of is determined by the current strategy πθ; Indicates that in state s t Take action a t The probability of old Decide; represents the advantage estimate under the old policy, that is, the difference between the value of taking a specific action in this state and the average value; Represents the Kullback-Leibler divergence between two probability distributions, which is used to measure the distance or similarity between them.

5. A complex task decomposition, classification and optimization method as claimed in claim 1, characterized in that: The process of using the vectorization model to vectorize the knowledge that has not been learned by the large language model and use it as an external knowledge base of the large language model is as follows: Vectorize the knowledge that has not been learned by the large language model and convert the text data into a fixed-length digital form; The text data converted into digital form is stored in the knowledge base for enhanced generation of large language models.

6. A complex task decomposition, classification and optimization method as claimed in claim 5, characterized in that: The enhanced generation of the large language model is specifically to retrieve relevant contexts in the knowledge base according to the semantic similarity between the user query and the embedded block based on vector similarity retrieval, obtain knowledge blocks related to the input question, so as to enhance the generation capability of the large language model.

7. A complex task decomposition, classification and optimization method as claimed in claim 1, characterized in that: The process of implementing complex task splitting and optimization by calling related functions based on the Action tool is as follows: Based on the function description, request parameter description, and response parameter description of the function to be called, the large language model can appropriately select which function to call based on the user's input, understand the user's natural language, and convert it into request parameters for calling the function; based on the function name and parameters returned by the large language model, the function is called and a response is obtained.

8. A complex task decomposition, classification and optimization system, characterized in that: include: The preprocessing module is configured to: obtain the user's learning data and perform preprocessing; The model training and tuning module is configured to: train and tune the large language model based on the preprocessed data; The retrieval enhancement generation module is configured to: use the vectorization model to vectorize the knowledge that has not been learned by the large language model and use it as an external knowledge base of the large language model; The task decomposition, classification and optimization module is configured to: build an intelligent agent architecture to decompose, classify and optimize the input complex tasks; Among them, the intelligent agent architecture parses the prompt words of the input complex tasks; based on the trained and tuned large language model, the parsed prompt words are deeply analyzed to generate solutions for complex tasks; the knowledge base is called to obtain information related to the current complex task to assist decision-making and task execution; Planning task planning is used to decompose complex tasks into multiple subtasks and formulate effective strategies; based on the Action tool, related functions are called to realize complex task disassembly, classification and optimization.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in a complex task decomposition method based on a large model and an intelligent agent as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the steps in a complex task decomposition method based on a large model and an intelligent agent as described in any one of claims 1-7.

Images