An experience extraction and task planning method for large language model agents

Abstract

The present invention provides an experience extraction and task planning method for a large language model intelligent agent. The method first constructs an execution history library and provides an execution history upload interface to the intelligent agent. Then, subtask decomposition experience, task execution plan experience, and error-prone point reminder experience are extracted. Then, data enhancement is performed on the task description to reduce the impact of task description diversity on semantic similarity search. Finally, an experience library is constructed and an experience acquisition interface is provided to the intelligent agent. After receiving a task request, the intelligent agent calls the experience acquisition interface and obtains task-related experience data through semantic similarity retrieval. Based on the task-related experience data, the prompt structure of the intelligent agent is optimized.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence, and specifically relates to an experience extraction and task planning method for a large language model intelligent agent. Background Art

[0002] With the popularity of large language models, building intelligent agents using them has become a trend. The core process involves constructing prompts and leveraging large language models to plan tasks, invoke tools, and complete complex tasks. Currently, intelligent agents are widely used in various areas of production and life, such as intelligent report generation, intelligent sales advisors, intelligent database analysis, software deployment agents, digital employees, research assistants, and personal assistants.

[0003] AutoGPT, BabyAGI, and XAgent are currently well-known intelligent agents. However, they suffer from the problem that they must start from scratch each time they perform a task, unable to draw on past experience from similar cases for task execution and planning. Task planning trajectories in execution history are crucial for intelligent agents to migrate tasks, providing a reference for similar task plans and preventing them from repeatedly exploring task planning paths. Consequently, existing intelligent agents are unable to effectively mine historical experience data, resulting in frequent calls to large language models and high token consumption. Summary of the Invention

[0004] To address the shortcomings and deficiencies of the prior art, this paper proposes an experience extraction and task planning method for large language model agents. This method extracts experience from the agent's historical task planning trajectory and provides recommendations for future task execution. This experience includes subtask decomposition experience, task execution plan experience, and error-prone point reminder experience.

[0005] The method proposed in this paper can ultimately be presented as an SDK (Software Development Kit). The agent can upload its execution history through an interface and gain experience when subsequently executing similar tasks. The agent uses this experience to optimize its prompts, which in turn influences the formulation of subsequent specific task plans, thereby reducing the number of large language model calls generated by the agent's repeated exploration of execution paths.

[0006] The present invention proposes an experience extraction and task planning method for a large language model agent, comprising the following steps:

[0007] S1, build an execution history library and provide an execution history upload interface to the agent, wherein the execution history library stores the agent's task planning trajectory; the task planning trajectory includes the following attribute information: task description, task execution status, action or called tool, and action or tool operation result; the task execution status is used as an attribute to record whether the task execution result is successful; the action or tool operation result is used as an attribute to record whether the action execution or tool operation is successful;

[0008] S2, extracting several subtask decomposition experiences and several task execution plan experiences from the task planning trajectory in S1, each of which includes a task description and several subtasks obtained by its decomposition. The subtask decomposition experience is used to guide the intelligent agent on how to decompose a complex task into multiple subtasks. Each of which includes a task description and several actions and tools required to complete the task description. The task execution plan experience is used to guide the intelligent agent on how to sequentially call tools and take actions to complete the task.

[0009] S3, extract error-prone points and remind experience:

[0010] Using a large language model, we summarize the errors made by the agent from the execution history library and generate several error-prone experience reminders. Each error-prone experience reminder includes a task description and an error. The types of errors include: missing necessary parameters when calling a tool, ignoring dependencies between steps, and the agent lacking the knowledge required to perform the task.

[0011] S4, performing data enhancement on the task descriptions in the subtask planning experience, the task execution solution experience, and the error-prone point reminder experience to mitigate the impact of task description diversity on the semantic similarity search in S5. The data enhancement includes: using a large language model to expand the task description and generate descriptions from more perspectives;

[0012] S5, builds an experience database and provides the agent with an experience acquisition interface;

[0013] The experience data in the experience library includes: the subtask decomposition experience and task execution plan experience obtained in step S2, and the error-prone point reminder experience obtained in step S3; after receiving the task request, the intelligent agent calls the experience acquisition interface, obtains the task-related experience data through semantic similarity retrieval, and optimizes the Prompt structure of the intelligent agent based on the task-related experience data.

[0014] Preferably, in step S1, the format of the task planning trajectory is a structured tree structure; the task planning trajectory also includes the following attribute information: parent-child task affiliation; the structured tree structure includes three types of nodes, namely task nodes, underlying task nodes, and tool or action nodes; the three types of nodes all include a node status field, and the value of the node status field is success or failure; if the current node is a task node and an underlying task node, the node status field is used to mark the attribute information as the task execution status, and if the current node is a tool or action node, the node status field is used to mark the attribute information as the running result of the action or tool; the root node of the tree structure represents the task request received by the intelligent agent, and the type is any one of the task node and the underlying task node; the task node represents a task that can be decomposed into subtasks, and the child node of the task node that can decompose subtasks is a task node or an underlying task node; the underlying task node represents the task that is not It can be decomposed into subtasks, whose subnodes are tools or action nodes; the tool or action node serves as a leaf node, representing the action to be performed or the tool to be called to complete the underlying task; in step S2, the method of extracting several subtask decomposition experiences and several task execution plan experiences is a method of extracting experience from a structured tree-structured task planning trajectory, comprising the following steps: traversing downward from the root node of the tree until a leaf node is traversed; if the traversed node type is a task node and the node status is successful, a "subtask decomposition experience" is generated, and the structure of the "subtask decomposition experience" includes: the task description of the current node and all subtasks of the node; if the traversed node type is a bottom-level task node and the node status is successful, a "task execution plan experience" is generated, and the structure of the "task execution plan experience" includes: the task description of the current node and all tool or action nodes under the node with a successful node status.

[0015] Preferably, in step S1, the format of the task planning trajectory is: an unstructured text structure, presented in the form of continuous plain text; in step S2, the method of extracting several subtask decomposition experiences and several task execution plan experiences is to extract experience from the unstructured text task planning trajectory through a large language model, and the writing of the Prompt used by the large language model includes: defining a role substitution for the large language model; requiring the large language model to extract the subtask decomposition experience and task execution plan experience through the task planning trajectory; for the extraction of the subtask decomposition experience, find out the tasks that have been successfully executed and decomposed, and summarize their subtask decomposition methods; for the extraction of the task execution plan, find out the tasks that have been successfully executed and cannot be decomposed any further, and summarize the effective actions or tool calls of the task; requiring the output format of the large language model to be output in a structured format to facilitate subsequent program processing of the structured data.

[0016] Preferably, in step S3, the writing of the prompt used by the large language model includes: defining a role substitution for the large language model; requiring the large language model to extract the error-prone point reminder experience through task planning trajectory; the extraction method includes the following steps: requiring the large language model to refine the task and summarize the error-prone point reminder experience according to the type of error described in step S3; each task may contain multiple error-prone points, and the large language model is required to describe them in the same sentence; if no error is found, the task is ignored; and the output format of the large language model is required to be output in a structured form.

[0017] Preferably, in step S4, the writing of the prompt used by the large language model for the data enhancement includes: defining a role substitution for the large language model; requiring the large language model to expand the task description; the expansion method includes the following steps: erasing special parameter information in the task description; expanding the task description from several different angles; limiting the number of new task descriptions generated by the expansion; and requiring the output format of the large language model to be output in a structured form.

[0018] Preferably, in step S5, the construction of the experience library includes: using the embedding model to convert the subtask decomposition experience, task execution plan experience and error-prone point reminder experience into vectors and save them; calling the experience acquisition interface includes: using the embedding model to convert the task request received by the agent into a vector, and using cosine similarity to calculate the vector distance, so as to obtain several subtask decomposition experiences, task execution plan experiences and error-prone point reminder experiences that are most relevant to the current task; the agent optimizes its Prompt structure with reference to the most relevant several subtask decomposition experiences, task execution plan experiences and error-prone point reminder experiences.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] The present invention provides an SDK to asynchronously analyze the execution history of the agent in the background and provide relevant experience during the subsequent execution of the agent, thereby avoiding repeated exploration of task planning paths and consuming a large number of large language model tokens.

[0021] The experience extraction and task planning method of the present invention utilizes both success and failure cases: for success cases, subtask decomposition experience and task execution plan experience are extracted; for failure cases, error-prone point reminder experience is generated.

[0022] Experiments on the ALFWorld dataset show that compared with the ReAct task planning method, the task planning method optimized by the present invention reduces the average number of calls to the large language model per task from 22.9 to 17.75. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flow chart of the present invention.

[0024] Figure 2 This is a schematic diagram of using the experience extraction and task planning method for a large language model intelligent agent of the present invention to guide the work of the intelligent agent. DETAILED DESCRIPTION

[0025] To more clearly illustrate the embodiments of the present invention, specific embodiments of the present invention will be described below with reference to the accompanying drawings. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings and other embodiments can be obtained based on these drawings without inventive efforts.

[0026] This implementation provides an experience extraction and task planning method for large language model agents, the flow chart of which is as follows: Figure 1 As shown in the figure; the schematic diagram of guiding the work of the intelligent agent through this method is as follows Figure 2 shown.

[0027] This implementation demonstrates a real example of how to use the present invention to optimize an existing agent's task planning.

[0028] First, we build a simple large language model agent based on the LangChain framework as the agent to be optimized. Its original prompt template is as follows:

[0029]

[0030] The above template is built into the agent and remains consistent even when the task changes. {task} is a variable that changes automatically based on the task being executed. The agent sends the prompt, with the {task} variable embedded, to the large language model. Each time, the model generates a "thought" (or "action"), which the agent then executes, appending the result of the action as an "observation" to the end of the prompt. This cycle repeats until the task is finally completed.

[0031] In this example, the agent simulates a household robot and solves various tasks (derived from the ALFWorld dataset, used for agent task planning testing). We also limit the number of exploration steps in task planning to 30, meaning that the large language model can be called at most 30 times. This prevents the agent from retrying tasks indefinitely due to unsuccessful tasks.

[0032] First, if you are performing housework tasks for the first time and have no experience, the whole process is as follows:

[0033] Fill in the {task} field in the Prompt template. A task consists of two parts: a background description and a task description. The current task is "Wash some pans and place them on the counter."

[0034] Finally, the complete prompt is as follows:

[0035]

[0036] Using the above prompt to request the large language model, the resulting conversation is as follows (each action-observation pair is the result of a large language model call):

[0037]

[0038]

[0039] It can be seen that the current agent cannot complete the task within 30 steps. This is mainly because when searching for the frying pan, the agent tries to find it on three countertops, the sink, and two stoves in turn. It takes 17 steps to find it and ultimately fails.

[0040] First, step S1 is performed to save the above task planning trajectory into the execution history library through the execution history upload interface.

[0041] Then proceed to step S2 to extract subtask decomposition experience and task execution plan experience from the task planning trajectory. The specific prompts are as follows:

[0042]

[0043]

[0044] Among them, {log} represents the unstructured text-based task planning trajectory, and this field is automatically filled in each time the large language model is requested.

[0045] Using the above prompt to request a large language model, in this embodiment, step S2, the result obtained is as follows:

[0046]

[0047] As you can see, there are two types of experiences:

[0048] (1) Subtask decomposition experience: "Clean some pans and put them on the countertop" can be decomposed into three subtasks: "Find and pick up a pan", "Clean the pan", and "Put the pan on the countertop".

[0049] (2) Experience in mission execution:

[0050] ① To "find and pick up a frying pan," the possible actions are: "Go to Stove 2," "Pick up Pan 1 from Stove 2." Note that the agent previously searched for the pan in multiple locations before ultimately finding it on the stove. However, the generated experience directly skipped these ineffective intermediate steps.

[0051] ② Since "clean the pan" and "place the pan on the countertop" were not executed successfully, the valid sub-steps are empty and the experience will be eliminated by the system.

[0052] Then proceed to step S3 to extract error-prone reminder experience. The prompt is as follows:

[0053]

[0054] Where {task_and_logs} is a variable representing the agent execution history.

[0055] Using the above prompt to request a large language model, the result of step S3 is as follows:

[0056]

[0057] As you can see, two common mistakes were identified. The first corrects external domain knowledge (the possible locations of the pan), and the second warns the large language model against hallucinations, as the agent attempts to pick up a non-existent pan in action 8.

[0058] Then, step S4 is performed to enhance the task description in the subtask planning experience, the task execution plan experience, and the error-prone point reminder experience. The specific prompt is as follows:

[0059]

[0060] Among them, {original_task_descriptions} represents the list of task descriptions to be generalized. In this embodiment, its content is as follows: "Clean some pans and put them on the counter", "Find and pick up a pan", "Find a pan", "Pick up a pan".

[0061] Using the above prompt to request a large language model, the result of step S4 is as follows:

[0062]

[0063] It can be seen that each task description in the above three experiences generates several task description extensions from different angles, which can improve the recall rate of experience acquisition in subsequent similarity retrieval based on task descriptions.

[0064] Then, proceed to step S5.1 and use the embedding model to convert the above three experiences into vectors and save them.

[0065] Below, after the demonstration agent uses the task planning method of the present invention, the overall process is as follows:

[0066] To demonstrate the agent's transferability between tasks, "we tested it this time on a task that was similar but different from the previous one, which was to "cool down some pans and place them on the counter."

[0067] It should be noted that unlike the previous method of directly filling the task into the prompt, this time the agent first gains experience through this invention and then constructs the prompt. The steps are as follows:

[0068] Going to step S5.2, the agent calls the "Experience Acquisition Interface" with the parameter "Cool some pans and place them on the countertop." The interface returns the following:

[0069]

[0070]

[0071] The first is about subtask decomposition, the second is about task execution plans, and the last two are about error-prone areas. We've also discovered that sometimes multiple experiences complement each other. For example, the first and third experiences improved the efficiency of finding a frying pan from different perspectives.

[0072] The agent can then use this experience to optimize the original Prompt, and the final generated Prompt is as follows:

[0073]

[0074]

[0075] It's not hard to see that the above prompt mainly adds the "Here are some experiences you can refer to" part. After multiple requests to the large language model, the final results are as follows:

[0076]

[0077]

[0078] As you can see, the agent ultimately completed the task in 11 steps. This was primarily because, after gaining experience with the frying pan, the agent adjusted its search order, first searching the stove (rather than cabinets, countertops, etc.), ultimately finding the pan in just five steps. It's worth noting that, although the placement of the items in this task was different from the first, the generated task planning suggestions still effectively influenced the agent's task planning strategy.

[0079] This paper implements an experience extraction and task planning method for large language model agents, tested in real-world household chores scenarios. This method demonstrates its ability to influence the agent's task planning by extracting multiple task planning suggestions from its execution history. This ultimately reduces the agent's incorrect task explorations, improves its task completion rate, and effectively reduces token consumption in large language models.

[0080] It should be noted that the above is only a detailed description of the preferred embodiments and principles of the present invention. For ordinary technicians in this field, based on the ideas provided by the present invention, there will be changes in the specific implementation methods, and these changes should also be regarded as the scope of protection of the present invention.

Claims

1. A method for experience extraction and task planning for a large language model agent, characterized in that: The following steps are involved: S1, builds the execution history library and provides the execution history upload interface to the agent, The execution history library stores the task planning trajectory of the intelligent agent; The task planning trajectory includes the following attribute information: task description, task execution status, action or called tool, and action or tool operation results; The task execution status is used as an attribute to record whether the task execution result is successful; The result of the action or tool is used as an attribute to record whether the action is executed successfully or the tool is run successfully; S2, extracts several subtask decomposition experiences and several task execution plan experiences from the task planning trajectory in S1, Each subtask decomposition experience includes a task description and several subtasks obtained by decomposition. The subtask decomposition experience is used to guide the agent on how to decompose a complex task into multiple subtasks. Each piece of task execution plan experience includes a task description and a number of actions and tools required to complete the task description. The task execution plan experience is used to guide the agent on how to sequentially call tools and take actions to complete the task; S3, extract error-prone points and remind experience: Using a large language model, we summarize the errors made by the agent from the execution history library and generate several error-prone experience reminders. Each error-prone experience reminder includes a task description and an error. The types of errors include: missing necessary parameters when calling a tool, ignoring dependencies between steps, and the agent lacking the knowledge required to perform the task. S4, performing data enhancement on the task descriptions in the subtask planning experience, the task execution solution experience, and the error-prone point reminder experience to mitigate the impact of task description diversity on the semantic similarity search in S5. The data enhancement includes: using a large language model to expand the task description and generate descriptions from more perspectives; S5, builds an experience database and provides the agent with an experience acquisition interface; The experience data in the experience database includes: the subtask decomposition experience and task execution plan experience obtained in step S2, and the error-prone point reminder experience obtained in step S3; After receiving the task request, the intelligent agent calls the experience acquisition interface, obtains the task-related experience data through semantic similarity retrieval, and optimizes the Prompt structure of the intelligent agent based on the task-related experience data.

2. The experience extraction and task planning method for a large language model agent according to claim 1, characterized in that: In step S1, the format of the task planning trajectory is a structured tree structure; the task planning trajectory also includes the following attribute information: parent-child task affiliation; The structured tree structure includes three types of nodes, namely task nodes, underlying task nodes, and tool or action nodes; all three types of nodes include a node status field, the value of which is success or failure; if the current node is a task node or an underlying task node, the node status field is used to mark the attribute information as the task execution status; if the current node is a tool or action node, the node status field is used to mark the attribute information as the running result of the action or tool; The root node of the tree structure represents the task request received by the agent, and its type is either a task node or a bottom-level task node; The task node represents a task that can be decomposed into subtasks, and the child node of the task node that can be decomposed into subtasks is a task node or an underlying task node; The bottom-level task node represents a task that cannot be decomposed into subtasks, and its subnodes are tool or action nodes; The tool or action node is a leaf node, representing the action to be performed or the tool to be called to complete the underlying task; In step S2, the method for extracting a plurality of subtask decomposition experiences and a plurality of task execution plan experiences is a method for extracting experiences from a structured tree-structured task planning trajectory, comprising the following steps: Traverse down from the root node of the tree until you reach the leaf node; If the traversed node type is a task node and the node status is successful, a "subtask decomposition experience" is generated. The structure of the "subtask decomposition experience" includes: the task description of the current node and all subtasks of the node; if the traversed node type is a bottom-level task node and the node status is successful, a "task execution plan experience" is generated. The structure of the "task execution plan experience" includes: the task description of the current node and all tool or action nodes under the node with a successful status.

3. The experience extraction and task planning method for a large language model agent according to claim 1, characterized in that: In step S1, the format of the task planning trajectory is: an unstructured text structure, presented in the form of continuous plain text; In step S2, the method for extracting several subtask decomposition experiences and several task execution plan experiences is to extract experiences from unstructured text-based task planning trajectories through a large language model. The writing of prompts used by the large language model includes: Define a role substitution for the large language model; The large language model is required to extract the subtask decomposition experience and task execution plan experience through the task planning trajectory; Extracting the subtask decomposition experience, finding the tasks that have been successfully executed and decomposed, and summarizing their subtask decomposition methods; For the extraction of the task execution plan, find the tasks that are successfully executed and cannot be further decomposed, and summarize the effective actions or tool calls of the tasks; The output format of the large language model is required to be in a structured format to facilitate subsequent program processing of the structured data.

4. The experience extraction and task planning method for a large language model agent according to claim 1, characterized in that: In step S3, the writing of the Prompt used by the large language model includes: Define a role substitution for the large language model; The large language model is required to extract the error-prone points through task planning trajectories and remind users of these errors. The extraction method includes the following steps: requiring the large language model to refine the task, and summarizing the common error reminder experience according to the error type described in step S3; Each task may contain multiple error-prone points, requiring the large language model to describe them in the same sentence; if no errors can be found, the task is ignored; The output format of the large language model is required to be in a structured form.

5. The experience extraction and task planning method for a large language model agent according to claim 1, characterized in that: In step S4, the writing of the Prompt used by the large language model for the data enhancement includes: Define a role substitution for the large language model; Requires large language models to extend the task description; The expansion method includes the following steps: erasing special parameter information in the task description; expanding the task description from several different angles; limiting the number of new task descriptions generated by the expansion; The output format of the large language model is required to be in a structured form.

6. The experience extraction and task planning method for a large language model agent according to claim 1, characterized in that: In the step S5, The construction of the experience library includes: using an embedding model to convert the subtask decomposition experience, task execution plan experience, and error-prone point reminder experience into vectors and save them; Calling the experience acquisition interface includes: using the embedding model to convert the task request received by the agent into a vector, using cosine similarity to calculate the vector distance, so as to obtain several subtask decomposition experiences, task execution plan experiences and error-prone point reminder experiences that are most relevant to the current task; the agent refers to the several most relevant subtask decomposition experiences, task execution plan experiences and error-prone point reminder experiences to optimize its Prompt structure.

Images