Intelligent agent high-level task planning method and device based on task graph enhancement, equipment and storage medium
By using task graph augmentation, logical and physical constraints between subtasks are constructed to guide the large language model to generate verifiable subtask sequences. The model is then optimized through a reward function designed using the task graph, which solves the problems of instruction misunderstanding and insufficient generalization ability of embodied agents in long-cycle tasks, and achieves more efficient planning results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- RENMIN UNIVERSITY OF CHINA
- Filing Date
- 2026-03-16
- Publication Date
- 2026-06-16
AI Technical Summary
Existing embodied agents based on large language models suffer from instruction misunderstanding and planning illusion when performing long-cycle complex tasks, and their generalization ability is insufficient due to supervised fine-tuning on data within a limited domain.
The task graph augmentation method is adopted to represent the logical and physical constraints between subtasks by constructing a task graph, which guides the large language model to generate verifiable subtask sequences. The model is optimized by the reward function designed by the task graph, and combined with the iterative optimization process, the feasibility and consistency of the plan are ensured.
It improves the planning feasibility and generalization ability of embodied agents in long-term tasks, reduces planning illusions, and increases the success rate in unseen tasks.
Smart Images
Figure CN122222010A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of embodied artificial intelligence technology, and in particular to a method, apparatus, device and storage medium for high-level task planning of intelligent agents based on task graph enhancement. Background Technology
[0002] Currently, research on embodied agents that execute human commands to complete long-term daily tasks is increasing. In the field of embodied artificial intelligence, command following requires the agent to possess three key capabilities: parsing natural language, utilizing egocentric visual observation, and performing physical actions to navigate and interact in the environment.
[0003] One direct approach is to train agents in an end-to-end supervised manner using large-scale datasets with labeled instructions and sequences of actions from underlying experts. However, this paradigm is extremely resource-intensive: it heavily relies on task-specific data and generalizes poorly to unseen scenarios. In contrast, data-efficient hierarchical approaches have emerged as a promising alternative: a high-level planner first breaks down instructions into subtasks, and then the lower-level executor invokes a skill library to translate these subtasks into actionable actions in the environment.
[0004] In recent years, thanks to the powerful language understanding and reasoning capabilities of Large Language Models (LLMs), a trend in hierarchical approaches has been to utilize LLMs for high-level planning. Nevertheless, long-term planning remains a major challenge for embodied agents, with the following key limitations: First, LLM-based planning lacks a physical foundation, making it prone to "planning illusions." While general-purpose LLMs possess rich knowledge and powerful reasoning capabilities, their lack of understanding of the physical environment and the robot's action space often leads to misunderstandings of instructions or planning illusions during plan generation, preventing the robot from executing in real-world scenarios. Furthermore, the failure rate of task planning increases significantly with task complexity and the extension of the field of view.
[0005] Secondly, models fine-tuned under supervision on data within a limited domain lack generalization ability. To adapt to specific task domains, existing methods typically require supervised fine-tuning on large-scale labeled datasets containing expert action sequences. This approach relies on costly labeled data and generalizes poorly to unseen long-term tasks, resulting in a significant drop in planning performance.
[0006] Therefore, a new solution is needed to provide a physical basis for LLM planning and to have the ability to plan long-term tasks. Summary of the Invention
[0007] This application provides a method, apparatus, device, and storage medium for high-level task planning of intelligent agents based on task graph enhancement, to address the following technical shortcomings of existing embodied intelligent agent planners based on large language models, despite significant progress: (i) Although general-purpose LLMs possess powerful reasoning capabilities, they lack a physical basis, leading to instruction misunderstandings, planning illusions, and an increased failure rate as task complexity increases. (ii) Supervised fine-tuning on limited domain data results in poor generalization ability to unseen long-cycle tasks unless extremely costly labeled data is used.
[0008] Firstly, this application provides a high-level task planning method for intelligent agents based on task graph enhancement, including: Obtain the task graph, which is represented as follows: ,in Nodes represent various types of subtasks. Represents the set of edges, each edge This indicates that it is allowed from subtasks. Switch to subtask ; Obtaining natural language instructions τ ; Based on the task graph With the natural language instructions An initial sequence of subtasks is generated using a large language model, where each subtask in the sequence corresponds to a node in the task graph, and the subtask sequence is located within the task graph. G This forms a path , , and These represent the first, second, and Tth subtasks, respectively. The subtask sequence is verified and iteratively optimized to obtain an executable planning sequence.
[0009] In one possible design, the subtask Represented as or ,in Indicates advanced actions, Representing related objects; using metaclasses in task graphs. To replace the specific object name ,in Indicates non-container items. Indicates a fixed container. Indicates a movable container. Represents items required for a specific subtask; during the planning and generation process, the metaclass is mapped to a set of environment objects. The specific object name.
[0010] In one possible design, the method for constructing the edges of the task graph includes: if the previous subtask The final low-level action and subsequent sub-tasks If the initial action is functionally compatible within the capabilities of the agent, then at the node... and Add directed edges between .
[0011] In one possible design, the large language model is guided to generate the initial subtask sequence through a prompting strategy. The prompting strategy includes: encoding the nodes of the task graph as action-target pairs with unique labels, encoding the edges as symbolic connections, and integrating the encoded node and edge information, agent roles, planning rules, and expected output format into prompt words to guide the model to strictly follow the topological structure of the task graph and select the next subtask only from the neighboring nodes of the previous subtask node each time.
[0012] In one possible design, the large language model is a model pre-trained through reinforcement learning; the reinforcement learning training process includes: based on the task graph... G Constructing a reward function To optimize model parameters so that the model generates a graph that conforms to the task. G The ability of the constrained sub-task sequence; the reward function Including format rewards Node-level rewards Border-level rewards and instructions follow the reward ,and .
[0013] In one possible design, the format reward The determination method includes: if the output structure of the large language model satisfies the preset first and second formats, then let the format reward... If the output structure of the large language model only satisfies one of the preset first and second formats, then the format reward is set to the first value. If the output structure of the large language model does not satisfy either the preset first or second format, then the format reward is set to the second value. It equals the third value, where the first value is greater than the second value, and the second value is greater than the third value; The node-level reward The calculation formula is: in, For meta-class dynamic weights, For metaclass, The number of metaclasses, The efficiency of an object of a certain metaclass is calculated as follows: in, For indexing objects in the plan, For the set of objects in the plan, Metaclasses in the environment The corresponding actual collection of objects, For indicator functions; The side-level reward The calculation formula is: in, For adjacent subtasks, For the set of nodes in the task graph, Let be the set of edges in the task graph. Index for subtasks Number of subtasks; The instruction follows the reward The determination methods include: extracting key sub-tasks from the baseline plan; if the generated plan covers all key sub-tasks, then setting a reward for compliance. The first score is equal to the first score; if the generated plan perfectly matches the baseline plan, then the follow-through reward is given. The second score is equal to the second score; if the generated plan does not perfectly match the baseline plan and does not cover all key sub-tasks, then the follow-through reward is set. It equals the third score, where the second score is greater than the first score, and the first score is greater than the third score.
[0014] In one possible design, the subtask sequence is validated and iteratively optimized to obtain an executable planning sequence, including: Perform node-level and edge-level validity verification on the initial subtask sequence; The node-level validity verification function for: The verification function for the edge-level validity verification for: If the verification fails, the corresponding error message is fed back to the large language model to guide it in correcting the subtask sequence and iterating until the verification is passed or the preset maximum number of iterations is reached.
[0015] Secondly, this application provides a high-level task planning device for intelligent agents based on task graph enhancement, the device comprising: The task graph acquisition module is configured to acquire a task graph, which is represented as follows: ,in Nodes represent various types of subtasks. Represents the set of edges, each edge This indicates that it is allowed from subtasks. Switch to subtask ; The instruction acquisition module is configured to acquire natural language instructions. ; The path generation module is configured to be based on the task graph. With the natural language instructions An initial sequence of subtasks is generated using a large language model, where each subtask in the sequence corresponds to a node in the task graph, and the subtask sequence is located within the task graph. G This forms a path , , and These represent the first, second, and Tth subtasks, respectively. The iterative optimization module is configured to verify and iteratively optimize the subtask sequence to obtain an executable planning sequence.
[0016] Thirdly, embodiments of this application provide an electronic device, including: at least one processor and a memory; the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, causing the at least one processor to perform the high-level task planning method for intelligent agents based on task graph enhancement as described in the first aspect and various possible designs of the first aspect.
[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the high-level task planning method for intelligent agents based on task graph enhancement as described in the first aspect and various possible designs of the first aspect.
[0018] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the high-level task planning method for intelligent agents based on task graph enhancement as described in the first aspect and various possible designs of the first aspect.
[0019] The high-level task planning method, apparatus, device, and storage medium for intelligent agents based on task graph enhancement provided in this application have at least the following beneficial effects: This application introduces a unified task graph model to model the logical and physical constraints between subtasks, and uses it as the core of structured knowledge throughout the entire planning process: in the inference phase, prompts guide the generation of verifiable subtask sequences; in the training phase, a reward function is designed based on the task graph to drive the model to learn diverse feasible paths, improving long-field reasoning and generalization capabilities; and in the verification phase, it is used as an external module for iterative optimization. This systematically improves the executability and goal consistency of the planning process. Attached Figure Description
[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0021] Figure 1 A flowchart illustrating a high-level task planning method for intelligent agents based on task graph enhancement, provided for embodiments of this application; Figure 2 A detailed planning flowchart of a high-level task planning method for intelligent agents based on task graph enhancement provided in this application embodiment; Figure 3 This is a structural diagram of a high-level task planning device for intelligent agents based on task graph enhancement, provided in an embodiment of this application.
[0022] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation
[0023] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0024] The collection, storage, use, processing, transmission, provision, and disclosure of financial data or user data involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0025] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.
[0026] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0027] Using large visual language models (VLMs) for progress estimation in complex multi-step tasks remains extremely challenging, primarily for two reasons. First, embodied tasks typically consist of multiple subtasks with complex temporal dependencies, requiring the model to infer these dependencies and align them with visual observations. Second, processing long video trajectories via VLMs incurs significant computational and time overhead, rendering such methods impractical for real-world tasks.
[0028] To address the above issues, this application provides a high-level task planning method for intelligent agents based on task graph enhancement. The task graph provides a physical basis for high-level planning, mitigating the planning illusion inherent in large language models. The task graph, as a unified representation, models the sequential dependencies between subtasks. It permeates the entire reasoning process: first, by explicitly incorporating the task graph into prompts, it guides the LLM to generate verifiable plans; then, task graph-based rewards are used for group-relative policy optimization to enhance the LLM's reasoning capabilities; finally, it serves as an external verification module for iteratively optimizing the subtask sequence.
[0029] It should be noted that the intelligent agent described in this application refers to an autonomous or semi-autonomous system capable of perception, decision-making, and execution in a specific environment. Specific implementations include, but are not limited to: physical robots with physical actuators, such as home service robots, industrial robotic arms, autonomous vehicles, or drones; software programs running in virtual or simulated environments, such as virtual characters in digital twin systems or non-player characters in games; and composite systems that achieve remote operation through a combination of hardware and software. Any entity or system capable of receiving instructions, planning based on environmental information, and controlling execution units to complete objectives can be considered an intelligent agent as described in this method.
[0030] High-level tasks refer to complex instructions issued to achieve a macro-level goal, which need to be broken down into multiple ordered sub-steps. These tasks cannot be completed directly by a single atomic action, and their typical characteristics are temporality, composability, and logical dependency. For example, in a home environment, "preparing breakfast" is a high-level task; in an industrial setting, "assembling a product component" is a high-level task; and in the service industry, "guiding guests to their rooms and providing them with items" is also a high-level task.
[0031] To more clearly illustrate the relationship between the two, several examples of their integration are given below: A mobile service robot, as an intelligent agent, can be assigned the high-level task of preparing breakfast. This task needs to be broken down into a series of sub-steps, such as going to the kitchen, identifying ingredients, and operating kitchen utensils. An industrial robotic arm, as an intelligent agent, can be assigned the high-level task of assembling product components. This task involves several logically interconnected sub-steps, such as grasping parts, precise positioning, performing assembly, and quality inspection. An autonomous vehicle, as an intelligent agent, can be assigned the high-level task of point-to-point delivery of goods. This task encompasses a long-cycle sequence of operations, including path planning, dynamic obstacle avoidance, destination docking, and goods handover.
[0032] It must be specifically noted that the above enumeration of agent forms, high-level task types, and their combination methods is merely illustrative and intended to help understand the possible application scenarios of the method in this application, and is not intended to limit the scope of protection of this application in any way. The high-level task planning method for agents based on task graph enhancement provided in this application is universal in that it can plan various complex tasks that require multi-step collaboration for different types of agents.
[0033] like Figure 1 As shown, the high-level task planning method for agents based on task graph enhancement can be implemented through the following steps S10-S20.
[0034] S10: Obtain the task map G .
[0035] The task graph G mentioned in step S10 can be obtained through methods including, but not limited to, the following exemplary approaches: calling a predefined or pre-stored graph structure file, which encodes the logical relationships and transition constraints between subtasks within a specific task domain; automatically constructing or updating the task graph structure based on the analysis and summarization of historical task execution trajectories; dynamically generating nodes and edges of the task graph by analyzing the functional compatibility relationships between the underlying action strategies corresponding to the subtasks; and loading descriptive information of the task graph from an external knowledge base, domain model, or configuration interface. Regardless of the specific method used, the obtained task graph... G All are represented as a directed graph structure, which provides a physical and logical constraint basis for subsequent high-level mission planning.
[0036] In this embodiment, the constraint space of the subtask can be represented as a directed graph. , where nodes Each edge represents various types of subtasks. The allowed subtask transformations are encoded, namely... It is possible Then execute.
[0037] In one exemplary implementation, subtask nodes Represented as or ,in Indicates advanced actions, while Metaclasses are used in task graphs to refer to relevant objects. To encourage higher-level planners to focus on object roles within subtasks, metaclasses are used in task graphs. To replace the specific object name ,in Indicates non-container items. Indicates a fixed container. Indicates a movable container. This contains items required for specific subtasks; for example, a "refrigerator" is used for the "cooling" task. During plan generation, the metaclass is mapped to a set of environment objects. The specific object names; this set contains all visible objects identified by the pre-trained object detection module. For convenience, use... Represents each metaclass A corresponding set of environment objects. Each subtask is mapped to a serialized low-level action policy that specifies the primitive actions required to complete the task and is dynamically instantiated through environment awareness. For example, this embodiment uses 12 subtask nodes, which have proven to be an appropriate granularity to ensure that high-level planning can be reliably mapped to executable low-level actions. If necessary, the subtask nodes can be expanded as the low-level policy evolves.
[0038] In one exemplary implementation, when constructing edges on the task graph, one approach is to construct edges between subtasks by traversing the training trajectory. However, the objective of this application is to construct the task graph as a general representation that can generalize beyond the baseline task. To achieve this objective, this embodiment automatically constructs the graph by analyzing the functional compatibility between the low-level action policies corresponding to the subtasks. Specifically, for the current subtask... The final low-level action (e.g., grab) and subsequent subtasks When the initial action (e.g., placement) is functionally compatible within the agent's capabilities, a directed edge is added between the two nodes. This mechanism can not only generalize to complex task combinations beyond the training scope, but also be applied to new tasks that do not require expert demonstration.
[0039] S20: Obtain natural language instructions τ .
[0040] The natural language instructions described in step S20 can be acquired through various means, including but not limited to the following exemplary methods: receiving user voice input via a speech recognition module and converting it into text; receiving directly input text instructions via a graphical user interface, command-line interface, or application programming interface; reading stored instruction text from a preset instruction library, task file, or database; or receiving structured task description information from other upstream systems or network services and parsing or converting it into a natural language text sequence. Regardless of the specific channel or form through which the instruction is acquired, it is ultimately processed into a natural language text sequence that the intelligent agent's planning system can understand and process, serving as input for high-level task planning. It should be noted that the above examples of acquisition methods are merely illustrations, and this application does not limit the specific source and transmission method of the natural language instructions.
[0041] S30: Based on task graph G With natural language instructions An initial sequence of subtasks is generated using a large language model.
[0042] In this embodiment, based on a given natural language instruction The goal of advanced planning is to generate a sequence of subtasks. This guides the agent to complete the tasks specified in the instructions. The sub-task sequence should be within the diagram. This constitutes a feasible path And each subtask All objects in the context should belong to the collection of context objects. .
[0043] In one exemplary implementation, to mitigate the planning illusion problem in task planning based on large language models, this invention proposes a novel cueing strategy that incorporates robotics domain knowledge into the task graph. This strategy guides the agent to strictly follow the graph's topology to generate a sequence of high-level subtasks, selecting the next subtask only from the neighboring nodes of the previous node. Specifically, nodes in the task graph are encoded as action-object pairs and assigned unique labels (e.g., `[A]PickupObject(object)`), while edges are converted into symbolic connections (e.g., `A→B`). Furthermore, static cueing components, such as agent roles, planning rules, and expected output formats, are introduced to facilitate contextual learning.
[0044] Existing research has shown that combining rule-based rewards with reinforcement learning can significantly enhance reasoning capabilities. Task planning often involves multiple feasible paths, and it is difficult to evaluate the quality of the generated plan based solely on expert trajectories. However, task graphs encapsulate multiple feasible paths, making them suitable for multi-solution task planning problems. Therefore, in an exemplary implementation, a task graph-based reward function is designed and integrated into the Group Relative Policy Optimization (GRPO) reinforcement learning framework to enhance the long-range reasoning capabilities of LLMs. Simultaneously, format and instruction compliance rewards are integrated to guide LLMs in generating high-quality plans that conform to physical constraints and accurately align with instruction intent. The reward functions are described in detail below: Format rewards Its core function is to constrain the standardization of the LLM output structure, ensuring that the planning sequence can be accurately parsed by downstream execution modules. Its definition rule is: if the output simultaneously satisfies the XML format (using `...`) <plan> ...< / plan> If the package contains a JSON format (including the key name "high_level", corresponding to an array of subtask sequences), then... When only the XML format is satisfied but the JSON structure is invalid, If neither of these conditions is met, .
[0045] Node-level rewards To ensure the legitimacy of objects in each subtask of the plan and achieve physical grounding to avoid "illusory objects," its core logic is based on the four types of object metaclasses defined in the task graph. Indicates non-container items. Indicates a fixed container. Indicates a movable container. The validity weight of an object (representing items required for a specific task) is dynamically calculated using the following formula: in For dynamic weights of metaclasses, unused metaclass weights are set to 0, and used metaclass weights are normalized and summed to 1. For metaclass, The number of metaclasses, For objects of a certain metaclass to be efficient, that is: in For the set of objects in the plan, This refers to the actual collection of objects corresponding to this metaclass in the environment. This is an indicator function that returns 1 if the condition is met, and 0 otherwise.
[0046] Border-level rewards Its purpose is to ensure that subtask transitions conform to the constraints of the task graph, avoid "illegal action sequences," and enhance the physical feasibility of the plan. Its calculation formula is: in For adjacent subtasks, For the set of nodes in the task graph, Let be the set of edges in the task graph. Index for subtasks Let be the number of subtasks; if all adjacent subtasks are nodes in the task graph and the transitions satisfy the edge constraints, then Otherwise, it is 0.
[0047] Instructions followed are rewarded To address the instruction alignment problem in scenarios with multiple solutions and avoid planning that "meets constraints but deviates from the intent," a two-stage design is adopted: first, a baseline plan is extracted. Key subtasks in That is, the core steps necessary to complete the instruction, if a plan is generated. 0.5 points are awarded for covering all key sub-tasks; 1.0 point is awarded for a perfect match with the baseline plan; otherwise, 0 points are awarded.
[0048] The total reward function integrates the above four types of components to form the final optimization objective, namely... .
[0049] The above design possesses four core innovations and technological advantages: First, the task graph enables multi-solution compatibility, encapsulating all legal subtasks and transition paths, allowing the reward function to support multiple valid planning solutions; second, dynamic weights and physical grounding, the dynamic weight design of node-level rewards can adapt to the object requirements of different tasks, and is combined with environmental object set verification; third, bidirectional constraints prevent reward hacking, edge-level rewards provide structural constraints, and instruction-following rewards provide semantic constraints, the combination of the two avoids the model generating invalid plans that only satisfy constraints but deviate from instructions; fourth, enhanced generalization ability, through the universal and functionally compatible characteristics of the task graph, the reward function can adapt to unseen long-term tasks without relying on a large amount of labeled data.
[0050] S40: Verify and iteratively optimize the subtask sequence to obtain an executable planning sequence.
[0051] In one exemplary implementation, the task graph is treated as an external validation tool, and validation feedback is used to improve the response of the large language model. The validation function consists of two parts: First, to determine whether each object in the initial subtask sequence is valid, node-level validity validation is performed: .
[0052] Secondly, to verify whether all action transitions satisfy the edge constraints of the task graph, the definition of edge-level validity check is similar to that of edge-level reward. : .
[0053] Validation passed means: node-level validity verification function The calculation result is 1, and the edge-level legality verification function The calculation result is 1; the verification failed because the node-level legality verification function... The calculation result is 0, or the edge-level legality verification function The calculation result is 0.
[0054] For any invalid subtask or transformation, the corresponding error message is fed back to the large language model to guide its correction. This verification → feedback → correction loop iterates until the plan passes verification or the maximum number of iterations is reached; in experiments, a maximum of 3 iterations are typically set.
[0055] In one exemplary embodiment, the high-level task planning method provided in this application can be achieved through... Figure 2 The process shown is implemented in detail. This process is illustrated based on a specific natural language instruction example, which is: "Cut the potato, put the knife beside the bread on the black table and place a clean slice in the fridge."
[0056] The execution process of the method is as follows: First, the natural language instruction is received. Simultaneously, the system acquires or invokes a predefined task graph and generates structured graph hints based on this graph. The graph hints include at least the encoded representations of nodes and edges in the task graph. Subsequently, the natural language instruction, the graph hints, and a graph reward function constructed based on the same task graph are input into the Large Language Model (LLM). The graph reward function comprises node-level rewards and edge-level rewards, and its design and calculation method are as described above. The system uses the Group RelativePolicy Optimization framework (GPPO) to train or guide inference for the Large Language Model. Based on the above input, the Large Language Model generates a preliminary sub-task sequence. Next, the system performs graph verification on the sub-task sequence. The graph verification also includes node-level verification and edge-level verification, used to check the legality of objects referenced by each sub-task in the sequence and whether the transitions between adjacent sub-tasks conform to the constraints of the task graph. If the verification results show that the subtask sequence fully satisfies all node-level and edge-level constraints, the verification is deemed valid, and the sequence is output as the final high-level planning result. If the verification results show that the subtask sequence does not fully satisfy the above constraints, the verification is deemed invalid. In this case, the system will generate specific verification feedback information based on the verification results, which clearly indicates the type and location of the constraint violation (e.g., invalid object or illegal transfer). This verification feedback information is fed back to the large language model as input for the next round of planning generation to correct previous errors. This "generation-verification-feedback" loop will continue until a verified subtask sequence is output, or the preset maximum number of iterations is reached.
[0057] This example demonstrates the complete workflow and internal loop optimization mechanism of the method through a specific instruction.
[0058] This application's embodiments were experimented on ALFRE, a challenging robot instruction following benchmark. The primary evaluation metric is Task Success Rate (SR), which measures the percentage of tasks completed completely. Additionally, Goal Condition Success Rate (GC) is used to evaluate the percentage of goal conditions met. This application is compared with leading work in the test set reported on the ALFRED public leaderboard. Table 1 reports two baseline scenarios: using only goal instructions; and using both goal instructions and step-by-step instructions. As shown in Table 1, the method of this application significantly outperforms previous results on unseen tasks, exceeding them by 6.17%, while achieving state-of-the-art results on all metrics in both unseen and seen environments (67.71% and 68.52%, respectively), demonstrating the effectiveness of the method. Furthermore, the method of this application surpasses all previous methods regardless of whether step-by-step instructions are used or only goal instructions are used, highlighting the superiority of the method. It is worth emphasizing that the method of this application does not rely on metadata for contextual grounding, indicating stronger generalization ability in real-world scenarios.
[0059] Table 1: Comparison results with state-of-the-art methods on the ALFRED test set. Baseline results were obtained from the official ALFRED leaderboard or papers. The "Step-by-Step Instructions" and "LLMs" columns indicate whether step-by-step instructions and large language models were used in high-level planning, respectively.
[0060] To evaluate the effectiveness of the proposed method in large-scale graph augmentation planning, it was compared with several robust baseline large-scale models, including API-based closed-source and open-source models under different inference settings. Since ALFRED includes a limited and relatively simple range of task types, this method also constructs a new long-term dataset with 1396 samples to better evaluate the capabilities of graph augmentation planning on unseen long-term tasks. This dataset adds 17 more complex long-term task types to the original seven short-term task types. For fair comparison, three representative closed-source models (GPT-4o, Deepseek-r1, and Gemini-2.5-Pro) were considered to reflect state-of-the-art performance in general inference and language understanding. To improve the adaptability of the general agent to planning tasks, common inference strategies (zero-shot, thought chain, few-shot, and retrieval-augmented generation) were applied to the agent. Furthermore, using the same backbone network Qwen2.5-Instruct-7B as the proposed method, comparisons were made under different inference methods and supervised fine-tuning to evaluate the specific contributions of the proposed method. The method described in this application outperforms a range of leading API-based LLMs, particularly on complex, long-cycle tasks, achieving a success rate of 90.04%, significantly exceeding the best baseline (44.91%).
[0061] Table 2: Success rates of various methods in high-level planning on validation sets and long-term tasks. For simplicity, the abbreviations Zs and Fs are used to represent zero samples and few samples, respectively.
[0062] This application also provides a high-level task planning device for intelligent agents based on task graph enhancement, used to implement the methods described in any of the above embodiments, such as... Figure 3 As shown, the high-level task planning device for intelligent agents based on task graph enhancement includes: Task graph acquisition module 301 is configured to acquire a task graph, wherein the task graph is represented as follows: ,in Nodes represent various types of subtasks. Represents the set of edges, each edge This indicates that it is allowed from subtasks. Switch to subtask ; The instruction acquisition module 302 is configured to acquire natural language instructions. ; Path generation module 303 is configured to be based on the task graph With the natural language instructions An initial sequence of subtasks is generated using a large language model, where each subtask in the sequence corresponds to a node in the task graph, and the subtask sequence is located within the task graph. G This forms a path , , and These represent the first, second, and Tth subtasks, respectively. The iterative optimization module 304 is configured to verify and iteratively optimize the subtask sequence to obtain an executable planning sequence.
[0063] This application provides an electronic device. The electronic device may include a processor and a memory, wherein the processor and the memory can communicate; exemplarily, the processor and the memory communicate via a communication bus.
[0064] The processor executes computer execution instructions stored in memory, causing the processor to perform the scheme in the above embodiments. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0065] The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.
[0066] The electronic device provided in this application embodiment can be the terminal device described in the above embodiments.
[0067] This application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on a computer, the computer performs the technical solution of the high-level task planning method for intelligent agents based on task graph enhancement described in the above embodiments.
[0068] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, it can implement the technical solution of the high-level task planning method for intelligent agents based on task graph enhancement in the above embodiments.
[0069] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.
[0070] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.
[0071] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit composed of the above modules can be implemented in hardware or in the form of hardware plus software functional units.
[0072] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.
[0073] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.
[0074] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.
[0075] Buses can be Industry Standard Architecture (ISA) buses, Peripheral Component Interconnect (PCI) buses, or Extended Industry Standard Architecture (EISA) buses, etc. Buses can be categorized into address buses, data buses, control buses, etc.
[0076] The aforementioned storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.
[0077] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic control unit or main control device.
[0078] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.
[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.
Claims
1. A high-level task planning method for intelligent agents based on task graph enhancement, characterized in that, The method includes: Obtain the task graph, which is represented as follows: ,in Nodes represent various types of subtasks. Represents the set of edges, each edge This indicates that it is allowed from subtasks. Transform execution to subtask ; Obtaining natural language instructions τ ; Based on the task graph With the natural language instructions An initial sequence of subtasks is generated using a large language model, where each subtask in the sequence corresponds to a node in the task graph, and the subtask sequence is located within the task graph. G This forms a path , , and These represent the first, second, and Tth subtasks, respectively. The subtask sequence is verified and iteratively optimized to obtain an executable planning sequence.
2. The high-level task planning method for intelligent agents based on task graph enhancement according to claim 1, characterized in that, The subtask Represented as or ,in Indicates advanced actions, Representing related objects; using metaclasses in task graphs. To replace the specific object name ,in Indicates non-container items. Indicates a fixed container. Indicates a movable container. Represents items required for a specific subtask; during the planning and generation process, the metaclass is mapped to a set of environment objects. The specific object name.
3. The high-level task planning method for intelligent agents based on task graph enhancement according to claim 1, characterized in that, The method for constructing the edges of the task graph includes: if the previous subtask The final low-level action and subsequent sub-tasks If the initial action is functionally compatible within the capabilities of the agent, then at the node... and Add directed edges between .
4. The high-level task planning method for intelligent agents based on task graph enhancement according to claim 1, characterized in that, The large language model is guided to generate the initial subtask sequence through a prompting strategy, which includes: encoding the nodes of the task graph as action-target pairs with unique labels, encoding the edges as symbolic connections, and integrating the encoded node and edge information, agent roles, planning rules, and expected output format into prompt words to guide the model to strictly follow the topological structure of the task graph and select the next subtask only from the neighboring nodes of the previous subtask node each time.
5. The high-level task planning method for intelligent agents based on task graph enhancement according to claim 1, characterized in that, The large language model is a model pre-trained through reinforcement learning; the reinforcement learning training process includes: based on the task graph G Constructing a reward function To optimize model parameters so that the model generates a graph that conforms to the task. G The ability of the constrained sub-task sequence; the reward function Including format rewards Node-level rewards Border-level rewards and instructions follow the reward ,and .
6. The high-level task planning method for intelligent agents based on task graph enhancement according to claim 5, characterized in that, The format reward The determination method includes: if the output structure of the large language model satisfies the preset first and second formats, then let the format reward... If the output structure of the large language model only satisfies one of the preset first and second formats, then the format reward is set to the first value. If the output structure of the large language model does not satisfy either the preset first or second format, then the format reward is set to the second value. It equals the third value, where the first value is greater than the second value, and the second value is greater than the third value; The node-level reward The calculation formula is: in, For meta-class dynamic weights, For metaclass, The number of metaclasses, The efficiency of an object of a certain metaclass is calculated as follows: in, For indexing objects in the plan, For the set of objects in the plan, Metaclasses in the environment The corresponding actual collection of objects, For indicator functions; The side-level reward The calculation formula is: in, For adjacent subtasks, For the set of nodes in the task graph, Let be the set of edges in the task graph. Index for subtasks Number of subtasks; The instruction follows the reward The determination methods include: extracting key sub-tasks from the baseline plan; if the generated plan covers all key sub-tasks, then setting a reward for compliance. The first score is equal to the first score; if the generated plan perfectly matches the baseline plan, then the follow-through reward is given. The second score is equal to the second score; if the generated plan does not perfectly match the baseline plan and does not cover all key sub-tasks, then the follow-through reward is set. It equals the third score, where the second score is greater than the first score, and the first score is greater than the third score.
7. The high-level task planning method for intelligent agents based on task graph enhancement according to claim 6, characterized in that, The subtask sequence is validated and iteratively optimized to obtain an executable planning sequence, including: Perform node-level and edge-level validity verification on the initial subtask sequence; The node-level validity verification function for: The verification function for edge-level validity verification for: If the verification fails, the corresponding error message is fed back to the large language model to guide it in correcting the subtask sequence and iterating until the verification is passed or the preset maximum number of iterations is reached.
8. A high-level task planning device for intelligent agents based on task graph enhancement, characterized in that, The device includes: The task graph acquisition module is configured to acquire a task graph, which is represented as follows: ,in Nodes represent various types of subtasks. Represents the set of edges, each edge This indicates that it is allowed from subtasks. Transform execution to subtask ; The instruction acquisition module is configured to acquire natural language instructions. ; The path generation module is configured to be based on the task graph. With the natural language instructions An initial sequence of subtasks is generated using a large language model, where each subtask in the sequence corresponds to a node in the task graph, and the subtask sequence is located within the task graph. G This forms a path , , and These represent the first, second, and Tth subtasks, respectively. The iterative optimization module is configured to verify and iteratively optimize the subtask sequence to obtain an executable planning sequence.
9. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes the computer execution instructions stored in the memory to implement the agent high-level task planning method based on task graph enhancement as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the high-level agent task planning method based on task graph enhancement as described in any one of claims 1-7.