Robot task planning enhancement method and system based on large language model

Abstract

The invention discloses a robot task planning enhancement method and system based on a large language model, and the method comprises the steps: constructing an action tree which comprises a plurality of candidate action sequences, and each candidate action sequence comprises a plurality of action functions; obtaining a long sequence task, inputting the long sequence task into the first large language model for task planning, and outputting a plurality of subtasks; respectively inputting each subtask into a second large language model for task scheduling, and obtaining an action function for driving the robot and a first execution result corresponding to the action function according to the action tree; when the first execution result is execution failure, a local solution and a global solution are obtained through the first large language model and the second large language model, and then the robot is driven to complete corresponding subtasks according to the local solution and / or the global solution. The task execution efficiency of the robot in a complex environment can be improved, and the method can be widely applied to the technical field of task planning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of task planning, in particular to a robot task planning enhancement method and system based on a large language model. BACKGROUND

[0002] When deploying a large language model on a mobile robot platform, there are many technical challenges. First, the large language model has high requirements for computing resources and video memory during the inference stage, resulting in slow inference speed and significant increase in running cost. Existing solutions usually need to trade off between efficiency and cost to meet the actual application requirements. Second, although the large language model can align with the robot system to some extent, it can parse human instructions into executable tool functions and call related functions to complete tasks, but it has obvious limitations in handling long sequence tasks. Specifically, the large language model has difficulty in simultaneously considering high-level abstract task planning, low-level specific function calling, and integration of real-time feedback in dynamic working environments, which poses a serious challenge to the efficient execution of complex tasks. SUMMARY

[0003] The main purpose of the embodiments of the present application is to provide a robot task planning enhancement method and system based on a large language model, which can improve the task execution efficiency of robots in complex environments.

[0004] To achieve the above-mentioned purpose, one aspect of the embodiments of the present application provides a robot task planning enhancement method based on a large language model, comprising the following steps: constructing an action tree, the action tree comprising a plurality of candidate action sequences, each of the candidate action sequences comprising a plurality of action functions; obtaining a long sequence task, inputting the long sequence task into a first large language model for task planning, and outputting a plurality of subtasks; inputting each of the subtasks into a second large language model for task scheduling, and obtaining an action function for driving a robot and a first execution result corresponding to the action function according to the action tree; when the first execution result is execution failure, obtaining a local solution and a global solution through the first large language model and the second large language model, and then driving the robot to complete the corresponding subtask according to the local solution and / or the global solution.

[0005] In some embodiments, the constructing an action tree specifically comprises: obtaining a plurality of target task samples, inputting each of the target task samples into the first large language model, and outputting a plurality of candidate action sequences; pruning and optimizing each of the candidate action sequences, constructing a first action tree based on the candidate action sequences after pruning and optimizing; performing weight distribution on the first action tree to obtain the constructed action tree.

[0006] In some embodiments, the first action tree includes a plurality of nodes and a plurality of edges, one node corresponding to one action function, and one edge corresponding to a transition relationship between nodes, and the performing weight distribution on the first action tree to obtain the constructed action tree specifically includes: driving the robot according to the action function in the first action tree to obtain an execution result sample; performing weight distribution on all edges of the first action tree according to the execution result sample to obtain the action tree.

[0007] In some embodiments, the inputting each of the sub-tasks into a second large language model for task scheduling, and obtaining an action function for driving the robot and a first execution result corresponding to the action function according to the action tree, specifically includes: inputting the sub-tasks into the second large language model for task scheduling, and outputting the action function corresponding to the sub-tasks; driving the robot according to the action function, and recording the first execution result corresponding to the action function; when the first execution result is execution success, inputting a next sub-task in the action tree into the second large language model for task scheduling, outputting the action function corresponding to the sub-task, and returning to the step of driving the robot according to the action function until task scheduling of all sub-tasks is completed.

[0008] In some embodiments, when the first execution result is execution failure, the first large language model and the second large language model are used to obtain a local solution and a global solution: when the first execution result is execution failure, performing local analysis on the first execution result through the second large language model to obtain the local solution; sending the first execution result to the first large language model through the second large language model; performing global analysis on the first execution result through the first large language model to obtain the global solution.

[0009] In some embodiments, the driving the robot to complete the corresponding sub-task according to the local solution and / or the global solution specifically includes: According to the local solution, the robot is driven to perform the corresponding subtask, and a second execution result corresponding to the local solution is obtained; When the second execution result is an execution failure, the action tree is structurally adjusted according to the global solution, and then the robot is driven to complete the corresponding subtask based on the action tree after the structural adjustment.

[0010] In some embodiments, the structural adjustment of the action tree according to the global solution specifically includes: The effectiveness of the node in the action tree that fails to execute is evaluated, and then the node is marked according to the effectiveness evaluation result, and a node marking result is obtained; According to the global solution and the long sequence task, a plurality of new subtasks are generated; According to the node marking result and the new subtasks, the action tree is structurally adjusted.

[0011] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application proposes a robot task planning enhancement system based on a large language model, comprising: An action tree construction module is configured to construct an action tree, wherein the action tree comprises a plurality of candidate action sequences, and each candidate action sequence comprises a plurality of action functions; A task planning module is configured to obtain a long sequence task, input the long sequence task into a first large language model for task planning, and output a plurality of subtasks; A subtask execution module is configured to input each subtask into a second large language model for task scheduling, and obtain an action function for driving a robot and a first execution result corresponding to the action function according to the action tree; An error processing module is configured to, when the first execution result is an execution failure, obtain a local solution and a global solution through the first large language model and the second large language model, and then drive the robot to complete the corresponding subtask according to the local solution and / or the global solution.

[0012] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application proposes an electronic device, which comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the method as described above when executing the computer program.

[0013] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application proposes a computer program product, which comprises a computer program, and the computer program implements the method as described above when executed by a processor.

[0014] The embodiments of the present application at least have the following beneficial effects: the robot task planning enhancement method and system based on a large language model provided by the present application firstly constructs an action tree, the action tree includes a plurality of candidate action sequences, and each candidate action sequence includes a plurality of action functions; then a long sequence task is obtained, the long sequence task is input into a first large language model for task planning, and a plurality of subtasks are output; then each subtask is input into a second large language model for task scheduling, and an action function for driving the robot and a first execution result corresponding to the action function are obtained according to the action tree; when the first execution result is an execution failure, a local solution and a global solution are obtained through the first large language model and the second large language model, and then the robot is driven to complete the corresponding subtask according to the local solution and / or the global solution. The present application adopts two large language models, the first large language model is used to decompose a complex long sequence task into a plurality of subtasks, and then the second large language model is used to extract the action functions corresponding to the subtasks according to the constructed action tree, so that the task execution efficiency of the robot in a complex environment can be improved; in addition, when the subtask execution fails, the local solution and the global solution are generated through the first large language model and the second large language model, so that the reliability and response speed of the robot can be further improved. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following introduces the drawings needed to be used in the embodiments of the present application. It should be understood that the drawings introduced below are only for facilitating the clear description of part of the embodiments of the technical solutions of the present application, and other drawings can be obtained by those skilled in the art without paying creative labor on the premise of the drawings.

[0016] Figure 1 The step flow chart of the robot task planning enhancement method based on a large language model provided by an embodiment of the present application is shown in the following figure: Figure 2 The flowchart of the candidate action sequence generation provided by an embodiment of the present application is shown in the following figure: Figure 3 The schematic diagram of the action tree with weights provided by an embodiment of the present application is shown in the following figure: Figure 4 The flowchart of the hierarchical cooperation provided by an embodiment of the present application is shown in the following figure: Figure 5 The structural schematic diagram of the robot task planning enhancement system based on a large language model provided by an embodiment of the present application is shown in the following figure: Figure 6 The hardware structural schematic diagram of the electronic device provided by an embodiment of the present application is shown in the following figure. DETAILED DESCRIPTION

[0017] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application. When the following description refers to the accompanying drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to limit the present application.

[0019] A large language model (LLM) is a deep learning-based neural network decoder architecture whose core mechanism is to learn the patterns and structures of natural language by training on vast amounts of text data, thereby possessing the ability to generate coherent natural language text or understand semantics. After receiving user input, the large language model can predict the next possible phrase or word according to the learned language distribution patterns, thereby realizing continuous text generation.

[0020] Most mainstream large language models are currently based on Meta's LLaMA model, which consists of multiple stacked decoder modules. Each decoder module sequentially performs self-attention computation, feature compression and expansion, and residual connection operations. When the parameter size and training data volume of the model reach a certain level, the capabilities of large language models will "emerge", showing significant performance improvements on tasks such as text generation, text translation, and question-answering systems, and even surpassing human level in some scenarios. As a result, by introducing appropriate agent datasets, large language models can further learn to call various functions and complete complex tasks with the help of various tool functions. This expansion of capabilities provides important support for the potential of large language models in practical application scenarios such as mobile robots, as large language models can think like the human brain and call appropriate functions to drive robot movement and complete corresponding tasks. However, there are still many technical challenges when deploying large language models on mobile robot platforms. First, large language models have high requirements for computing resources and memory during inference, resulting in slower inference speed and significantly increased running costs. Existing solutions usually need to trade off between efficiency and cost to meet the actual application requirements. Second, although large language models can align with robot systems to some extent, they can parse human instructions into executable tool functions and call relevant functions to complete tasks, but they have obvious limitations when handling long sequence tasks. Specifically, large language models have difficulty in simultaneously considering high-level abstract task planning, low-level specific function calling, and real-time feedback integration in dynamic working environments, which poses a serious challenge to the efficient execution of complex tasks.

[0021] Robot task planning refers to the process of generating a series of motion instructions and function calls to drive the robot to complete a specific task. This process can be formalized as a partially observable Markov decision process (POMDP) and described by the tuple (S, A, T, O, R, γ, h).

[0022] where S represents the current state of the robot and its environment; A represents the set of executable actions for the robot; T is the state transition function, which describes the change in state after executing an action in a certain state; O is the observation function, which provides information related to the current state; R is the reward function, which represents the immediate reward obtained by executing action A in state S; γ is the discount factor, which is usually used in combination with the reward function R to weigh the importance of current rewards and future rewards; h represents historical information, including all past observation and action sequences, i.e., ht=(o0,a0,o1,a1...).

[0023] Assuming that the final goal of the task is g, the core of the task planning is to find an optimal strategy so that the cumulative return of the strategy under the given goal is maximized. This optimization process not only needs to consider the immediate reward of the current state and action, but also needs to combine historical information and possible state transitions in the future to achieve the globally optimal task execution effect.

[0024] In the robot-based task planning process, the state space S can be defined by the current pose P of the robot and the angles θ of each joint of the robot arm. The action space A is composed of a set of motion primitives executable by the robot, such as basic operations such as grasping, placing, and moving, and can also be extended to include some auxiliary functions such as object recognition, obstacle detection, and dialogue functions for human interaction. The observation space O represents the perception range of the robot to the environment state, which usually includes the spatial position, geometric properties and other related feature information of the objects in the environment.

[0025] Therefore, an embodiment of the present application proposes a robot task planning enhancement method based on a large language model. First, an action tree is constructed, which includes a plurality of candidate action sequences, each of which includes a plurality of action functions. Then, a long sequence task is obtained, which is input into a first large language model for task planning, outputting a plurality of sub-tasks. Then, each sub-task is input into a second large language model for task scheduling, and the action function for driving the robot and the first execution result corresponding to the action function are obtained according to the action tree. When the first execution result is execution failure, a local solution and a global solution are obtained through the first large language model and the second large language model, and then the robot is driven to complete the corresponding sub-task according to the local solution and / or the global solution. The present application uses two large language models, which can improve the task execution efficiency of the robot in a complex environment by decomposing the complex long sequence task into multiple sub-tasks through the first large language model, and then extracting the action functions corresponding to these sub-tasks through the second large language model according to the constructed action tree. In addition, when the sub-task execution fails, the local solution and the global solution are generated through the first large language model and the second large language model, which can further improve the reliability and response speed of the robot.

[0026] Reference Figure 1 , Figure 1 The steps of the robot task planning enhancement method based on a large language model provided by an embodiment of the present application are shown in the flowchart. An embodiment of the present application proposes a robot task planning enhancement method based on a large language model, which can include but is not limited to the following steps S101 to S104: Step S101, constructing an action tree, the action tree including a plurality of candidate action sequences, each candidate action sequence including a plurality of action functions; Specifically, by generating an action tree T(c)=(V,E) through a Large Language Model (LLM), the robot will select the corresponding action node in the action tree according to the current state and call the corresponding action function to execute it, thereby realizing the gradual transition of the state.

[0027] As an optional implementation, step S101 can be further divided into the following steps S1011 to S1013: Step S1011: Obtain multiple target task samples, input each target task sample into the first large language model, and output several candidate action sequences; In some alternative embodiments, such as Figure 2 The diagram illustrates the process of generating candidate action sequences. In the initial stage of task planning, candidate action sequences are generated through a sampling mechanism. Specifically, key environmental information and the robot's own state S are first used as input prompts for the large language model. This key information includes, but is not limited to, the robot's current environmental state, available resources, and task objectives. Subsequently, a clear target task sample is provided to the large language model, which generates N possible candidate action sequences C={ci,i∈[0,i)}, where each candidate action sequence ci represents a potential solution to complete the task. These candidate action sequences consist of a series of basic action functions, such as grasping, placing, or moving operations, and also include some auxiliary function functions, such as object recognition or obstacle detection. These action functions have been ensured to be implemented and run on the robot system.

[0028] Step S1012: Prune and optimize each candidate action sequence, and construct the first action tree based on the pruned and optimized candidate action sequences; Specifically, to improve planning efficiency and reduce redundancy, this embodiment considers the possibility of duplicate action functions in different action sequences. Therefore, by pruning and optimizing the generated candidate action sequences, a compact action tree structure (i.e., the first action tree) is constructed. In this first action tree, identical action functions are merged into a single node, while different action functions are expanded as child nodes of that node. Formally, the first action tree can be represented as T(c)=(V,E), where V and E represent the set of nodes and the set of edges of the first action tree, respectively. Node V corresponds to a specific action function, while edge E represents the transition relationship between states, that is, the execution path from one action function to another. The robot's sequential execution of action functions on the first action tree is essentially a traversal of a multi-branch tree.

[0029] It is understandable that by pruning the generated candidate action sequences, this embodiment of the application can not only effectively reduce redundant calculations in task planning, but also clearly capture the logical relationships of task execution in the structured representation of the action tree, thereby providing a more efficient solution for subsequent path optimization and task scheduling. Furthermore, this task planning method based on a large language model has strong versatility and scalability, and can adapt to the robot task requirements in various complex scenarios.

[0030] Step S1013: Perform weight allocation on the first action tree to obtain the constructed action tree.

[0031] It should be noted that, in order to obtain the optimal execution order and reduce backtracking during action tree traversal, different weights need to be assigned to different edges to reduce the possibility of executing low-quality action sequences during task planning.

[0032] As an optional implementation, the first action tree includes several nodes and several edges, where each node corresponds to an action function and each edge corresponds to a transition relationship between nodes. Step S1013 can be further divided into the following steps S10131 and S10132: Step S10131: Drive the robot according to the action functions in the first action tree to obtain the execution result sample; Step S10132: Based on the execution result sample, assign weights to all edges of the first action tree to obtain the action tree.

[0033] Specifically, in this embodiment, each edge in the first action tree is assigned a weight. If multiple task sequences pass through a certain edge, it indicates that the edge is more reasonable, and a higher weight is assigned to that edge. When the robot executes the action function, choosing to enter that edge will also yield greater benefits. Conversely, if an edge appears less frequently in action sequence C, it indicates that its quality is lower and it is not conducive to achieving the final goal. After this step, the following is obtained: Figure 3 The weighted action tree T(c) shown can efficiently guide the robot to quickly complete a given task.

[0034] Furthermore, since the robot's interaction with the environment causes dynamic changes in the environmental state, the action tree needs to be adjusted in real time to adapt to these changes. After executing each action function, the robot perceives the environment through sensors or other observation methods to obtain the impact of the action on the environment and an assessment of the quality of execution. For example, the robot needs to determine whether it successfully grasped the target object (such as an apple) or accurately moved to the designated location. This observation data is passed as feedback information to the large language model in the task planning module, which then determines whether the structure of the action tree needs to be adjusted.

[0035] Step S102: Obtain long sequence tasks, input the long sequence tasks into the first large language model for task planning, and output several sub-tasks; It's important to note that existing task planning models based on large language models have significant limitations when handling long-sequence tasks. First, long-sequence tasks typically result in large and complex action tree structures, increasing the difficulty of pruning or dynamically modifying the tree and potentially reducing system stability. Especially when facing environmental changes, frequent adjustments to complex action trees can introduce additional computational overhead and potential error risks. Second, decomposing and analyzing long-sequence tasks consumes substantial time and computational resources. This high resource requirement can become a bottleneck in practical deployments, making it difficult to meet real-time and efficiency requirements.

[0036] To address the aforementioned issues, this application proposes a layered architecture-based large language model working mode. This mode achieves efficient task decomposition and execution by introducing two layers of large language models. First, upon receiving a long sequence of tasks from the user, the system invokes a large-scale first language model A, leveraging its powerful semantic understanding and logical reasoning capabilities to decompose the long sequence of tasks into several independent but logically coherent subtasks. These subtasks have clear goals and boundaries, facilitating subsequent modular processing.

[0037] Step S103: Input each subtask into the second large language model for task scheduling, and obtain the action function for driving the robot and the first execution result corresponding to the action function based on the action tree; Specifically, after decomposing the long sequence task into several subtasks using the first large language model A, the system deploys a second large language model B with a smaller parameter scale, which is specifically responsible for processing these subtasks and recording the first execution results corresponding to these subtasks in real time.

[0038] As an optional implementation, step S103 can be further divided into the following steps S1031 to S1033: Step S1031: Input the subtask into the second language model for task scheduling and output the action function corresponding to the subtask; Step S1032: Drive the robot according to the action function and record the first execution result corresponding to the action function; Step S1033: When the first execution result is successful, input the next subtask in the action tree into the second language model for task scheduling, output the action function corresponding to the subtask, and return to the step of driving the robot according to the action function, until the task scheduling of all subtasks is completed.

[0039] Specifically, the main function of the second language model B is to generate specific action sequences based on the requirements of the current subtask, combined with environmental information and available tool functions, and drive the robot to complete the corresponding operations. After the second language model B completes the current subtask, the system will automatically jump to the next action node in the action tree and extract the corresponding subtask from it for the second language model B to continue executing.

[0040] Step S104: When the first execution result is execution failure, the local solution and the global solution are obtained through the first and second language models, and then the robot is driven to complete the corresponding sub-task based on the local solution and / or the global solution. Specifically, when the action function fails to execute, this embodiment of the application handles the issue through a layered collaboration approach. The second major language model B first performs a local analysis of the specific reasons for the execution failure and attempts to propose a local solution for the current subtask. At the same time, the first major language model A re-evaluates the feasibility of the task planning from a global perspective and proposes a global solution.

[0041] As an optional implementation, when the first execution result is execution failure, the step of obtaining the local solution and the global solution through the first and second large language models can be further divided into the following steps S1041 to S1043: Step S1041: When the first execution result is execution failure, perform local analysis on the first execution result using the second language model to obtain a local solution; Step S1042: Send the first execution result to the first language model through the second language model; Step S1043: Perform a global analysis of the first execution result using the first major language model to obtain a global solution.

[0042] Specifically, during the execution of an action function, various reasons may cause it to fail, such as failing to successfully grab the target object, failing to navigate to the designated location, or the action becoming infeasible due to dynamic changes in the environment. For example... Figure 4 The diagram shows a layered collaboration process. When such a failure occurs, the system will handle it through layered collaboration: The second language model B is first responsible for performing a local analysis of the specific reasons for the execution failure and trying to propose a solution for the current subtask, thus obtaining a local solution; at the same time, the second language model B will also feed back the reasons for the failure to the first language model A, so as to re-evaluate the feasibility of the task planning from a global perspective and obtain a global solution.

[0043] The responsibilities of the second major language model, B, include, but are not limited to: attempting to change the calling interface of utility functions, adjust parameter configurations, or select alternative functional functions based on failure scenarios, in order to complete the objective within the current subtask framework as much as possible. Through these local solutions, the system can quickly respond to execution failures and improve its real-time responsiveness.

[0044] As an optional implementation, the step of driving the robot to complete the corresponding sub-task based on the local solution and / or global solution can be further divided into the following steps S1044 to S1045: Step S1044: Based on the local solution, drive the robot to execute the corresponding sub-task to obtain the second execution result corresponding to the local solution; Step S1045: When the second execution result is execution failure, the action tree is structurally adjusted according to the global solution, and then the robot is driven to complete the corresponding sub-task based on the structurally adjusted action tree.

[0045] It should be noted that the robot prioritizes executing the local solution provided by the second largest language model B, while the first largest language model A runs in the background and generates the global solution. Since the computation time of the first largest language model A may be relatively long, its operation overlaps with the robot's motion control in time, thus effectively hiding the impact of the high latency of the first largest language model A on the system response speed.

[0046] Ultimately, the system decides which solution to adopt based on the outputs of the two models: if the local solution of the second largest language model B successfully resolves the problem of the current function's execution failure, the global solution of the first largest language model A will be discarded; conversely, if the local solution of the second largest language model B fails, the system will adopt the global solution provided by the first largest language model A. This hierarchical collaboration mechanism not only improves the system's fault tolerance but also significantly enhances the flexibility and adaptability of task planning.

[0047] As an optional implementation, the step of structurally adjusting the action tree according to the global solution can be further divided into the following steps S10451 to S10453.

[0048] Step S10451: Evaluate the effectiveness of nodes that fail to execute in the action tree, and then mark the nodes according to the effectiveness evaluation results to obtain the node marking results; Step S10452: Based on the global solution and the long sequence of tasks, generate several new sub-tasks; Step S10453: Adjust the structure of the action tree based on the node marking results and the new sub-tasks.

[0049] Specifically, after receiving feedback of execution failure, the first major language model A will conduct an in-depth analysis of the reasons for the failure from the perspective of global task planning, and take the following measures to optimize the action tree structure: Node validity assessment: First, assess the validity of the currently failed node and its child nodes. If it is confirmed that the node can no longer be executed, mark it as an invalid node and backtrack to the previous feasible state.

[0050] Subtask sequence update: Based on environmental changes and reasons for failure, a new subtask sequence is regenerated to ensure that task planning can adapt to the needs of dynamic environments.

[0051] Action tree structure optimization: When necessary, optimize the overall structure of the action tree, such as merging redundant nodes, adjusting branch priorities, or introducing new path selection strategies, thereby improving the robustness and efficiency of planning.

[0052] The above describes the robot task planning enhancement method based on a large language model according to embodiments of this application. It can be recognized that, compared with existing robot task planning methods that solely use a large language model, embodiments of this application have the following advantages: First, when processing long sequence tasks, the complex long sequence task is decomposed into multiple sub-tasks by the first large language model, and the action functions corresponding to these sub-tasks are extracted sequentially by the second large language model according to the constructed action tree. This avoids excessive task decomposition and frequent function calls, thereby significantly reducing the running time of the large language model and improving the overall response speed of the robot.

[0053] Second, by decomposing complex tasks into multiple subtasks and completing these subtasks sequentially, this method demonstrates stronger robustness and reliability compared to directly decomposing the task sequence all at once, and can effectively cope with the uncertainty and complexity in task execution.

[0054] Third, when a task fails, the second largest language model B, with fewer parameters, performs the initial analysis. Because of its smaller computational scale, the second largest language model B can quickly generate a corresponding solution. Simultaneously, the second largest language model B will determine whether the problem needs to be fed back to the first largest language model A to adjust the action tree structure. During this process, the robot's motion control and the computation of the large language model can partially overlap in time, thus mitigating the robot's response delay caused by the long execution time of the large language model.

[0055] Reference Figure 5 This application also provides a robot task planning enhancement system based on a large language model, including: The action tree construction module is used to construct an action tree, which includes several candidate action sequences, and each candidate action sequence includes several action functions. The task planning module is used to obtain long sequence tasks, input the long sequence tasks into the first large language model for task planning, and output several sub-tasks. The subtask execution module is used to input each subtask into the second large language model for task scheduling, and obtain the action function for driving the robot and the first execution result corresponding to the action function based on the action tree; The error handling module is used to obtain local and global solutions through the first and second major language models when the first execution result is execution failure, and then drive the robot to complete the corresponding sub-task based on the local and / or global solutions.

[0056] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0057] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0058] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0059] Please see Figure 6 , Figure 6 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 1001 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1002 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1002 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002 and is called and executed by the processor 1001 using the methods described in the embodiments of this application. Input / output interface 1003 is used to implement information input and output; The communication interface 1004 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 1005 transmits information between various components of the device (e.g., processor 1001, memory 1002, input / output interface 1003, and communication interface 1004); The processor 1001, memory 1002, input / output interface 1003 and communication interface 1004 are connected to each other within the device via bus 1005.

[0060] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0061] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0062] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0063] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0064] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0065] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0066] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0067] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0068] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0069] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0070] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0071] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0072] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0073] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0074] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0075] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A robot task planning enhancement method based on a large language model, characterized in that, Includes the following steps: Construct an action tree, which includes several candidate action sequences, each of which includes several action functions; Obtain a long sequence of tasks, input the long sequence of tasks into the first large language model for task planning, and output several sub-tasks; Each of the subtasks is input into the second large language model for task scheduling, and the action function for driving the robot and the first execution result corresponding to the action function are obtained according to the action tree. When the first execution result is execution failure, a local solution and a global solution are obtained through the first large language model and the second large language model, and then the robot is driven to complete the corresponding sub-task based on the local solution and / or the global solution.

2. The method according to claim 1, characterized in that, The construction of the action tree specifically includes: Multiple target task samples are obtained, and each target task sample is input into the first large language model to output several candidate action sequences. Each candidate action sequence is pruned and optimized, and a first action tree is constructed based on the pruned and optimized candidate action sequences; The first action tree is weighted to obtain the constructed action tree.

3. The method according to claim 2, characterized in that, The first action tree includes several nodes and several edges. Each node corresponds to an action function, and each edge corresponds to a transition relationship between the nodes. The step of weighting the first action tree to obtain the constructed action tree specifically includes: The robot is driven according to the action function in the first action tree to obtain an execution result sample; Based on the execution result sample, weights are assigned to all edges of the first action tree to obtain the action tree.

4. The method according to claim 1, characterized in that, The step of inputting each of the subtasks into the second large language model for task scheduling, and obtaining the action function for driving the robot and the first execution result corresponding to the action function based on the action tree, specifically includes: The subtask is input into the second large language model for task scheduling, and the action function corresponding to the subtask is output. The robot is driven according to the action function, and the first execution result corresponding to the action function is recorded; When the first execution result is successful, the next subtask in the action tree is input into the second language model for task scheduling, the action function corresponding to the subtask is output, and the step of driving the robot according to the action function is returned until the task scheduling of all subtasks is completed.

5. The method according to claim 1, characterized in that, When the first execution result is execution failure, a local solution and a global solution are obtained through the first large language model and the second large language model: When the first execution result is execution failure, the second large language model is used to perform local analysis on the first execution result to obtain the local solution; The first execution result is sent to the first large language model through the second large language model; The global solution is obtained by performing a global analysis of the first execution result using the first large language model.

6. The method according to claim 1, characterized in that, The step of driving the robot to complete the corresponding sub-task based on the local solution and / or the global solution specifically includes: Based on the local solution, the robot is driven to execute the corresponding sub-task to obtain the second execution result corresponding to the local solution; If the second execution result is an execution failure, the action tree is structurally adjusted according to the global solution, and then the robot is driven to complete the corresponding sub-task based on the structurally adjusted action tree.

7. The method according to claim 6, characterized in that, The structural adjustment of the action tree based on the global solution specifically includes: The nodes that fail to execute in the action tree are evaluated for validity, and then the nodes are marked according to the validity evaluation results to obtain the node marking results; Based on the global solution and the long sequence of tasks, generate several new sub-tasks; Based on the node labeling results and the new subtasks, the action tree is structurally adjusted.

8. A robot task planning enhancement system based on a large language model, characterized in that, include: An action tree construction module is used to construct an action tree, which includes several candidate action sequences, and each candidate action sequence includes several action functions. The task planning module is used to obtain long sequence tasks, input the long sequence tasks into the first large language model for task planning, and output several sub-tasks. The subtask execution module is used to input each of the subtasks into the second large language model for task scheduling, and to obtain the action function for driving the robot and the first execution result corresponding to the action function based on the action tree; An error handling module is used to obtain a local solution and a global solution through the first large language model and the second large language model when the first execution result is an execution failure, and then drive the robot to complete the corresponding sub-task based on the local solution and / or the global solution.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method of any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images