A multi-robot cooperation method, system and electronic device in a dynamic environment

Abstract

This invention provides a method, system, and electronic device for multi-robot collaboration in dynamic environments, relating to the field of robotics. The method includes: generating a task dependency graph represented by a directed acyclic graph (DAG) based on task description information; generating a hierarchical DAG for each sub-task using large language models (LLMs) based on first environment description information, functional information of each robot, and the task-corresponding DAG, where the hierarchical DAG represents the execution strategy of the sub-task; obtaining a task plan for each robot based on its functional information, first state information, and the sub-task execution strategy; and assigning each robot's task plan to the robots so that each robot executes the task plan. This improves the flexibility of task planning and allocation during multi-robot collaboration, better adapting to dynamic environments.

Description

Technical Field

[0001] This invention relates to the field of robotics, and in particular to a method, system, and electronic device for multi-robot collaboration in dynamic environments. Background Technology

[0002] In multi-robot collaboration, task planning and allocation are crucial processes. In robot task planning and allocation, tasks typically need to be decomposed into a series of subtasks that can be executed by the deployed robots. Related technologies generally address static and known environments, where task decomposition is usually achieved through classic methods, such as manual planning. This involves defining the task decomposition logic using manually set rules, and typically, the planning and allocation strategy is constructed manually based on domain expertise and hard-coded. While simple and intuitive, this method lacks flexibility and struggles to adapt to dynamic environments. Summary of the Invention

[0003] The purpose of this invention is to provide a method, system, and electronic device for multi-robot collaboration in dynamic environments, so as to improve the flexibility of task planning and allocation during multi-robot collaboration and better adapt to dynamic environments. The specific technical solution is as follows:

[0004] Firstly, a method for multi-robot collaboration in a dynamic environment is provided, including:

[0005] Receive task description information;

[0006] Using the task description information, a task dependency graph represented by a directed acyclic graph (DAG) corresponding to the task is generated. Each node in the DAG represents a subtask, and the edge between any two nodes represents the dependency relationship between the subtasks represented by the two nodes.

[0007] Based on the first environment description information of the current environment, the functional information of each robot and the DAG corresponding to the task, a hierarchical DAG corresponding to each subtask is generated through large language models LLMs. The hierarchical DAG represents the execution strategy of the subtask.

[0008] Based on the functional information and first state information of each robot, as well as the subtask execution strategy corresponding to each subtask, the task plan of each robot is obtained.

[0009] Assign a task plan to each robot so that each robot can execute the task plan.

[0010] Optionally, when the task description information includes a task description represented in natural language, the method further includes, after receiving the task description information:

[0011] The task description represented in natural language is converted into task description information in linear temporal logic (LTL) form.

[0012] Optionally, the step of generating a hierarchical DAG for each subtask based on the first environment description information, the functional information of each robot, and the DAG corresponding to the task, using large language models (LLMs), includes:

[0013] The first environment description information, the functional information of each robot and the DAG corresponding to the task are input into the large language model LLMs, and the first execution strategy is output through the large language model LLMs.

[0014] Obtain the historical execution strategy of a historical task with the same task type as the current task, and input the historical execution strategy into the Large Language Model (LLMs). Adjust the first execution strategy through the Large Language Model (LLMs) to obtain the second execution strategy.

[0015] Obtain resource conflict constraints and time conflict constraints, and input the resource conflict constraints and time conflict constraints into the Large Language Model (LLMs). Adjust the second execution strategy through the Large Language Model (LLMs) to obtain the hierarchical DAG corresponding to each subtask.

[0016] Optionally, the step of obtaining the task plan for each robot based on the functional information and first state information of each robot, as well as the subtask execution strategy corresponding to each subtask, includes:

[0017] Based on the functional information and first state information of each robot, as well as the subtask execution strategy corresponding to each subtask, the task plan of each robot is obtained by using the branch and bound search method and the integer programming method, with minimizing the overall task completion time as the optimization objective.

[0018] Optionally, after assigning the task plan for each robot to the robot, the method further includes:

[0019] When the first preset condition is met, the second environment description information is obtained. The first preset condition includes at least one of the following conditions: a new operation object corresponding to the task type of the current task is detected in the current environment; a new resource type is detected in the current environment; a new resource of an existing resource type is detected in the current environment; a new task type is detected. The second environment description information is the updated environment description information compared with the first environment description information.

[0020] Based on the second environmental description information, the functional information of each robot, and the updated DAG, the hierarchical DAG corresponding to each subtask is regenerated using large language models (LLMs).

[0021] Optionally, after assigning the task plan for each robot to the robot, the method further includes:

[0022] When the second preset condition is met, the second state information of each robot is obtained. The second preset condition includes: no plan completion information is received from the robot that executes the task plan within a preset time range, or a robot malfunction is detected. The second state information is the updated state information compared with the first state information.

[0023] Based on the functional information and second state information of each robot, as well as the subtask execution strategy corresponding to each subtask, the updated task plan for each robot is obtained.

[0024] Optionally, after generating the task dependency graph represented by the directed acyclic graph (DAG) corresponding to the task, the method further includes: displaying the DAG to allow a human operator to verify the DAG;

[0025] After generating the hierarchical DAG corresponding to each subtask, the method further includes: displaying the hierarchical DAG corresponding to each subtask so that a human operator can verify the hierarchical DAG corresponding to each subtask.

[0026] After obtaining the task plans for each robot, the method further includes: displaying the task plans for each robot to allow a human operator to verify the task plans for each robot;

[0027] After assigning the task plan to each robot, the method further includes: receiving the task execution progress information fed back by the robot, and displaying the task execution progress information so that a human operator can verify the task plan of each robot based on the task execution progress information.

[0028] Optionally, the first environment description information includes: task object information of the operation object corresponding to the task type of the current task and resource information of resources in the environment; the robot's functional information indicates the types of operations that the robot can perform and the types of resources that it can operate.

[0029] Secondly, a multi-robot collaborative system in a dynamic environment is provided, including: a task understanding module, a subtask generation module, and a subtask allocation module;

[0030] The task understanding module is used to receive task description information; and use the task description information to generate a task relationship dependency graph represented by a directed acyclic graph (DAG) corresponding to the task, wherein each node in the DAG represents a subtask, and the edge between any two nodes represents the dependency relationship between the subtasks represented by the two nodes.

[0031] The subtask generation module is used to generate a hierarchical DAG corresponding to each subtask based on the first environment description information of the current environment, the functional information of each robot and the DAG corresponding to the task, using large language models LLMs. The hierarchical DAG represents the execution strategy of the subtask.

[0032] The subtask allocation module is used to obtain the task plan for each robot based on the functional information and first state information of each robot, as well as the subtask execution strategy corresponding to each subtask; and to allocate the task plan of each robot to the robot so that each robot can execute the task plan.

[0033] Thirdly, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0034] Memory, used to store computer programs;

[0035] When a processor executes a program stored in memory, it implements any of the steps described in the first aspect.

[0036] This invention also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute any of the multi-robot collaboration methods described above in a dynamic environment.

[0037] Beneficial effects of the embodiments of the present invention:

[0038] In this embodiment of the invention, task description information is received and used to generate a task dependency graph represented by a directed acyclic graph (DAG) corresponding to the task. Each node in the DAG represents a subtask, and the edge between any two nodes represents the dependency relationship between the subtasks represented by the two nodes. Then, based on the first environment description information of the current environment, the functional information of each robot, and the task-corresponding DAG, a hierarchical DAG corresponding to each subtask is generated using a large language model (LLMs). The hierarchical DAG represents the execution strategy of the subtask. Furthermore, based on the functional information and first state information of each robot, as well as the subtask execution strategy corresponding to each subtask, the task plan of each robot is obtained. The task plan of each robot is then assigned to the robot so that each robot can execute the task plan. Thus, unlike related technologies that require hard-coding strategies based on domain expertise for an existing environment, where the strategy is specific to that environment and needs to be re-engineered and hard-coded when the environment changes, this invention only requires task description information. It automatically represents the relationships between subtasks using a Directed Acyclic Graph (DAG) and generates execution strategies for each subtask using Large Language Models (LLMs). Based on these execution strategies, task plans for each robot can be obtained and assigned to each robot, improving the flexibility of task planning and allocation in multi-robot collaboration and better adapting to dynamic environments.

[0039] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0041] Figure 1 This is a flowchart of a multi-robot collaboration method in a dynamic environment, as described in an embodiment of the present invention.

[0042] Figure 2 This is a schematic diagram of the hierarchical DAG corresponding to the subtasks in an embodiment of the present invention;

[0043] Figure 3 A schematic diagram illustrating the framework of the multi-robot collaboration method in a dynamic environment as described in this embodiment of the invention;

[0044] Figure 4This is a flowchart of the dynamic adaptation verification process in an embodiment of the present invention;

[0045] Figure 5 This is a schematic diagram of a multi-robot collaboration method in a dynamic environment, as described in an embodiment of the present invention.

[0046] Figure 6 This is a schematic diagram of the structure of a multi-robot collaborative system in a dynamic environment according to an embodiment of the present invention;

[0047] Figure 7 This is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on the present invention are within the scope of protection of the present invention.

[0049] Reference Figure 1 This invention provides a method for multi-robot collaboration in a dynamic environment, including:

[0050] S11, Receive task description information;

[0051] S12. Using the task description information, generate a task dependency graph represented by a Directed Acyclic Graph (DAG) corresponding to the task. Each node in the DAG represents a subtask, and the edge between any two nodes represents the dependency relationship between the subtasks represented by the two nodes.

[0052] S13. Based on the first environment description information of the current environment, the functional information of each robot and the DAG corresponding to the task, a hierarchical DAG corresponding to each subtask is generated through a large language model (LLMs). The hierarchical DAG represents the execution strategy of the subtask.

[0053] S14. Based on the functional information and first state information of each robot, as well as the sub-task execution strategy corresponding to each sub-task, the task plan of each robot is obtained.

[0054] S15, assign the task plan of each robot to the robot so that each robot can execute the task plan.

[0055] In this embodiment of the invention, task description information is received and used to generate a task dependency graph represented by a directed acyclic graph (DAG) corresponding to the task. Each node in the DAG represents a subtask, and the edge between any two nodes represents the dependency relationship between the subtasks represented by the two nodes. Then, based on the first environment description information of the current environment, the functional information of each robot, and the task-corresponding DAG, a hierarchical DAG corresponding to each subtask is generated using a large language model (LLMs). The hierarchical DAG represents the execution strategy of the subtask. Furthermore, based on the functional information and first state information of each robot, as well as the subtask execution strategy corresponding to each subtask, the task plan of each robot is obtained. The task plan of each robot is then assigned to the robot so that each robot can execute the task plan. Thus, unlike related technologies that require hard-coding strategies based on domain expertise for an existing environment, where the strategy is specific to that environment and needs to be re-engineered and hard-coded when the environment changes, this invention only requires task description information. It automatically represents the relationships between subtasks using a Directed Acyclic Graph (DAG) and generates execution strategies for each subtask using Large Language Models (LLMs). Based on these execution strategies, task plans for each robot can be obtained and assigned to each robot, improving the flexibility of task planning and allocation in multi-robot collaboration and better adapting to dynamic environments.

[0056] The multi-robot collaboration method in dynamic environments provided in this invention can be applied to electronic devices, specifically servers, controllers, etc. This multi-robot collaboration method can be applied to scenarios such as disaster response, search and rescue, etc.

[0057] In S11, the task description information can be a task description represented in natural language, or it can be a task description information in the form of linear temporal logic (LTL).

[0058] Specifically, it can receive task instructions, which contain task description information.

[0059] When the task description information includes a task description in natural language representation, after receiving the task description in natural language representation, the task description in natural language representation is first converted into task description information in the form of Linear Temporal Logic (LTL).

[0060] Task description information can be determined based on the actual application scenario. For example, in a fire response scenario, task description information can represent information describing a task with fire rescue as its objective. Specifically, task description information may include the operations to be performed, the objects targeted by the operations, and the operation area. Alternatively, task description information may also include the priority of each operation, etc.

[0061] For example, in a fire response scenario, the task description in Natural Language Representation (NLP) could be: "Completely explore the area. After exploration and inspection, always extinguish detected fires, rescue detected victims, prioritize large fires over small fires, and severely injured victims over those with minor injuries." The LTL (Low-Temperature Language) form of this task description would be: □◇Explore∧□(Detect∧(Flame→◇Extinguish)∧(Human→◇Rescue)∧((Human∧Flame)→◇Rescue∧◇Extinguish))∧□((Large Fire∧Small Fire)→◇Extinguish The equation is: .Small fire ∧◇Extinguish.Large fire) ∧□((Seriously injured ∧ Minorly injured) →◇(Rescue.Seriously injured ∧Rescue.Minorly injured)), where □ indicates that a condition must be true at all future moments (e.g., "□φ" requires φ to always be true); ◇ indicates that a condition will be true at some future moment (e.g., "◇φ" indicates that φ will eventually be true); ∧ indicates conjunction, which is the logical AND, indicating that multiple conditions must be satisfied simultaneously; → indicates implication, meaning "if...then...", for example, "φ→ψ" is equivalent to "if φ is true, then ψ must be true". Here, φ and ψ represent specific conditions.

[0062] In S12, the generated task dependency graph can also be understood as an ordered set of subtasks.

[0063] Dependencies can represent the priority and exclusion relationships between subtasks. The priority of subtasks can be determined based on their urgency, such as rescuing seriously injured people having a higher priority than rescuing slightly injured people. Exclusion relationships between subtasks indicate whether two subtasks can be executed simultaneously.

[0064] This step can also be understood as performing semantic understanding and semantic modeling on the task description information, and the generated task dependency graph is the result of semantic modeling. Specifically, step S12 can be executed through the task understanding module.

[0065] In S13, the first environment description information includes: task object information of the operation object corresponding to the task type of the current task and resource information of resources in the environment.

[0066] The objects and resources to be operated on are determined based on the actual application scenario. For example, in a fire response scenario, the objects to be operated on may include fire and injured persons, with injured persons further subdivided into seriously injured and slightly injured persons; resources may include water, sand, etc.

[0067] The first environment description information mentioned here can also be understood as the initial environment information of the current environment when the task description information is received.

[0068] The robot's functional information indicates the types of operations the robot can perform and the types of resources it can manipulate.

[0069] Simply put, a robot's functional information refers to the skills it possesses.

[0070] Among them, the types of operations that the robot can perform are the actions that the robot can perform, and the types of resources that the robot can operate on are the resources that each action it can perform can operate on.

[0071] In one example, there are three sub-tasks: firefighting, rescuing seriously injured people, and rescuing slightly injured people. The resulting hierarchical DAG graphs for each sub-task are as follows: Figure 2 As shown. For the sub-task of firefighting, the execution strategy includes filling the tank with water and then spraying water, or filling the tank with sand and then spraying sand. Filling the tank with sand and water is performed only by the fire truck. For the sub-task of rescuing seriously injured people, the execution strategy includes performing the actions: grabbing, stopping, moving, and placing. For the sub-task of rescuing slightly injured people, the execution strategy includes performing the actions: grabbing, stopping, moving, and placing.

[0072] In an optional embodiment, S13 includes the following steps:

[0073] Step 1: Input the first environment description information, the functional information of each robot and the DAG corresponding to the task into the large language model LLMs, and output the first execution strategy through the large language model LLMs;

[0074] Step 2: Obtain the historical execution strategy of historical tasks with the same task type as the current task, and input the historical execution strategy into the Large Language Model (LLMs). Adjust the first execution strategy through the Large Language Model (LLMs) to obtain the second execution strategy.

[0075] Step 3: Obtain resource conflict constraints and time conflict constraints, and input them into the Large Language Model (LLMs). Adjust the second execution strategy using the LLMs to obtain the hierarchical DAG corresponding to each subtask.

[0076] Specifically, step S13 can be executed through a subtask generation module, such as an LLMs online subtask generator. This step leverages the powerful reasoning capabilities of LLMs to generate specific execution strategies for subtasks in real time. Unlike traditional one-time reasoning, this embodiment of the invention employs a multi-stage reasoning method to progressively optimize the generation process of specific execution strategies for subtasks.

[0077] Subtask generation: Based on environmental description, robot capabilities, and historical logs, multi-stage reasoning is performed through the subtask generation module in DEXTER-LLM (Lucky Big Language Model).

[0078] LLMs are used to analyze scene semantics, generating feasible subtask decomposition strategies and their time logic constraints. A hierarchical Directed Acyclic Graph (DAG) is output. Based on the task-related DAG generated in the task understanding phase, each subtask node is expanded into a specific subtask strategy. This process may include the following stages:

[0079] (1) Context analysis: Filter out irrelevant historical observations and events, and extract key information about the current environment.

[0080] This stage inputs to LLMs include: general instructions, which may include the LLM's role, tasks, and desired actions; a description of the robot group's capabilities, specifically the skills possessed by each type of robot, i.e., what types of actions they can perform and what resources each action can manipulate, such as spraying water; specific examples of contextual analysis; existing resources in the environment, such as water and fire extinguishers; and existing tasks in the environment, such as small fires, large fires, and casualties.

[0081] Historical observations and events can be environmental descriptions, tasks, etc., obtained from historical task analysis.

[0082] The key information of the current environment may include the task object information of the operation object corresponding to the task type of the current task and the resource information of the resources in the environment, such as existing resources in the environment, such as water, fire extinguishers, etc.; and existing tasks in the environment, such as small fire, large fire, and lightly and seriously injured people.

[0083] Contextual analysis is a specific example of what you want LLMs to normalize the output of at this stage.

[0084] (2) Meta-policy adjustment: Generate new policies based on newly discovered resources or features, and update the meta-policy to guide LLMs inference.

[0085] Meta-strategy refers to a plan for completing a task. For example, if a fire is discovered and water is also available, then the priority is to use a drone or vehicle to fill it with water and then go to the fire to spray water.

[0086] The input to LLMs in this stage may include: the overall instructions and robot capability descriptions from the previous stage; the original meta-policy, which is mainly used to store the experience generated by subtasks; and the outputs from the previous stage.

[0087] The adjusted execution strategy is output through LLMs.

[0088] (3) Subtask guidance: Consider resource conflicts and time relationships to generate a consistent sequence of subtasks.

[0089] Resource conflict refers to a sequence of subtasks in a task that requires the same resource to complete the task; time conflict refers to whether several subtasks should be executed sequentially or in parallel.

[0090] Consistent tasks refer to a sequence of subtasks without resource or time conflicts, which is the generation of a layered DAG of subtasks.

[0091] The inputs to LLMs in this stage may include: general instructions from the previous two stages, robot capability descriptions, and contextual analysis; and the results output from the previous stage.

[0092] LLMs output subtask-guided cases, which are sequences of subtasks described in natural language.

[0093] (4) Subtask serialization: Encode the subtask strategy into hierarchical DAG and JSON formats for subsequent allocation.

[0094] The inputs to LLMs in this stage may include: general instructions and robot capability descriptions from the previous two stages; and the results output from the previous stage.

[0095] The LLMs output a task sequence example, which is the final structured subtask sequence.

[0096] In S14, the robot's state information can be understood as information indicating whether the robot is available. The first state information mentioned here can be understood as the robot's initial state information when the task description information is received.

[0097] S14 essentially involves assigning the execution strategies of each subtask to each robot. Each robot's task plan can include the execution strategies of the subtasks assigned to that robot. The robot's task plan can be the entire execution strategy of the subtask, or it can be split into subtask execution strategies. The robot's task plan is the local execution strategy obtained from the split.

[0098] In one optional embodiment, based on the functional information and first state information of each robot, and the subtask execution strategy corresponding to each subtask, a task plan for each robot is obtained, including:

[0099] Based on the functional information and first state information of each robot, as well as the subtask execution strategy corresponding to each subtask, the task plan of each robot is obtained by using the branch and bound search method and the integer programming method, with minimizing the overall task completion time as the optimization objective.

[0100] In this embodiment of the invention, the input to the branch-bound search method and the integer programming method is the subtask execution strategy, specifically a series of subtask sequences for each task, and the output is the robot allocation result for each subtask, specifically one subtask sequence for each task under the above-mentioned DAG dependency relationship, and the robot allocation result for each subtask.

[0101] Specific process:

[0102] (1) First, it is explained that the branch and bound search method specifically includes node construction, search tree expansion method, search direction, and determination of upper and lower bounds.

[0103] (2) Node construction: Each node contains a list, and inside each list is a smaller list representing each robot. Then, the tasks that need to be performed by the robot are placed in each smaller list.

[0104] (3) The search tree expansion method is to put all the alternative tasks into the tree in a combination.

[0105] (4) Search direction: Since the optimization objective is a min-max problem, the most important thing is to have a common upper bound. Then, by judging that the lower bound of each node is greater than the common upper bound, there is no need to explore and it can be directly deleted.

[0106] (5) Upper bound: It can be solved by putting them in one by one using a greedy algorithm and then using integer programming.

[0107] (6) Lower bound: By assuming no movement distance and no waiting time, an integer programming problem is constructed, and then a solver such as the mathematical programming optimizer (gurobi) can be used to solve it, and then the upper bound can be obtained.

[0108] (7) It can be done by width search, and then convergence is achieved based on the termination condition, the number of searches or the basic bound remains unchanged.

[0109] Specifically, step S14 can be executed through a subtask allocation module, which can be a subtask allocator and scheduler. After generating a sequence of subtasks (the execution strategies of the serialized subtasks), the subtask allocator and scheduler allocates these serialized subtask execution strategies to each robot and formulates a detailed execution plan. Considering the heterogeneity of the robots and the time constraints of the tasks, an integer programming algorithm based on branch and bound can be used to optimize the allocation and scheduling of subtasks. The optimization objectives are: to ensure that all task constraints (such as time order, resource limitations, etc.) are met; and to minimize the overall task completion time (i.e., the total project duration).

[0110] This step considers constraints such as robot state, explored map, and task distribution to minimize the overall task completion time (makespan) and outputs a local plan for each robot, i.e., each robot's task plan, which contains a sequence of actions with time constraints. Specifically, branch-and-bound search and integer programming methods are used to optimize subtask allocation and scheduling. After representing the execution strategies of subtasks in a hierarchical DAG graph, the subtask allocation and scheduler minimizes the overall task completion time based on constraints such as the robot state in the current robot cluster, individual robot skills, task distribution, and temporal relationships. The output of the subtask allocation and scheduler is the local plan for each robot, containing a sequence of actions with time constraints.

[0111] In summary, the embodiments of the present invention realize multi-robot collaborative tasks, such as... Figure 3 As shown, this can be accomplished through the following steps:

[0112] A1. Task Understanding and Semantic Modeling: Receives task descriptions in natural language or linear temporal logic (LTL) form.

[0113] This step can be specifically implemented through S12 as described above.

[0114] A2. Subtask Generation: Based on the environment description, robot capabilities, and historical logs, multi-stage reasoning is performed through the subtask generation module in DEXTER-LLM.

[0115] This step is specifically implemented through S13 described above.

[0116] Leveraging the powerful reasoning capabilities of LLMs, specific execution strategies for subtasks are generated in real time. Unlike traditional one-time reasoning, a multi-stage reasoning method is employed to progressively optimize the subtask generation process. This includes: context analysis, meta-policy adjustment, subtask guidance, and subtask serialization.

[0117] A3. Subtask Allocation and Scheduling: Optimize subtask allocation and scheduling using branch-and-bound search and integer programming methods. Consider constraints such as robot state, explored map, and task distribution, and minimize the overall completion time (makespan). Output the local plan for each robot, including a sequence of actions with time constraints.

[0118] This step is specifically implemented through S14 described above.

[0119] In this way, by completing the multi-robot collaborative task planning, the robots in the robot cluster can execute tasks according to the execution plan allocated by A3, thus realizing the task execution process of the cluster robots.

[0120] In one alternative embodiment, after assigning the task plan for each robot to the robot, the following may also be included:

[0121] When the first preset condition is met, the second environment description information is obtained; based on the second environment description information, the functional information of each robot and the updated DAG, the hierarchical DAG corresponding to each subtask is regenerated through LLMs.

[0122] The first preset condition includes at least one of the following conditions: detecting a new operation object corresponding to the task type of the current task in the current environment, detecting a new resource type in the current environment, detecting a new resource of an existing resource type in the current environment, or detecting a new task type; the second environment description information is the updated environment description information compared to the first environment description information.

[0123] A new operation object can also be understood as a new task instance, that is, a new instance of the current task. For example, detecting new trapped personnel, new fire sources, etc. in the environment.

[0124] New operation objects can include those with higher priority than existing operation objects. For example, a new seriously injured person is detected in the environment.

[0125] New resource types refer to new resource types that can be used during task execution. They can also be understood as new feature types, such as fire extinguishing resource types such as water tanks in addition to fire extinguishers.

[0126] New resources of existing resource types, such as newly detected reservoirs.

[0127] In one alternative embodiment, after assigning the task plan for each robot to the robot, the following may also be included:

[0128] When the second preset condition is met, the second state information of each robot is obtained; based on the functional information and second state information of each robot, as well as the sub-task execution strategy corresponding to each sub-task, the updated task plan of each robot is obtained.

[0129] The second preset condition includes: no plan completion information is received from the robot executing the task plan within the preset time range, or a robot malfunction is detected. The second status information is the updated status information compared to the first status information.

[0130] The preset time range can be determined based on actual needs or experience. Plan completion information indicates that the robot has completed its assigned execution plan.

[0131] For example, whenever the robot completes a subtask, relevant information such as planned completion information is fed back to the subtask allocation module to monitor the progress of the plan. If a long delay occurs, the subtask allocation module will be retried to update the robot's execution plan.

[0132] Alternatively, when a robot malfunctions, the current plan becomes infeasible under unchanged global information. Therefore, the subtask assignment module is retried to find a feasible local robot plan, i.e., the robot's execution plan.

[0133] The process described above, where the first preset condition is met, and based on the second environmental description information, the functional information of each robot, and the updated DAG, hierarchical DAGs corresponding to each subtask are regenerated using LLMs; and where the second preset condition is met, and based on the functional information and second state information of each robot, as well as the subtask execution strategy corresponding to each subtask, the updated task plan for each robot is obtained, can also be understood as the dynamic adaptation verification stage. This allows for the automatic generation and optimization of subtask allocation schemes in the face of dynamically changing environmental characteristics and the triggering of new tasks, enabling better adaptation to dynamic environments.

[0134] Specifically, such as Figure 4 As shown, dynamic adaptation verification may include the following steps:

[0135] B1. New Task Instance Detection: This detection can be invoked in real time during task execution. The detection mechanism is as follows: the environment is monitored in real time through sensors such as robot radar and cameras, or external inputs (such as manual alarms). When a new instance (such as trapped personnel or fire source) matching the known task type (i.e., the current task) is detected, an event is triggered. When a new task instance is discovered, the parameters of the new instance (location, urgency, resource requirements) are parsed and compared with the coverage of the currently planned tasks. If the new instance is within the reachable area of ​​the current path, it is directly inserted into the task queue; if it is outside the range, the path replanning algorithm is initiated.

[0136] The local plan is the robot's execution plan.

[0137] B2. New High-Priority Task Instance Detection: This detection can be invoked in real time during task execution. The priority determination criteria are based on multi-dimensional feature analysis (such as vital sign intensity, environmental crisis level, etc.) to assign priority to each task point. If a new high-priority task instance is detected, the current task graph DAG needs to be adjusted.

[0138] The DAG update process can be specifically as follows: compare the priority relationship between the newly detected task and the tasks in the task sequence, and insert the high-priority task into the original DAG. In this embodiment of the invention, the current task execution will not be interrupted by default; instead, the high-priority task will be executed as the next task. Priorities can be dynamically assigned manually in the meta-policy, or fixed priorities can be assigned before the project starts, such as: severely wounded personnel have higher priority than slightly wounded personnel, and large fires have higher priority than small fires.

[0139] B3. New Task Type Detection: Whenever a new task type is specified in a new task, the task understanding module is re-triggered to update the task DAG, and then fed into the task assignment module to finally update the local plan.

[0140] B4. New Feature Type Detection: During task execution, whenever a robot detects a new type of feature (e.g., fire-fighting resource types such as water tanks in addition to fire extinguishers), this indicates the potential for a new strategy to decompose the task using new features. Therefore, the subtask generation module is re-triggered to update the hierarchical DAG, and then forwarded to the next module in the subtask assignment to update the local plan.

[0141] B5. New Feature Instance Detection: Whenever a new instance of a known feature is detected (e.g., a new reservoir is detected), it indicates that some subtasks may complete earlier. Therefore, the map is updated using these features, and the subtask assignment module is re-triggered to update the local plan.

[0142] In summary, the process involves detecting whether a new instance has appeared. If so, the DAG graph is updated and the subtask allocation module is retried to update the local plan. If not, the task continues to execute, which can also be understood as not performing the update process, allowing the task to continue executing.

[0143] B6. Subtask Execution Status Update: Whenever the robot completes a subtask, this information is fed back to the subtask allocation module to monitor the progress of the plan execution. This process can also be called the subtask execution status detection process. It determines whether a task has experienced a long delay. If a task has been delayed due to prolonged execution, the subtask allocation module is re-triggered to update the local plan; otherwise, if no long delay has occurred, the task continues to execute. This can also be understood as not performing an update process, allowing the task to continue execution.

[0144] B7. Robot Fault Detection: During the overall task execution, the robot may malfunction due to various practical situations (such as signal loss, excessive temperature, inability to move, etc.). To determine if a robot fault has occurred, if so, it is assumed that the robot's current plan is unexecutable without altering the global plan. Therefore, the subtask allocation module is re-triggered to update the local plan. After a fault occurs, the fault information is transmitted to the lower-level subtask allocation module to find a feasible execution plan for the robot. If no robot fault has occurred, the task continues to execute; this can also be understood as not updating the process, allowing the task to continue.

[0145] In one optional embodiment, after generating the task dependency graph represented by the directed acyclic graph (DAG) corresponding to the task, the method further includes: displaying the DAG to allow a human operator to verify the DAG;

[0146] After generating the hierarchical DAG corresponding to each subtask, the process also includes: displaying the hierarchical DAG corresponding to each subtask so that human operators can verify the hierarchical DAG corresponding to each subtask.

[0147] After obtaining the task plans for each robot, the process also includes: displaying the task plans for each robot so that human operators can verify them;

[0148] After assigning the task plan to each robot, the process also includes: receiving and displaying the task execution progress information from the robots, so that human operators can verify the task plans of each robot based on the task execution progress information.

[0149] In this embodiment of the invention, the process of verification by a human operator can also be referred to as the human-in-the-loop verification stage.

[0150] The results generated at each stage can be verified, such as the DAG generated during task understanding, the hierarchical DAG representing the execution strategies of subtasks generated during subtask generation, and the task plans for each robot generated during subtask allocation. Additionally, the progress of the robots in executing their task plans can also be verified.

[0151] Specifically, during the human-loop verification phase, the following steps can be performed:

[0152] C1. Human operators can choose to review the topology of the DAG corresponding to each task through a visual interface, use the timeline tool to check whether the critical path length is reasonable, and use the dependency matrix to verify the task priority settings. Specifically, it is necessary to confirm whether urgent tasks have the highest priority and whether there are resource conflicts among parallel tasks.

[0153] This process involves generating the DAG corresponding to the task and then displaying the DAG so that a human operator can verify it. Specifically, the task understanding module can generate a task dependency graph represented by a directed acyclic graph (DAG) corresponding to the task. Therefore, this process can also be understood as verifying a portion of the sorting in the task understanding module.

[0154] C2. In the generated hierarchical DAG, human operators examine the decomposition strategy (i.e., the execution strategy of subtasks) at each level to check its rationality. They can evaluate the specific settings of each decomposition step to confirm whether the task has been accurately decomposed into reasonable subtasks. If some tasks are overly complex, broken down into many unnecessary small tasks, or overly simplified, human feedback is needed for strategy adjustments. Humans can offer suggestions on the decomposition strategy based on practical experience to optimize task complexity and execution efficiency.

[0155] This process involves generating the hierarchical DAG corresponding to each subtask, and then displaying the hierarchical DAG corresponding to each subtask so that human operators can verify the hierarchical DAG corresponding to each subtask.

[0156] Specifically, a hierarchical DAG (representing the execution strategy of the subtask) can be generated for each subtask through the subtask generation module. The subtask generation module can also be called the task decomposition module, etc. Therefore, this process can also be understood as verifying the strategy in the task decomposition module.

[0157] C3. For the generated local plan, verify its consistency with constraints by progressively reviewing each subtask and its allocation. Human operators examine the sequence of subtasks assigned to each robot (the serialized subtask execution strategy) to confirm whether these sequences comply with all constraints, such as resource conflicts, time windows, and equipment availability, ensuring that the assigned subtask sequences do not cause conflicts or delays in actual operation. Simultaneously, the workload balance and rationality of each robot task must be evaluated.

[0158] This process involves obtaining the task plans for each robot and then displaying those plans to allow human operators to verify them.

[0159] Specifically, the task plan for each robot can be obtained through the subtask allocation module, and this task plan can also be called a local plan. Therefore, this process can also be understood as verifying the local plan in the subtask allocation module.

[0160] C4. Human operators monitor and review the execution feedback of each subtask to ensure that the subtasks are executed according to the predetermined plan and that the results match expectations. In scenarios with complex tasks and dense dependencies, special attention must be paid to the "illusion" phenomenon of large language models, i.e., the generated results do not conform to the actual situation. By carefully verifying and comparing the execution status, these potential errors can be detected and corrected in a timely manner, thereby ensuring the correctness and success rate of the overall task.

[0161] This process involves assigning task plans to each robot, receiving and displaying the robot's progress information, allowing human operators to verify the task plans based on this information. Alternatively, it can be understood as verifying the actual execution status of sub-tasks. It supports human-machine interaction for verification and adjustment, improving the success rate and efficiency of task execution.

[0162] In complex, dynamic, and unknown environments, such as disaster response, search and rescue scenarios, multi-robot collaboration is often employed to perform complex tasks. These complex tasks typically require multiple heterogeneous robots to work together to achieve objectives such as firefighting, personnel rescue, and environmental monitoring. However, due to the uncertainty of unknown environments and the diversity and dynamism of tasks, efficiently coordinating multiple robots becomes a major challenge. Complex, dynamic, and unknown environments can be collectively referred to as dynamic environments.

[0163] This invention aims to address the challenges of dynamic task coordination among multiple robots in dynamic environments. It enables dynamic task planning and allocation in dynamic environments, achieving dynamic and interpretable multi-robot collaboration using a large language model in dynamic environments.

[0164] In the traditional field of multi-robot task planning, a robot's task needs to be decomposed into a series of sub-tasks, which must be executable by the deployed robot. For static and known environments, this task decomposition can be achieved using classical methods.

[0165] This invention avoids the need for traditional methods to construct symbolic representations based on extensive domain expertise, which are often hard-coded. This makes them difficult to adapt to new features, tasks, or environmental changes in open and dynamic environments, thus improving the adaptability of task planning and allocation in multi-robot collaboration. Furthermore, it avoids the limitations of traditional optimization methods based on fixed rule sets or constraints, which make them inflexible in dynamic environments to handle unexpected situations or new tasks. For example, in disaster relief scenarios, if new victims are suddenly discovered or resources are scarce, traditional methods struggle to adjust task allocation in real time; this invention is not limited by hard constraints. It also avoids the limitation that traditional methods are suitable for small, well-defined task scenarios but become inadequate for large-scale, complex tasks. As the number of tasks increases or environmental complexity rises, the computational complexity of traditional methods rises rapidly, leading to decreased planning efficiency. This invention improves scalability; as the number of tasks increases or environmental complexity rises, it adaptively updates the DAG corresponding to the task, the hierarchical DAG of subtasks, and the robot's execution plan by detecting changed environmental description information, enabling rapid task planning and allocation.

[0166] Furthermore, the embodiments of this invention, through a human-in-the-loop verification process, can verify the execution strategies and allocated execution plans generated by LLMs, avoiding the problem of insufficient formal verification in related technologies. For example, while LLM-based methods in related technologies can generate diverse task decomposition strategies, the quality of their generated plans lacks rigorous mathematical or logical verification. This means that in high-risk scenarios, unreliable or suboptimal solutions may be generated. The embodiments of this invention can automatically generate and optimize sub-task allocation schemes under dynamically changing environmental characteristics and the triggering of new tasks, thus better adapting to dynamic environments.

[0167] Furthermore, the embodiments of the present invention are sufficiently adaptable to situations involving dynamic resource constraints or frequent changes in task priorities. For example, when resource distribution changes or task urgency is adjusted, it can respond promptly and adaptively update the DAG corresponding to the task, the hierarchical DAG of subtasks, and the robot's execution plan by detecting the changed environmental description information, enabling rapid task planning and allocation.

[0168] Furthermore, the embodiments of this invention address the problem of insufficient interpretability in related technologies. Although LLM can generate complex task decomposition schemes, its internal reasoning process is often black-box, making it difficult to explain to users why a particular strategy is chosen. The embodiments of this invention, on the one hand, use DAG to intuitively and logically represent the relationships between subtasks, and on the other hand, demonstrate the results generated at each step to human operators during human-in-the-loop verification, thereby facilitating user understanding of the entire task planning and allocation process. This improves the interpretability of the task planning and allocation process and overcomes the deficiency of insufficient interpretability in scenarios requiring human participation in decision-making.

[0169] To better understand the multi-robot collaboration method in a dynamic environment provided by the embodiments of the present invention, an example from a real-world scenario is used for illustration.

[0170] For ease of description, the above-described multi-robot collaboration method in a dynamic environment is implemented in this embodiment of the invention using the DEXTER-LLM (Lucky Big Language Model) framework or system. The DEXTER-LLM framework includes: a task understanding module, a subtask generation module, and a subtask allocation module.

[0171] When a fire alarm sounds in the central control room of the chemical plant, the emergency command system immediately initiates a multi-robot collaborative rescue process. (See diagram below.) Figure 5 As shown, the scene may include elements such as the starting point, rescue station, slightly injured, seriously injured, sand, storage tanks, and flames, and includes storage tank area A, storage tank area B, storage tank area C, and storage tank area D.

[0172] The initial mission instructions are: "Fully explore the area. After exploration and inspection, always extinguish any detected fires, rescue any detected victims, prioritize large fires over small fires, and severely injured persons over those with minor injuries."

[0173] Assume the available robots include: ① Drones: Equipped with flexible robotic arms and dexterous hands, capable of moving, grasping objects or people, placing objects or people, and using tools. ② Transport vehicles: Small vehicles used to transport people or goods, but unable to load or unload themselves; their primary action is movement. ③ Fire trucks: Vehicles used to extinguish large fires; their primary actions are movement and spraying water to extinguish the fire.

[0174] The task understanding module of DEXTER-LLM started operating, identifying several core elements through semantic parsing: fire fighting priority, personnel rescue priority, area exploration, fire detection, flame extinguishing, and casualty rescue. The task understanding module performed semantic parsing and logical modeling on the natural language description of the task, breaking down the complex rescue task into several clear sub-tasks, including "completely explore the chemical plant area," "detect fire and personnel," "prioritize extinguishing large fires," and "prioritize rescuing seriously injured personnel." The system converted the natural language instructions into LTL-format task descriptions: □◇Explore∧□(Detect∧(Flame→◇Extinguish)∧(Humans→◇Rescue)∧((Humans∧Flame)→◇Rescue∧◇Extinguish))∧□((Large Fire∧Small Fire)→◇Extinguish.Small Fire∧◇Extinguish.Large Fire)∧□((Seriously Injured∧Minorly Injured)→◇(Rescue.Seriously Injured∧Rescue.Minorly Injured)).

[0175] After the initial exploration mission was launched, the multi-robot system began to systematically allocate tasks and carry out exploration work. The drone, as a flexible and efficient detection device, was assigned the task of exploring the chemical plant area. Relying on sensor and camera path coverage and real-time environmental scanning, the drone gradually created a map of the chemical plant. Shortly after the exploration began, the drone detected a fire at tank area A. At this point, a new task instance needed to be called for detection. The detected new task instance was: extinguish the fire at tank area A. Next, the task decomposition module needed to be called, and the LLMs were used to understand the task requirement as extinguishing the fire, decomposing it into two sub-tasks: refill with water → spray water (see details). Figure 2 Then the task allocation module is called to assign the subtask to fire truck No. 1. The robot's actual action sequence is: move to the water tank → fill with water → move to tank area A → extinguish the flames in tank area A.

[0176] Immediately during the operation of fire truck No. 1, a new task type, "Rescue Minor Injuries," was discovered. The task decomposition module (i.e., the sub-task generation module) was then invoked, and LLMs were used to understand the task requirements and break them down into multiple sub-tasks: Staying at the bottom of the injured person → Grabbing the injured person → Putting the injured person down → Moving the injured person. Next, the sub-task allocation module was invoked, assigning this sub-task to transport vehicle No. 1 and operating drone No. 1. The specific operational flow is as follows: the transport vehicle and operating drone arrive at the injured person's location; the drone grabs the injured person and places them on the transport vehicle; the transport vehicle then transfers the injured person to the fire station.

[0177] During the execution of the "Rescue the Minorly Wounded" task, a seriously wounded person was discovered, triggering a high-priority task instance, "Rescue the Seriously Wounded." This high-priority task instance refers to the new operation object corresponding to the current task type detected in the current environment, and its priority is higher than existing operation objects. The task decomposition module is invoked, and LLMs are used to understand the task requirements and decompose it into multiple sub-tasks: Stay at the bottom of the wounded person → Grab the wounded person → Put the wounded person down → Move the wounded person. Furthermore, the task of rescuing the seriously wounded person is required to be executed as quickly as possible, and this task is prioritized and inserted into the existing task sequence.

[0178] During the exploration, a new rescue station No. 2 was discovered, meaning there was a rescue station closer to the wounded. It was no longer necessary to transfer the wounded from a distance to a rescue station near the starting point. By changing the location information in the previous mission plan, the slightly wounded and seriously wounded could be transferred to the rescuer No. 2. The LLMs multi-level task planning module was invoked to re-plan the mission (modify the target point), which can also be understood as re-execution strategy of sub-tasks, thus completing the rescue mission more efficiently and minimizing medical rescue time.

[0179] As the fire truck proceeded to tank area A as planned, the drone detected a new type of firefighting resource—a "sand pile"—in the western area of ​​the chemical plant. This triggered a "new feature type detection," and the task understanding module analyzed the usability of the sand pile (e.g., whether it could replace water for firefighting). After updating the DAG hierarchical structure of the firefighting task, another feasible sub-task decomposition route was added: "Fire the sand" → "Proceed to the fire point and spray the sand pile to bury the burning materials." Subsequent analysis by the sub-task allocation module revealed that since the fire truck had its own transport capabilities, no additional equipment was needed to perform the sand pile loading, unloading, and spraying tasks. Ultimately, the fire truck was given the ability to flexibly switch between water and sand firefighting methods.

[0180] During the mission, the system monitored the status of each robot in real time. During the firefighting operation in the tank area, fire truck No. 1 experienced a technical malfunction that caused its water spray system to fail. The system immediately triggered the "robot fault detection" module, and task allocation was quickly adjusted. The firefighting task in the tank area was temporarily transferred to fire truck No. 2. Fire truck No. 1, as instructed, returned to the maintenance area to restore its functionality as quickly as possible and rejoin subsequent tasks. Simultaneously, an independent and efficient task allocation module reassigned subsequent sub-tasks to other robots to ensure that the completion of the overall mission was not affected by the failure of a single robot.

[0181] During rescue and firefighting operations, human operators at the control center also participated in the "human-in-the-loop verification" phase, verifying and adjusting the tasks executed by each module. In the task DAG generated by the task understanding module, human operators discovered potential conflicts in the order of certain tasks, such as prioritizing certain branches of the task while neglecting the time-sensitive nature of the fire. After verifying the algorithm's logic, the human operators adjusted the node order in the DAG to make the task dependencies more reasonable. In the task decomposition module, human operators found that the decomposition strategies for some subtasks were not concise enough, such as the drones having excessive workloads causing delays. Through feedback, the subtask decomposition logic was adjusted, allowing transport vehicles to proactively undertake a larger proportion of the tasks. In the task allocation phase, human operators also checked the local planning, verifying whether the plans for each robot met the actual resource and environmental constraints.

[0182] Corresponding to the multi-robot collaboration method in a dynamic environment provided in the above embodiments, this invention also provides a multi-robot collaboration system in a dynamic environment, such as... Figure 6 As shown, it includes: a task understanding module 601, a subtask generation module 602, and a subtask allocation module 603;

[0183] The task understanding module 601 is used to receive task description information; and use the task description information to generate a task relationship dependency graph represented by a directed acyclic graph (DAG) corresponding to the task. Each node in the DAG represents a subtask, and the edge between any two nodes represents the dependency relationship between the subtasks represented by the two nodes.

[0184] The subtask generation module 602 is used to generate a hierarchical DAG for each subtask based on the first environment description information of the current environment, the functional information of each robot and the DAG corresponding to the task, through a large language model LLMs. The hierarchical DAG represents the execution strategy of the subtask.

[0185] The subtask allocation module 603 is used to obtain the task plan of each robot based on the functional information and first state information of each robot, as well as the subtask execution strategy corresponding to each subtask; and to allocate the task plan of each robot to the robot so that each robot can execute the task plan.

[0186] Optionally, the task understanding module 601 is further configured to, when the task description information includes a task description in natural language representation, convert the task description in natural language representation into task description information in linear temporal logic (LTL) form after receiving the task description information.

[0187] Optionally, the subtask generation module 602 is specifically used to input the first environment description information, the functional information of each robot, and the DAG corresponding to the task into the large language model LLMs, and output the first execution strategy through the large language model LLMs; obtain the historical execution strategy of the historical task with the same task type as the current task, and input the historical execution strategy into the large language model LLMs, and adjust the first execution strategy through the large language model LLMs to obtain the second execution strategy; obtain the resource conflict constraint relationship and the time conflict constraint relationship, and input the resource conflict constraint relationship and the time conflict constraint relationship into the large language model LLMs, and adjust the second execution strategy through the large language model LLMs to obtain the hierarchical DAG corresponding to each subtask.

[0188] Optionally, the subtask allocation module 603 is specifically used to obtain the task plan for each robot based on the functional information and first state information of each robot, as well as the subtask execution strategy corresponding to each subtask, by using the branch and bound search method and the integer programming method, with minimizing the overall task completion time as the optimization objective.

[0189] Optionally, the subtask generation module 602 is further configured to, after assigning the task plan of each robot to the robot, obtain second environment description information when a first preset condition is met, wherein the first preset condition includes at least one of the following conditions: detecting a new operation object corresponding to the task type of the current task in the current environment, detecting a new resource type in the current environment, detecting a new resource of an existing resource type in the current environment, or detecting a new task type; the second environment description information is an updated environment description information compared to the first environment description information; based on the second environment description information, the functional information of each robot, and the updated DAG, a hierarchical DAG corresponding to each subtask is regenerated using a large language model LLMs.

[0190] Optionally, the subtask allocation module 603 is further configured to, after allocating the task plan of each robot to the robot, obtain the second state information of each robot when a second preset condition is met, wherein the second preset condition includes: no plan completion information is received from the robot executing the task plan within a preset time range, or a robot malfunction is detected, and the second state information is the updated state information compared with the first state information; based on the functional information and second state information of each robot, and the subtask execution strategy corresponding to each subtask, the updated task plan of each robot is obtained.

[0191] Optionally, the multi-robot collaborative system in this dynamic environment also includes:

[0192] The human-loop verification module is used to: 1) generate a directed acyclic graph (DAG) representing the task dependency graph for each task; 2) generate hierarchical DAGs for each subtask; 3) obtain task plans for each robot; and 4) receive and display the task execution progress information from the robots after assigning their task plans to them, allowing human operators to verify the task plans based on this information.

[0193] Optionally, the first environment description information includes: task object information of the operation object corresponding to the task type of the current task and resource information of resources in the environment; the robot's functional information indicates the types of operations that the robot can perform and the types of resources that it can operate.

[0194] This invention also provides an electronic device, such as... Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703, and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0195] Memory 703 is used to store computer programs;

[0196] The processor 701, when executing the program stored in the memory 703, implements the method steps of the multi-robot cooperation method in the above dynamic environment.

[0197] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0198] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0199] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0200] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0201] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of the multi-robot collaboration method in any of the above-described dynamic environments.

[0202] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the multi-robot collaboration method in any of the above embodiments under dynamic environments.

[0203] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0204] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0205] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system, electronic device, computer-readable storage medium, and computer program product embodiments are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0206] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for multi-robot collaboration in a dynamic environment, characterized in that, include: Receive task description information; Using the task description information, a task dependency graph represented by a directed acyclic graph (DAG) corresponding to the task is generated. Each node in the DAG represents a subtask, and the edge between any two nodes represents the dependency relationship between the subtasks represented by the two nodes. The first environmental description information of the current environment, the functional information of each robot and the DAG corresponding to the task are input into the large language model LLMs, and the first execution strategy is output through the large language model LLMs. Obtain the historical execution strategy of a historical task with the same task type as the current task, and input the historical execution strategy into the Large Language Model (LLMs). Adjust the first execution strategy through the Large Language Model (LLMs) to obtain the second execution strategy. Obtain resource conflict constraints and time conflict constraints, and input the resource conflict constraints and time conflict constraints into the Large Language Model (LLMs). Adjust the second execution strategy through the Large Language Model (LLMs) to obtain a hierarchical DAG corresponding to each subtask. The hierarchical DAG represents the execution strategy of the subtask. Based on the functional information and first state information of each robot, as well as the subtask execution strategy corresponding to each subtask, the task plan of each robot is obtained. Assign a task plan to each robot so that each robot can execute the task plan.

2. The method according to claim 1, characterized in that, When the task description information includes a task description represented in natural language, the method further includes, after receiving the task description information: The task description represented in natural language is converted into task description information in linear temporal logic (LTL) form.

3. The method according to claim 1, characterized in that, The process of obtaining the task plan for each robot based on its functional information and first state information, as well as the subtask execution strategy corresponding to each subtask, includes: Based on the functional information and first state information of each robot, as well as the subtask execution strategy corresponding to each subtask, the task plan of each robot is obtained by using the branch and bound search method and the integer programming method, with minimizing the overall task completion time as the optimization objective.

4. The method according to claim 1, characterized in that, After assigning the task plan for each robot to the robot, the method further includes: When the first preset condition is met, the second environment description information is obtained. The first preset condition includes at least one of the following conditions: a new operation object corresponding to the task type of the current task is detected in the current environment; a new resource type is detected in the current environment; a new resource of an existing resource type is detected in the current environment; a new task type is detected. The second environment description information is the updated environment description information compared with the first environment description information. Based on the second environmental description information, the functional information of each robot, and the updated DAG, the hierarchical DAG corresponding to each subtask is regenerated using large language models (LLMs).

5. The method according to claim 1, characterized in that, After assigning the task plan for each robot to the robot, the method further includes: When the second preset condition is met, the second state information of each robot is obtained. The second preset condition includes: no plan completion information is received from the robot that executes the task plan within a preset time range, or a robot malfunction is detected. The second state information is the updated state information compared with the first state information. Based on the functional information and second state information of each robot, as well as the subtask execution strategy corresponding to each subtask, the updated task plan for each robot is obtained.

6. The method according to claim 1, characterized in that, After generating the task dependency graph represented by the directed acyclic graph (DAG) corresponding to the task, the method further includes: displaying the DAG to allow a human operator to verify the DAG; After generating the hierarchical DAG corresponding to each subtask, the method further includes: displaying the hierarchical DAG corresponding to each subtask so that a human operator can verify the hierarchical DAG corresponding to each subtask. After obtaining the task plans for each robot, the method further includes: displaying the task plans for each robot to allow a human operator to verify the task plans for each robot; After assigning the task plan to each robot, the method further includes: receiving the task execution progress information fed back by the robot, and displaying the task execution progress information so that a human operator can verify the task plan of each robot based on the task execution progress information.

7. The method according to any one of claims 1 to 6, characterized in that, The first environment description information includes: task object information of the operation object corresponding to the task type of the current task and resource information of resources in the environment; the robot's functional information indicates the types of operations that the robot can perform and the types of resources that it can operate.

8. A multi-robot collaborative system in a dynamic environment, characterized in that, include: Task understanding module, subtask generation module, subtask allocation module; The task understanding module is used to receive task description information; Using the task description information, a task dependency graph represented by a directed acyclic graph (DAG) corresponding to the task is generated. Each node in the DAG represents a subtask, and the edge between any two nodes represents the dependency relationship between the subtasks represented by the two nodes. The subtask generation module is used to input the first environment description information of the current environment, the functional information of each robot, and the DAG corresponding to the task into a large language model LLMs, and output a first execution strategy through the large language model LLMs; obtain the historical execution strategy of a historical task with the same task type as the current task, and input the historical execution strategy into the large language model LLMs, and adjust the first execution strategy through the large language model LLMs to obtain a second execution strategy; obtain resource conflict constraint relationships and time conflict constraint relationships, and input the resource conflict constraint relationships and time conflict constraint relationships into the large language model LLMs, and adjust the second execution strategy through the large language model LLMs to obtain a hierarchical DAG corresponding to each subtask, wherein the hierarchical DAG represents the execution strategy of the subtask; The subtask allocation module is used to obtain the task plan for each robot based on the functional information and first state information of each robot, as well as the subtask execution strategy corresponding to each subtask. Assign a task plan to each robot so that each robot can execute the task plan.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-7.

Images