Multi-robot collaborative operation method and system

Abstract

The invention provides a multi-robot collaborative operation method and system, and the method comprises the steps: receiving an input instruction through a global agent, and uploading the input instruction to a cloud brain model; task-level disassembly is carried out on the instruction through the cloud brain model, a task topological graph comprising a plurality of sub-tasks is generated, and a robot body for executing each sub-task is determined; respectively transmitting each sub-task to a corresponding end-side cerebellum model through the global agent by a cloud brain model; after each end-side cerebellum model receives the subtask, continuing to call the cloud brain model to disassemble the subtask into a skill sequence, and controlling the corresponding robot body to execute the skill sequence through the end-side cerebellum model, the intelligent distribution mechanism solves the problem of uneven robot load caused by task distribution based on simple rules in a traditional system, and improves the overall efficiency of multi-robot collaborative operation.

Description

Technical Field

[0001] The present invention relates to the technical field of embodied intelligent robots, and in particular to a method and system for multi-robot collaborative operation. Background Art

[0002] With the continuous updating of embodied intelligent robot technology, embodied intelligent robots have been involved in all aspects of human life and are playing an increasingly important role in people's lives.

[0003] Traditional robot control systems typically adopt a layered architecture consisting of two main components: the cloud and the edge. The cloud is responsible for complex computations and global planning, while the edge is responsible for detailed execution control and real-time decision-making. Traditional robot control systems have significant limitations in task execution and multi-machine collaboration. These limitations include significant cloud processing latency, insufficient edge computing power that limits the robot's autonomous decision-making capabilities, and an inability to accurately understand complex human commands in task planning and command parsing. These issues collectively limit the overall performance of traditional robot control systems when faced with complex tasks and changing environments.

[0004] Especially in multi-robot collaborative work scenarios, due to the lack of effective collaborative planning mechanisms, task allocation is often based on simple rules, failing to intelligently allocate tasks based on task complexity and robot capabilities. This can lead to overloading some robots and leaving others idle. Furthermore, the lack of global topological monitoring during task execution prevents effective coordination of the movement sequences between robots, making task conflicts and incorrect execution sequences prone to occur. These issues not only limit the collaborative efficiency of multi-robot systems but also expose structural flaws in existing systems when handling complex collaborative tasks. Summary of the Invention

[0005] This invention provides a method and system for multi-robot collaborative operations, addressing existing shortcomings in multi-robot collaborative operations, such as cloud-based processing delays, insufficient client-side computing power, low multi-machine collaborative efficiency, and imperfect exception handling mechanisms. The method and system of this invention enable efficient task allocation, real-time monitoring, and flexible exception handling, significantly improving the efficiency and robustness of multi-robot collaborative operations.

[0006] The present invention provides a method for multi-robot collaborative operation, which is used in a multi-robot collaborative operation system. The multi-robot collaborative operation system includes a cloud-based brain model, a global intelligent agent, a terminal-side cerebellum model, and multiple robot bodies. Each robot body is deployed with a corresponding terminal-side cerebellum model, and the cloud-based brain model and the multiple terminal-side cerebellum models are respectively connected to the global intelligent agent. The method includes: Receive input instructions through the global agent and upload the input instructions to the cloud brain model; The cloud-based brain model is used to decompose the instructions at the task level, generate a task topology diagram including multiple subtasks, and determine the robot body that executes each subtask. The task topology diagram defines the execution order and logical relationship of each subtask; Each subtask is transmitted to the corresponding terminal cerebellum model through the global agent via the cloud brain model; After each end-side cerebellum model receives a subtask, it continues to call the cloud-side brain model to decompose the subtask into a skill sequence, and controls the corresponding robot body to execute the skill sequence through the end-side cerebellum model.

[0007] According to a multi-robot collaborative operation method provided by the present invention, the global agent receives input instructions and uploads the input instructions to a cloud brain model, specifically comprising: The global agent receives input instructions through the user interaction module, converts the input instructions into instruction text, and then uploads the instruction text to the cloud brain model.

[0008] According to a multi-robot collaborative operation method provided by the present invention, a cloud-based brain model is used to decompose instructions at the task level to generate a task topology diagram including multiple subtasks, specifically including: Perform semantic understanding of the instruction text using natural language processing technology through a cloud-based brain model to identify key information in the instruction text; Combined with the current scenario and historical task information, the context of the instruction text is analyzed based on the key information, the instruction text is broken down into multiple subtasks, and the corresponding task topology diagram is generated according to the logical relationship and execution order of the subtasks.

[0009] According to a method for multi-robot collaborative operation provided by the present invention, the global topological order of each subtask is monitored by the cloud-side brain model to ensure that the subtasks are executed in the order of execution; the topological order of each skill execution is monitored by the end-side cerebellum model to ensure that the skills are executed in the topological order.

[0010] According to a multi-robot collaborative operation method provided by the present invention, the skill sequence includes multiple skills. After each end-side cerebellum model receives a subtask, it continues to call the cloud-side brain model to decompose the subtask into a skill sequence, specifically including: After receiving the subtask, each end-side cerebellum model sends a request to the cloud-side brain model and uploads the current environment perception data and robot status information; The subtasks are analyzed by the cloud brain model and, combined with the environmental perception data and robot capabilities, the subtasks are broken down into corresponding skill sequences.

[0011] According to a multi-robot collaborative operation method provided by the present invention, the corresponding robot bodies are controlled by a terminal-side cerebellum model to execute the skill sequence, specifically including: Based on the multiple skills, the local path planning algorithm is called by the end-side cerebellum model, and combined with the local environment perception data to generate an initial path plan; According to the planned initial path, the corresponding robot body is controlled by the end-side cerebellum model to perform the skills in sequence.

[0012] According to a multi-robot collaborative operation method provided by the present invention, in the process of each end-side cerebellum model controlling the corresponding robot body to execute the skill sequence, the method includes: Monitor the execution of the path in real time through the end-to-end cerebellum model to detect whether any abnormalities are encountered; If an abnormal situation is encountered during the path execution, the path is replanned according to the current environmental perception data through the end-to-end cerebellum model; If the path replanning fails continuously, the current environmental perception data is uploaded to the cloud brain model via the global agent through the end-side cerebellum model, the path is replanned according to the current environmental perception data through the cloud brain model, and the replanned path is sent to the end-side cerebellum model via the global agent.

[0013] According to a multi-robot collaborative operation method provided by the present invention, after the corresponding robot bodies are controlled by the end-side cerebellum model to execute the skill sequence, the method further includes: The task completion status is fed back to the cloud brain layer via the global agent through the end-side cerebellum model, and the task completion status is fed back to the user via the global agent and user interaction module.

[0014] The present invention discloses a multi-robot collaborative operation system, which includes a cloud-based brain model, a global intelligent agent, a terminal-side cerebellum model, and multiple robot bodies. Each robot body is deployed with a corresponding terminal-side cerebellum model, and the cloud-based brain model and the multiple terminal-side cerebellum models are respectively connected to the global intelligent agent. The global agent is used to receive input instructions and upload the input instructions to the cloud brain model; The cloud brain model is used to perform task-level decomposition of instructions, generate a task topology diagram including multiple subtasks, and determine the robot body that executes each subtask. The task topology diagram defines the execution order and logical relationship of each subtask; The cloud-side brain model is used to transmit each subtask to the corresponding end-side cerebellum model via the global agent; The end-side cerebellum model is used to continue calling the cloud-side brain model to decompose the subtask into a skill sequence after receiving the subtask, and control the corresponding robot body to execute the skill sequence.

[0015] The present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for multi-robot collaborative operation as described above is implemented.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for collaborative operation of multiple robots.

[0017] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-described methods for collaborative operation of multiple robots.

[0018] The method and system for multi-robot collaborative operation provided by the present invention uses a cloud-based brain model to perform task-level decomposition of input instructions, generate a task topology diagram, and allocate tasks based on task complexity and robot capabilities, ensuring the rationality and efficiency of task allocation. The cloud-based brain model then transmits each subtask via a global agent to the corresponding end-to-end cerebellum model. Each end-to-end cerebellum model calls the cloud-based brain model to decompose the subtasks into skill sequences, and the end-to-end cerebellum model controls the corresponding robot body to execute the skill sequence. This intelligent allocation mechanism solves the problem of uneven robot load caused by simple rule-based task allocation in traditional systems, and improves the overall efficiency of multi-robot collaborative operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 It is a schematic diagram of the framework of the multi-robot collaborative operation system provided by the present invention.

[0021] Figure 2 It is a flow chart of the multi-robot collaborative operation method provided by the present invention.

[0022] Figure 3This is a schematic diagram of a specific scenario of multi-robot collaborative operation provided by the present invention.

[0023] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0025] For existing technologies, in multi-robot collaborative operations, traditional systems mainly face the following challenges: Cloud processing delay: The cloud is responsible for complex task planning and global monitoring, but processing delays are prominent, resulting in insufficient real-time execution of tasks.

[0026] Insufficient computing power on the edge: The computing power of edge devices (such as robots) is limited, making it difficult to make real-time decisions and plan complex tasks.

[0027] Unreasonable task allocation: Existing systems usually allocate tasks based on simple rules and are unable to intelligently allocate tasks based on task complexity and robot capabilities, resulting in some robots being overloaded and others being idle.

[0028] Imperfect exception handling mechanism: The existing system has limited exception handling capabilities, lacks hierarchical processing and dynamic adjustment capabilities, and has a low task success rate.

[0029] When delving deeper into complex task scenarios, the shortcomings of existing systems become even more apparent. When executing tasks in dynamic environments, robots need to possess real-time perception, rapid decision-making, and flexible adjustment capabilities. However, the existing system's cloud-based processing latency and insufficient on-device computing power make it difficult for robots to respond to environmental changes quickly, significantly compromising the real-time and accuracy of task execution. Furthermore, complex tasks often involve the coordinated execution of multiple subtasks and skills. The existing system's task planning and instruction parsing capabilities are insufficient to support this complexity, making it impossible to effectively decompose and allocate tasks, nor to ensure the topological order of task execution. The simplistic exception handling mechanism further renders the system helpless in the face of complex exceptions, preventing dynamic adjustment and optimization through intelligent algorithms.

[0030] For example, in warehousing and logistics scenarios, robots need to perceive environmental changes in real time, make rapid decisions, and flexibly adjust task execution paths. Traditional systems often struggle to meet these requirements. These issues collectively limit the application scope and effectiveness of existing robot control systems in complex task scenarios. There is an urgent need for a new multi-robot collaborative operation system that can overcome these limitations.

[0031] In order to overcome the limitations of existing technologies and meet the complex needs of multi-robot collaborative operations, the embodiments of the present invention achieve efficient task allocation, real-time monitoring and flexible exception handling through the collaborative work of the cloud-based brain model, the global intelligent agent and the terminal-side cerebellum model, significantly improving the efficiency and robustness of multi-robot collaborative operations.

[0032] Before delving into the specific implementation of the present invention, we first briefly introduce the key components and terminology of the system to better understand its working principles and innovations. These components include the cloud-based brain model, the global agent, the client-side cerebellum model, the robot body, subtasks, and skill sequences. Through the collaborative operation of these components, the present invention can effectively solve many problems in the existing technology and provide an efficient, flexible, and reliable solution for multi-robot collaborative operations.

[0033] Cloud Brain Model: The Cloud Brain Model is an intelligent module deployed in the cloud, responsible for complex task planning, global monitoring, and exception handling. Equipped with powerful computing capabilities and a global perspective, it decomposes input instructions into task-level components, generates a task topology, and assigns tasks to the appropriate robots. Furthermore, it monitors the global execution order of tasks, ensuring they are completed as planned and performing global optimizations when necessary.

[0034] Global Agent: The global agent is the coordination module in the system, serving as a bridge between the cloud-based brain model and the on-device cerebellum model. It receives user commands and uploads them to the cloud-based brain model. It also passes subtasks assigned by the cloud-based brain model to the corresponding on-device cerebellum model. The global agent also helps monitor task execution status and assists in handling abnormal situations.

[0035] On-device cerebellum model: The on-device cerebellum model is a local control module deployed on each robot, responsible for specific task execution and real-time decision-making. It receives subtasks from the global agent and calls the cloud-based brain model to break them down into skill sequences. The on-device cerebellum model controls the robot's execution of the skill sequence, monitors the order of skill execution, and, if an anomaly is encountered, attempts a local solution or reports it to the cloud-based brain model.

[0036] Robot Body: The robot body is the device that actually performs physical operations, possessing basic functions such as navigation, grasping, and carrying. Each robot body is deployed with a corresponding on-device cerebellum model, which receives tasks and executes specific skill sequences. The robot body perceives the environment through sensors and feeds this data back to the on-device cerebellum model.

[0037] Subtasks: Subtasks are independent units broken down from complex tasks. They are generated by the cloud-based brain model based on input instructions. Subtasks define the specific operations that the robot must complete and define their execution order and logical relationships with other subtasks within the task topology. Subtasks are the result of task-level decomposition and serve as the fundamental unit for robots to execute complex tasks.

[0038] Skill Sequences: Skill sequences are the specific execution steps of subtasks, further broken down by the cloud-based brain model and combining environmental perception data with the robot's capabilities. Skill sequences define the series of specific actions a robot must perform to complete a subtask, such as navigation, grasping, and carrying. Skill sequences, the result of skill-level decomposition, serve as a detailed guide for the robot to perform subtasks.

[0039] The following is a detailed introduction to the multi-robot collaborative operation system disclosed in an embodiment of the present invention. The multi-robot collaborative operation system of the embodiment of the present invention achieves efficient task allocation, real-time monitoring and flexible exception handling through a carefully designed architecture and module collaboration.

[0040] like Figure 1 As shown, the core components of the multi-robot collaborative operation system of an embodiment of the present invention include a cloud-based brain layer, a user interaction module, a global agent, an end-side cerebellum model, and a robot body. These components work together to ensure the smooth completion of complex tasks.

[0041] The cloud-based brain layer serves as the system's intelligent core, responsible for task planning and global monitoring. The cloud-based brain model (RoboBrain) leverages its powerful visual model and knowledge graph to semantically understand input commands and perform task-level decomposition, generating a detailed task topology. This task topology not only defines the execution order and logical relationships of subtasks but also provides a foundation for subsequent task allocation. The collaboration module and task-level exception handling module ensure coordination and robustness during task execution, enabling timely strategy adjustments even in the event of anomalies. The data storage and update module maintains configuration files for scenarios, robots, and tools, providing real-time data support for the system.

[0042] The user interaction module is the bridge between the system and the user. It receives user voice or text commands and converts them into text. This module ensures that users can interact with the system conveniently and is the starting point of the entire system process.

[0043] The Global Agent, as the intermediary layer, plays a crucial role. It not only receives command text from the user interaction module and uploads it to the cloud-based brain layer, but also bridges the gap between the cloud-based brain model and the client-side cerebellum model. The Global Agent ensures efficient transmission of commands and data between the cloud and client, while also assisting in monitoring task execution status and promptly addressing exceptions.

[0044] The end-to-end cerebellum model, deployed on each robot, serves as the local control center for task execution. It includes real-time decision-making, skill execution, and environmental perception modules, enabling the robot to make decisions and execute corresponding skills based on real-time information. The anomaly detection and skill-level exception handling modules monitor the skill execution process, promptly identifying and addressing anomalies to ensure smooth task execution. The scene cache data module stores scene maps and related data, providing environmental awareness support for the robot.

[0045] The robot itself, as the device that actually performs physical operations, possesses basic functions such as navigation, grasping, and carrying. It receives tasks from the on-device cerebellum model and executes specific skill sequences. Sensors on the robot itself can detect environmental changes and feed this data back to the on-device cerebellum model for path planning and real-time decision-making.

[0046] In the system's workflow, users enter commands through the user interaction module. These commands are converted into text and passed to the global agent. The global agent then uploads the commands to the cloud-based brain layer, which decomposes them into task-level components, generates a task topology, and assigns tasks based on task complexity and robot capabilities. The assigned subtasks are then passed through the global agent to the corresponding on-device cerebellum model. After receiving the subtasks, the on-device cerebellum model calls the cloud-based brain layer to further decompose the subtasks into skill sequences and control the robot to execute these skills.

[0047] During execution, the on-device cerebellum model monitors the execution of the path in real time, detecting and handling exceptions. If an unresolvable exception is encountered, the on-device cerebellum model reports it to the cloud-based brain layer, which then performs global optimization. Upon task completion, the on-device cerebellum model reports the task completion status to the global agent, which then reports the status back to the cloud-based brain layer and the user.

[0048] The multi-robot collaborative operation system of this embodiment addresses many of the shortcomings of traditional systems through global planning at the cloud-based brain layer, coordination of global agents, and local execution of the client-side cerebellum model. This system not only improves the efficiency and accuracy of task execution but also enhances the system's robustness and adaptability, providing an efficient, flexible, and reliable solution for multi-robot collaborative operations. This system is applicable to a variety of scenarios, such as warehousing and logistics, and industrial production, significantly improving operational efficiency and quality.

[0049] Specifically, as the on-device cerebellum model controls the robot to execute a skill sequence, it uses sensors (such as lidar and cameras) to monitor the robot's progress along the path in real time, detecting obstacles or other anomalies. The robot uses sensors to detect environmental changes, such as the appearance of dynamic obstacles or the movement of obstacles along the path, and feeds this data back to the on-device cerebellum model.

[0050] The end-side cerebellum model uses sensor data to detect whether the robot deviates from the planned path. If it detects that the path deviation exceeds the preset threshold, the exception handling mechanism is triggered; or the end-side cerebellum model detects whether new obstacles appear on the path, such as other robots, dropped cargo, etc.; or the end-side cerebellum model monitors the progress and status of task execution, such as whether the skill execution is successful or whether it has timed out.

[0051] If an obstacle or other anomaly is detected on the path, the lateral cerebellum model first attempts a local solution, such as replanning the path to circumvent the obstacle. This local path replanning is typically based on algorithms such as Rapidly Expanding Random Trees (RRT) or A*. If a skill fails, the lateral cerebellum model attempts to retry the skill, typically with a set number of retries (e.g., three).

[0052] If the on-device cerebellum model fails to resolve the exception after multiple attempts, it reports the exception (including current environmental perception data and robot status information) to the cloud-based brain layer. Upon receiving the exception report, the cloud-based brain layer leverages its powerful computing power and global perspective to perform global optimization. For example, it may replan the path to avoid obstacles or adjust task allocation to optimize resource utilization. The cloud-based brain layer then sends the optimized path or task adjustment plan to the on-device cerebellum model to ensure continued task execution.

[0053] After completing a task, the end-to-end cerebellum model will feedback the task completion status (such as success, failure, partial completion, etc.) to the global intelligent agent. The feedback information includes detailed information such as the time of task execution, path execution status, encountered exceptions, and processing results.

[0054] The global agent aggregates the task completion status of all robot entities to form an overall task execution report. This global agent feeds task completion status and detailed information back to the cloud-based brain layer, ensuring the cloud has comprehensive information on task execution. The global agent also reports task completion status to users via the user interaction module, allowing them to view task execution results and detailed information through the interactive interface.

[0055] Furthermore, the cloud brain layer stores detailed information and feedback data on task execution for subsequent task optimization and system improvement. It can also update scene configuration, robot configuration, and skill library information based on task execution status and feedback information to ensure continuous optimization of the system.

[0056] Through this mechanism, the system of this embodiment of the present invention can ensure efficient and robust task execution. The real-time monitoring and local processing capabilities of the end-to-end cerebellum model, combined with the global optimization capabilities of the cloud-based brain layer, enable the system to flexibly respond to various complex situations and ensure the successful completion of tasks. Furthermore, the feedback mechanism after task completion ensures that users can promptly understand the task execution status and provides data support for the continuous improvement of the system.

[0057] Furthermore, the system design of the embodiment of the present invention fully considers scalability, supporting the easy addition of new robot bodies or updating of the functions of existing robot bodies. Through simple configuration changes, the new robot body can be quickly integrated into the system and participate in task execution. This scalability not only reduces the maintenance cost of the system, but also improves the long-term availability of the system. Specifically, the new robot body needs to have basic hardware devices such as processors, memory, sensors (such as lidar, cameras) and actuators (such as robotic arms, wheels), and be able to communicate with the system through common network protocols (such as TCP / IP, MQTT). In terms of software, the new robot body needs to install an OS that supports robot operations (such as ROS) and other necessary drivers to ensure that the hardware devices can work properly and are compatible with the system.

[0058] After hardware and software preparation is complete, the new robot connects to the system via the network and registers with the cloud-based brain layer. During registration, the system updates the robot configuration file, recording the new robot's ID, type, capabilities, and other information to ensure it can be included in task assignments. The global agent also updates its configuration accordingly to identify and manage the new robot.

[0059] After a new robot joins the system, it can immediately participate in task execution. The cloud-based brain layer considers the new robot's capabilities and status when generating the task topology and assigns it appropriate tasks. Upon receiving a subtask, the new robot invokes the cloud-based brain model to break it down into skill sequences and execute specific actions based on these sequences. Throughout this process, the system's real-time monitoring and adjustment mechanisms ensure the new robot's smooth integration and collaborative operation with other robots.

[0060] Furthermore, the system of this embodiment has robust fault tolerance, allowing it to continue operating even if some components fail. For example, if a robot fails, the system automatically reallocates its tasks to other available robots, ensuring that the overall mission is not impacted. This fault-tolerance mechanism significantly improves system reliability and mission success rates.

[0061] After a detailed introduction to the multi-robot collaborative operation system of the present invention, we will next explore how the system implements its functions. This will be explained in detail through the methods of the present invention's embodiments, which cover the entire process from task reception, breakdown, and allocation to execution and feedback. Through these methods and steps, the system can efficiently handle complex tasks and ensure the smooth operation of multi-robot collaborative operations. These methods and steps not only demonstrate the system's functional design but also illustrate its operational details in practical applications.

[0062] Figure 2 This is one of the flow charts of the multi-robot collaborative operation method provided by the present invention, such as Figure 2 As shown, the method includes the following: Step 201: Receive input instructions through the global agent and upload the input instructions to the cloud brain model.

[0063] The global agent serves as the system's coordination hub, responsible for receiving input commands from the user interaction module. Users can issue commands via voice or text, which the user interaction module converts into a unified text format. Upon receiving the command text, the global agent preprocesses it to ensure its completeness and accuracy before uploading it to the cloud-based brain model. This process ensures that user commands are accurately received and processed by the system and serves as the starting point for the entire task execution process.

[0064] Step 202: Decompose the instructions at the task level through the cloud brain model to generate a task topology diagram including multiple subtasks, and determine the robot body that executes each subtask. The task topology diagram defines the execution order and logical relationship of each subtask.

[0065] After receiving the instruction text, the cloud brain model first uses natural language processing (NLP) technology to understand the semantics of the instruction. This process includes the following key steps: (1) Instruction analysis: Lexical analysis: Breaking down instruction text into words or phrases, identifying verbs, nouns, and other key elements.

[0066] Syntactic analysis: Analyze the grammatical structure of the instruction and determine the subject, predicate, and object of the sentence to understand the logical structure of the instruction.

[0067] Semantic understanding: Combining context and domain knowledge to understand the intent and goal of a command. For example, the command "Transfer all electronic products from shelves in Area A to the quality inspection table in Area B" is interpreted as a handling task involving the transfer of goods from Area A to Area B.

[0068] (2) Extraction of key information: Action recognition: Identify the actions involved in instructions, such as "carry", "grab", "place", etc.

[0069] Object recognition: Identify objects involved in instructions, such as "electronic products", "shelves", "quality inspection tables", etc.

[0070] Position identification: Identify the position information involved in the instruction, such as "Area A", "Area B", etc.

[0071] Time constraints: Identify time limits or other constraints involved in the instruction, such as "complete by 5 pm."

[0072] (3) Task breakdown: Task decomposition: Break down complex tasks into multiple subtasks. For example, the above-mentioned transport task can be broken down into the following subtasks: Subtask 1: Navigate to shelf in area A; Subtask 2: Grasping electronic products; Subtask 3: Transport to the quality inspection station in Area B; Subtask 4: Place electronic products.

[0073] Subtask definition: Each subtask has clear goals and execution requirements to ensure that the robot can accurately understand and execute them.

[0074] (4) Task topology generation: Dependency analysis: Analyze the dependencies and execution order between subtasks. For example, picking up electronic products must be done after navigating to the shelf in Area A, and placing electronic products must be done after transporting them to the quality inspection station in Area B.

[0075] Topology Construction: Based on the dependencies between subtasks, a task topology is constructed. The task topology is a directed acyclic graph (DAG), where nodes represent subtasks and edges represent dependencies between subtasks. The topology ensures that tasks are executed in the correct order, avoiding logical errors or execution conflicts.

[0076] (5) Task allocation: Robot capability assessment: Based on the capabilities of the robot itself (such as load capacity, navigation accuracy, operation accuracy, etc.), evaluate the type of subtask that each robot is suitable for performing.

[0077] Task allocation strategy: Intelligent algorithms (such as genetic algorithms and ant colony algorithms) are used to allocate subtasks to ensure rationality and efficiency. For example, robots with greater load capacity can be assigned to heavy lifting tasks, while robots with higher navigation accuracy can be assigned to tasks requiring precise path planning.

[0078] Resource optimization: Considering the current position and load of the robot body, optimize the task allocation to avoid overloading some robots while others are idle.

[0079] Through the above steps, the cloud-based brain model can break down complex user instructions into multiple executable subtasks and generate a clear task topology, ensuring the logical order and efficiency of task execution. At the same time, through reasonable task allocation, each subtask is ensured to be executed by the most suitable robot, thereby improving the success rate and efficiency of overall task execution.

[0080] Step 203: Each subtask is transmitted to the corresponding terminal cerebellum model through the global agent via the cloud-side brain model.

[0081] After completing task-level decomposition and generating a task topology, the cloud-based brain model packages each subtask and its associated information (such as execution requirements and dependencies) into data packets. These packets contain detailed descriptions of the subtasks, ensuring that the client-side cerebellum model can accurately understand and execute them.

[0082] The cloud-based brain model determines the target robot for each subtask based on the task assignment results. It associates the subtask data packet with the target robot's identifier to ensure that the data packet is accurately delivered to the corresponding end-to-end cerebellum model.

[0083] The global agent plays a coordination and delivery role. It receives subtask data packets from the cloud-based brain model. It verifies the integrity and accuracy of the packets to ensure no data loss or corruption. Based on the target robot's identifier, the global agent delivers the subtask data packets to the corresponding end-to-end cerebellum model. It uses efficient communication protocols (such as TCP / IP and MQTT) to ensure reliable and real-time data transmission.

[0084] The global agent monitors the delivery process of the subtasks to ensure that each data packet reaches the target end-side cerebellum model accurately. If a delivery failure is detected, it will automatically retry or take other remedial measures.

[0085] The end-to-end cerebellar model receives a subtask data packet from the global agent. It parses the data packet, extracts the subtask details, and confirms receipt. After successfully receiving the subtask, the end-to-end cerebellar model sends a confirmation message to the global agent, indicating successful receipt. The global agent records the confirmation information to ensure the integrity of the task transfer process.

[0086] During task transfers, the global agent schedules tasks based on the priority of the subtasks. High-priority tasks are prioritized, ensuring timely execution of critical tasks. Furthermore, the global agent monitors the robot load in real time and dynamically adjusts task transfer strategies. If a robot is overloaded, it will transfer some tasks to other available robots to ensure overall system efficiency.

[0087] In addition, the global agent implements data encryption and authentication during task delivery to ensure the security and integrity of task data. This prevents data from being tampered with or stolen during transmission.

[0088] Through the above steps, step 203 ensures that the subtasks can be efficiently and accurately transferred from the cloud-side brain model to the corresponding end-side cerebellum model, laying the foundation for subsequent skill-level disassembly and task execution.

[0089] Step 204: After each end-side cerebellum model receives a subtask, it continues to call the cloud-side brain model to decompose the subtask into a skill sequence, and controls the corresponding robot body to execute the skill sequence through the end-side cerebellum model.

[0090] Specifically, after receiving a subtask, the client-side cerebellum model first sends a skill sequence decomposition request to the cloud-based brain model. This request includes detailed subtask information, current environmental perception data (such as sensor data from lidar and cameras), and the robot's state information (such as position, payload, and battery level). This data helps the cloud-based brain model more accurately generate skill sequences.

[0091] After receiving the request, the cloud-based brain model further analyzes the subtask, combining its objectives, environmental perception data, and the robot's capabilities. It uses path planning algorithms (such as A* and RRT) and task planning algorithms to generate a specific skill sequence. This skill sequence defines the specific actions the robot must perform to complete the subtask, such as navigation, grasping, carrying, and placing.

[0092] The cloud brain model also considers the priority and time constraints of task execution, optimizes the execution order of skill sequences, and ensures that tasks can be completed efficiently.

[0093] The cloud-based brain model sends the generated skill sequence to the client-side cerebellum model. The skill sequence includes detailed parameters for each skill, such as the navigation target coordinates, grasping force and angle, and handling speed.

[0094] After receiving the skill sequence, the cerebellum model on the end side controls the robot to execute each skill in sequence. It calls on local control algorithms (such as PID controllers) and actuators (such as robotic arms and wheels) to ensure that the robot can accurately complete each action.

[0095] During execution, the lateral cerebellum model monitors the execution of tasks in real time, ensuring that skills are executed in sequence and making fine-tuning adjustments based on environmental changes. For example, if a slight path deviation is encountered during navigation, the lateral cerebellum model will adjust the robot's direction to ensure it returns to the correct path.

[0096] The on-device cerebellum model uses sensors to monitor task execution in real time and detect any anomalies (such as blocked paths, dropped cargo, etc.). If an anomaly is encountered, the on-device cerebellum model first attempts a local solution, such as replanning the path or adjusting the grasping strategy.

[0097] If the local solution fails to resolve the problem, the on-device cerebellum model reports the anomaly (including current environmental perception data and robot status information) to the cloud-based brain model. Upon receiving the anomaly report, the cloud-based brain model performs global optimization, generates a new skill sequence, and sends it to the on-device cerebellum model.

[0098] After the skill sequence is completed, the client-side cerebellum model feeds back the task completion status to the global agent. This feedback includes detailed information such as the task execution result (e.g., success, failure, partial completion), execution time, any exceptions encountered, and the handling results.

[0099] The global agent summarizes the task completion status of all robots, generates an overall task execution report, and feeds the report back to the cloud-based brain layer. The cloud-based brain layer updates the task topology based on this feedback, ensuring that the global order and logical relationships of the tasks are maintained.

[0100] Finally, the global agent feeds back the task completion status to the user through the user interaction module, and the user can view the task execution results and detailed information through the interactive interface.

[0101] Through the above steps, step 204 ensures that subtasks can be efficiently broken down into specific skill sequences and accurately executed by the robot itself. This mechanism not only improves the efficiency and accuracy of task execution, but also enhances the robustness and adaptability of the system, ensuring the smooth operation of multi-robot collaborative operations.

[0102] The multi-robot collaborative operation method provided by the embodiment of the present invention uses a cloud-based brain model to perform task-level decomposition of input instructions, generate a task topology diagram, and allocate tasks based on task complexity and robot capabilities to ensure the rationality and efficiency of task allocation. The cloud-based brain model then transmits each subtask via the global intelligent agent to the corresponding end-side cerebellum model. Each end-side cerebellum model calls the cloud-based brain model to decompose the subtasks into skill sequences, and the end-side cerebellum model controls the corresponding robot body to execute the skill sequence. This intelligent allocation mechanism solves the problem of uneven robot load caused by simple rule-based task allocation in traditional systems, and improves the overall efficiency of multi-robot collaborative operation.

[0103] To ensure the efficiency and accuracy of multi-robot collaborative operations, the method of the embodiment of the present invention not only covers the basic task execution process, but also further improves the performance of the system through a series of monitoring and optimization mechanisms. During the execution of step 204, the system's monitoring and exception handling capabilities are particularly critical.

[0104] First, the cloud-based brain model continuously monitors the global topological order of each subtask, ensuring that subtasks are executed according to the predetermined logic and sequence. This global monitoring mechanism promptly detects deviations or conflicts during task execution and makes appropriate adjustments. Simultaneously, the client-side cerebellum model also monitors the topological order of each skill execution, ensuring that skills are executed in the correct order. This two-tiered monitoring mechanism provides a strong guarantee for the stable operation of the system.

[0105] For path planning, the on-device cerebellum model invokes a local path planning algorithm based on the skill sequence, combining it with real-time environmental perception data to generate an initial path plan. This local path planning rapidly responds to environmental changes, ensuring the robot's flexibility and adaptability when performing tasks. Based on the planned initial path, the on-device cerebellum model controls the robot to sequentially execute each skill in the skill sequence, ensuring successful task completion.

[0106] However, in a complex dynamic environment, the robot may encounter various abnormal situations when executing path planning, such as blocked paths or obstacles. To this end, the end-side cerebellum model has the ability to monitor the execution of the path in real time and can detect abnormal situations in a timely manner. When encountering an abnormality, the end-side cerebellum model first attempts to replan the path based on the current environmental perception data. If the local replanning fails continuously, the end-side cerebellum model uploads the current environmental perception data to the cloud brain model via the global intelligent agent. The cloud brain model performs global optimization processing based on the uploaded data, generates a new path plan, and sends the replanned path to the end-side cerebellum model to ensure that the task can continue to be executed.

[0107] This mechanism enables the system to flexibly adjust its strategies when encountering complex situations, ensuring successful task completion. This end-cloud collaborative path planning and exception handling mechanism not only improves the success rate of task execution but also enhances the system's robustness and adaptability. Ultimately, the end-side cerebellum model feeds back the task completion status to the global agent, which then feeds back the status to the cloud-side brain layer and the user, forming a complete closed-loop control process.

[0108] In order to facilitate understanding of the solution of the embodiment of the present invention, a specific application scenario is used as an example for schematic description. The application scenario is a restaurant scenario. Figure 3 As shown, in a restaurant scenario, the multi-robot collaborative operation system of an embodiment of the present invention can efficiently manage multiple robots to ensure that the entire process from guest reception to food delivery is smooth and unobstructed.

[0109] Guests place their orders in the restaurant dining area using interactive terminals (such as touch screens or voice devices). The user interaction module receives these orders and converts them into text. This text is then passed to the global agent, serving as the starting point for the entire task processing process.

[0110] The global agent uploads the command text to the cloud-based brain model. The cloud-based brain model uses natural language processing technology to parse the command and identify key information, such as dish name, quantity, and special requirements. Combining the hotel's scenario configuration (dining area table layout, kitchen workspace, etc.) and historical task information, the cloud-based brain model breaks down the complex task into multiple subtasks and generates a task topology diagram, defining the execution order and logical relationships of the subtasks.

[0111] Subtasks may include: The robot body Agent-1 receives guests in the dining area and guides them to their seats.

[0112] The robot body Agent-2 interacts with guests in the dining area and records order information.

[0113] The robot Agent-3 sorts ingredients in the kitchen.

[0114] The robot body Agent-4 prepares meals in the kitchen area and delivers the dishes to the designated delivery point.

[0115] The cloud-based brain model intelligently assigns tasks based on the complexity of the subtasks and the robot's capabilities. For example, robots with navigation and interaction capabilities are assigned to reception and ordering tasks, while robots with operational precision are assigned to serving meals.

[0116] The global agent passes the subtask to the corresponding on-device cerebellum model. After receiving the subtask, the on-device cerebellum model calls the cloud-based brain model to further decompose the subtask into a skill sequence. For example, the meal delivery task of Agent-4 is broken down into specific skills, such as navigating to the kitchen to pick up the food, grabbing the food, transporting it to the designated table in the dining area, and placing the food.

[0117] The cloud-based brain model combines real-time environmental perception data (such as the layout of the kitchen and dining area, the current location of other robots, etc.) and robot capabilities to generate a detailed skill sequence and send it to the end-side cerebellum model.

[0118] The cerebellum model on the end side controls the robot to execute a skill sequence. For example, the robot Agent-4 performs the food delivery task according to the skill sequence: Navigate to the kitchen to pick up food: Use path planning algorithms to avoid other robots and obstacles.

[0119] Grabbing food: Adjust the gripping force according to the shape and weight of the food.

[0120] Transport to the designated table in the dining area: adjust the route in real time to ensure smooth arrival.

[0121] Place dishes: Place them precisely to avoid collisions.

[0122] During execution, the lateral cerebellum model monitors the task execution in real time to ensure that the skills are executed in order. For example, if the robot body Agent-4 encounters an obstacle during transportation, it will adjust the path based on real-time perception data to ensure the successful completion of the task.

[0123] If Agent-4 encounters an unresolved exception during execution (such as a persistently blocked path), the on-device cerebellum model reports the exception (including current environmental perception data and robot status information) to the cloud-based brain model. The cloud-based brain model then replans the path and sends the new path to Agent-4's corresponding on-device cerebellum model to ensure continued mission execution.

[0124] After the task is completed, the client-side cerebellum model feeds back the task completion status to the global agent. The global agent aggregates the task completion status of all robots, generates an overall task execution report, and feeds this report back to the cloud-based brain layer. The cloud-based brain layer updates the task topology based on this feedback, ensuring that the global order and logical relationships of the tasks are maintained.

[0125] Finally, the global agent feeds back the task completion status to the guest through the user interaction module, and the guest can check the order progress and delivery status through the interactive interface.

[0126] Through the above process, the system of the present invention realizes efficient and flexible multi-robot collaborative operation in the hotel scene, improving service efficiency and customer experience.

[0127] Figure 4 An example of a physical structure diagram of an electronic device is shown below. Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 may call logic instructions in the memory 430 to execute a method for multi-robot collaborative operation, which includes: receiving input instructions through the global agent and uploading the input instructions to a cloud-based brain model; performing task-level decomposition of the instructions through the cloud-based brain model to generate a task topology graph including multiple subtasks, and determining a robot body to execute each subtask, wherein the task topology graph defines the execution order and logical relationship of each subtask; transmitting each subtask through the cloud-based brain model to the corresponding end-side cerebellum model via the global agent; after each end-side cerebellum model receives a subtask, continuing to call the cloud-based brain model to decompose the subtask into a skill sequence, and controlling the corresponding robot body to execute the skill sequence through the end-side cerebellum model.

[0128] Furthermore, the logic instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0129] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multi-robot collaborative operation method provided by the above methods, which includes: receiving input instructions through the global intelligent agent and uploading the input instructions to the cloud brain model; performing task-level decomposition of the instructions through the cloud brain model to generate a task topology diagram including multiple subtasks, and determining the robot body that executes each subtask, wherein the task topology diagram defines the execution order and logical relationship of each subtask; transmitting each subtask to the corresponding end-side cerebellum model via the global intelligent agent through the cloud brain model; after each end-side cerebellum model receives the subtask, continue to call the cloud brain model to decompose the subtask into a skill sequence, and control the corresponding robot body to execute the skill sequence through the end-side cerebellum model.

[0130] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for multi-robot collaborative operation provided by the above-mentioned methods, the method comprising: receiving input instructions through the global intelligent agent, and uploading the input instructions to a cloud brain model; performing task-level decomposition of the instructions through the cloud brain model, generating a task topology diagram including multiple subtasks, and determining a robot body to execute each subtask, wherein the task topology diagram defines the execution order and logical relationship of each subtask; transmitting each subtask to the corresponding end-side cerebellum model via the global intelligent agent through the cloud brain model; after each end-side cerebellum model receives the subtask, continuing to call the cloud brain model to decompose the subtask into a skill sequence, and controlling the corresponding robot body to execute the skill sequence through the end-side cerebellum model.

[0131] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0132] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for multi-robot collaborative operation, characterized in that: A system for multi-robot collaborative operation, the multi-robot collaborative operation system comprising a cloud-based brain model, a global intelligent agent, a terminal-side cerebellum model, and multiple robot bodies, each robot body being deployed with a corresponding terminal-side cerebellum model, and the cloud-based brain model and the multiple terminal-side cerebellum models being respectively connected to the global intelligent agent; The method comprises: Receive input instructions through the global agent and upload the input instructions to the cloud brain model; The cloud-based brain model is used to decompose the instructions at the task level, generate a task topology diagram including multiple subtasks, and determine the robot body that executes each subtask. The task topology diagram defines the execution order and logical relationship of each subtask; Each subtask is transmitted to the corresponding terminal cerebellum model through the global agent via the cloud brain model; After each end-side cerebellum model receives a subtask, it continues to call the cloud-side brain model to decompose the subtask into a skill sequence, and controls the corresponding robot body to execute the skill sequence through the end-side cerebellum model.

2. The multi-robot collaborative operation method according to claim 1, characterized in that: Receiving input instructions through the global agent and uploading the input instructions to the cloud brain model specifically includes: The global agent receives input instructions through the user interaction module, converts the input instructions into instruction text, and then uploads the instruction text to the cloud brain model.

3. The method for multi-robot collaborative operation according to claim 2, characterized in that: The cloud-based brain model decomposes the instructions at the task level and generates a task topology diagram consisting of multiple subtasks, including: Perform semantic understanding of the instruction text using natural language processing technology through a cloud-based brain model to identify key information in the instruction text; Combined with the current scenario and historical task information, the context of the instruction text is analyzed based on the key information, the instruction text is broken down into multiple subtasks, and the corresponding task topology diagram is generated according to the logical relationship and execution order of the subtasks.

4. The method for multi-robot collaborative operation according to claim 1, characterized in that: Monitoring the global topological order of each subtask through the cloud brain model to ensure that the subtasks are executed in the execution order; The topological order of each skill execution is monitored by the telencephalic cerebellar model to ensure that the skills are performed in a topological order.

5. The multi-robot collaborative operation method according to claim 1, characterized in that: The skill sequence includes a plurality of skills; After receiving the subtask, each client-side cerebellum model continues to call the cloud-side brain model to decompose the subtask into a skill sequence, specifically including: After receiving the subtask, each end-side cerebellum model sends a request to the cloud-side brain model and uploads the current environment perception data and robot status information; The subtasks are analyzed by the cloud brain model and, combined with the environmental perception data and robot capabilities, the subtasks are broken down into corresponding skill sequences.

6. The method for multi-robot collaborative operation according to claim 5, characterized in that: Controlling the corresponding robot body to execute the skill sequence through the terminal cerebellum model specifically includes: Based on the multiple skills, the local path planning algorithm is called by the end-side cerebellum model, and combined with the local environment perception data to generate an initial path plan; According to the planned initial path, the corresponding robot body is controlled by the end-side cerebellum model to perform the skills in sequence.

7. The multi-robot collaborative operation method according to claim 1 or 6, characterized in that: In the process of each end-side cerebellum model controlling the corresponding robot body to execute the skill sequence, the method includes: Monitor the execution of the path in real time through the end-to-end cerebellum model to detect whether any abnormalities are encountered; If an abnormal situation is encountered during the path execution, the path is replanned according to the current environmental perception data through the end-to-end cerebellum model; If the path replanning fails continuously, the current environmental perception data is uploaded to the cloud brain model via the global agent through the end-side cerebellum model, the path is replanned according to the current environmental perception data through the cloud brain model, and the replanned path is sent to the end-side cerebellum model via the global agent.

8. The multi-robot collaborative operation method according to claim 1, characterized in that: After the corresponding robot body is controlled by the end-side cerebellum model to execute the skill sequence, the method further includes: The task completion status is fed back to the cloud brain layer via the global agent through the end-side cerebellum model, and the task completion status is fed back to the user via the global agent and user interaction module.

9. A multi-robot collaborative operation system, characterized in that: The multi-robot collaborative system includes a cloud-based brain model, a global agent, a terminal-side cerebellum model, and multiple robot bodies. Each robot body is deployed with a corresponding terminal-side cerebellum model, and the cloud-based brain model and the multiple terminal-side cerebellum models are respectively connected to the global agent. The global agent is used to receive input instructions and upload the input instructions to the cloud brain model; The cloud brain model is used to perform task-level decomposition of instructions, generate a task topology diagram including multiple subtasks, and determine the robot body that executes each subtask. The task topology diagram defines the execution order and logical relationship of each subtask; The cloud-side brain model is used to transmit each subtask to the corresponding end-side cerebellum model via the global agent; The end-side cerebellum model is used to continue calling the cloud-side brain model to decompose the subtask into a skill sequence after receiving the subtask, and control the corresponding robot body to execute the skill sequence.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for multi-robot collaborative operation as described in any one of claims 1 to 8 is implemented.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for multi-robot collaborative operation according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for multi-robot collaborative operation according to any one of claims 1 to 8 is implemented.

Images