A multi-agent cooperative control system and method, and a storage medium

Abstract

The application provides a multi-agent cooperative control system and method and a storage medium. The system comprises a cloud planning layer, an edge cooperation layer and an agent execution layer. The cloud planning layer is configured to assign a corresponding subtask to each agent based on state information of each agent in the agent execution layer, and send all subtasks to the edge cooperation layer. The edge cooperation layer is configured to determine a control strategy based on abnormal information when an abnormality occurs in the execution of the subtask sent by the edge cooperation layer by the agent execution layer, and send the control strategy to the agent execution layer. The agent execution layer is configured to perform cooperative control of the multi-agent according to the control strategy.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a multi-agent collaborative control system, method, and storage medium. Background Technology

[0002] Global decision-making and optimization via the cloud suffer from communication latency, failing to meet the real-time control requirements of collaborative control. Decision-making via edge devices based on local information struggles to achieve global optimization for complex tasks, and the need for multiple communications between agents to reach consensus also hinders the real-time control capabilities required for collaborative control. Summary of the Invention

[0003] This application provides a multi-agent cooperative control system, method, and storage medium that can meet the real-time control requirements of cooperative control and improve the real-time performance of cooperative control.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a multi-agent cooperative control system, which includes: a cloud planning layer, an edge collaboration layer, and an agent execution layer; wherein... The cloud planning layer is used to assign corresponding subtasks to each agent based on the state information of each agent in the agent execution layer, and send all subtasks to the edge collaboration layer; The edge collaboration layer is used to determine a control strategy based on the anomaly information when an anomaly is detected in the execution of a subtask sent by the edge collaboration layer by the agent execution layer, and then send the control strategy to the agent execution layer. The agent execution layer is used to perform collaborative control of multiple agents according to the control strategy.

[0005] In this embodiment, a three-layer collaborative control architecture consisting of a cloud planning layer, an edge collaboration layer, and an agent execution layer is constructed. The cloud planning layer is responsible for assigning corresponding sub-tasks to each agent for macro-level task decomposition and planning. The edge collaboration layer monitors the agent execution layer and determines control strategies based on anomaly information when anomalies occur, thus relieving the real-time control pressure on the cloud planning layer. Real-time control is then performed by the edge collaboration layer. Through architectural innovation, millisecond-level real-time control (handled by the edge collaboration layer) is decoupled from complex global planning (handled by the cloud planning layer), which satisfies the real-time control requirements of collaborative control and improves its real-time performance.

[0006] Furthermore, the subtask is a collaborative performance; the edge collaboration layer is also used to parse the subtask to obtain multiple instructions to be executed and the corresponding issuance time of each instruction to be executed, and to send the corresponding instruction to be executed to the agent execution layer at the issuance time; the agent execution layer is used to control the corresponding agent to execute the instruction to be executed when it receives the instruction to be executed.

[0007] Based on the above technical means, the edge collaboration layer relieves the real-time control pressure of the cloud planning layer. Even if the cloud planning layer network is temporarily interrupted, the edge collaboration layer can still rely on the locally cached instruction scripts to continue to command the intelligent agent to complete the predetermined task for a period of time, avoiding anomalies due to network interruption.

[0008] Furthermore, the edge collaboration layer is also used to determine a control strategy that includes at least one of the following when the abnormal information is that the instruction to be executed at the current moment has failed to be executed or the instruction to be executed at the current moment has not been executed within a preset time: re-execute the instruction to be executed at the current moment, skip the instruction to be executed at the current moment, and execute the instruction to be executed at the next moment.

[0009] Based on the above technical means, the edge collaboration layer determines the corresponding control strategy based on the abnormal information, so that the system can tolerate the brief abnormality or failure of a single agent, and ensure the graceful degradation of the overall task through the corresponding control strategy, rather than a cascading collapse.

[0010] Furthermore, the sub-task is material handling; the cloud planning layer is also used to allocate corresponding driving paths for each sub-task based on the map and the location of each agent, and send all driving paths to the edge collaboration layer; the edge collaboration layer is also used to send all driving paths to the agent execution layer; the agent execution layer is also used to control multiple agents to execute corresponding sub-tasks according to the corresponding driving paths.

[0011] Based on the above technical means, the cloud planning layer allocates corresponding driving paths for each sub-task based on the map and the location of each agent, makes global decisions, and sends them to the agent execution layer through the edge collaboration layer. This enables the agent execution layer to control multiple agents to execute corresponding sub-tasks according to the corresponding driving paths, which can avoid systemic deadlock and long-term inefficiency.

[0012] Furthermore, the edge collaboration layer is also used to determine the control strategy, in the case of failure to pick up or put down materials, including at least re-picking or re-placing materials; in the case of agent failure, to determine the control strategy, including at least reassigning the subtasks of the failed agent and replanning the travel paths for other agents around the failed agent; and in the case of conflicting travel paths between two agents, to determine the control strategy, including at least replanning the travel paths of the two conflicting agents and sending adjustment instructions to the two conflicting agents based on the replanned travel paths.

[0013] Based on the aforementioned technical means, the edge collaboration layer performs real-time obstacle avoidance to ensure operational safety, maximizing warehouse throughput and minimizing the accident rate. Furthermore, by separating time-consuming global replanning from rapid local adjustments, most dynamic disturbances are absorbed by the edge collaboration layer in a very short time, with only major anomalies affecting the entire system being reported to the cloud, resulting in agile system response.

[0014] Furthermore, the sub-task is collaborative reconnaissance; the intelligent agent includes multiple drones and multiple robots; the edge collaboration layer is also used to lock onto the target drone when the target drone among the multiple drones identifies the reconnaissance target, and determine the target robot and the verification path of the target robot among the multiple robots; send the verification path of the target robot to the intelligent agent execution layer; the intelligent agent execution layer is also used to control the target robot to conduct reconnaissance according to the verification path of the target robot.

[0015] Based on the above technical means, when a target drone is identified among multiple drones through the edge collaboration layer, the target drone is locked and the target robot and the verification path of the target robot are determined among multiple robots. This perfectly integrates the "wide-area, fast, and overhead" perspective of drones and the "close, detailed, and multimodal perception" capabilities of ground robots, and realizes collaborative reconnaissance between drones and robots.

[0016] Furthermore, the edge collaboration layer is also used to determine the control strategy, at least including reassigning the robot, when the abnormal information is that the robot cannot identify the reconnaissance target due to a malfunction; and to determine the control strategy, at least including commanding the drone with insufficient power to return to base and having a backup drone take over the unreconnaissance area, when the abnormal information is that there is a drone with insufficient power.

[0017] Based on the above technical means, when the edge collaboration layer encounters robot or drone anomalies, the formulation of control strategies can overcome the latency bottleneck of centralized scheduling in the cloud planning layer, adapt to the high real-time reconnaissance operation requirements of robot and drone swarms, and reduce the latency of collaborative scheduling.

[0018] Furthermore, the state information of the agent includes at least one or more of the following: the agent's identifier, the agent's type, the agent's static state, and the agent's dynamic state; the agent's static state represents the actions that the agent can perform; the agent's dynamic state includes at least one or more of the following: the agent's operating state, the agent's battery level, and the agent's location.

[0019] This application provides a multi-agent cooperative control method applied to edge devices, the method comprising: Send corresponding subtasks to multiple agents; the subtasks are assigned to each agent by the cloud based on the state information of each agent. When an anomaly is detected in the execution of the corresponding sub-task by the multi-agent, a control strategy is determined based on the anomaly information and sent to the multi-agent; the multi-agent is then used to perform coordinated control according to the control strategy.

[0020] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the above-described method when executed by a processor.

[0021] This application provides a computer program product, including a computer program or computer executable instructions, which, when executed by a processor, implement the above-described method.

[0022] In this embodiment, a three-layer collaborative control architecture consisting of a cloud planning layer, an edge collaboration layer, and an agent execution layer is constructed. The cloud planning layer is responsible for assigning corresponding sub-tasks to each agent for macro-level task decomposition and planning. The edge collaboration layer monitors the agent execution layer and determines control strategies based on anomaly information when anomalies occur, thus relieving the real-time control pressure on the cloud planning layer, which then performs real-time control. Through architectural innovation, millisecond-level real-time control (handled by the edge collaboration layer) is decoupled from complex global planning (handled by the cloud planning layer), which satisfies the real-time control requirements of collaborative control and improves its real-time performance. Attached Figure Description

[0023] Figure 1 This application provides a multi-agent cooperative control system. Figure 2 This application provides a multi-agent cooperative control method. Figure 3 This application provides a more detailed multi-agent cooperative system in its embodiments; Figure 4 This is a flowchart illustrating the steps of a collaborative dancing scene provided in an embodiment of this application; Figure 5This is a flowchart illustrating the steps of a real-time fault-tolerant processing method provided in an embodiment of this application. Figure 6 This is a flowchart illustrating the steps of a material handling scenario provided in an embodiment of this application; Figure 7 This is a flowchart illustrating the steps of a collaborative reconnaissance scenario provided in an embodiment of this application. Detailed Implementation

[0024] To gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference only and are not intended to limit the embodiments of this application.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing the embodiments only and is not intended to limit the scope of this application.

[0026] In the following description, references to "some embodiments," "this embodiment," "this embodiment," and examples, etc., describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subset of all possible embodiments and may be combined with each other without conflict.

[0027] If similar descriptions such as "first / second" appear in the application documents, the following explanation shall be added: In the following description, the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0028] In this embodiment, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0029] Figure 1 This application provides a multi-agent cooperative control system, see [link to relevant documentation]. Figure 1 The system 10 includes: a cloud planning layer 101, an edge collaboration layer 102, and an intelligent agent execution layer 103; among which, The cloud planning layer 101 is used to assign corresponding sub-tasks to each intelligent agent based on the state information of each intelligent agent in the intelligent agent execution layer, and send all sub-tasks to the edge collaboration layer; The edge collaboration layer 102 is used to determine a control strategy based on the abnormal information when an abnormality is detected in the execution of the sub-task sent by the edge collaboration layer by the agent execution layer, and to send the control strategy to the agent execution layer. The agent execution layer 103 is used to perform collaborative control of multiple agents according to the control strategy.

[0030] In some embodiments, the state information of an agent includes at least one or more of the following: the agent's identifier, the agent's type, the agent's static state, and the agent's dynamic state.

[0031] The types of intelligent agents include, but are not limited to, vehicles, robots, and drones.

[0032] In this context, the static state of an agent represents the actions that the agent can perform. In some embodiments, the static state of an agent includes, but is not limited to, moving, turning on a light, and playing music.

[0033] The dynamic state of the agent includes at least one or more of the following: the agent's operating state, the agent's battery level, and the agent's location. In some embodiments, the agent's operating state includes an online state and an offline state.

[0034] In some embodiments, the cloud planning layer can first decompose the total task requirements into multiple sub-tasks, and then perform optimal task allocation based on the identifier, type, static state, and dynamic state of each agent, assigning a corresponding sub-task to each agent.

[0035] For example, the overall task requirement could be to complete a 5-minute collaborative light show. The cloud-based planning layer, based on the workflow engine built into the task management and orchestration engine, automatically decomposes the overall task requirement, generating a set of subtasks with logical order and dependencies. For instance, the collaborative light show task can be broken down into subtasks such as vehicle positioning, light initialization, main show sequence execution, and closing actions.

[0036] In some embodiments, after the edge collaboration layer receives all the subtasks sent by the cloud planning layer, it converts all the subtasks into micro-instructions that can be executed by the corresponding intelligent agent, and sends all the micro-instructions to the corresponding intelligent agent in the intelligent agent execution layer.

[0037] In some embodiments, the intelligent agents are equipped with an intelligent agent agent program, which has the ability to report its own location, battery level, etc., and to receive and execute instructions. For example, the ability to execute instructions such as moving, lifting, loading / unloading.

[0038] In some embodiments, after the agent execution layer receives a micro-instruction, it is executed by the corresponding agent. During the execution process, the edge collaboration layer monitors the execution process of the agent execution layer. If an abnormality is detected, a control strategy is determined based on the abnormality information, and the control strategy is sent to the agent execution layer. The agent execution layer then performs collaborative control of multiple agents according to the control strategy.

[0039] The abnormal information includes, but is not limited to, execution lag, failure of non-critical instructions, failure or timeout of critical instructions. Control strategies include, but are not limited to, delay compensation, instruction retries, and dynamic jumps. In some embodiments, when a delay in the execution of an agent is detected, the edge collaboration layer issues an instruction to accelerate its execution to compensate for the delay; when a failure to execute a non-critical instruction is detected, a limited number of retries are performed; when a failure or timeout of a critical instruction is detected, and retries are ineffective, the current step is skipped according to a preset strategy, and the process jumps to a subsequent step to ensure the continuity of the overall task, rather than a global interruption.

[0040] In some embodiments, each agent in the agent execution layer is equipped with a capability wrapper to encapsulate the agent's hardware functions (such as vehicle headlight control, door opening and closing, and movement; robot joint movement and grasping) into a series of standardized instruction interfaces that can be remotely invoked (such as turn_on_light(color,duration), move_to_position(x,y,theta)). After receiving micro-instructions or control strategies, the instruction executor in the agent calls the corresponding method in the local capability wrapper to execute them.

[0041] In this embodiment, a three-layer collaborative control architecture consisting of a cloud planning layer, an edge collaboration layer, and an agent execution layer is constructed. The cloud planning layer is responsible for assigning corresponding sub-tasks to each agent for macro-level task decomposition and planning. The edge collaboration layer monitors the agent execution layer and determines control strategies based on anomaly information when anomalies occur, thus relieving the real-time control pressure on the cloud planning layer, which then performs real-time control. Through architectural innovation, millisecond-level real-time control (handled by the edge collaboration layer) is decoupled from complex global planning (handled by the cloud planning layer), which satisfies the real-time control requirements of collaborative control and improves its real-time performance.

[0042] In some embodiments, the subtask is a collaborative performance; the edge collaboration layer is further configured to parse the subtask to obtain multiple instructions to be executed and the issuance time corresponding to each instruction to be executed, and send the corresponding instruction to be executed to the agent execution layer at the issuance time; the agent execution layer is configured to control the corresponding agent to execute the instruction to be executed when it receives the instruction to be executed.

[0043] In some embodiments, collaborative performance can involve multiple intelligent connected vehicles and robots dancing in unison. In this case, the intelligent agents can be vehicle Agent_V1, Agent_V2 and robot Agent_R1, Agent_R2. In a collaborative performance scenario, the overall task requirement can be a structured performance task file that describes the overall performance flow and the multimedia resources used, such as background music.

[0044] The cloud-based planning layer can break down the performance into a series of sub-tasks arranged chronologically, based on the overall task requirements received from the upstream performance choreography system and the music's timeline. For example, T0-T10s: Vehicles enter their designated positions, and robots take their places. T10s-T60s: The main performance phase; vehicle lights change with the music rhythm, and robots perform dance moves. T60s-T65s: The performance ends, and all agents return to their initial states. Then, based on the agents' static states, each sub-task is assigned to its corresponding agent, and an instruction script is generated. For example, if a sub-task requires an agent with lighting and movement capabilities, it is assigned to an agent with those capabilities; if a sub-task requires an agent with joint movement capabilities, it is assigned to an agent with those capabilities.

[0045] Instruction scripts serve as the link between the cloud planning layer and the edge collaboration layer. Their structured design is fundamental to achieving precise collaboration. A simplified script instance primarily includes the following key components to support complex collaborative tasks: Script Meta-information: This includes the script's unique identifier (script_id), version number, and total duration, used for full lifecycle tracking and management. Trigger Rules: These define the conditions for script activation, such as based on absolute time, specific events, or external commands, ensuring tasks are executed at predetermined times. Execution Flow: This is the core of the script, consisting of a list of sequential or parallel steps. Each step explicitly specifies the execution time (achieving millisecond-level synchronization), the executing agent (task allocation), and the standardized instructions and parameters to be executed (action definition). The flow supports complex logic such as loops and conditional branches. Exception Handling Strategy: Fault tolerance mechanisms (such as retry, skip, or pause reporting) are pre-defined for the script or specific steps. This is crucial for ensuring the overall robustness of collaborative tasks, enabling the system to automatically handle the failure of some agents. Through this structure, the script decouples complex collaborative logic from specific code, forming a configurable, predictable, and easily debuggable task blueprint. See below for the specific form of the instruction script.

[0046]

[0047] In some embodiments, the cloud planning layer securely distributes the generated instruction script to the edge collaboration layer via HTTPS. The edge collaboration layer's instruction script parser loads the instruction script, and when the clock reaches the T0 time specified in the instruction script, it publishes instruction messages to the corresponding agent topic (e.g., down / Agent_V1 / execute / request) via MQTT protocol according to the timestamp sequence in the script. The instruction message body is in JSON format, for example:

[0048] In some embodiments, after the agent executes the corresponding instruction, the execution result is published through its corresponding status topic (such as up / Agent_V1 / status / update):

[0049] Based on the above technical means, the edge collaboration layer relieves the real-time control pressure of the cloud planning layer. Even if the cloud planning layer network is temporarily interrupted, the edge collaboration layer can still rely on the locally cached instruction scripts to continue to command the intelligent agent to complete the predetermined task for a period of time, avoiding anomalies due to network interruption.

[0050] In some embodiments, the edge collaboration layer is further configured to determine a control strategy that includes at least one of the following when the abnormal information indicates that the instruction to be executed at the current moment has failed to be executed or the instruction to be executed at the current moment has not been executed within a preset time: re-execute the instruction to be executed at the current moment, skip the instruction to be executed at the current moment, and execute the instruction to be executed at the next moment.

[0051] In some embodiments, the state monitor of the edge collaboration layer continuously listens to the state feedback of all agents. Suppose that at T=30s, the monitor detects that the robot Agent_R1 has either failed to respond to a dance move instruction or has timed out. The fault-tolerant controller will, according to a preset strategy (e.g., retry once), if the retry also fails, trigger a "dynamic jump," sending an instruction to Agent_R1 to "skip the current action and prepare to execute the next action." Simultaneously, it may notify the vehicle Agent_V1 to also skip its corresponding action, thus ensuring that the performance continues synchronously at T=31s, almost imperceptibly to the audience.

[0052] Based on the above technical means, the edge collaboration layer determines the corresponding control strategy based on the abnormal information, so that the system can tolerate the brief abnormality or failure of a single agent, and ensure the graceful degradation of the overall task through the corresponding control strategy, rather than a cascading collapse.

[0053] In some embodiments, the subtask is material handling; the cloud planning layer is also used to allocate a corresponding driving path for each subtask based on the map and the location of each agent, and send all driving paths to the edge collaboration layer; the edge collaboration layer is also used to send all driving paths to the agent execution layer; the agent execution layer is also used to control multiple agents to execute the corresponding subtasks according to the corresponding driving paths.

[0054] In some embodiments, material handling can be the material handling of a modern smart warehouse, where the intelligent agent can be an Automated Guided Vehicle (AGV). In the material handling scenario, the cloud-based planning layer interfaces with the Warehouse Management System (WMS) and Enterprise Resource Planning (ERP) systems. The overall task requirements it receives are a structured set of inbound / outbound orders, such as "Within one hour, move the materials on shelves P1001-P1010 to workstations A1-A10 in the packing area."

[0055] The cloud-based planning layer receives a batch of orders from the WMS, including information such as material number, source location, target location, and priority. The scheduler in the cloud-based planning layer merges multiple orders into more efficient handling task batches. Based on the warehouse's static map (shelves, aisles, charging station locations) and the real-time locations of all AGVs, a conflict-free initial path is calculated for each task. Advanced algorithms (such as spatiotemporal scheduled joint path planning) can be used here to avoid potential deadlocks and collisions. Based on the AGV's current location, battery level, load capacity, and task priority, the task is assigned to the most suitable AGV to maximize overall handling efficiency (e.g., shortest total travel distance, fastest task completion time) and generate an instruction script.

[0056] In some embodiments, the instruction script for material handling differs from that for collaborative performance. The timing constraints of this material handling instruction script are more about logical sequence dependencies between tasks and spatially based reservation time windows, rather than strict absolute timestamps. The script contains a sequence of waypoints that each AGV needs to visit sequentially.

[0057] In some embodiments, after receiving the instruction script, the edge collaboration layer converts the path point sequence in the instruction script into a real-time motion instruction (such as "move to coordinates (X,Y) at a speed of 0.5m / s") and sends it to the AGV.

[0058] In some embodiments, AGVs report their LiDAR / vision sensor data in real time. When the edge collaboration layer detects unforeseen dynamic obstacles (such as unexpected personnel or scattered packages) or when the trajectory predictions of two AGVs indicate a potential conflict, the edge collaboration layer immediately performs millisecond-level local path replanning locally and issues adjustment instructions (such as deceleration, stopping, or detouring) to the affected AGVs, without having to upload information to the cloud planning layer, greatly reducing response latency.

[0059] Based on the above technical means, the cloud planning layer allocates corresponding driving paths for each sub-task based on the map and the location of each agent, makes global decisions, and sends them to the agent execution layer through the edge collaboration layer. This enables the agent execution layer to control multiple agents to execute corresponding sub-tasks according to the corresponding driving paths, which can avoid systemic deadlock and long-term inefficiency.

[0060] In some embodiments, the edge collaboration layer is further configured to: determine a control strategy that includes at least re-picking or re-placing materials when the abnormal information is a failure to pick up or place materials; determine a control strategy that includes at least reassigning the subtasks of the faulty agent and replanning the driving paths for other agents around the faulty agent when the abnormal information is a conflict between the driving paths of two conflicting agents; and determine a control strategy that includes at least replanning the driving paths of the two conflicting agents and sending adjustment instructions to the two conflicting agents based on the replanned driving paths.

[0061] In some embodiments, the edge collaboration layer monitors whether each AGV is traveling along the planned path, its task execution status (successful / failed pickup, successful / failed unloading), and its battery status. When an anomaly is detected (such as an AGV stopping abruptly due to a malfunction or a pickup failure), a control strategy is determined.

[0062] In some embodiments, if material retrieval or placement fails, the AGV is instructed to retry, i.e., retrieve or place the material again. For AGV malfunctions, the edge collaboration layer marks the malfunctioning AGV's incomplete tasks as "pending reallocation" and quickly notifies the cloud planning layer of the new task allocation scheme and path adjustment information. After updating the global view, the cloud planning layer can generate new adjustment instructions and replan paths for AGVs around the malfunction point to avoid congestion. If two AGVs have conflicting travel paths, the travel paths of both AGVs are replanned, and adjustment instructions are sent to both AGVs.

[0063] Based on the aforementioned technical means, the edge collaboration layer performs real-time obstacle avoidance to ensure operational safety, maximizing warehouse throughput and minimizing the accident rate. Furthermore, by separating time-consuming global replanning from rapid local adjustments, most dynamic disturbances are absorbed by the edge collaboration layer in a very short time, with only major anomalies affecting the entire system being reported to the cloud, resulting in agile system response.

[0064] In some embodiments, the subtask is collaborative reconnaissance; the agent includes multiple drones and multiple robots; the edge collaboration layer is also used to, when a target drone among the multiple drones identifies a reconnaissance target, lock onto the target drone, and determine the target robot among the multiple robots and the verification path of the target robot; send the verification path of the target robot to the agent execution layer; the agent execution layer is also used to control the target robot to conduct reconnaissance according to the verification path of the target robot.

[0065] In some embodiments, collaborative reconnaissance can be a drone-ground robot collaborative reconnaissance in the public safety field, where the intelligent agent can be a drone or a robot. In the scenario of collaborative reconnaissance, the cloud planning layer interfaces with the command center system. The macro-level tasks it receives are usually target-oriented, such as "searching for and identifying suspicious targets in area X (given geographical boundaries)" or "conducting peripheral patrols at location Y".

[0066] The drones are equipped with high-definition zoom cameras and infrared thermal imagers; the ground robots are equipped with high-definition cameras, robotic arms, and hazardous gas sensors. Both are heterogeneous intelligent agents with completely different capability models.

[0067] In some embodiments, the cloud planning layer receives reconnaissance orders from the command center, which include geographic information of the target area, reconnaissance priorities (such as searching for signs of life or detecting specific chemicals), and mission priorities.

[0068] In some embodiments, based on a digital map, the target area is divided into multiple sub-regions, and a preliminary search route for the UAV is planned to cover the entire area. Appropriate agent types are automatically matched according to task requirements. For example, a UAV may be designated to handle wide-area rapid scanning, while a ground robot serves as a standby precise verification unit. A dynamic script combining event-driven and state-triggered mechanisms is generated. The initial stage of the script primarily defines the UAV's search route. Simultaneously, triggering rules are predefined in the script, such as: "When the UAV identifies a 'suspected target' (confidence > 80%) at coordinates (Px, Py), trigger the subtask: guide the nearest ground robot to approach coordinates (Px, Py) for verification."

[0069] In some embodiments, the cloud planning layer distributes reconnaissance scripts and digital map data to the edge collaboration layer on the mobile command vehicle. The edge collaboration layer directs the drone to fly automatically along a predetermined route according to the script and receives the video stream and metadata (position, attitude, detection results) transmitted back in real time.

[0070] In some embodiments, the edge collaboration layer runs a lightweight AI model (such as a target detection algorithm) to analyze the drone video stream in real time. Once a pre-defined "suspected target" is identified, a pre-defined trigger rule in the script is activated. Based on the trigger rule, the edge collaboration layer automatically performs the following operations: locks onto the drone, instructs it to hover and continuously track and zoom to confirm the target. Immediately selects the nearest and reachable ground robot from among the available ones and plans a safe arrival path (avoiding known obstacles) for it from its current location to the target point. The target coordinates, the visual references continuously provided by the drone, and the movement path are sent to the ground robot in the form of a command stream.

[0071] Based on the above technical means, when a target drone is identified among multiple drones through the edge collaboration layer, the target drone is locked and the target robot and the verification path of the target robot are determined among multiple robots. This perfectly integrates the "wide-area, fast, and overhead" perspective of drones and the "close, detailed, and multimodal perception" capabilities of ground robots, and realizes collaborative reconnaissance between drones and robots.

[0072] In some embodiments, the edge collaboration layer is also used to determine, when the abnormal information is that the robot cannot identify the reconnaissance target due to a malfunction, that the control strategy includes at least reassigning the robot; and when the abnormal information is that there is a drone with insufficient power, that the control strategy includes at least instructing the drone with insufficient power to return to base and having a backup drone take over the unreconnaissance area.

[0073] In some embodiments, the edge collaboration layer can monitor drone battery level, communication quality, robot movement status, etc. If the assigned ground robot cannot reach the target due to obstacles, the edge collaboration layer immediately reassigns another robot. If the drone's battery is critically low, the edge node directs it to return to base and automatically dispatches a backup drone to take over the reconnaissance mission, while synchronizing the mission context (such as unsearched areas) to the replacement drone. Throughout the process, the edge collaboration layer synchronizes key statuses (such as "target detected," "robot dispatched," "target confirmed") to the cloud planning layer in real time for the command center to monitor globally.

[0074] Based on the above technical means, when the edge collaboration layer encounters robot or drone anomalies, the formulation of control strategies can overcome the latency bottleneck of centralized scheduling in the cloud planning layer, adapt to the high real-time reconnaissance operation requirements of robot and drone swarms, and reduce the latency of collaborative scheduling.

[0075] Figure 2 This application provides a multi-agent cooperative control method applied to edge devices. The method includes: S201, send the corresponding subtasks to the multi-agent team; the subtasks are assigned to each agent by the cloud based on the state information of each agent. S202, when an anomaly is detected in the execution of the corresponding sub-task by the multi-agent, a control strategy is determined based on the anomaly information and the control strategy is sent to the multi-agent; the multi-agent is used to perform coordinated control according to the control strategy.

[0076] In some embodiments, the subtask is a collaborative performance; the method further includes: parsing the subtask to obtain multiple instructions to be executed and the issuance time corresponding to each instruction to be executed, and sending the corresponding instruction to be executed to the multi-agent at the issuance time; the multi-agent is used to control the corresponding agent to execute the instruction to be executed when it receives the instruction to be executed.

[0077] In some embodiments, determining a control strategy based on anomaly information includes: when the anomaly information is that the instruction to be executed at the current moment has failed to be executed or the instruction to be executed at the current moment has not been executed within a preset time period, determining a control strategy includes at least any one of the following: re-executing the instruction to be executed at the current moment, skipping the instruction to be executed at the current moment, and executing the instruction to be executed at the next moment.

[0078] In some embodiments, the subtask is material handling; the method further includes: sending all driving paths to multiple agents; the driving paths are assigned to each subtask by the cloud based on a map and the location of each agent; the multiple agents are used to execute the corresponding subtasks according to the corresponding driving paths.

[0079] In some embodiments, determining a control strategy based on anomaly information includes: S211, In the case of abnormal information indicating failure to pick up or place materials, the control strategy shall include at least picking up or placing materials again. S212, In the case of abnormal information indicating agent failure, the control strategy is determined to include at least the reallocation of subtasks of the failed agent and the replanning of driving paths for other agents around the failed agent. S213, if the abnormal information indicates that the driving paths of two conflicting agents are in conflict, determine that the control strategy includes at least replanning the driving paths of the two conflicting agents, and sending adjustment instructions to the two conflicting agents based on the replanned driving paths.

[0080] In some embodiments, the subtask is collaborative reconnaissance; the intelligent agent includes multiple drones and multiple robots; the method further includes: In the case where a target drone identifies a reconnaissance target among multiple drones, the target drone is locked, and the target robot and its verification path are determined among the multiple robots. The verification path of the target robot is sent to the multi-agent system; the multi-agent system is used to control the target robot to conduct reconnaissance according to the verification path.

[0081] In some embodiments, determining a control strategy based on anomaly information includes: S221, If ​​the abnormal information indicates that the robot is unable to identify the reconnaissance target due to a malfunction, determine that the control strategy includes at least reassigning the robot. S222, In the event that the abnormal information indicates that there is a drone with insufficient power, the control strategy shall include at least instructing the drone with insufficient power to return to base and having a backup drone take over the unreconnoitered area.

[0082] In some embodiments, the state information of the agent includes at least one or more of the following: the agent's identifier, the agent's type, the agent's static state, and the agent's dynamic state; the agent's static state characterizes the actions that the agent can perform; the agent's dynamic state includes at least one or more of the following: the agent's operating state, the agent's battery level, and the agent's location.

[0083] Research on multi-agent systems (MAS) aims to solve complex problems that a single agent cannot accomplish independently through collaboration among multiple autonomous or semi-autonomous agents. However, traditional MAS collaborative architectures have revealed a series of irreconcilable contradictions when dealing with highly dynamic, real-time, and heterogeneous integration scenarios. In recent years, industry and academia have proposed various improvement schemes. Through analysis of related technologies, the common challenges currently faced by these technologies can be more clearly revealed.

[0084] 1. Centralized control architecture and its shortcomings: This architecture relies on a powerful central controller (usually located in the cloud), where all environmental perception, information fusion, decision-making, planning, and motion control are handled centrally, with intelligent agents acting as simple execution terminals. A related solution constructs a cloud-based collaborative testing system, where the cloud control platform uses distributed scheduling and feedback learning mechanisms to achieve real-time coordination and long-term self-optimization of multi-agent vehicle behavior patterns. This represents an intelligent evolution of the traditional centralized architecture, but its core still relies on the powerful computing power of the cloud for global decision-making and optimization. However, the inherent limitations of this architecture are amplified in real-time control scenarios requiring millisecond-level response times: 1) High communication latency: The intelligent agent and the cloud need to go through multiple network hops for transmission, and the latency is usually on the order of hundreds of milliseconds, which cannot meet the real-time control requirements of collaborative control and drone formation that require millisecond or even sub-millisecond response.

[0085] 2) Single point of failure risk: If the central controller fails, the entire system will be paralyzed, resulting in poor robustness.

[0086] 3) Network bandwidth bottleneck: All raw sensing data needs to be uploaded to the cloud, and the massive data transmission puts enormous pressure on network bandwidth.

[0087] 4) Limited scalability: As the number of agents increases, the computing and communication load of the central controller grows exponentially, which can easily become a system bottleneck.

[0088] 2. Fully Distributed Control Architecture and its Limitations: In this architecture, agents communicate and negotiate point-to-point through a self-organizing network, eliminating the need for a central node. Related solutions propose a scheme combining edge computing nodes. This application deploys models at edge nodes to process sensor data, identify anomalies, and generate collaborative decision-making strategies using digital twin models. This represents a trend towards offloading computation to the edge and pursuing low latency. However, such schemes typically position edge nodes as "data processors" or "local decision-makers," which has limitations: 1) Global consistency is difficult to guarantee: Lacking a global perspective, the agent makes decisions based on local information, making it difficult to achieve global optimization for complex tasks and prone to decision conflicts.

[0089] 2) Low collaboration efficiency: The consensus between intelligent agents is reached through multiple communication iterations, which results in a slow convergence speed and is not suitable for scenarios that require rapid response.

[0090] 3) High system complexity: It requires extremely high autonomous decision-making ability of intelligent agents, and the system behavior is difficult to predict and monitor globally.

[0091] 4) Difficulty in connecting heterogeneous systems: Different manufacturers and types of intelligent agents have differences in communication protocols, data models, and decision-making logic, making it difficult to achieve seamless interconnection and interoperability in a distributed architecture.

[0092] 3. Attempts and shortcomings of other technical approaches: Furthermore, related solutions enhance the local perception and communication capabilities of intelligent agents, achieving collaboration through semantic compression and knowledge graph sharing. While these fully distributed methods avoid central nodes, they place extremely high demands on the autonomous intelligence of individual agents, suffer from low collaboration efficiency, struggle to quickly reach consensus in dynamic environments, and are unsuitable for scenarios with strict constraints on execution timing.

[0093] Analysis of the aforementioned technologies reveals two main technical paths in the current multi-agent collaboration field: one is to strengthen the cloud-based brain, sacrificing real-time performance for global optimization; the other is to strengthen edge or terminal capabilities, sacrificing global collaboration for low latency. A significant capability gap exists between these two approaches. Related technologies fail to achieve a good balance between "system global planning and optimization capabilities" and "real-time and reliable control," and lack a mature architecture for unified, efficient, and robust scheduling of large-scale, highly heterogeneous agent clusters.

[0094] Therefore, there is an urgent need for an innovative collaborative paradigm that can fundamentally resolve the aforementioned contradictions and provide a feasible technical path for the practical implementation of large-scale, heterogeneous, real-time multi-agent systems. The primary objective of this application is to propose a layered, decoupled multi-agent collaborative control system and method. By introducing the key role of "edge collaborative nodes," complex global task planning is decoupled from low-latency real-time control, thus combining centralized global optimization capabilities with the advantages of distributed real-time response. Another objective of this application is to enable the system to possess high heterogeneity compatibility, allowing it to access agents with different communication protocols, data formats, and capability models, and to achieve unified management and scheduling through standardized interfaces. A further objective of this application is to improve the system's robustness and scalability, avoid single points of failure, and flexibly expand the system scale by adding edge nodes.

[0095] To achieve the above objectives, the core concept of this application is to construct a three-layer collaborative control architecture consisting of a "cloud-based task planning layer, an edge-based real-time collaboration layer, and an agent execution layer." The essence of this architecture is to separate and redistribute the "decision-making intelligence" and "control intelligence" of the agent: the cloud is responsible for "slow thinking" (macro-level task decomposition and planning) that requires a global perspective and complex computation; edge nodes are responsible for "fast response" (micro-level instruction synchronization and fault tolerance) that requires low latency and determinism; and the agent focuses on high-fidelity "precise execution."

[0096] Figure 3 This application provides a more detailed multi-agent cooperative system, see embodiments thereof. Figure 3 This includes: a cloud-based collaborative platform 301 (i.e., the aforementioned cloud planning layer), an edge collaborative node 302 (i.e., the aforementioned edge collaborative layer), and multiple embodied intelligent agents 303 (embodied intelligent agent #1 to embodied intelligent agent #N) (i.e., the aforementioned intelligent agent collaborative layer). The cloud-based collaborative platform 301 issues tasks / queries status to the edge collaborative node 302, the edge collaborative node 302 issues control commands to the multiple embodied intelligent agents 303, the multiple embodied intelligent agents 303 upload status feedback to the edge collaborative node 302, and the edge collaborative node 302 reports execution results / abnormal alarms to the cloud-based collaborative platform 301.

[0097] 1. Cloud-based collaboration platform 301: Functional positioning: As the "strategic brain" of the system, it is responsible for macro-management functions that require global information but are not real-time or near real-time.

[0098] Core modules: (1) Task Management and Scheduling Engine: Receives macro-level task descriptions (e.g., "Complete a 5-minute collaborative light show") from upstream business systems (such as performance scheduling systems and logistics dispatching systems). This engine has a built-in workflow engine that can automatically decompose complex tasks, generating a set of sub-tasks with logical order and dependencies. For example, the light show task can be decomposed into sub-stages such as "vehicle positioning," "light initialization," "main show sequence execution," and "closing actions."

[0099] (2) Resource Management and Scheduler: Maintains a global pool of intelligent agent resources, recording each agent's unique identifier, type (e.g., vehicle, robot), static capabilities (e.g., executable actions: move, turn on lights, play music), and dynamic status (e.g., online / offline, battery level, location). Based on task requirements and agent status, performs optimal task allocation.

[0100] (3) Data Management and Analysis Center: Aggregates status data and task execution logs reported by all agents for offline analysis, performance optimization and digital twin updates.

[0101] 2. At least one edge collaboration node 302: Functional positioning: As a "tactical command center" deployed locally in the scenario, it is the key to achieving low-latency collaboration in this application. It acts as a bridge between the cloud and intelligent agents, transforming macroscopic subtasks in the cloud into microscopic instructions that can be executed by intelligent agents, and ensuring the strict timing of instruction execution.

[0102] Core modules: (1) Instruction Script Parser and Scheduler: Receives subtask descriptions from the cloud (usually in a structured instruction script format, such as JSON or XML). The core function of this scheduler is timing management. It parses the time constraints in the script (such as absolute timestamps and relative time intervals) and generates an atomic instruction stream with precise execution times based on a high-precision local clock. For example, the script specifies that "at time T0, the robot begins action A; at time T0+500ms, the vehicle turns on its lights B." The scheduler is responsible for issuing the corresponding instructions at these two precise times, T0 and T0+500ms.

[0103] (2) Protocol Adaptation and Communication Gateway: Built-in adapters for various communication protocols (such as MQTT, CoAP, WebSocket, A2A protocol, etc.). This module implements the following functions: 1) Unified access: Adapt and convert different communication protocols for different intelligent agents, and provide a unified internal API for the upper layer.

[0104] 2) Message Routing: The system supports multiple communication paradigms through a protocol adaptation layer to adapt to the needs of different scenarios and agents. The publish / subscribe pattern is one core implementation method. In this pattern, each agent is assigned a unique topic, such as down / agent_id / execute / request for issuing commands, and up / agent_id / status / update for receiving status. Edge nodes, acting as coordinators, can publish messages to specific command topics and subscribe to the status topics of all agents for centralized monitoring. The system also supports point-to-point communication and request / response modes. For example, for commands requiring direct interaction or confirmation, a request / response mechanism similar to the A2A (Agent-to-Agent) protocol can be used; for simple queries on resource-constrained devices, the CoAP protocol can be used. The underlying communication gateway (such as MQTTBroker, HTTPServer, etc.) and protocol adapters are responsible for uniformly abstracting different modes and providing a consistent interface to the upper layers, thereby achieving flexibility and scalability in communication modes.

[0105] (3) Status monitoring and fault-tolerant controller: Real-time monitoring of the instruction execution status (e.g., "Executing", "Execution successful", "Execution failed", "Execution timed out") of each agent. This is the core of achieving collaborative robustness. Its built-in fault-tolerant strategies may include: 1) Delay compensation: If a delay is detected in the execution of a certain agent, an "accelerate execution" command can be issued to it.

[0106] 2) Instruction retry: When a non-critical instruction fails to execute, a limited number of retries are performed.

[0107] 3) Dynamic jump: When a critical instruction fails or times out and retries are ineffective, the current step is skipped according to a preset strategy, and the process jumps to a subsequent safe step in the script to ensure the continuity of the overall task rather than a global interruption.

[0108] 3. Multiple homogeneous or heterogeneous embodied intelligent agents 303: Functional role: The final execution unit of a task. Each agent must be equipped with an "agent agent program".

[0109] Core modules: (1) Capability Encapsulator: This encapsulates the hardware functions of an intelligent agent (such as vehicle headlight control, door opening and closing, and movement; robot joint movement and grasping) into a series of standardized instruction interfaces that can be remotely invoked. As follows: turn_on_light(color,duration),move_to_position(x,y,theta).

[0110] (2) Instruction executor: Receives atomic instructions from edge nodes, calls the corresponding methods in the local capability wrapper to execute them, and monitors the execution process.

[0111] (3) Status reporter: Periodically or event-triggered, it reports its own status information to the edge node, including health status (battery, signal strength), pose status, and the execution result of the current command.

[0112] Compared with related technologies, the significant technical effects of this application are reflected in the following aspects: It fundamentally solves the contradiction between control precision and system scale: Through architectural innovation, it decouples millisecond-level real-time control (handled by edge nodes) from complex global planning (handled by the cloud), achieving "plug-and-play" unified collaboration of heterogeneous intelligent agents: The protocol adapter of the edge node effectively shields the heterogeneity of the underlying intelligent agents. Regardless of the communication protocol (MQTT, CoAP, etc.) or internal implementation used by the intelligent agent, as long as its agent agent program implements the standard interaction interface with the edge node, it can quickly access the system and participate in collaboration. This greatly reduces the complexity of system integration and expansion. It significantly improves the reliability and robustness of the system: Avoiding single points of failure: The edge node shares the real-time control pressure of the cloud. Even if the cloud network is temporarily interrupted, the edge node can still rely on locally cached instruction scripts to continue directing the intelligent agent to complete the predetermined task for a period of time; Strong fault tolerance: The real-time status monitoring and dynamic compensation mechanism of the edge node enables the system to tolerate the brief abnormality or failure of a single intelligent agent, and ensures the "graceful degradation" of the overall task through strategies such as script jumps, rather than a "avalanche" collapse. It offers excellent scalability: the system can adopt a "cloud-multi-edge" architecture. Each edge node is responsible for a group of intelligent agents within a local area (such as a parking lot or a workshop). When the system needs to be expanded, only new edge nodes need to be added and connected to the cloud platform; the overall system architecture remains unchanged.

[0113] Example 1: A vehicle and robot collaborative performance system for exhibition demonstrations This embodiment uses the collaborative dancing of multiple intelligent connected vehicles and robots as a specific scenario to illustrate the implementation process of this application in detail.

[0114] 1. System component instantiation: Cloud-based collaboration platform: Can be deployed on Company A's cloud. The task description is a macro-level task titled "Exhibition Opening Show".

[0115] Edge collaboration node: A high-performance edge server (such as one with an Intel i7 processor and 32GB of memory) deployed at the conference site, which connects to the intelligent agent via a 5G private network or high-speed Wi-Fi.

[0116] Embodied intelligent agents: including two intelligent cars (Agent_V1, Agent_V2) and two humanoid robots (Agent_R1, Agent_R2).

[0117] 2. Detailed Workflow Explanation: Task Reception and Parsing: The cloud platform receives a structured performance task file from the upstream "Performance Choreography System". This file describes the overall performance flow, the multimedia resources used (background music), etc.

[0118] Global task planning: The cloud-based platform's task orchestration engine, combined with the music's timeline, breaks down the performance into a series of sub-tasks strictly arranged chronologically. For example: 1) T0-T10s: The vehicle drives into the designated position and the robot is in place.

[0119] 2) T10s-T60s: During the main performance phase, the vehicle lights change with the rhythm of the music, and the robots perform dance moves.

[0120] 3) T60s-T65s: The performance ends, and all agents return to their initial state.

[0121] The platform assigns subtasks to specific agents based on their capabilities (V1 / V2 has lighting and movement capabilities, R1 / R2 has joint movement capabilities) and generates an instruction script.

[0122] Script distribution: The cloud-generated instruction script is securely distributed to the edge collaboration nodes on-site via HTTPS protocol.

[0123] Command Scheduling and Issuance (Core): The command script parser on the edge node loads the script. Internally, it maintains a high-precision timer. When the system clock reaches the T0 time specified in the script, the edge node begins issuing command messages according to the timestamp sequence in the script, via the MQTT protocol to the corresponding agent topics (e.g., down / Agent_V1 / execute / request). The command message body is in JSON format, for example:

[0124] Status monitoring and feedback: After executing a command, the agent will immediately publish the execution result through its corresponding status topic (such as up / Agent_V1 / status / update):

[0125] Real-time fault tolerance: The edge node's state monitor continuously listens to the state feedback of all agents. Suppose that at T=30s, the monitor detects that the robot Agent_R1 has either failed to respond to a dance move instruction or has timed out. The fault tolerance controller will, according to a preset strategy (e.g., retry once), if the retry also fails, trigger a "dynamic jump," sending an instruction to Agent_R1 to "skip the current action and prepare to execute the next action." Simultaneously, it may notify the vehicle Agent_V1 to also skip its corresponding action, thus ensuring that the performance continues synchronously at T=31s, almost imperceptibly for the audience.

[0126] Figure 4 This is a flowchart illustrating the steps of a collaborative dancing scene provided in an embodiment of this application. See also... Figure 4 Specifically, it can include: S401, Begin.

[0127] S402, the cloud platform receives macro-level collaborative tasks.

[0128] S403 performs task decomposition and global planning in the cloud to generate instruction scripts.

[0129] S404, issue the instruction script to the edge collaboration node.

[0130] S405, edge node resolves script and initializes local scheduler.

[0131] S406, waiting for the script trigger conditions to be met (such as the absolute time to arrive).

[0132] S407 issues atomic instructions to the corresponding intelligent agents according to the script sequence.

[0133] S408, the edge node monitors the status feedback of all agents.

[0134] S409, determine if the status is normal? If not, jump to S410, otherwise jump to S417.

[0135] S410 triggers the fault tolerance processing logic.

[0136] S411, determine the exception type. If the exception type is instruction execution failure, jump to S412. If the exception type is instruction delay, jump to S416.

[0137] S412, Retry? If it still fails, jump to S413, otherwise jump to S414.

[0138] S413, skip the current instruction and notify the relevant agent.

[0139] S414, issue a retry command.

[0140] S415, Retry successful? If not, proceed to S413; otherwise, proceed to S417.

[0141] S416, issue acceleration command.

[0142] S417, continue with the subsequent instruction sequence.

[0143] S418, Has the script been executed? If not, proceed to S407; otherwise, proceed to S419.

[0144] S419 reports task completion to the cloud.

[0145] S420, End.

[0146] Figure 5 This is a flowchart illustrating the steps of real-time fault-tolerant processing provided in an embodiment of this application. See also... Figure 5 Specifically, it can include: S501, at time T0, the edge collaboration node issues instruction 1, sends command X to agent A (at time T0), and sends command Y to agent B (at time T0).

[0147] S502, Agent A executes normally and sends a message to the edge collaboration node that Action X was successfully executed (50 milliseconds after T0), i.e., ACK: Action_X Success (T0+50ms).

[0148] S503, Agent B executes slowly / is unresponsive. The edge collaboration node's monitor detects a timeout (preset timeout: 100 milliseconds) and sends the command to Agent B: Action Y (Retry) (150 milliseconds after T0), i.e., CMD: Action_Y(RETRY)( T0+150ms).

[0149] S504, if agent B fails to retries again or times out, send an ACK to the edge collaboration node: Action-Y Failure (or Timeout).

[0150] S505, the edge collaboration node sends a fault-tolerant decision to agent B: skip action Y or adjust action X (because action Y failed). That is, CMD: Skip_Action_Y, CMD: Adjust_Action_X (because Y failed).

[0151] S506, at time T1, the edge collaboration node continues to issue instruction 2, issuing action X2 (at time T1) to agent A, i.e., CMD: Action_X2(T1), and issuing action Y2 (at time T1) to agent B, i.e., CMD: Action_Y2(T1).

[0152] S507, Agent A sends Action X2 to the edge collaboration node and executes it successfully, i.e., ACK: Action_X2 Success. Agent B sends Action Y2 to the edge collaboration node and executes it successfully, i.e., ACK: Action_Y2 Success.

[0153] 3. Key Data Structure: Instruction Script Instruction scripts serve as the link between cloud-based planning and edge execution. Their structured design forms the basis for precise collaboration, and they primarily comprise the following key components to support complex collaborative tasks: (1) Script meta information: includes the script's unique identifier (script_id), version number (version) and total duration (total_duration), etc., for tracking and management throughout the entire lifecycle.

[0154] (2) Triggering rules: Define the conditions for the script to start, such as based on absolute time, specific events or external commands, to ensure that the task is executed at the predetermined time.

[0155] (3) Execution Flow: This is the main body of the script, consisting of a list of steps arranged sequentially or in parallel. Each step clearly specifies the execution time (achieving millisecond-level synchronization), the executing agent (task allocation), and the standardized instructions and parameters to be executed (action definition). The flow supports complex logic such as loops and conditional branches.

[0156] (4) Exception handling strategy: Fault tolerance mechanism (such as retry, skip or pause reporting) is preset for the script or specific steps. This is the key to ensuring the overall robustness of collaborative tasks and enables the system to automatically respond to the execution failure of some agents.

[0157] Through the above structure, the script decouples the complex collaborative logic from the specific code, forming a configurable, predictable, and easy-to-debug task blueprint.

[0158] Example 2: Multi-AGV Collaborative Scheduling System Applied to Intelligent Warehousing and Logistics This embodiment uses material handling in a modern intelligent warehouse as a scenario to illustrate in detail how this application solves the classic problem of efficient and conflict-free path planning and real-time scheduling of multiple AGVs (Automated Guided Vehicles) in a dynamic environment. This system, through a cloud-edge-device architecture, achieves fully automated and intelligent collaboration throughout the entire process from order receipt to goods delivery.

[0159] 1. System component instantiation: Cloud-based collaborative platform: It interfaces with warehouse management systems (WMS) and enterprise resource planning (ERP) systems. The macro-level tasks it receives are structured sets of inbound / outbound orders, such as "transfer the materials on shelves P1001-P1010 to workstations A1-A10 in the packing area within 1 hour".

[0160] Edge collaboration nodes: One or more high-performance edge servers are deployed within the warehouse. Each node is responsible for managing all AGVs, elevators, conveyor belt interfaces, and other equipment within a physical area (such as a warehouse area), forming a "collaborative control unit".

[0161] Embodied intelligent agents: These are multiple AGVs within the warehouse. Each AGV is equipped with an "intelligent agent program," capable of reporting its own location, power level, and load status, as well as receiving and executing standardized commands such as moving, lifting, loading / unloading.

[0162] 2. Detailed Workflow Explanation: Task Reception and Parsing: The cloud platform receives a batch of orders from the WMS. The orders contain information such as material number, source location, target location, and priority.

[0163] Global task planning and resource allocation: The cloud platform's scheduler performs the following core calculations: Order batch consolidation: Combine multiple orders into more efficient handling task batches.

[0164] Global path pre-planning: Based on the static map of the warehouse (shelves, aisles, charging station locations) and the real-time locations of all AGVs, a conflict-free initial path is calculated for each task. Advanced algorithms (such as spatiotemporal reservation-based joint path planning) can be used here to avoid potential deadlocks and collisions.

[0165] Task-AGV optimal matching: Based on the AGV's current position, battery level, load capacity, and task priority, the task is assigned to the most suitable AGV to maximize overall handling efficiency (such as the shortest total travel distance and the fastest task completion time).

[0166] Generate a scheduling script: The planning results above are transformed into a structured scheduling script. Unlike a performance script, this script's timing constraints are more about logical sequential dependencies between tasks and spatially based reservation time windows, rather than strict absolute timestamps. The script contains the sequence of path points that each AGV needs to visit sequentially.

[0167] Script distribution: The cloud will distribute the scheduling script to the edge collaboration nodes responsible for the corresponding database area.

[0168] Real-time scheduling and command issuance: Edge nodes load scripts, becoming the "real-time traffic command center" for the reservoir area. Its core responsibilities are: 1) Refined path tracking and command issuance: Convert the path point sequence in the script into real-time motion commands (such as "move to coordinates (X,Y) at a speed of 0.5m / s") and issue them to the AGV.

[0169] 2) Dynamic obstacle avoidance and local replanning: This is where the value of edge nodes is concentrated. AGVs report their LiDAR / vision sensor data in real time. When an edge node detects unforeseen dynamic obstacles (such as unexpected personnel or scattered packages) or when the trajectory predictions of two AGVs indicate a potential conflict, the edge node immediately performs millisecond-level local path replanning locally and issues adjustment instructions (such as deceleration, stopping, or detouring) to the affected AGVs, without having to upload information to the cloud for decision-making, greatly reducing response latency.

[0170] Status monitoring and fault tolerance: (1) Status monitoring: The edge node monitors whether each AGV travels along the planned path, the task execution status (successful / failed pickup, successful / failed unloading), and the power status.

[0171] (2) Fault tolerance: When an anomaly is detected (such as an AGV stopping suddenly due to a malfunction or failing to pick up goods), the edge node activates a preset strategy: 1) Task retry: If the pickup or delivery fails, instruct the AGV to retry.

[0172] 2) Task Reassignment: For AGV malfunctions, edge nodes mark the incomplete tasks of the malfunctioning AGV as "pending reassignment" and quickly notify the cloud of the new task assignment scheme and path adjustment information. After the cloud updates the global view, it can generate new adjustment instructions and issue them.

[0173] 3) Traffic management: Immediately reroute the AGVs around the fault point to avoid congestion.

[0174] Figure 6 This is a flowchart illustrating the steps of a material handling scenario provided in an embodiment of this application. See also... Figure 6 Specifically, it can include: S601, Begin.

[0175] S602 receives a batch of logistics orders in the cloud.

[0176] S603 performs global order merging and route planning in the cloud.

[0177] S604 generates scheduling scripts in the cloud and distributes them to edge nodes.

[0178] S605, edge node parsing and refreshing, starts scheduling AGV to execute tasks.

[0179] S606, the edge node monitors the status of all AGVs and environmental sensor data.

[0180] S607, Has a conflict or dynamic obstacle been detected? If yes, proceed to S608; otherwise, proceed to S613.

[0181] S608, edge nodes initiate local real-time replanning.

[0182] S609, determine the conflict type.

[0183] S610 calculates priority in the event of path conflict between AGVs and replans a local path for one party (such as yielding or slowing down); in the event of a sudden static obstacle, it plans a detour path for the affected AGV; in the event of AGV failure / stuck, it marks the AGV's task as failed and plans a detour path for the AGVs behind it.

[0184] S611 immediately issues the new local path instruction to the relevant AGV.

[0185] S612: Edge nodes asynchronously report local adjustment information (not the original data) to the cloud.

[0186] S613, continue to issue movement instructions as originally planned.

[0187] S614, Have all tasks in the current batch been completed? If so, proceed to S615; otherwise, proceed to S606.

[0188] S615, report task completion to the cloud.

[0189] S616, End.

[0190] 3. Key advantages of this application highlighted in this embodiment: Balancing efficiency and security: Global optimization in the cloud avoids systemic deadlocks and long-term inefficiency; real-time obstacle avoidance at the edge ensures operational safety. The combination of these two approaches maximizes warehouse throughput while minimizing incident rates.

[0191] High adaptability to dynamic environments: Separating time-consuming global replanning from rapid local adjustments. Most dynamic disturbances are absorbed by edge nodes in a very short time, and only major anomalies affecting the whole system are reported to the cloud, resulting in agile system response.

[0192] Scalable cluster management: By adding edge nodes, clusters of hundreds of AGVs across multiple large warehouse areas can be easily managed. Each edge node manages a subset, while the cloud is responsible for cross-regional collaboration and resource allocation.

[0193] Example 3: Unmanned Aerial Vehicle-Ground Robot Collaborative Reconnaissance System Applied to Public Safety This embodiment uses a collaborative reconnaissance mission in public safety emergency response as a scenario to illustrate in detail how this application achieves efficient heterogeneous collaboration between aerial drones and ground robots, solving the problem of rapid, large-scale search and precise close-range reconnaissance in complex, unknown, and potentially communication-limited environments.

[0194] 1. System component instantiation: Cloud-based collaborative platform: It interfaces with the command center system. The macro-level tasks it receives are usually target-oriented, such as "searching for and identifying suspicious targets in area X (given geographical boundaries)" or "conducting peripheral patrols of location Y".

[0195] Edge Collaboration Node: As a forward command hub, this node has powerful local computing capabilities and multi-mode communication links (5G / 4G, dedicated data transmission, Mesh self-organizing network) to ensure reliable connectivity in complex environments such as the field or city.

[0196] Embodied intelligent agents include multi-rotor drones and all-terrain ground robots. Drones are equipped with high-definition zoom cameras and infrared thermal imagers; ground robots are equipped with high-definition cameras, robotic arms, and hazardous gas sensors. Both are heterogeneous intelligent agents with drastically different capability models.

[0197] 2. Detailed Workflow Explanation: Mission reception and analysis: The cloud platform receives reconnaissance orders issued by the command center. The orders include the geographic information of the target area, the reconnaissance focus (such as searching for signs of life, detecting specific chemicals) and mission priority.

[0198] Global task planning and resource dispatch: The cloud platform performs the following analysis: 1) Regional analysis and preliminary planning: Based on the digital map, the target area is divided into multiple sub-regions, and preliminary search routes for UAVs are planned to cover the entire area.

[0199] 2) Heterogeneous capability matching: Automatically match the appropriate intelligent agent type according to task requirements. For example, designate a drone to be responsible for wide-area rapid scanning, and a ground robot as a standby precise verification unit.

[0200] 3) Generate a collaborative reconnaissance script: Generate a dynamic script that combines event-driven and state-triggered approaches. The initial stage of the script mainly defines the UAV's search route. At the same time, the script predefines triggering rules, such as: "When the UAV identifies a 'suspected target' (confidence > 80%) at coordinates (Px, Py), trigger the sub-task: guide the nearest ground robot to approach coordinates (Px, Py) for verification."

[0201] Script distribution: The cloud distributes the reconnaissance script and digital map data to the edge collaboration nodes on the mobile command vehicle.

[0202] Real-time task execution and dynamic command: Edge nodes become the "task commanders" on site.

[0203] (1) Drone control: The edge node commands the drone to fly automatically along the predetermined route according to the script, and receives the video stream and metadata (position, attitude, detection results) transmitted back in real time.

[0204] (2) Intelligent analysis and event triggering: The edge node runs a lightweight AI model (such as a target detection algorithm) to perform real-time analysis of the drone video stream. Once a preset "suspected target" is identified, the pre-set triggering rules in the script are activated.

[0205] (3) Chained task scheduling: Edge nodes automatically perform the following operations according to the triggering rules: 1) Lock onto the drone, instruct it to hover and continuously track and zoom to confirm the target.

[0206] 2) Immediately select the nearest and reachable robot from the available ground robots and plan a safe arrival path for it from its current location to the target point (avoiding known obstacles).

[0207] 3) The target coordinates, the visual references continuously provided by the UAV, and the movement path are sent to the ground robot in the form of a command stream.

[0208] Status monitoring and collaborative fault tolerance: (1) Status monitoring: Edge nodes monitor the drone's battery level, communication quality, robot movement status, etc.

[0209] (2) Dynamic fault tolerance and adjustment: 1) If the assigned ground robot is unable to reach the target due to an obstacle, the edge node immediately reassigns another robot.

[0210] 2) If the drone's battery is low, the edge node will instruct it to return to base and automatically dispatch a backup drone to take over the reconnaissance mission, while synchronizing the mission context (such as the unsearched area) to the replacement drone.

[0211] 3) Throughout the process, edge nodes will synchronize key statuses (such as "target detected", "robot dispatched", "target confirmed") to the cloud platform in real time for the command center to monitor globally.

[0212] Figure 7 This is a flowchart illustrating the steps of a collaborative reconnaissance scenario provided in an embodiment of this application. See also... Figure 7 Specifically, it can include: S701, begin.

[0213] S702, cloud-based reception area reconnaissance mission.

[0214] S703, cloud-based analysis area, generates initial reconnaissance script (drone flight path + event triggering rules).

[0215] S704, scripts are distributed from the cloud to mobile edge nodes.

[0216] S705, the edge node commands the drone to initiate reconnaissance according to the route.

[0217] S706 edge nodes perform real-time analysis of video / data transmitted from drones.

[0218] S707, Has a preset "suspected target" event been identified? If yes, proceed to S708; otherwise, proceed to S716.

[0219] S708, the event triggers the activation of the response rules in the script.

[0220] S709: The edge node instructs the drone to hover and continuously track the target; the edge node selects the nearest available ground robot; the edge node plans an approach path for the ground robot.

[0221] S710, edge nodes form a collaborative command flow.

[0222] S711, edge node synchronously guides drones and ground robots. The drones provide an aerial view, and the robots perform ground verification.

[0223] S712, Verify the result. If the target is confirmed, proceed to S713; otherwise, proceed to S715.

[0224] S713, edge nodes report "target confirmation" and detailed information to the cloud.

[0225] S714: Edge nodes determine subsequent actions (such as continuous monitoring or expanded search) based on new instructions or preset rules from the cloud.

[0226] S715, edge node, instructs the UAV to resume its original reconnaissance route and instructs the robot to return to the standby area.

[0227] The S716 drone continued to execute its original reconnaissance route.

[0228] S717, edge node monitoring status (battery, communication, basic status).

[0229] S718, Has the reconnaissance mission been completed? If yes, proceed to S719; otherwise, proceed to S706.

[0230] S719, report task completion to the cloud.

[0231] S720, end.

[0232] 3. Key advantages of this application highlighted in this embodiment: Heterogeneous collaboration and complementary capabilities: It perfectly integrates the "wide-area, fast, and overhead" perspective of drones with the "close, detailed, and multimodal perception" capabilities of ground robots, and achieves a synergistic effect of 1+1>2 through architecture.

[0233] Event-driven dynamic intelligence: The script is not a completely static timeline, but rather an event-triggered rule that integrates a closed loop of "perception-decision-execution", enabling the system to intelligently respond to unexpected situations on the spot and achieve true autonomous collaboration.

[0234] Adaptability to harsh environments: Edge nodes are deployed on mobile platforms and form a local autonomous network with intelligent agents, which greatly reduces the dependence on stable, high-bandwidth communication from the cloud, making them very suitable for emergency communication in poor field conditions.

[0235] In summary, this application ingeniously solves the core problem that has long plagued the development of multi-agent systems through an innovative cloud-edge-device three-layer collaborative architecture. The above embodiments fully demonstrate its technical versatility and powerful value. Any variation, substitution, or combination based on the core concept of this application—namely, bridging cloud planning and terminal execution through edge nodes to achieve efficient and reliable collaboration—falls within the scope of protection claimed by this application.

[0236] In an exemplary embodiment, this application also provides a computer-readable storage medium for storing a computer program.

[0237] Optionally, the computer-readable storage medium can be applied to any of the methods in the embodiments of this application, and the computer program causes the computer to execute the corresponding processes implemented by the processor in the various methods of the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0238] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0239] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0240] Furthermore, in the various embodiments of this application, all functional units can be integrated into one processing module, or each unit can be a separate unit, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units. Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0241] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0242] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0243] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0244] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multi-agent cooperative control system, characterized in that, The system comprises: a cloud planning layer, an edge collaboration layer, and an intelligent agent execution layer; wherein... The cloud planning layer is used to allocate corresponding sub-tasks to each intelligent agent based on the state information of each intelligent agent in the intelligent agent execution layer, and send all the sub-tasks to the edge collaboration layer; The edge collaboration layer is used to determine a control strategy based on the abnormal information when it detects an abnormality in the execution of the subtask sent by the intelligent agent execution layer, and then send the control strategy to the intelligent agent execution layer. The agent execution layer is used to perform collaborative control of the multiple agents according to the control strategy.

2. The system according to claim 1, characterized in that, The sub-task is collaborative performance; The edge collaboration layer is also used to parse the subtasks to obtain multiple instructions to be executed and the issuance time corresponding to each instruction to be executed, and to send the corresponding instruction to be executed to the agent execution layer at the issuance time; The agent execution layer is used to control the corresponding agent to execute the instruction to be executed when the instruction to be executed is received.

3. The system according to claim 2, characterized in that, The edge collaboration layer is further configured to, when the abnormal information is that the instruction to be executed at the current moment has failed to be executed or the instruction to be executed at the current moment has not been executed within a preset time period, determine that the control strategy includes at least one of the following: re-execute the instruction to be executed at the current moment, skip the instruction to be executed at the current moment, and execute the instruction to be executed at the next moment.

4. The system according to claim 1, characterized in that, The sub-task is material handling; The cloud planning layer is also used to allocate a corresponding driving path for each subtask based on the map and the location of each of the intelligent agents, and send all the driving paths to the edge collaboration layer; The edge collaboration layer is also used to send all the driving paths to the agent execution layer; The agent execution layer is also used to control the multiple agents to execute corresponding sub-tasks according to the corresponding driving paths.

5. The system according to claim 4, characterized in that, The edge collaboration layer is also used to determine, in the case that the abnormal information is a failure to pick up or place materials, that the control strategy includes at least picking up or placing the materials again; and in the case that the abnormal information is an agent failure, that the control strategy includes at least reassigning the subtasks of the failed agent and replanning the travel paths for other agents around the failed agent. If the abnormal information indicates that the driving paths of two conflicting agents are in conflict, the control strategy is determined to include at least replanning the driving paths of the two conflicting agents and sending adjustment instructions to the two conflicting agents based on the replanned driving paths.

6. The system according to claim 1, characterized in that, The sub-task is collaborative reconnaissance; the intelligent agent includes multiple drones and multiple robots. The edge collaboration layer is also used to, when a target drone among the plurality of drones identifies a reconnaissance target, lock the target drone, determine the target robot among the plurality of robots and the verification path of the target robot; and send the verification path of the target robot to the agent execution layer; The intelligent agent execution layer is also used to control the target robot to conduct reconnaissance according to the target robot's verification path.

7. The system according to claim 6, characterized in that, The edge collaboration layer is also used to determine, when the abnormal information is that the robot cannot identify the reconnaissance target due to a malfunction, that the control strategy includes at least reassigning the robot; and when the abnormal information is that there is a drone with insufficient power, that the control strategy includes at least instructing the drone with insufficient power to return to base and having a backup drone take over the unreconnaissance area.

8. The system according to any one of claims 1 to 7, characterized in that, The state information of the agent includes at least one or more of the following: the identifier of the agent, the type of the agent, the static state of the agent, and the dynamic state of the agent; The static state of the agent represents the actions that the agent can perform; the dynamic state of the agent includes at least one or more of the following: the operating state of the agent, the battery level of the agent, and the position of the agent.

9. A multi-agent cooperative control method applied to edge devices, characterized in that, The method includes: Send corresponding subtasks to the multi-agent system; the subtasks are assigned to each agent by the cloud based on the state information of each agent. When an anomaly is detected in the execution of the corresponding sub-task by the multi-agent, a control strategy is determined based on the anomaly information, and the control strategy is sent to the multi-agent; the multi-agent is used to perform coordinated control according to the control strategy.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the multi-agent cooperative control method of claim 9.

Images