Multi-agent low-delay cooperative communication method and system based on task-level instruction and dynamic relay

Abstract

The invention discloses a multi-agent low-delay cooperative communication method and system, and the method comprises the steps: transmitting a task-level control instruction frame to a heterogeneous agent through a single control source device in a one-to-many radio frequency broadcast mode, and achieving the effect that the instruction synchronization delay is smaller than 500 microseconds; each agent keeps relative position constraint based on a virtual rigid body model, a dynamic relay module is started when it is detected that the quality of a communication link is lower than a threshold value, an adjacent agent forwards an instruction in a point-to-point mode, and the forwarding delay is smaller than 2 milliseconds; a pre-training language model is built in the system to analyze natural language input, and a task-level instruction sequence verified through geometric collision detection is generated. According to the invention, a unified message bus interface is adopted to standardize different platform control data, microsecond-level cooperative control of heterogeneous intelligent agents such as unmanned aerial vehicles, robots and model airplanes is supported, and the system is suitable for scenes such as light show and air-ground cooperation.

Description

Technical Field

[0001] This invention relates to the field of multi-agent collaborative control technology, specifically to a method and system for achieving low-latency collaborative communication among multiple intelligent agents such as drones, robots, and model aircraft through a single control source, applicable to application scenarios such as light show performances, collaborative operations, and air-ground collaboration. Background Technology

[0002] With the development of drone and robotics technologies, multi-agent collaborative control has become a hot topic in industry and scientific research. Existing technical solutions have the following four shortcomings:

[0003] First, the communication architecture suffers from high latency. Existing patent documents such as CN120276464A (a multi-UAV collaborative hoisting system) and CN120178916A (a multi-UAV collaborative transportation control method, device, equipment, and storage medium) mostly adopt a "ground base station + distributed computing" architecture, relying on Wi-Fi, 4G, 5G cellular networks, or dedicated data links for communication. The end-to-end latency generally exceeds 10 milliseconds, making it impossible to achieve microsecond-level high-precision synchronization control. Especially in complex environments such as urban canyons, mountainous areas, and indoor spaces, signal obstruction exacerbates latency jitter, and synchronization errors can reach the order of hundreds of milliseconds.

[0004] Second, the control commands have a low level of abstraction. Existing technologies mostly use low-level control commands (such as PWM duty cycle signals) or preset waypoint trajectories, lacking high-level task semantic understanding capabilities. Although some solutions, such as CN120952060A (a method for multi-agent interaction intent understanding and cooperative control based on a large model), introduce large language models for intent understanding, they fail to achieve end-to-end conversion of natural language commands to real-time control commands, still relying on manual coding or offline planning;

[0005] Third, there is insufficient support for heterogeneous platforms. Existing solutions are usually designed for a single platform (drones or ground robots only), lacking a unified control abstraction layer and data protocol, making it difficult to achieve cross-platform state estimation and collaborative control, thus limiting the scalability and compatibility of heterogeneous intelligent agent clusters;

[0006] Fourth, poor scenario adaptability. Existing patents mainly focus on specific fields such as collaborative hoisting and logistics transportation. For emerging application scenarios such as light show performances, interactive formation reconstruction, and air-ground collaborative exploration, there is a lack of dynamic topology adaptation and real-time security verification mechanisms.

[0007] Therefore, there is an urgent need for a multi-agent collaborative communication scheme that can achieve microsecond-level synchronization, support natural language interaction, adapt to heterogeneous platforms, and has dynamic relay capabilities. Summary of the Invention

[0008] The technical problem this invention aims to solve is to overcome the shortcomings of existing technologies, such as high communication latency, low command abstraction level, poor platform compatibility, and weak environmental adaptability, and to provide a low-latency collaborative communication method and system for multiple intelligent agents. This system can achieve microsecond-level command synchronization in complex environments, support high-level task semantics and natural language interaction, and is adaptable to various heterogeneous intelligent agent platforms such as UAVs, ground robots, watercraft, and model aircraft. To solve the above technical problems, this invention provides several methods.

[0009] The first aspect of this invention provides a low-latency cooperative communication method for multiple agents, comprising the following steps:

[0010] S10, the control source device sends task-level control command frames to all agents in the cluster in a one-to-many broadcast mode through a wireless radio frequency communication module operating in the unlicensed frequency band. The command frame contains a unified timestamp and the target state vector of all agents. The delay difference between the synchronous arrival of the command to each agent is less than a first preset delay threshold. );

[0011] S20, after each intelligent agent receives the task-level control command frame, it synchronizes the clock based on the local clock and timestamp correction, with a synchronization error of less than 50 microseconds.

[0012] S30, each agent parses the task-level control command and extracts the target position-velocity-attitude triplet of its own agent. Where i is the agent number, It is a three-dimensional position vector. For velocity vectors, The attitude quaternion is used to perform cooperative motion based on a virtual rigid body model, while maintaining the relative positional constraints in a rigid coordinate system. ( );

[0013] S40, each agent periodically sends a heartbeat packet containing signal strength (RSSI) and timestamp to the control source, when the control source reaches the second preset delay threshold ( If no heartbeat response is received from a certain agent within 50ms, or if the confirmation timeout of the agent's command frame is detected, the direct communication is determined to have failed, and a dynamic relay communication mode is activated. The control command is forwarded point-to-point by a nearby agent with a signal strength RSSI ≥ -60dBm, and the forwarding delay is less than a third preset delay threshold. =2ms), and the number of relay hops does not exceed 1 hop;

[0014] The S50, with its built-in pre-trained language model, parses user-input natural language commands in real time and generates task-level control command sequences. Each instruction It includes motion type, parameter vector and safety constraints, and verifies the safety of the instruction sequence through a geometric collision detection algorithm. After successful verification, step S10 is executed.

[0015] Furthermore, the task-level control commands include, but are not limited to: formation overall translation commands. Virtual rigid body rotation command Commands to surround points of interest and overall takeoff and landing commands ,in It is a translation vector. For the axis of rotation, The rotation angle is... To surround the center, The radius of the circle. This is the height value.

[0016] Furthermore, the dynamic relay communication mode adopts a command frame copying and forwarding mechanism. After receiving the broadcast command, the relay agent does not parse its content, but directly attaches the relay identifier and timestamp and retransmits it to the target agent, ensuring that the end-to-end delay is controllable.

[0017] A second aspect of the present invention provides a multi-agent low-latency cooperative communication system, comprising:

[0018] A single control source device is configured to send task-level control commands in a one-to-many broadcast mode via a wireless radio frequency communication module, and has a built-in pre-trained language model, geometric collision detection engine and communication quality monitoring unit.

[0019] Multiple heterogeneous intelligent agent devices are configured to receive the task-level control commands and perform cooperative motion. The heterogeneous intelligent agents include at least two of the following: aerial drones, ground mobile robots, watercraft, and fixed-wing model aircraft. Each intelligent agent is equipped with a unified message bus interface, a clock synchronization unit, and a virtual rigid body motion controller.

[0020] The dynamic relay module is configured to automatically activate a nearby agent with a signal strength RSSI ≥ -60dBm and remaining energy E ≥ 20% to relay the signal when the control source does not receive a heartbeat response from an agent within 50 milliseconds, with a relay delay of less than 2 milliseconds.

[0021] A unified message bus interface is configured to standardize control data from different platforms into "{timestamp, agent ID, location}" format. , speed , attitude quaternion Data packets in the format of "}";

[0022] The AI ​​planning module is configured to convert natural language commands into a sequence of task-level control commands and verify collision risks in real time based on a 3D spatial grid map.

[0023] Furthermore, the heterogeneous intelligent agent device also includes a sensor fusion module, which fuses measurement data from an optical flow sensor, an inertial measurement unit (IMU), and a multi-directional obstacle avoidance sensor using an extended Kalman filter algorithm to calculate the intelligent agent's pose estimate. This makes the estimation error covariance matrix traces ≤0.01 .

[0024] The beneficial effects of this invention are: ultra-low synchronization latency, achieving command synchronization latency of less than 500 microseconds and synchronization accuracy of less than 50 microseconds through single-hop radio frequency broadcasting and a high-precision timestamp mechanism, significantly better than the millisecond level of existing technologies, providing a technical foundation for high-precision light synchronization and close formation flight; intelligent environmental adaptation, with a dynamic relay mechanism extending communication coverage while ensuring low latency, and relay forwarding latency of less than 2 milliseconds and no more than 1 hop, effectively solving the signal obstruction problem in complex environments such as urban canyons and indoor spaces, improving communication robustness by more than 60%; natural language interaction, with a large language model parsing natural language commands in real time and generating task-level control code end-to-end, reducing the operation threshold by 8%. 0%, supporting high-level semantic commands such as "create a circular formation and rotate clockwise"; cross-platform compatibility, with a unified message bus interface standardizing heterogeneous platform control data into position, velocity, and attitude triples, enabling plug-and-play collaboration between drones, robots, and model aircraft, improving system scalability by 3 times; pure software implementation, core functions do not require dedicated ASIC or FPGA hardware, implemented based on general-purpose microprocessors and software-defined radio (SDR), reducing deployment costs by 50% and significantly improving flexibility; multiple security guarantees, with built-in geometric collision detection and heartbeat timeout mechanisms to ensure mission execution safety, fault detection response time of less than 100 milliseconds, and cluster operation security reaching SIL-2 level. Attached Figure Description

[0025] Figure 1 This is a block diagram of the overall architecture of the multi-agent low-latency cooperative communication system of the present invention, showing the connection topology of a single control source with multiple agents via radio frequency broadcast, and the bypass path of the dynamic relay module in the case of signal obstruction.

[0026] Figure 2 This is a flowchart of the task instruction generation and distribution process based on a pre-trained language model, illustrating the complete data flow from natural language input to agent execution, including security verification, clock synchronization, and communication quality feedback loops.

[0027] Figure 3This is a schematic diagram of the formation motion of the multi-agent system based on the virtual rigid body model of the present invention, showing the topological constraint relationship in which each agent maintains a fixed relative offset under the three task commands of translation, rotation and orbit.

[0028] Figure 4 This is a timing diagram of the switching of the dynamic relay communication mode of the present invention, showing the complete time sequence and state machine transition of the heartbeat packet detection mechanism, relay trigger condition judgment and instruction forwarding. Detailed Implementation

[0029] The present invention will be further described below with reference to specific embodiments.

[0030] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms "and" and "or" as used herein include any and all combinations of one or more of the associated listed items.

[0031] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0032] Embodiment 1 of the present invention is a drone light show formation performance system. This embodiment describes the application of the present invention in a drone swarm light show performance. The scenario requires the LED light synchronization error of 32 drones to be less than 1 millisecond and the formation geometric accuracy to be less than 0.5 meters.

[0033] The hardware configuration of this embodiment is as follows: the control source device is equipped with a ground station computer, running a local instance of the Llama-3-8B pre-trained language model, and connected to a software-defined radio (SDR) transmitter module (Ettus USRPB210) via a USB 3.0 interface. The operating frequency band is set to ISM 915MHz and the transmit power is 20dBm.

[0034] The communication module in this embodiment is configured with GFSK modulation, a baud rate of 2Mbps, a command frame length of 256 bytes, a broadcast period of T=10ms, and a theoretical air interface delay. ≈1.28ms, actual measured synchronization arrival delay difference =320±50μs (first preset delay threshold).

[0035] The intelligent agent device in this embodiment consists of 32 quadcopter drones (Ardupilot flight control open source platform), each equipped with an ARM Cortex-M7 microprocessor (480MHz), an optical flow sensor (PX4FLOW), an IMU unit (BMI270, sampling rate 1kHz), and an 8-way ToF obstacle avoidance sensor (VL53L1X).

[0036] The specific implementation steps of this embodiment one are as follows:

[0037] S101, Natural Language Analysis, the operator inputs natural language commands through the ground station software: "Create a circular horizontal formation with a diameter of 20 meters, hovering height of 50 meters, move forward 10 meters as a whole, and rotate 90 degrees clockwise in a rigid body form, with a translation speed of 2 m / s and a rotation angular velocity of 30° / s".

[0038] S102, AI planning module processing: pre-trained language model parses instructions and generates secondary task sequences. ;

[0039] S103, Geometric Collision Detection: The collision detection engine verifies all trajectory pairs in the three-dimensional spatiotemporal domain. The minimum distance;

[0040] S104, Command Encoding and Broadcasting: The task sequence is encoded into standardized command frames, which are then transmitted at a high frequency with a period of T=10ms by the radio frequency broadcasting module. All agents are in... Receive instructions within the time window ( );

[0041] S201, intelligent agent-side processing: after receiving instructions, each UAV calculates pose estimation by fusing data from IMU and optical flow sensors;

[0042] S202, Virtual rigid body motion execution, each UAV calculates the offset of the rigid coordinate system. And execute the PID control law;

[0043] S301, heartbeat packet monitoring, each drone periodically... Send a heartbeat packet containing: {Agent ID, Local RSSI, Battery Voltage, Pose Checksum}. If the ground station detects a heartbeat at the second preset delay threshold... If no response is received from a certain drone, it is determined to be out of contact;

[0044] S302, Dynamic Relay Trigger: When UAV #15's RSSI falls below -85dBm due to building obstruction, the ground station detects a heartbeat timeout and immediately activates the relay mechanism. Neighboring UAV #14 (RSSI=-55dBm, Energy E=75%) receives the relay command and... =Within 1.2ms, append the relay header "{Relay source ID=14, destination ID=15, timestamp" to the original instruction frame. + After forwarding to #15, the forwarding was delayed. =1.2ms, which is less than the 2ms threshold. Throughout the process, the cluster synchronization accuracy remains less than 50μs.

[0045] The secondary task sequence in step S102 ,in: This is a formation initialization command, which generates the initial positions of 32 agents on a circle with a radius of R = 10m. (i=1~32); It is a composite motion command that includes a translation vector. (Unit: meters) and quaternion about the z-axis .

[0046] The detection engine in step S103 verifies the minimum distance of all trajectory pairs (i,j) in the three-dimensional spatiotemporal domain, in order to solve the optimization problem:

[0047] st ( )

[0048] Where T=10s is the task duration. The minimum distance is calculated. =1.2m>safety threshold =0.8m, deemed safe.

[0049] The instruction frame structure in step S104 is: "Timestamp: uint64_t" (μs-level precision), Agent ID mask: uint32_t (Bitmap representation), target state vector: float μ

[32] [7] = { , , , , , ,ψ}”.

[0050] The formula for calculating the pose estimation in step S201 is as follows:

[0051]

[0052] Kalman gain The position estimation error is ensured by dynamic calculation of the error covariance matrix Pᵢ. .

[0053] The PID control law in step S202 has the following formula:

[0054]

[0055] Among them, control gain , This ensures that the geometric error of the formation is less than 0.3m.

[0056] Embodiment 2 of the present invention is an air-ground collaborative exploration system. This embodiment demonstrates the collaborative control of heterogeneous platforms (drone + ground robot dog) and verifies the unified interface and relative position maintenance capabilities.

[0057] System composition of this embodiment two:

[0058] The control source is a handheld terminal (processor is Horizon Sunrise X3 RDK), running a lightweight pre-trained language model GLM, and integrating a 920MHz LoRa RF module (transmit power 17dBm).

[0059] The drone platform consists of one hexacopter drone (600mm wheelbase), equipped with an RTK-GNSS module (positioning accuracy σ=0.02m) and a downward-facing camera (for visual tracking).

[0060] Ground robot dog, one quadruped robot (Unitree Go1), equipped with a wheel encoder (odometer accuracy ±2%) and 2D laser SLAM (Hector SLAM algorithm, map resolution 0.05m).

[0061] A unified message bus interface is used, and both subscribe to the ROS topic " / multi_agent / cmd". The data format is the ROS2 standard "geometry_msgs / PoseStamped" message, and the timestamp synchronization adopts the PTP protocol (with an accuracy of less than 1μs).

[0062] The collaborative process in this second embodiment:

[0063] S401, voice command analysis, operator voice input: "The drone stays 2 meters above the robot dog, both move forward in sync at a speed of 1 m / s. When encountering an obstacle, the robot dog turns left and the drone follows."

[0064] S402, the AI ​​planning module generates instructions, and the pre-trained language model outputs a bi-objective instruction sequence;

[0065] S403, Timestamp Synchronization and Execution, Control Source Broadcast Command Frame, Timestamp =0x5F5E0A1C (μs level). After receiving the data from the robot dog and drone, the local clock is corrected based on the PTP protocol, and the clock error is... Less than 5μs;

[0066] S404, relative position closed-loop feedback: the drone detects the Apriltag marker on the top of the robot dog using a downward-facing camera and calculates the relative pose. And run the visual servo control law;

[0067] S405, dynamic frequency adjustment, robot dog frequency control. =20Hz (based on wheel encoder), UAV control frequency =50Hz (based on IMU + vision), the two are synchronized asynchronously through timestamp alignment to ensure system consistency;

[0068] S406, Communication Recovery Detection: When the robot dog turns an angle greater than 30 degrees, the drone may briefly lose its visual anchor point. At this time, the dynamic relay module is activated: the robot dog acts as a relay node, forwarding its SLAM pose to the drone via LoRa, with a delay of [missing information]. =1.8ms, ensuring uninterrupted collaboration. Automatically switches back to direct connection mode upon successful visual re-identification;

[0069] The dual-target instruction sequence in step S402 contains the following:

[0070] Robot dog commands: = {Movement type: "FORWARD", Speed: 1.0m / s, Duration: ∞, Obstacle response: "TURN_LEFT"}”;

[0071] Drone command: = {Motion type: "RELATIVE_POSITION", Anchor point: "machine_dog", Offset: [0,0,2](unit: METER)}.

[0072] The visual servo control law in step S404 has the following formula:

[0073]

[0074] in =0.8, ensuring relative position error | <0.15m.

[0075] While preferred embodiments of the invention have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, modifications, and substitutions will now occur to those skilled in the art without departing from the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed to practice the invention. The following claims are intended to define the scope of the invention, and methods and structures within the scope of these claims and their equivalents are also covered.

Claims

1. A low-latency cooperative communication method for multiple agents, characterized in that, Includes the following steps: (a) The control source device sends task-level control command frames to all heterogeneous agents in the cluster in a one-to-many broadcast mode via a wireless radio frequency communication module operating in the unlicensed frequency band. The command frame contains a unified timestamp and the target state vector of all agents. The delay difference between the synchronous arrival of the command to each agent is less than a first preset delay threshold. ,in (a) After receiving the task-level control command frame, each agent synchronizes its clock based on its local clock and timestamp correction, with a synchronization error of less than 50 microseconds; (b) Each agent parses the task-level control command and extracts the target position-velocity-attitude triplet of its own body. μ i = p i v i θ i T Where i is the agent number, It is a three-dimensional position vector. For velocity vector, The attitude quaternion is used to perform cooperative motion based on a virtual rigid body model, while maintaining the relative positional constraints in a rigid coordinate system. ( (d) Each agent periodically sends a heartbeat packet containing the signal strength RSSI and a timestamp to the control source. When the control source reaches the second preset delay threshold... If no heartbeat response is received from a certain agent within 50 milliseconds, or if the confirmation timeout of the agent's command frame is detected, the direct communication is determined to have failed, and a dynamic relay communication mode is activated. The control command frame is forwarded point-to-point by a nearby agent with a signal strength RSSI ≥ -60dBm, and the forwarding delay is less than a third preset delay threshold. ,in =2 milliseconds, and the number of relay hops does not exceed 1 hop; (e) The pre-trained language model built into the control source parses the natural language commands input by the user in real time and generates a sequence of task-level control commands. Each instruction It includes motion type, parameter vector and safety constraints, and verifies the safety of the instruction sequence C through a geometric collision detection algorithm. After the verification is successful, step (a) is executed.

2. The method according to claim 1, characterized in that, The wireless radio frequency communication module adopts GFSK or LoRa modulation, operates in the ISM 915MHz or 2.4GHz unlicensed frequency band, does not establish a mesh network topology, and has a broadcast period T∈[5ms,20ms].

3. The method according to claim 1, characterized in that, The task-level control command frame includes the following types: formation overall translation command. Virtual rigid body rotation command Commands to surround points of interest and overall take-off and landing commands ,in It is a translation vector. For the axis of rotation, The rotation angle is... To surround the center, The radius of the circle. This is the height value.

4. The method according to claim 1, characterized in that, The dynamic relay communication mode employs a command frame copying and forwarding mechanism. Upon receiving a broadcast command, the relay agent does not parse its content but directly appends a relay identifier and timestamp before retransmitting it to the target agent, ensuring controllable end-to-end latency and minimizing relay delay. <2 milliseconds.

5. The method according to claim 1, characterized in that, The geometric collision detection algorithm verifies all agent trajectory pairs in the three-dimensional spatiotemporal domain. Find the minimum distance and solve the optimization problem. m i n t ∈ 0 T | | p i t - p j t | | 2 And ensure that the minimum distance is greater than the safety threshold. ≥0.8 meters.

6. A multi-agent low-latency cooperative communication system, characterized in that, include: A single control source device is configured to transmit task-level control command frames in a one-to-many broadcast mode via a wireless radio frequency communication module. It incorporates a pre-trained language model, a geometric collision detection engine, and a communication quality monitoring unit, wherein the synchronization arrival delay difference of the command frames is less than 500 microseconds. Multiple heterogeneous intelligent agent devices are configured to receive and execute the task-level control commands. These heterogeneous intelligent agents include at least two of the following: aerial drones, ground mobile robots, watercraft, and fixed-wing model aircraft. Each intelligent agent is equipped with a unified message bus interface, a clock synchronization unit, and a virtual rigid body motion controller, maintaining a relative position constraint Δpᵢⱼ=constant. A dynamic relay module is configured to automatically activate a nearby intelligent agent with a signal strength RSSI ≥ -60dBm and remaining energy E ≥ 20% for relay forwarding when the control source does not receive a heartbeat response from an intelligent agent within 50 milliseconds, with a forwarding delay of less than 2 milliseconds. The unified message bus interface is configured to standardize control data from different platforms into a format: "{timestamp, agent ID, location"}. speed , attitude quaternion Data packets in the format of "}"; AI planning module, configured to convert natural language instructions into a sequence of task-level control instructions, and to verify collision risks in real time based on a 3D spatial grid map.

7. The system according to claim 6, characterized in that, The heterogeneous intelligent agent device also includes a sensor fusion module, which fuses measurement data from an optical flow sensor, an inertial measurement unit (IMU), and a multi-directional obstacle avoidance sensor using an extended Kalman filter algorithm to calculate the intelligent agent's pose estimate. This makes the estimation error covariance matrix traces ≤0.01 .

8. The system according to claim 6, characterized in that, The dynamic relay module is configured to periodically send heartbeat packets and determines the communication link status based on a dual judgment mechanism of timestamp comparison and RSSI threshold. Relay is triggered only when two conditions are met simultaneously: ① heartbeat packet timeout; ② target agent RSSI setting threshold.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the multi-agent low-latency cooperative communication method as described in any one of claims 1-5.

Images