Multi-agent natural language formation control method and system based on large language model

Abstract

The invention discloses a multi-agent natural language formation control method and system based on a large language model. The method comprises the steps of receiving a natural language instruction; calling a large language model to generate a target state sequence containing a space-time stamp; collision detection is carried out based on time sequence deduction and geometric calculation, and the minimum safety distance between the intelligent agents is verified; and after the verification is passed, synchronously issuing a control instruction in a direct connection or relay mode through radio frequency communication. The system comprises a ground station, a radio frequency communication module and a heterogeneous agent cluster. Microsecond-level synchronization precision, cross-platform compatibility and full-closed-loop AI planning are achieved, and the method is suitable for collaborative operation scenes of unmanned aerial vehicles, robots and the like.

Description

Technical Field

[0001] This invention relates to the field of multi-agent cooperative control technology, specifically to a method and system for heterogeneous intelligent agent natural language formation planning and collision avoidance based on a large language model. Background Technology

[0002] With the development of artificial intelligence and low-latency communication technology, multi-agent collaborative control is widely used in scenarios such as industrial inspection, light shows, and military exercises. However, the inventors of this invention discovered the following defects in the existing technology during the implementation of this invention: human-computer interaction is not intuitive, and traditional methods rely on pre-programmed or distributed algorithms, requiring operators to have professional programming skills; communication latency is too high: Wi-Fi, 4G, and 5G latency is usually >10ms, making it difficult to meet the requirements of high-precision synchronization; AI planning is incomplete: existing technologies, such as CN120952060A (a method for understanding and collaborative control of multi-agent interaction intent based on a large model), only focus on intent recognition and do not achieve a closed loop of "language → trajectory → verification → execution"; cross-platform compatibility is poor, and heterogeneous intelligent agents (drones, ground robots, etc.) lack a unified control interface; application scenarios are limited, and existing hoisting patents, such as CN120276464A (a multi-drone collaborative hoisting system), are based on mechanical devices and are not suitable for emerging scenarios such as light shows.

[0003] The technical problem to be solved by this invention is to provide a complete closed-loop control scheme with low latency, high semantics, and cross-platform compatibility, so as to achieve end-to-end coordination between natural language input and anti-collision trajectory generation. Summary of the Invention

[0004] To achieve the above objectives, this invention provides a multi-agent natural language formation control method based on a large language model, characterized by comprising the following steps:

[0005] Step S1: Receive formation control instructions in natural language form, including task-level operation instructions or complete performance scripts;

[0006] Step S2: Call the large language model to process the instructions and generate the target state sequence of each agent. The sequence includes position, velocity, attitude parameters and timestamp.

[0007] Step S3: Collision detection is performed based on temporal deduction and geometric calculation to verify that the Euclidean distance between any two agents at any time is not less than a preset safety threshold. ;

[0008] Step S4: If the verification fails, construct a feedback prompt and return to step S2 to regenerate; if the verification passes, proceed to step S5 according to the selected execution mode.

[0009] Step S5: The verified target state sequence is encoded into control commands and synchronously sent to each intelligent agent for execution via radio frequency communication mechanism.

[0010] The formation control command in step S1 supports text input, voice recognition (STT conversion), or graphical touch screen interaction.

[0011] The large language model in step S2 is a generative model based on the Transformer architecture, which adopts at least one of the following: commercial API interfaces (such as Kimi, Qwen, DeepSeek, GPT), open source models (such as the LLaMA series), or locally deployed models.

[0012] Optionally, the ground station integrates a search-enhanced generation module to improve generation accuracy by searching UAV API documentation and formation rule knowledge bases.

[0013] The target state sequence in step S2 is represented as follows:

[0014]

[0015] Where i is the agent number. For position vectors, For velocity vectors, t is the attitude angle vector, and t is the UTC timestamp.

[0016] The collision detection in step S3 calculates the temporal minimum distance using the following formula:

[0017]

[0018] , , must meet

[0019] in A preset safety threshold is used, typically 1–5 m. Detection employs discrete-time sampling, with a sampling interval Δt ≤ 100 ms.

[0020] The radio frequency communication mechanism in step S5 includes the following two modes:

[0021] In direct connection mode, the ground station radio frequency module communicates directly with the intelligent agent, with a one-way delay of ≤200μs and support for clock synchronization error of <1ms.

[0022] In relay mode, when the distance between the agent and the ground station exceeds the effective range of radio frequency, an agent with a delay of <200μs and a signal strength of >-70dBm is selected as the relay node, and the forwarding delay is ≤5ms.

[0023] The relay selection algorithm for the relay mode is as follows:

[0024] st

[0025] Where R is the set of relay nodes.

[0026] The intelligent agents in step S5 include aerial mobile platforms (multi-rotor UAVs, fixed-wing aircraft), ground mobile platforms (wheeled / tracked robots, legged robots), and surface / underwater mobile platforms. Each platform connects via a unified message bus interface, and the bus data format is defined as:

[0027] Message={header:{type,timestamp,source_id},payload:{position,velocity,attitude,control_mode}}

[0028] The agent in step S5 executes the action in a dual-mode mechanism: real-time mode, which sends instructions immediately via radio frequency communication after verification; and pre-planning mode, which stores the verified complete trajectory sequence in a non-volatile memory and triggers the execution of steps N to M according to the operation instructions (where 1≤N≤M≤total number of steps).

[0029] The beneficial effects of this invention are: intuitive interaction, with natural language input lowering the operational threshold and eliminating the need for professional programming knowledge; high synchronization accuracy, with microsecond-level radio frequency communication and global clock calibration ensuring that the synchronization error of multi-agent actions is <1ms in direct connection mode and ≤5ms in relay mode; complete AI closed loop: achieving full-link automation of "language understanding → trajectory generation → geometric verification → synchronous execution" for the first time; cross-platform compatibility: the unified message bus abstraction layer supports plug-and-play functionality on heterogeneous platforms; strong security: temporal geometric collision detection ensures no conflicts throughout the process; flexible deployment: real-time / offline dual modes adapt to diverse application scenarios. Attached Figure Description

[0030] Figure 1 This is a system architecture diagram of the present invention (corresponding to claim 8);

[0031] Figure 2 This is a flowchart of the method of the present invention (corresponding to claim 1);

[0032] Figure 3 This is a communication timing diagram (corresponding to claim 5);

[0033] Figure 4 This is a block diagram of sensor fusion for an intelligent agent (corresponding to an intelligent agent module).

[0034] Figure 5 This is a schematic diagram of a cross-platform message bus (corresponding to claim 9). Detailed Implementation

[0035] The present invention will be further described below with reference to specific embodiments.

[0036] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms "and" and "or" as used herein include any and all combinations of one or more of the associated listed items.

[0037] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0038] Embodiment 1 of the present invention is a real-time control of a drone formation light show. This embodiment describes an application that enables drone formation performances by inputting natural language commands.

[0039] The hardware configuration of this embodiment is as follows: 4 quadcopter drones (using STM32H750VBT6 processor and NRF24L01 radio frequency module) and ground station control terminal (using KimiAPI).

[0040] The steps in this first embodiment are as follows:

[0041] The operator inputs the text command: "Four drones form a circle with a radius of 10m and a height of 50m for 30 seconds."

[0042] The LLM processing unit generates JSON format trajectory data “{"drone_id":1,"waypoints":[{"x":10,"y":0,"z":50,"t":0},{"x":7.07,"y":7.07,"z":50,"t":5}]}”;

[0043] The collision detection module is based on the formula. Calculate at t=5s =14.14m, =5m, > Verification passed;

[0044] The radio frequency transmitter broadcasts commands with a 130μs delay, and UAVs 5 and 6 execute them synchronously with a clock error of 0.8ms, forming a perfect circular formation.

[0045] Embodiment 2 of the present invention is a heterogeneous air-ground collaborative control. In this embodiment, the robot dog performs ground inspection, and the drone follows at a relative altitude of 5m.

[0046] The steps in this second embodiment are as follows:

[0047] The voice command was converted into text via STT: "Drone follows robot dog, stays 5m above."

[0048] The unified message bus will locate the robot dog. The relative offset (0,0,5) from the drone is encapsulated as a standard message;

[0049] The LLM processing unit generates dynamic targets: = =Δp, where Δp=(0,0,5);

[0050] The dynamic target production loop executes once every 100ms to ensure that the UAV maintains a relative position error of <10cm when the UWB positioning accuracy is ±5cm.

[0051] Embodiment 3 of the present invention is a large-scale formation relay communication. In this embodiment, a formation of 50 UAVs is used, with the ground station located at the edge of the site.

[0052] The steps in this third embodiment are as follows:

[0053] The signal strength detector measures the RSSI of each unit and selects relay nodes that meet the criteria (RSSI>-70dBm and delay<200μs);

[0054] The communication is divided into three groups: Group 1 (20 aircraft) has a direct connection delay of 130μs; Groups 2 and 3 (15 aircraft each) are relayed, with a total delay of 4.13ms.

[0055] The clock synchronization module calculates the clock offset Δk for each group and broadcasts the calibration value to ensure that the synchronization error is less than 5ms.

[0056] Embodiment 4 of the present invention is an offline pre-planning fault-tolerant mechanism. This embodiment is a 20-step light show performance with a total duration of 10 minutes.

[0057] The steps in this fourth embodiment are as follows:

[0058] Generate trajectories in batches and verify for conflicts, with step 16 being particularly problematic due to existing issues. =3.2m, =5m, < Validation failed and the code was marked as conflicting.

[0059] Construct feedback prompts to inform the LLM to regenerate step 15 until verification is successful;

[0060] The verified 20-step light show performance data is stored in an SQLite database, with each step containing a timestamp and trajectory.

[0061] If one of the intelligent agents loses contact during the performance, the system will mark it as offline, and the remaining 49 agents will be reassembled and continue to execute.

[0062] While preferred embodiments of the invention have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, modifications, and substitutions will now occur to those skilled in the art without departing from the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed to practice the invention. The following claims are intended to define the scope of the invention, and methods and structures within the scope of these claims and their equivalents are also covered.

Claims

1. A multi-agent natural language formation control method based on a large language model, characterized in that, The method comprises the following steps: Step S1, receiving a platoon control instruction in natural language form; Step S2, calling a large language model to process the instruction to generate a target state sequence of each agent, the sequence including position, speed, attitude parameters and time stamp; Step S3, collision detection based on time sequence deduction and geometric calculation, verifying that the Euclidean distance between any two agents at any time is not less than a preset safety threshold ; Step S4, if the verification fails, a feedback prompt word is constructed and returned to step S2 to regenerate; if the verification passes, step S5 is entered according to the selected execution mode; Step S5, encoding the target state sequence that passes the verification into a control instruction, and synchronously issuing it to each agent for execution in direct mode or relay mode through radio communication.

2. The method of claim 1, wherein, The large language model is a generation model based on a Transformer architecture, adopts a cloud API, an open source model or a local deployment model, and integrates a retrieval enhancement generation module to access a domain knowledge base.

3. The method of claim 1, wherein, The collision detection is computed by the formula The timing minimal distance is computed, where is the position vector of agent i at time t, with a sampling interval At < 100 ms.

4. The method of claim 1, wherein, The direct connection mode radio frequency communication one-way delay ≤200μs, relay mode through the preferred relay node forwarding, total delay ≤5ms; relay node according to Select, constraint condition is signal strength .

5. The method of claim 1, wherein, The intelligent agent includes an air mobile platform, a ground mobile platform, a water surface or underwater mobile platform, each platform accesses through a unified message bus, and the bus data format is Wherein the payload contains a standardized position-velocity-pose vector.

6. The method of claim 1, wherein, The execution mode comprises: a real-time mode, in which an instruction is issued immediately after verification; and a pre-planning mode, in which a track sequence that passes verification is stored in a non-volatile memory and execution is triggered according to a specified step N to M.

7. A large language model-based multi-agent natural language formation control system, characterized in that, The method comprises the following steps: An instruction receiving module is configured to receive a natural language instruction and convert the instruction into a text sequence; a large language model processing module is connected to the instruction receiving module and configured to generate a target state sequence; and a collision detection module is connected to the large language model processing module and configured to perform time sequence geometry verification. A mode selection module is configured to switch between a real-time mode and a pre-planning mode; a radio frequency communication module is configured to support a direct connection mode and a relay mode and is configured to synchronize and issue a control instruction; and a heterogeneous intelligent agent cluster is configured to include a sensor fusion unit, a control unit and a communication unit.

8. The system of claim 7, wherein, The radio frequency communication module comprises a radio frequency transmitter, an intelligent agent address encoder, a relay judgment and forwarding unit and a clock synchronization calibration unit. 9.A computer readable storage medium having stored thereon a computer program, wherein the program is executed by a processor to implement the method of any one of claims 1-6.

Images