Unmanned aerial vehicle cluster autonomous navigation control system based on multi-mode state perception and collaborative navigation decision

Abstract

The invention discloses an unmanned aerial vehicle cluster autonomous navigation control system based on multi-mode state perception and collaborative navigation decision, which comprises a multi-mode combination state perception information module, a collaborative decision navigation module and an algorithm evaluation module, and is characterized in that data passes through a graph attention network perception module through the multi-mode combination state perception module; grouping and aggregating the information of the neighbor agent group, the external obstacle group and the target group in the observation state; stacking a plurality of moments to obtain a multi-modal combination state description; and the collaborative decision-making module generates a motion instruction of each unmanned aerial vehicle in an end-to-end manner by extracting space and time sequence characteristics described by a multi-modal combination state, so that collision-free navigation of the unmanned aerial vehicles is ensured. According to the method, neighbor intelligent machine bodies, obstacles and target information are grouped and aggregated by using the image annotation sensing module, and reconstruction, splicing, gray scale and dimension reduction processing are performed on original visual data, so that the accuracy and robustness of environment sensing are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous navigation control of unmanned aerial vehicle (UAV) clusters, and in particular to an autonomous navigation control system for UAV clusters based on multimodal state perception and collaborative navigation decision-making. Background Art

[0002] Autonomous collaborative navigation and control of drone swarms refers to the ability of multiple unmanned aerial vehicles to collaborate and complete complex tasks without human intervention. The core challenge lies in achieving efficient information sharing, task allocation, and collaborative decision-making in dynamic and uncertain environments. Traditional centralized control approaches present the risk of single points of failure in response to the demands of large-scale swarms and complex environments. Distributed control approaches, while highly robust and scalable, are extremely challenging to maintain global optimality in dynamic environments.

[0003] Autonomous navigation of drone swarms based on environmental perception. Traditional methods rely on sensors such as GPS and lidar for environmental perception, but suffer from low accuracy and susceptibility to interference in complex and dynamic environments. In recent years, environmental perception methods based on graph attention networks have gradually emerged. They can effectively model the topology of drone swarms and adaptively capture the interactions between individuals through an attention mechanism, thereby improving the accuracy and robustness of environmental perception. Researchers have proposed a multi-UAV collaborative target tracking algorithm based on a graph attention network. This algorithm uses GAT to encode the relative position and velocity information between drones, achieving high-precision tracking of dynamic targets. However, this algorithm only considers the relative position information between drones and fails to fully utilize environmental information, resulting in reduced tracking accuracy in complex scenarios. Some researchers have proposed constructing a drone-obstacle interaction graph, but the shapes and sizes of obstacles are highly presumptive, making it difficult to adapt to complex and changing obstacle environments.

[0004] Target allocation and path planning are key aspects of cooperative navigation for drone swarms. Traditional methods often employ centralized or distributed optimization algorithms, but these methods suffer from high computational complexity and poor real-time performance in large-scale swarms and dynamic environments. In recent years, Transformer-based target allocation and path planning methods have gained increasing attention. These methods leverage self-attention mechanisms to capture global information and generate more optimal decision-making solutions. Some researchers have proposed Transformer-based multi-drone task allocation algorithms, which achieve efficient task allocation by encoding task requirements and drone state information. However, if dependencies between tasks are not considered, the task allocation results may be illogical. Leveraging the sequence modeling capabilities of Transformers, safe and efficient flight paths were generated. However, dynamic environmental changes make them less adaptable, making it difficult to respond to unexpected situations.

[0005] With collaborative control as the core, autonomous flight of drone swarms is achieved. Traditional methods mostly use model-based control methods, but they suffer from problems such as model mismatch and low control accuracy in complex dynamic environments. Collaborative control methods based on deep reinforcement learning are able to autonomously learn optimal control strategies through interaction with the environment and have gradually emerged in recent years. Based on deep reinforcement learning, a drone swarm formation control algorithm for stable formation flight of drone swarms is proposed, which uses a reasonable reward function design. However, reward functions that are highly dependent on the design are difficult to adapt to complex and changing control tasks. Another proposed drone swarm collaborative search algorithm improves search efficiency through experience sharing and collaborative learning, based on deep reinforcement learning of multiple intelligent agents. However, it has high requirements for communication bandwidth and computing resources, making it difficult to apply to large-scale swarms. Summary of the Invention

[0006] Purpose of the invention: The purpose of the present invention is to provide an autonomous navigation and control system for UAV clusters based on multimodal state perception and collaborative navigation decision-making.

[0007] Technical solution: The drone cluster autonomous navigation and control system based on multimodal state perception and collaborative navigation decision-making described in the present invention includes a multimodal combination state perception information module, a collaborative decision-making navigation module and an algorithm evaluation module. The data passed through the multimodal combination state perception module includes three parts: one part is the RGB data onboard the drone, which is reconstructed and spliced, and grayscaled and dimensionally reduced to obtain image information; the other part is the lidar data, which is grouped and aggregated by the graph attention network perception module for the information of the neighboring intelligent agent group, external obstacle group and target group in the observation state; the last part is the drone's body motion information, and the latter two parts are mapped together to the reconstructed image data, and multiple moments are stacked to obtain a multimodal combination state description; the collaborative decision-making module adopts a multi-agent deep reinforcement learning algorithm based on the Transformer encoder, and generates end-to-end motion instructions for each drone by extracting the spatial and temporal features of the multimodal combination state description, ensuring that the drone navigates to the target position without collision.

[0008] Furthermore, the construction of the graph attention network perception module includes dividing the information of the observation state into three groups: intelligent agents, obstacles and targets, using three graph attention perception modules and a dimensional mapping network to encode and process the neighbor intelligent agent observation information, obstacle observation information and target information respectively, and output features that are independent of the number and scale of intelligent agents and obstacles.

[0009] Furthermore, the elements in the graph attention perception module include Query, Key and Value. The position information of the current agent is regarded as Query and represented by Q, and the position information of the neighboring agents, neighboring obstacles and target points are regarded as Key and represented by K respectively. a , K o With K m Represents the high-dimensional features h of each vertex of the observation information ij =σ(o i W1) as Value, where σ(·) represents the activation function, the default activation function is the ReLU function, and W1 is the network weight.

[0010] Furthermore, the formula for calculating the attention weight in the graph attention perception module is as follows:

[0011]

[0012] in, are the attention weights of agent i to neighboring agents, obstacles, and target points, respectively. is the weight decay coefficient, and the softmax() function is used to normalize the attention, expressed as:

[0013]

[0014] By weighting the Value, we can obtain the aggregate characteristics of different groups:

[0015]

[0016] in, They are the high-dimensional features of agent j, obstacle k, and target g respectively; when obtaining the aggregated features After that, the output features of DAT are obtained through splicing operation, and the integration process is expressed as:

[0017]

[0018] in, is the network weight, (||) represents the concatenation operation.

[0019] Furthermore, the multimodal combined state generation method establishes an information description of multimodal combined state perception by splicing the perception description results of each drone, extracts and reconstructs key features in RGB visual perception, and grayscales and compresses the original data.

[0020] Furthermore, the multimodal combined state generation method comprises the following steps:

[0021] (1) Original visual data acquisition: RGB data is collected from the drone’s first-person perspective as the original visual perception input;

[0022] (2) The original visual image is input into the single-stage target detection model YOLOv5 based on deep learning to extract the pixel positions of navigation targets, obstacles and other drones in the image. Based on the positioning results, the three types of objects are visually reconstructed;

[0023] (3) Image dimensionality reduction and compression: grayscale processing of the reconstructed image;

[0024] (4) Image stitching: horizontal stitching of compressed images from multiple drones;

[0025] (5) Information fusion: After the image stitching is completed, the aggregated information about the three groups of drones, obstacles, and targets, as well as the action information, output by the image attention perception module model are stitched into an all-zero matrix. The result is then vertically spliced ​​with the multimodal image to obtain a multimodal perception image.

[0026] (6) Time series stacking: stacking the multimodal perception results of multiple moments as a multimodal combination state.

[0027] Furthermore, the collaborative decision-making navigation module uses a Transformer encoder to extract features describing multimodal perception state information, regards UAV collaborative navigation as a collaborative multi-intelligent agent reinforcement learning task, and regards the global action value as the aggregation of the action value of each UAV.

[0028] Furthermore, the collaborative decision-making navigation module defines three high-level instructions for drones, namely, for drone i, its action space A i ={a1, a2, a3, a4}, where a1 means going straight, a2 means turning left, a3 means turning right, and a4 means circling;

[0029] For the reward at time t, the reward function is defined as follows:

[0030]

[0031] Indicates the penalty when the drone collides, since the safe flight radius of the drone is d safe ,

[0032] The reward is used to train the drone's obstacle avoidance and collision avoidance capabilities. The calculation formula is:

[0033]

[0034] Where: k is the number of UAVs that collide; R colIt is the penalty value when a single drone collides; is the minimum distance between the drone and the surrounding obstacles; d1 is the threshold for the drone to determine collision with obstacles; d2 is the distance threshold for determining collision between drones, and its value is d safe ; It represents the sparse reward given when all drones in the environment correctly reach their respective target locations, calculated as:

[0035]

[0036] Among them, R all The reward value for all drones arriving correctly; is the distance between the UAV and the target; d goal is the threshold for determining whether the UAV reaches the target, r t ang The direction correction reward introduced to ensure that the drone can face the target during navigation is related to the angle of the drone's current position pointing to the target position and the drone's yaw angle, and is calculated as:

[0037]

[0038] Among them, w g Correction reward scaling factor for angle; is the two-dimensional coordinate position of the target at time t; is the projection of the drone's position on the XOY plane at time t; yaw t is the yaw angle of the UAV at time t; the expression of the function atan2 is as follows:

[0039]

[0040] During training, when a drone finds that a drone has already reached the target location it is heading towards, it is driven to continue exploring the environment and navigate to the correct target location.

[0041] Furthermore, the algorithm evaluation module uses the MADDPG algorithm, the VDN algorithm, and the TCDN algorithm as baseline algorithms. The Actor network and the Critic network of MADDPG and VDN are constructed using three-layer full connections.

[0042] Furthermore, the algorithm evaluation module uses the average cumulative system error and evaluation indicators to measure the control effect of the entire control process.

[0043] The mean cumulative systematic error is defined as follows:

[0044]

[0045] The evaluation index ξ can be used to evaluate the speed consistency and cluster topology stability of the system, and is defined as follows:

[0046]

[0047] When ζ(t) approaches 0, the speeds of the agents reach a consensus and the system structure is in a stable state.

[0048] Beneficial effects: Compared with the existing technology, the present invention has the following significant advantages: Compared with the existing technology, the present invention has the following significant advantages: The present invention uses the caption perception module to group and aggregate the information of neighboring intelligent bodies, obstacles, and targets, and realizes efficient environmental perception through reconstruction, splicing, grayscale, and dimensionality reduction of the original visual data, which significantly improves the accuracy and robustness of environmental perception. The multi-agent deep reinforcement learning algorithm based on the Transformer encoder extracts the spatial and temporal features of the multimodal combined state description, generates end-to-end motion instructions for the UAV, ensures collision-free navigation, and the designed action space and reward function optimize the obstacle avoidance, collision avoidance, target allocation, and autonomous navigation capabilities, generates a better collaborative strategy, and greatly improves the task completion rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is the architecture diagram of the UAV swarm perception-cooperative navigation decision algorithm;

[0050] Figure 2 Schematic diagram of the graph attention perception module;

[0051] Figure 3 is the average score curve of different algorithms;

[0052] Figure 4 Describe the average score curve of the input for different states;

[0053] Figure 5 is the average speed error curve;

[0054] Figure 6 is the average system loss curve;

[0055] Figure 7 It is the time consumption of different algorithms during the training process. DETAILED DESCRIPTION

[0056] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0057] Aiming at the problem of realizing efficient and robust autonomous cooperative navigation control of UAV clusters in random dynamic environments, this paper proposes a multi-modal combined state perception-cooperative navigation decision algorithm, such as Figure 1As shown in Figure 2, this algorithm aims to address the multiple challenges faced by drone swarms in complex environments, such as environmental perception, target allocation, path planning, and collaborative control. The multimodal combined state perception module comprises three components: first, image information obtained by reconstructing, splicing, grayscaling, and reducing the dimensionality of raw RGB visual data; second, a graph attention network perception module constructed from lidar data to group and aggregate information about neighboring agents, external obstacles, and targets in the observed state; and third, the drone's motion information. This information is mapped onto the visual data, and the results at multiple moments are stacked to form a multimodal combined state description. The collaborative decision-making module utilizes a Transformer encoder-based multi-agent deep reinforcement learning algorithm. By extracting spatial and temporal features from the multimodal combined state description, it generates motion commands for each drone in an end-to-end manner, ensuring collision-free navigation to the target location. To further optimize the drone's behavior, a reasonable action space and reward function are designed to ensure the drone's ability to avoid obstacles, collisions, allocate targets, and navigate autonomously.

[0058] The multimodal combined state perception module consists of three parts: 1. The drone's onboard RGB data, which is reconstructed and spliced, then grayscaled and reduced in dimension to create image information; 2. The LiDAR data, which is aggregated by the graph attention network perception module to group and aggregate information about neighboring agents, external obstacles, and targets in the observed state; and 3. The drone's motion information. These two components are mapped onto the reconstructed image data, and multiple moments are stacked to create a multimodal combined state description.

[0059] (1) Graph Attention Network Perception Module

[0060] The graph constructed by the multi-agent cluster system is represented as Among them, the vertex set Contains the set of vertices consisting of obstacles The vertex set of all agents The vertex set consisting of the target Edge Set Represents the interaction relationship between vertices in the vertex set. The set of neighboring agents of agent i It can be constructed based on the communication distance. The information interaction between neighboring agents is based on probability. Completed, so the neighbor set is a subset of and completes information interaction with agent i. According to the detection range of the agent, the adjacent obstacle set of agent i can be expressed as Therefore, the set of obstacles actually detected is within the detection range of agent i, with probability The detected obstacle set The target set of agent i It can be represented by the detection range of the agent, so the set of obstacles actually detected is a subset of this set. When the target is in the detection range of agent i, it will be detected with probability Detect the target by size.

[0061] Based on the above communication interaction relationship, the internal observation state of agent i is defined as:

[0062]

[0063] in, represents the motion state of agent j relative to agent i, which can be calculated by the following formula.

[0064]

[0065] in, Used to normalize the speed information of the agent, and also represents the agent with the maximum speed amplitude.

[0066] The observation state of the agent i and the adjacent obstacle j can be expressed as:

[0067]

[0068] in, represents the motion state of obstacle k relative to agent i, which can be calculated by the following formula:

[0069]

[0070] in, Indicates the maximum velocity amplitude of the obstacle.

[0071] Based on the detection results of agent i, the navigation goal g of agent i can be expressed as:

[0072]

[0073] in, represents the motion state of the target g relative to the agent i, which can be calculated by the following formula:

[0074]

[0075] in, Represents the maximum velocity amplitude of the target, and the observation state of agent i is defined as Based on the observation status of all agents, the observation status of the drone cluster is:

[0076] s=[o1,o2,…,o n ]

[0077] Construct a UAV cluster graph attention perception module to realize grouping and aggregation of information of neighboring agent groups, external obstacle groups, and target groups in the observation state. It divides the information of the observation state into three groups: agents, obstacles, and targets, and uses three graph attention perception modules and a dimensional mapping network to encode and process the neighboring agent observation information, obstacle observation information, and target information respectively, and outputs features that are independent of the number of agents and obstacles. Query, Key, and Value are the three major elements in the graph attention perception module. The location information of the current agent is regarded as Query, represented by Q, and the location information of neighboring agents, neighboring obstacles, and target points is regarded as Key, represented by K respectively. a , K o With K m Represents the high-dimensional features h of each vertex of the observation information ij =σ(o i W1) as the Value, where σ(·) represents the activation function, which defaults to the ReLU function, and W1 is the network weight. The following formulas are used to calculate the attention weights of the agent group, obstacle group, and target group respectively:

[0078]

[0079] in, are the attention weights of agent i to neighboring agents, obstacles, and target points, respectively. is the weight decay coefficient, and the softmax() function is used to normalize the attention, expressed as:

[0080]

[0081] By weighting the Value, we can obtain the aggregate characteristics of different groups:

[0082]

[0083] in, They are the high-dimensional features of agent j, obstacle k, and target g respectively; when obtaining the aggregated features After that, the output features of DAT are obtained through splicing operation, and the integration process is expressed as:

[0084]

[0085] in, is the network weight, (||) represents the concatenation operation.

[0086] Since the output features of the graph attention perception module are independent of the number of agents, obstacles, and targets, and the network weights and They are also independent of the number and scale of agents, obstacles, and targets. Therefore, the proposed graph attention perception module can enable the policy network to adapt to dynamic scale environments with different agents, obstacles, and targets.

[0087] (2) Generation of multimodal combined state information

[0088] To balance real-time performance with the efficiency of collaborative decision-making, the multimodal combined state information generation method described in this invention creates an information description of multimodal combined state perception by splicing the perception description results of each drone. This ensures that when making decisions, the intelligent agent not only considers its own local observations but also fully considers the perception information of other drones, thereby making decisions with higher collaborative efficiency. In this process, considering the communication limitations between drones, key features in RGB visual perception are extracted and reconstructed, and the raw data is grayscaled and compressed to reduce communication pressure and the computational burden of the decision-making network. The multi-agent multimodal combined state information generation process is as follows:

[0089] (2.1) Original visual data acquisition: RGB data is collected from the drone’s first-person perspective as the original visual perception input, with a size of 640×480×3.

[0090] (2.2) Object Detection and Reconstruction: The original visual image is fed into the deep learning-based single-stage object detection model YOLOv5 to extract the pixel locations of navigation targets, obstacles, and other drones in the image. Based on the positioning results, the three types of objects are visually reconstructed to ensure their accurate position information in the image.

[0091] Image dimensionality reduction and compression: To reduce communication pressure and the computational burden of the decision network, the reconstructed image is grayscaled to obtain a grayscale image of 640×480. Subsequently, the image is further compressed to 32×24 to reduce data transmission and computational burden.

[0092] (2.3) Image stitching: Horizontally stitch the compressed images from multiple drones. For two drones, the resulting stitching size is 64×24. This stitching method maintains image integrity while reducing data redundancy.

[0093] (2.4) Information fusion: After the image stitching is completed, the aggregated information about the three groups of drones, obstacles, and targets, as well as the action information, output by the graph attention perception module model are stitched into a 64×8 all-zero matrix. The result is then vertically spliced ​​with the multimodal image to obtain a 64×32 multimodal perception image.

[0094] (2.5) Time Series Stacking: Because UAV visual collaborative navigation requires spatial perception based on memorized state descriptions, this paper uses stacking multimodal perception results from multiple time steps as the multimodal combined state. For two UAVs, images from eight consecutive time steps are stacked to obtain a 512×32 multimodal combined state description, which serves as the input for the collaborative decision-making process.

[0095] Through the above steps, the multimodal combined state information generation method proposed in this paper can provide rich perception information while ensuring real-time performance, supporting efficient collaborative decision-making by drone swarms in complex dynamic environments. This method not only improves decision-making accuracy but also enhances the robustness and adaptability of the system, providing a solid foundation for autonomous navigation of drone swarms.

[0096] (3) Generation of multimodal combined state information

[0097] The UAV swarm collaborative decision-making model is designed based on a multi-agent reinforcement learning algorithm designed with a Transformer encoder for temporal feature extraction. It is designed to achieve adaptive target allocation and collaborative navigation for UAV swarms in partially observable complex environments, leveraging multimodal perception state information descriptions. The Transformer encoder is used to extract features from the multimodal perception state information descriptions. UAV collaborative navigation is formulated as a collaborative multi-agent reinforcement learning task. The global action value is considered the aggregate of each UAV's action value, with the following assumption: the global action value estimate can be considered the sum of each UAV's state-action value. To ensure that each UAV obtains the optimal action set when making a decision, each UAV's optimal action is a subset of the multimodal perception optimal action.

[0098] Discrete actions not only make the agent more robust to interference in the scene, noise in the control system, and the influence of different dynamic models, but also reduce the exponential growth of the action space caused by the increase in the number of drones. Therefore, the present invention defines three high-level instructions for drones, namely, for drone i, its action space A i ={a1, a2, a3, a4}, where a1 means going straight, a2 means turning left on the spot, a3 means turning right on the spot, and a4 means circling on the spot.

[0099] The overall idea behind the reward function design is to encourage all drones to reach their respective target locations without collision, while minimizing the time required to reach the target. However, since target assignment isn't done in advance, the reward function design must fully consider how autonomous target assignment can be achieved between drones, including penalties for multiple drones competing for the same target. It also requires consideration of obstacle avoidance between drones and obstacles, as well as collision avoidance in environments containing obstacles. For the reward at time t, the reward function is defined as follows:

[0100]

[0101] Indicates the penalty when the drone collides, since the safe flight radius of the drone is d safe Therefore, different thresholds need to be set for obstacle collision and drone collision during collision judgment. This reward is used to train the drone's obstacle avoidance and collision avoidance capabilities. The calculation formula is:

[0102]

[0103] Where: k is the number of UAVs that collide; R col It is the penalty value when a single drone collides; is the minimum distance between the drone and the surrounding obstacles; d1 is the threshold for the drone to determine collision with obstacles; d2 is the distance threshold for determining collision between drones, and its value is d safe ; It represents the sparse reward given when all drones in the environment correctly reach their respective target locations. The calculation formula is:

[0104]

[0105] in: is the coordinate value of the UAV in the x direction at time t: is the coordinate value of UAV i in the y direction at time t, It represents the sparse reward given when all drones in the environment correctly reach their respective target locations, calculated as:

[0106]

[0107] Where: R all The reward value for all drones arriving correctly; is the distance between the UAV and the target; d goal is the threshold for determining whether the drone reaches the target. In order to make the optimal situation that all drones arrive correctly, the final reward R all >n×R1, in actual training, R all=2n×R1,r t ang The direction correction reward introduced to ensure that the drone can face the target during navigation is related to the angle (azimuth) of the drone's current position pointing to the target position and the drone's yaw angle, and is calculated as:

[0108]

[0109] Where: w g Correction reward scaling factor for angle; is the two-dimensional coordinate position of the target at time t; is the projection of the drone's position on the XOY plane at time t; yaw t is the yaw angle of the drone at time t. The expression of the function atan2 is as follows:

[0110]

[0111] During training, when a drone discovers that a drone has already reached the target location it is heading towards, due to the maximum number of steps in the navigation task, the drone is driven to continue exploring the environment and navigate to the correct target location, thereby training the intelligent agent's ability to dynamically allocate tasks.

[0112] (4) Experiment and analysis

[0113] To validate the advantages of the proposed algorithm, experiments used the MADDPG, VDN, and TCDN algorithms as baselines. The Actor and Critic networks of MADDPG and VDN were constructed using a three-layer fully connected architecture. Furthermore, to demonstrate the importance of the DAT module in cluster collaborative control tasks in random dynamic environments, the graph attention perception module was removed for ablation experiments.

[0114] (5) Algorithm evaluation criteria

[0115] In the experiment, average speed error and average system loss were used as evaluation metrics to assess the control effectiveness of the cluster control algorithm at each iteration. These metrics can assess the system's speed consistency and cluster topology stability. These metrics measure the system's control error at each iteration and can be used as indicators for evaluating control effectiveness. The average cumulative system error was used to measure the control effectiveness of the entire control process. The average cumulative system error is defined as follows:

[0116]

[0117] The evaluation index η measures the control error of the system at each iteration and can be used as an indicator to evaluate the control effect. The evaluation index ξ can evaluate the speed consistency and cluster topology stability of the system and is defined as follows:

[0118]

[0119] When ζ(t) approaches 0, the speeds of the agents reach a consistent state and the system structure is in a stable state. Therefore, the stability of the system can be judged based on the ζ(t) curve.

[0120] (6) Experimental results and analysis

[0121] In reinforcement learning navigation training, a score curve is often used as an evaluation metric for training results. To verify the effectiveness of the designed navigation algorithm and its advantages over existing mainstream reinforcement learning frameworks, different algorithms were trained and compared. The score is calculated as the sum of the rewards for all drones. During training, the scores are averaged every 100 rounds, and the final average score curve is shown in the figure:

[0122] like Figure 3 The figure shows a comparison of the average score curves of our algorithm during training with those of the baseline algorithms MADDPG, VDN, and TCDN. Our algorithm's score improves more rapidly in the early stages of training, demonstrating superior exploration efficiency and policy optimization capabilities. The final score is significantly higher than that of the baseline algorithms, indicating that it can generate better collaborative strategies in complex dynamic environments and achieve higher task completion rates (e.g., target arrival and obstacle avoidance). The curves are more stable, demonstrating the algorithm's greater adaptability to random dynamic environments.

[0123] like Figure 4 The figure shows a comparison of ablation score curves for a full multimodal perception state description with and without the graph attention module. The full model achieves significantly higher scores than the ablation results, validating the critical role of the graph attention module in information aggregation. The graph attention module effectively captures the relationships between neighboring agents, obstacles, and targets through an attention mechanism, improving the accuracy of environmental perception and thus optimizing decision-making. The large fluctuations in ablation scores indicate that the strategy's adaptability to dynamic obstacles and targets is reduced when the graph attention module is missing.

[0124] like Figure 5 The figure shows the variation in error in the speed consistency of a drone swarm during training for different algorithms. Our algorithm achieves the lowest average speed error and the fastest convergence, demonstrating its ability to better coordinate swarm speeds and maintain formation stability. Baseline algorithms such as MADDPG lack global information fusion, resulting in large speed errors and prone to local oscillations. Low speed errors reflect the algorithm's ability to precisely control motion commands in a dynamic environment. Our algorithm exhibits faster convergence in the early stages of training and ultimately achieves a lower average speed error, demonstrating superior performance in speed consistency.

[0125] like Figure 6 The figure shows a comparison of the control errors (such as collision penalties and target deviation) of each algorithm during training. The system loss of our algorithm continues to decrease and stabilize at a low level, indicating that it can effectively reduce collisions and path deviations. The smoothness of the loss curve indicates that the reward function is well designed. The baseline algorithm has high loss and significant fluctuations due to the lack of multimodal perception (such as the vision + image attention perception module). The system loss of our algorithm gradually decreases during training and remains at a low level in the later stages, indicating its better performance in controlling error.

[0126] The experimental results above demonstrate that our algorithm performs well in the collaborative navigation task of drone swarms, particularly in terms of speed consistency, control error, cumulative error, and swarm topology stability. The introduction of a graph attention perception module and a global fusion reward mechanism further enhances the algorithm's adaptability and robustness in complex dynamic environments.

[0127] (7) Distributed training strategy

[0128] To further improve the training efficiency and adaptability of the autonomous collaborative navigation model for drone swarms, this paper introduces a data parallel strategy. Data parallelism divides the training data into multiple subsets and assigns them to different computing nodes. Each node independently calculates the gradient and then synchronously updates the model parameters using the AllReduce algorithm. The specific implementation steps are as follows:

[0129] Data partitioning: The multimodal collaborative perception state descriptions, actions, and reward data of the drone swarm are divided into multiple subsets by time step. Each subset contains data from consecutive time steps to preserve temporal information. Data partitioning is performed based on different task instances (such as different drone swarm configurations or environmental scenarios) to ensure uniform data distribution within each subset and avoid data bias.

[0130] Node allocation: Select appropriate computing nodes based on hardware resources. Each node is equipped with sufficient computing resources and memory to support large-scale data processing. Use a distributed data loader to ensure that the data subsets loaded by each node do not overlap and are evenly distributed.

[0131] Gradient synchronization: Using a parameter server architecture, each compute node calculates gradients and sends them to the parameter server, which aggregates the gradients and updates the global model parameters. After each training step, the AllReduce algorithm is used to synchronize gradients, exchanging them directly across all compute nodes and updating the global model parameters.

[0132] Optimizer configuration: Based on the characteristics of distributed training, adjust the learning rate appropriately. Generally, the learning rate should be proportional to the batch size to ensure convergence speed and stability. To avoid gradient explosion, use gradient clipping technology to limit the maximum norm of the gradient.

[0133] To validate the effectiveness of the data parallel strategy, we conducted a comparative experiment based on existing experiments. This experiment used eight NVIDIA V100 GPUs for distributed training, each equipped with 32GB of video memory. Using a dataset containing one million training examples, covering a variety of drone swarm missions in complex dynamic environments, we compared the performance of single-node training and data parallel training with respect to training time, focusing primarily on this metric.

[0134] Table 1. Time / speed performance comparison of different algorithms

[0135]

[0136] Table 1 shows the training time comparison of the four algorithms under the same training rounds. It can be seen that data parallel training significantly reduces the training time and shows higher training efficiency. Figure 7 The figures show the time consumed by different algorithms during training. This paper demonstrates that by introducing a data parallel strategy, the training efficiency and performance of the autonomous collaborative navigation model for drone swarms have been significantly improved. Data parallelism not only accelerates training but also generates a more optimized collaborative strategy for complex dynamic environments, improving mission completion rates and robustness. The experimental results validate the effectiveness and superiority of the data parallel strategy, providing a solid foundation for further research.

Claims

1. An autonomous navigation and control system for UAV swarms based on multimodal state perception and collaborative navigation decision-making, characterized by: It includes a multimodal combination state perception information module, a collaborative decision-making navigation module, and an algorithm evaluation module. The data passed through the multimodal combination state perception module consists of three parts. One part is the RGB data onboard the drone, which is reconstructed and spliced, and then grayscaled and reduced in dimension to obtain image information; the other part is the lidar data, which is grouped and aggregated by the graph attention network perception module for the information of the neighboring intelligent agent group, external obstacle group, and target group in the observation state; the last part is the drone's body motion information. The latter two parts are mapped together to the reconstructed image data, and multiple moments are stacked to obtain a multimodal combination state description; the collaborative decision-making module adopts a multi-agent deep reinforcement learning algorithm based on the Transformer encoder. By extracting the spatial and temporal features of the multimodal combination state description, it generates end-to-end motion instructions for each drone to ensure that the drone navigates to the target position without collision.

2. The UAV swarm autonomous navigation and control system based on multimodal state perception and collaborative navigation decision-making according to claim 1 is characterized in that: The construction of the graph attention network perception module includes dividing the observation state information into three groups: agents, obstacles and targets, using three graph attention perception modules and a dimensional mapping network to encode and process the neighboring agent observation information, obstacle observation information and target information respectively, and output features that are independent of the number and scale of agents and obstacles.

3. The UAV swarm autonomous navigation and control system based on multimodal state perception and collaborative navigation decision-making according to claim 1 is characterized in that: The elements in the graph attention perception module include Query, Key and Value. The location information of the current agent is regarded as Query and represented by Q. The location information of neighboring agents, neighboring obstacles and target points are regarded as Key and K respectively. a , K o With K m Represents the high-dimensional features h of each vertex of the observation information ij =σ(o i W1) as Value, where σ(·) represents the activation function, the default activation function is the ReLU function, and W1 is the network weight.

4. The UAV swarm autonomous navigation and control system based on multimodal state perception and collaborative navigation decision-making according to claim 1 is characterized in that: The formula for calculating the attention weight in the graph attention perception module is as follows: in, are the attention weights of agent i to neighboring agents, obstacles, and target points, respectively. is the weight decay coefficient, and the softmax() function is used to normalize the attention, expressed as: By weighting the Value, we can obtain the aggregate characteristics of different groups: in, They are the high-dimensional features of the agent, obstacle k, and target g respectively; in obtaining the aggregated features After that, the output features of DAT are obtained through splicing operation, and the integration process is expressed as: in, is the network weight, (||) represents the concatenation operation.

5. The UAV swarm autonomous navigation and control system based on multimodal state perception and collaborative navigation decision-making according to claim 1 is characterized in that: The multimodal combined state generation method establishes an information description of multimodal combined state perception by splicing the perception description results of each drone, extracts and reconstructs key features in RGB visual perception, and grayscales and compresses the original data.

6. The UAV swarm autonomous navigation and control system based on multimodal state perception and collaborative navigation decision-making according to claim 1 is characterized in that: The multimodal combined state generation method comprises the following steps: (1) Original visual data acquisition: RGB data is collected from the drone’s first-person perspective as the original visual perception input; (2) The original visual image is input into the single-stage target detection model YOLOv5 based on deep learning to extract the pixel positions of navigation targets, obstacles and other drones in the image. Based on the positioning results, the three types of objects are visually reconstructed; (3) Image dimensionality reduction and compression: grayscale processing of the reconstructed image; (4) Image stitching: horizontal stitching of compressed images from multiple drones; (5) Information fusion: After the image stitching is completed, the aggregated information about the three groups of drones, obstacles, and targets, as well as the action information, output by the image attention perception module model are stitched into an all-zero matrix. The result is then vertically spliced ​​with the multimodal image to obtain a multimodal perception image. (6) Time series stacking: stacking the multimodal perception results of multiple moments as a multimodal combination state.

7. The UAV swarm autonomous navigation and control system based on multimodal state perception and collaborative navigation decision-making according to claim 1 is characterized in that: The collaborative decision-making navigation module uses a Transformer encoder to extract features describing multimodal perception state information, treats UAV collaborative navigation as a collaborative multi-agent reinforcement learning task, and uses the global action value as the aggregation of the action value of each UAV.

8. The UAV swarm autonomous navigation and control system based on multimodal state perception and collaborative navigation decision-making according to claim 1 is characterized in that: The collaborative decision-making navigation module defines three high-level instructions for drones, namely, for drone i, its action space A i ={a1, a2, a3, a4}, where a1 means going straight, a2 means turning left, a3 means turning right, and a4 means circling; For the reward at time t, the reward function is defined as follows: Indicates the penalty when the drone collides, since the safe flight radius of the drone is d safe , The reward is used to train the drone's obstacle avoidance and collision avoidance capabilities. The calculation formula is: Where: k is the number of UAVs that collide; R col It is the penalty value when a single drone collides; is the minimum distance between the drone and the surrounding obstacles; d1 is the threshold for the drone to determine collision with obstacles; d2 is the distance threshold for determining collision between drones, and its value is d safe ; It represents the sparse reward given when all drones in the environment correctly reach their respective target locations, calculated as: Among them, R all The reward value for all drones arriving correctly; is the distance between the UAV and the target; d goal is the threshold for determining whether the UAV reaches the target, The direction correction reward introduced to ensure that the drone can face the target during navigation is related to the angle of the drone's current position pointing to the target position and the drone's yaw angle, and is calculated as: Among them, w g Correction reward scaling factor for angle; is the two-dimensional coordinate position of the target at time t; is the projection of the drone's position on the XOY plane at time t; yaw t is the yaw angle of the UAV at time t; the expression of the function atan2 is as follows: During training, when a drone finds that a drone has already reached the target location it is heading towards, it is driven to continue exploring the environment and navigate to the correct target location.

9. The UAV swarm autonomous navigation and control system based on multimodal state perception and collaborative navigation decision-making according to claim 1 is characterized in that: The algorithm evaluation module uses the MADDPG algorithm, the VDN algorithm, and the TCDN algorithm as baseline algorithms. The Actor network and the Critic network of MADDPG and VDN are constructed using three-layer full connections.

10. The UAV swarm autonomous navigation and control system based on multimodal state perception and collaborative navigation decision-making according to claim 1 is characterized in that: The algorithm evaluation module uses the average cumulative system error and evaluation indicators to measure the control effect of the entire control process. The mean cumulative systematic error is defined as follows: The evaluation index ξ can be used to evaluate the speed consistency and cluster topology stability of the system, and is defined as follows: When ζ(t) approaches 0, the speeds of the agents reach a consensus and the system structure is in a stable state.

Images