Machine Learning-Based Adaptive Optimization Method for UAV Swarm Environment

Abstract

This invention discloses a machine learning-based adaptive optimization method for UAV swarm environments, relating to the field of UAV swarm cooperative control technology. The method includes basic framework construction, training parameter optimization, dynamic policy adjustment, anti-interference communication enhancement, and fault-tolerant self-reconstruction. It couples an optimized federated learning model with a multimodal data fusion module, fuses time-synchronized sensor data using a self-attention mechanism, constructs a high-dimensional state vector containing agent states and environmental features, inputs it into a reinforcement learning framework to generate joint action decisions, implements fault detection through an LSTM network, and utilizes redundant sensor data and the multimodal fusion model for fault compensation. Through dynamic graph attention and multi-objective federated learning, the swarm can dynamically adjust its strategy to cope with complex environments such as electromagnetic interference and obstacle changes. Multi-agent cooperative decision-making and residual compensation fault self-healing mechanisms ensure that the swarm can still complete its tasks when nodes fail or communication is interrupted.

Description

Technical Field

[0001] This invention relates to the field of collaborative control technology for unmanned aerial vehicle (UAV) swarms, and more particularly to an adaptive optimization method for UAV swarm environments based on machine learning. Background Technology

[0002] Existing drone swarm control methods typically rely on preset rules or fixed strategies, which are difficult to cope with complex and ever-changing environmental conditions (such as electromagnetic interference, dynamic changes in obstacles, communication interruptions, etc.).

[0003] The existing technology has the following core defects:

[0004] Environmental modeling is crude: Traditional methods rely on manually designed environmental features, which makes it difficult to capture continuous spatial features such as dynamic electromagnetic spectrum and airflow disturbance, resulting in decision lag (average delay of more than 200ms).

[0005] Low collaboration efficiency: The centralized control architecture has a single point of failure (the mission failure rate exceeds 70% when the lead drone loses contact), while the distributed method has high communication overhead, with a single node transmitting data at a rate of 500KB per second, resulting in excessive energy consumption.

[0006] Insufficient robustness: Under strong electromagnetic interference of -20dBm, the bit error rate of traditional communication protocols exceeds 30%, and there is a lack of dynamic compensation mechanism when the sensor fails. When monocular vision fails, the positioning error increases by 5 times.

[0007] In view of this, the present invention is proposed to solve the above-mentioned technical problems. Summary of the Invention

[0008] The purpose of this invention is to provide a machine learning-based adaptive optimization method for UAV swarm environments, in order to solve the technical problems of existing UAV swarm optimization methods, such as extensive environmental modeling, low collaborative efficiency, and insufficient robustness.

[0009] The purpose of this invention is to provide a machine learning-based adaptive optimization method for drone swarm environments, comprising the following steps:

[0010] S1. Basic Framework Construction: The interaction relationships between agents and between agents and the environment are modeled through graph attention mechanism, and action policies are generated based on the hierarchical Actor-Critic network architecture to form a multi-agent reinforcement learning framework.

[0011] S2. Training parameter optimization: Each UAV uses environmental data collected by its local multimodal sensors to conduct distributed training through a federated learning mechanism to establish a multi-objective federated learning model, providing efficient model support for real-time decision-making;

[0012] S3. Dynamic policy adjustment: The optimized federated learning model is coupled with the multimodal data fusion module. The sensor data after time synchronization is fused by a self-attention mechanism to construct a high-dimensional state vector containing agent state and environmental features. The vector is then input into the reinforcement learning framework to generate joint action decisions.

[0013] S4. Enhanced anti-interference communication: During action decision-making and execution, the electromagnetic environment is monitored in real time through the spectrum sensing module. When the energy density in the 2.4GHz band is detected to be >−80dBm / MHz and lasts for more than 200ms, a three-layer communication adaptive mechanism is triggered. The three-layer communication adaptive mechanism is as follows:

[0014] Physical layer: Enable visible light communication modules with a wavelength of 850nm;

[0015] Data link layer: Time division multiple access (TDMA) is used to replace contention-based access, reducing the probability of collisions;

[0016] Network layer: Based on dynamic Bayesian network prediction of link state, optimize the Hello message interval of OLSR protocol;

[0017] S5. Fault-Tolerant Self-Reconfiguration: For scenarios involving potential sensor failures or node disconnections, fault detection is achieved through an LSTM network. Fault compensation is performed using redundant sensor data and a multimodal fusion model. The fault-tolerant self-reconfiguration employs fuzzy adaptive control and a distributed consensus algorithm to achieve sensor fault detection and cluster topology self-reconfiguration. A Long Short-Time Memory (LSTM) prediction model is constructed. When the residual data from the Inertial Measurement Unit (IMU) exceeds three times the standard deviation, fault isolation is triggered. The self-reconfiguration algorithm uses an improved distributed consensus protocol.

[0018] (k+1)=α (k) + (1-α) (k), where α is the inertia coefficient, The adjacency matrix weights are used to achieve smooth reconstruction of the cluster topology by dynamically adjusting α.

[0019] Furthermore, the multi-agent reinforcement learning framework employs a shared experience pool and distributed training, enabling agents to achieve collaborative optimization through joint action decisions.

[0020] Furthermore, the federated learning mechanism dynamically adjusts the number of local training rounds and convergence accuracy through a successive convex approximation algorithm, minimizing the total energy consumption of the cluster.

[0021] Furthermore, multimodal data fusion employs a self-attention mechanism to construct an environmental state vector to support the decision-making of the reinforcement learning model.

[0022] Furthermore, the anti-interference communication enhancement in step S4 adopts an adaptive communication protocol, including dynamic adjustment of OLSR parameters and switching of optical communication links, to cope with electromagnetic interference.

[0023] Furthermore, in the multi-agent reinforcement learning framework, a dynamic interaction graph containing environment nodes and agent nodes is constructed through a graph attention mechanism, and the agent collaborative weights are calculated in real time. The formula for calculating the collaborative weights is:

[0024] ,in, For collaborative attention weights among agents, Let be the state vector of agent i at time t. Let be the state vector of agent j at time t, and let leakyReLU be the activation function. Softmax is the activation function. j To perform Softmax normalization of the weights in dimension j, This is the weight matrix for the attention mechanism.

[0025] Federated learning mechanisms achieve energy-accuracy balance through a multi-objective optimization function, which is: + - A, and , representing the local training rounds and the convergence accuracy threshold, respectively. , , These are the weighting coefficients. Let A be the objective function term.

[0026] Furthermore, the switching conditions for the optical communication link are as follows: when the energy density of the 2.4GHz band is >−80dBm / MHz and the duration is >200ms, the 850nm visible light communication module is activated, and the OLSR protocol Hello interval is adjusted to: T hello =T0˙ ,in, Let T be the link connectivity probability, β be an adaptive coefficient ranging from 0.5 to 1.0, and T be the link connectivity probability. hello T0 represents the dynamically adjusted Hello message sending interval.

[0027] By adopting the above technical solution, the present invention has the following beneficial effects:

[0028] Environmental adaptability: Through dynamic graph attention mechanism and multi-objective federated learning, the cluster can dynamically adjust its strategy to cope with complex environments such as electromagnetic interference and obstacle changes, reducing task completion time by 40% and energy consumption by 35%.

[0029] Communication efficiency: Federated learning reduces data transmission by 92%, and visible light communication maintains a success rate of over 98% under strong interference, with latency reduced by 53% and energy consumption reduced by 63%.

[0030] Task reliability: Multi-agent collaborative decision-making and residual compensation fault self-healing mechanism ensure that the cluster can still complete the task when nodes fail or communication is interrupted. When the sensor fails, the positioning accuracy drops by only 15%, and the cluster's self-weight composition power reaches 99.2%. Attached Figure Description

[0031] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention, but do not constitute an undue limitation of the invention. Obviously, the drawings described below are merely some embodiments, and those skilled in the art can obtain other drawings based on these drawings without creative effort. In the drawings:

[0032] Figure 1 This is a framework diagram of the machine learning-based adaptive optimization method for drone swarm environment provided in this embodiment of the application.

[0033] It should be noted that these accompanying drawings and textual descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art by referring to specific embodiments. Detailed Implementation

[0034] This application provides a machine learning-based adaptive optimization method for drone swarm environments, including the following steps:

[0035] S1. Basic Framework Construction: The interaction between agents and between agents and the environment is modeled through graph attention mechanism. Action policies are generated based on the hierarchical Actor-Critic network architecture to form a multi-agent reinforcement learning framework. Each node in the drone swarm is an independent agent. A dynamic interaction graph containing environmental nodes (obstacles, interference sources) and agent nodes is constructed. The collaborative weights between agents are calculated in real time through graph attention mechanism (GAT).

[0036] Actor Network: Divided into a global policy layer and an individual execution layer. The global policy layer outputs a cluster-level task allocation matrix ϵ. The individual execution layer outputs flight control commands (speed, heading angle, obstacle avoidance acceleration) based on global decisions and local state.

[0037] Critic Network: Introduces a contrastive learning mechanism to improve the accuracy of value function estimation through positive and negative sample pairs, reducing training loss by 40% compared to traditional methods. The agent performs distributed training through a shared experience pool, dynamically adjusting its strategy to adapt to environmental changes.

[0038] S2. Training Parameter Optimization: Each UAV utilizes environmental data collected by its local multimodal sensors for distributed training via a federated learning mechanism to establish a multi-objective federated learning model. This provides efficient model support for real-time decision-making. The optimal number of local training rounds is determined using the Lagrange multiplier method. and convergence accuracy This reduces communication energy consumption and latency while ensuring the accuracy of the global model. An asynchronous aggregation mechanism is introduced, with the lead drone setting a dynamic aggregation window (100ms-500ms), allowing nodes with weaker computing power to prioritize uploading lightweight model parameters (transmitting only gradient symbols, reducing data volume by 90%).

[0039] S3. Dynamic policy adjustment: The optimized federated learning model is coupled with the multimodal data fusion module. The sensor data after time synchronization is fused by a self-attention mechanism to construct a high-dimensional state vector containing agent state and environmental features. The vector is then input into the reinforcement learning framework to generate joint action decisions.

[0040] S4. Enhanced anti-interference communication: During action decision-making and execution, the electromagnetic environment is monitored in real time through the spectrum sensing module. When the energy density in the 2.4GHz band is detected to be >−80dBm / MHz and lasts for more than 200ms, a three-layer communication adaptive mechanism is triggered. The three-layer communication adaptive mechanism is as follows:

[0041] Physical layer: Enable 850nm wavelength visible light communication module (communication distance 500m, bit error rate <10). -6 );

[0042] Data link layer: Time division multiple access (TDMA) is used to replace contention-based access, reducing the probability of collisions;

[0043] Network layer: Based on dynamic Bayesian network prediction of link state, optimize the Hello message interval of OLSR protocol;

[0044] S5. Fault-tolerant self-reconfiguration: For scenarios where sensors fail or nodes become disconnected, fault detection is achieved through an LSTM network, and fault compensation is performed using redundant sensor data and a multimodal fusion model.

[0045] The multi-agent reinforcement learning framework employs a shared experience pool and distributed training, with agents achieving collaborative optimization through joint action decisions.

[0046] The federated learning mechanism dynamically adjusts the number of local training rounds and convergence accuracy through a successive convex approximation algorithm, minimizing the total energy consumption of the cluster.

[0047] Multimodal data fusion employs a self-attention mechanism to construct an environmental state vector to support the decision-making of the reinforcement learning model. The UAV is equipped with multimodal sensors (LiDAR, camera, spectrum sensing module, etc.) to collect environmental data in real time. After time synchronization (accuracy <1ms), the data is fused using a self-attention mechanism to construct an 18-dimensional environmental state vector (including position, velocity, battery level, sensor status, environmental features, etc.). This vector is then input into the reinforcement learning model to generate joint action decisions, achieving collaborative optimization of path planning, task allocation, and obstacle avoidance. The processing time for a single frame is <15ms.

[0048] The anti-interference communication enhancement in step S4 adopts an adaptive communication protocol, including dynamic adjustment of OLSR parameters and switching of optical communication links, in order to cope with electromagnetic interference.

[0049] The fault-tolerant self-reconfiguration mechanism employs fuzzy adaptive control and a distributed consensus algorithm to achieve sensor fault detection and cluster topology self-reconfiguration. A Long Short-Time Memory (LSTM) predictive model is constructed. When the residual data from the Inertial Measurement Unit (IMU) exceeds three times the standard deviation, fault isolation is triggered (detection latency <50ms). The self-reconfiguration algorithm uses an improved distributed consensus protocol.

[0050] (k+1)=α (k) + (1-α) (k), where α is the inertia coefficient, As the adjacency matrix weight, α (0.3-0.7) is dynamically adjusted to achieve smooth reconstruction of the cluster topology (reconstruction time <200ms).

[0051] In the multi-agent reinforcement learning framework, a dynamic interaction graph containing environment nodes and agent nodes is constructed through a graph attention mechanism. The agent collaborative weights are calculated in real time, and the formula for calculating the collaborative weights is:

[0052] ,in, For collaborative attention weights among agents, Let be the state vector of agent i at time t. Let be the state vector of agent j at time t, and let leakyReLU be the activation function. Softmax is the activation function. j To perform Softmax normalization of the weights in dimension j, This is the weight matrix for the attention mechanism.

[0053] Federated learning mechanisms achieve energy-accuracy balance through a multi-objective optimization function, which is: + - A, and , representing the local training rounds and the convergence accuracy threshold, respectively. , , These are the weighting coefficients. Let A be the objective function term.

[0054] The conditions for switching optical communication links are as follows: when the energy density of the 2.4GHz band is >−80dBm / MHz and the duration is >200ms, the 850nm visible light communication module is activated, and the OLSR protocol Hello interval is adjusted to: T hello =T0˙ ,in, Let T be the link connectivity probability, β be an adaptive coefficient ranging from 0.5 to 1.0, and T be the link connectivity probability. hello T0 represents the dynamically adjusted Hello message sending interval.

[0055] 1. System Deployment

[0056] The drone swarm consists of k follower drones and 1 leader drone, with the following hardware configuration:

[0057] Table 1 Hardware parameters of drone swarm

[0058]

[0059] 2. Training Phase

[0060] (1) Initialization

[0061] State space: Define an 18-dimensional state vector S, which includes position (x, y, z), velocity (v_x, v_y, v_z), quaternion pose (q_1, q_2, q_3, q_4), remaining battery power E, sensor health status (3-dimensional one-hot encoding), and environmental features (6 dimensions such as obstacle distance and interference intensity).

[0062] Action space: Includes 12 discrete actions, including 4 types of mission assignment (reconnaissance, transport, relay, return) and 8 types of flight control (forward, backward, left, right translation, ascent, descent, and turning).

[0063] (2) Online learning

[0064] At each time step t, agent i collects its local state. Actions are generated using the DGAT-MARL network. Receive a reward after execution (The reward function includes five dimensions: task completion, energy consumption, and security, and employs a sparse reward + intrinsic incentive mechanism.) Experience tuple ( , , , The data is stored in the local experience pool. When the pool reaches 1000, the old experience is mixed with the training data at a ratio of 1:10 to avoid overfitting.

[0065] (3) Federated learning aggregation

[0066] The lead drone initiates parameter aggregation every 5 seconds, and each node dynamically adjusts its local training rounds based on its remaining battery power. (When battery level > 50%) =20, otherwise =5), and by using gradient compression technology (transmitting only the first 20% of the gradient in absolute value), the amount of communication was reduced, and the amount of data in a single aggregation was reduced from 500KB to 40KB.

[0067] 3. Task Execution Phase

[0068] (1) Environmental perception and decision making

[0069] Multimodal sensor data is fused through a self-attention mechanism to generate a 128-dimensional environmental state vector, which is then input into the DGAT-MARL network to output a joint action, including a task allocation matrix T and individual control commands. This action is converted into a PWM signal to drive the motor via a kinematic model, with the decision delay controlled within 80ms.

[0070] (2) Implementation of anti-interference communication

[0071] The spectrum analyzer scans the 2-6GHz frequency band at 100Hz. When interference power is detected to be greater than the threshold for three consecutive cycles, the visible light communication module is initialized (synchronization signal generation, light intensity modulation parameter configuration, time <50ms). At the same time, the OLSR protocol dynamically adjusts the Hello interval (default 1s, shortened to 0.2s during interference) according to the link quality, and enables network coding technology to improve throughput by 30%.

[0072] (3) Fault handling mechanism

[0073] When the prediction residual of the LSTM model exceeds 3σ for two consecutive cycles, a fault signal is sent to neighboring nodes, and sensor data fusion compensation is initiated (when GPS fails, positioning is achieved through visual SLAM + barometer fusion, maintaining an accuracy of ±50cm). When a node fails, neighboring nodes recalculate neighbor weights using a distributed consensus algorithm, completing topology reconstruction within three communication cycles (100ms per cycle).

[0074] This specific embodiment is merely an explanation of the invention and is not intended to limit the invention. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but as long as they are within the scope of protection of this invention, they are protected by patent law.

Claims

1. A machine learning-based adaptive optimization method for unmanned aerial vehicle (UAV) swarm environments, characterized in that, Includes the following steps: S1. Basic Framework Construction: The interaction relationships between agents and between agents and the environment are modeled through graph attention mechanism, and action policies are generated based on the hierarchical Actor-Critic network architecture to form a multi-agent reinforcement learning framework. S2. Training parameter optimization: Each UAV uses environmental data collected by its local multimodal sensors to conduct distributed training through a federated learning mechanism to establish a multi-objective federated learning model, providing efficient model support for real-time decision-making; S3. Dynamic policy adjustment: The optimized federated learning model is coupled with the multimodal data fusion module. The sensor data after time synchronization is fused by a self-attention mechanism to construct a high-dimensional state vector containing agent state and environmental features. The vector is then input into the reinforcement learning framework to generate joint action decisions. S4. Enhanced anti-interference communication: During action decision-making and execution, the electromagnetic environment is monitored in real time through the spectrum sensing module. When the energy density in the 2.4GHz band is detected to be >−80dBm / MHz and lasts for more than 200ms, a three-layer communication adaptive mechanism is triggered. The three-layer communication adaptive mechanism is as follows: Physical layer: Enable visible light communication modules with a wavelength of 850nm; Data link layer: Time division multiple access (TDMA) is used to replace contention-based access, reducing the probability of collisions; Network layer: Based on dynamic Bayesian network prediction of link state, optimize the Hello message interval of OLSR protocol; S5. Fault-Tolerant Self-Reconfiguration: For scenarios involving potential sensor failures or node disconnections, fault detection is achieved through an LSTM network. Fault compensation is performed using redundant sensor data and a multimodal fusion model. The fault-tolerant self-reconfiguration employs fuzzy adaptive control and a distributed consensus algorithm to achieve sensor fault detection and cluster topology self-reconfiguration. A Long Short-Time Memory (LSTM) prediction model is constructed. When the residual data from the Inertial Measurement Unit (IMU) exceeds three times the standard deviation, fault isolation is triggered. The self-reconfiguration algorithm uses an improved distributed consensus protocol. (k+1)=α (k) + (1-α) (k), where α is the inertia coefficient. The adjacency matrix weights are used to achieve smooth reconstruction of the cluster topology by dynamically adjusting α.

2. The machine learning-based adaptive optimization method for UAV swarm environments according to claim 1, characterized in that, The multi-agent reinforcement learning framework employs a shared experience pool and distributed training, with agents achieving collaborative optimization through joint action decisions.

3. The machine learning-based adaptive optimization method for UAV swarm environments according to claim 2, characterized in that, The federated learning mechanism dynamically adjusts the number of local training rounds and convergence accuracy through a successive convex approximation algorithm, minimizing the total energy consumption of the cluster.

4. The machine learning-based adaptive optimization method for UAV swarm environments according to claim 3, characterized in that, The multimodal data fusion employs a self-attention mechanism to construct an environmental state vector to support the decision-making of the reinforcement learning model.

5. The machine learning-based adaptive optimization method for UAV swarm environments according to claim 1, characterized in that, The anti-interference communication enhancement in step S4 adopts an adaptive communication protocol, including dynamic adjustment of OLSR parameters and switching of optical communication links, in order to cope with electromagnetic interference.

6. The machine learning-based adaptive optimization method for UAV swarm environment according to claim 2, characterized in that, In the multi-agent reinforcement learning framework, a dynamic interaction graph containing environment nodes and agent nodes is constructed through a graph attention mechanism, and the agent collaborative weights are calculated in real time. The formula for calculating the collaborative weights is: ,in, For collaborative attention weights among agents, Let be the state vector of agent i at time t. Let be the state vector of agent j at time t, and let leakyReLU be the activation function. Softmax is the activation function. j To perform Softmax normalization of the weights in dimension j, This is the weight matrix for the attention mechanism.

7. The machine learning-based adaptive optimization method for UAV swarm environments according to claim 3, characterized in that, The federated learning mechanism achieves energy-precision balance through a multi-objective optimization function, which is: + - A, and , representing the local training rounds and the convergence accuracy threshold, respectively. , , These are the weighting coefficients. Let A be the objective function term.

8. The machine learning-based adaptive optimization method for UAV swarm environments according to claim 5, characterized in that, The optical communication link switching conditions are as follows: when the energy density of the 2.4GHz band is >−80dBm / MHz and the duration is >200ms, the 850nm visible light communication module is activated, and the OLSR protocol Hello interval is adjusted to: T hello =T0˙ ,in, Let T be the link connectivity probability, β be an adaptive coefficient ranging from 0.5 to 1.0, and T be the link connectivity probability. hello T0 represents the dynamically adjusted Hello message sending interval.

Images