Unmanned aerial vehicle cluster anti-jamming communication method based on cross-layer cooperation and attention mechanism

Abstract

This invention discloses an anti-jamming communication method for UAV swarms based on cross-layer collaboration and attention mechanisms. The method includes: constructing a communication scenario for UAV swarms under complex electromagnetic interference sources; establishing a UAV swarm communication link model and a swept-frequency comb interference model; constructing a node state awareness model and a multi-hop data transmission model based on fixed routing constraints; designing evaluation indices for transmission link quality and UAV swarm topology maneuver efficiency; establishing a UAV swarm topology control model based on potential energy field and signal-to-interference-plus-noise ratio feedback; designing a deep recurrent Q-network based on an attention mechanism; designing a UAV swarm topology scaling and movement control algorithm based on the deep Q-network; constructing a cross-layer collaborative joint optimization framework and designing a multi-agent collaborative training method. This invention can significantly improve the data transmission success rate of UAV swarms from the source node to the destination node via multi-hop routing in strong interference environments, ensuring the stability of multi-hop data transmission.

Description

Technical Field

[0001] This invention relates to the field of wireless communication and UAV swarm collaborative control technology, specifically to an anti-interference communication method for UAV swarms based on cross-layer collaboration and attention mechanisms. Background Technology

[0002] In recent years, with the rapid development of unmanned aerial vehicle (UAV) technology, the application value of UAV swarms in military reconnaissance, emergency communication restoration, and collaborative operations has become increasingly prominent. As aerial information exchange nodes, UAV swarms heavily rely on highly reliable, low-latency wireless communication links to maintain formation and transmit critical intelligence data. However, with the upgrading of electronic warfare technology, the communication environment has become increasingly complex. In particular, malicious interference from the enemy can lead to frequent communication link interruptions, severely limiting the swarm's mission execution capabilities. Furthermore, the highly dynamic flight characteristics of UAV swarms, intertwined with a complex electromagnetic environment, make maintaining stable end-to-end data transmission extremely challenging.

[0003] In anti-jamming communication scenarios of UAV swarms, there is a strong coupling relationship between communication resource allocation and trajectory control, but existing solutions often struggle to achieve both simultaneously, specifically in the following aspects:

[0004] Regarding the intelligence of anti-interference decision-making, when faced with interference sources that possess periodic frequency sweeping or comb-like suppression capabilities, traditional frequency hopping techniques typically rely on preset pseudo-random sequences for spectrum switching. This lacks dynamic perception and adaptive capabilities to environmental changes, making them susceptible to tracking or large-area coverage by interference sources. Although deep reinforcement learning has been introduced into spectrum decision-making, in multi-drone scenarios, the number of neighboring nodes dynamically fluctuates with topology changes. Traditional fully connected neural networks struggle to handle variable-length inputs, preventing the agent from effectively utilizing cooperative information from local neighbors and resulting in blind spectrum decision-making.

[0005] In cross-layer collaboration, existing research mostly adopts a "divide and conquer" strategy: physical layer research focuses only on power control and channel selection, while trajectory planning research focuses only on obstacle avoidance and reaching the destination. In practical applications, if UAVs only adjust their frequency points without maneuvering to avoid areas with strong interference, the anti-interference effect is limited; conversely, if they simply move blindly without considering the connectivity of communication links and routing constraints, it may lead to a dispersed formation, making it impossible to maintain an effective data transmission link. Existing technologies lack a cross-layer collaboration mechanism that combines distributed spectrum decision-making with a centralized cluster topology space.

[0006] Regarding the reliability of multi-hop routing, drone swarms typically employ a multi-hop self-organizing network mode for data transmission. Existing anti-interference algorithms often only optimize the signal-to-interference-plus-noise ratio (SIR) of a single-hop link, neglecting the cascading effect of routing. In fixed routes or specific task flows, a spectral decision error or improper location of an intermediate node can directly lead to the breakdown of the entire link.

[0007] In general, current anti-jamming communication methods for UAV swarms in complex electromagnetic environments mainly face the following three problems: 1) Insufficient perception of dynamic and variable-length neighbor characteristics: Existing DRL algorithms struggle to handle dynamic neighbor state inputs caused by swarm topology changes, resulting in low efficiency of multi-agent collaborative anti-jamming. 2) Lack of deep coordination between frequency decision-making, power selection, and topology trajectory: Communication resource allocation and spatial trajectory planning are disconnected, making it impossible to maximize anti-jamming performance through a closed-loop mechanism of "situational awareness - position adjustment - spectrum avoidance". 3) Lack of end-to-end guarantee mechanism with route awareness: Optimization of a single link often ignores the dependencies between hops, making it difficult to maintain the integrity and stability of multi-hop routes under strong interference. Summary of the Invention

[0008] In complex electromagnetic environments, UAV swarms face numerous challenges, including high interference intensity and easily broken communication links. Traditional methods have significant limitations in handling dynamic neighbor characteristics and cross-layer collaborative optimization. To address these challenges in existing UAV swarm communication technologies, this invention provides an anti-interference communication method for UAV swarms based on cross-layer collaboration and attention mechanisms. This method significantly improves the data transmission success rate of UAV swarms from the source node to the destination node in strong interference environments, ensuring the stability of multi-hop data transmission and efficient task completion.

[0009] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:

[0010] A method for anti-jamming communication of UAV swarms based on cross-layer collaboration and attention mechanisms, characterized in that the method includes the following steps:

[0011] Step S1: Construct a communication scenario for a drone swarm under complex electromagnetic interference sources.

[0012] Step S2: Construct a UAV swarm communication link model and a frequency-sweeping comb interference model;

[0013] Step S3: Construct a node state awareness model and a multi-hop data transmission model based on fixed routing constraints;

[0014] Step S4: Design evaluation indicators for transmission link quality and UAV swarm topology maneuver efficiency;

[0015] Step S5: Construct a topology control model for UAV swarm based on potential energy field and signal-to-interference-plus-noise ratio feedback;

[0016] Step S6: Design a deep recurrent Q-network based on the attention mechanism to handle dynamic neighbor features, frequency decision and power selection problems;

[0017] Step S7: Design an algorithm for topology scaling and mobility control of UAV swarms based on deep Q-networks;

[0018] Step S8: Construct a cross-layer collaborative joint optimization framework, in which the UAV swarm topology control in step S5 and the UAV frequency decision and power selection in step S6 are executed alternately and mutually fed back in time steps, and the optimal anti-interference path planning and resource allocation strategy is obtained through multi-agent collaborative training.

[0019] Step S1 specifically includes:

[0020] Set up a simulation scenario, which includes... A swarm of drones A set of interference sources, starting points, and target endpoints, represented as a collection of UAVs. The set of interference sources is represented as The drone can be represented by a logical channel as The set of selectable transmit power levels represents The communication routing table of the drone swarm is This includes multiple fixed multi-hop links, and each drone uses a routing table... Determine its predecessor and successor nodes, and in the logical channel set With power set The process involves independently selecting actions to establish a communication link.

[0021] Discretize the duration of the drone swarm flight mission into Equal time slots, in the first In each time slot, a cluster topology model based on a virtual center point is established, and the cluster state is determined by the coordinates of the virtual center point. and topology scaling radius Jointly decided, the The _th time slot, the _th ... The location coordinates of the drone Represented as:

[0022]

[0023] in, Indicates the first The offset of the drone relative to the virtual center point is determined by the topology scaling radius. and the preset relative angle Decision; Virtual Center Point The update follows the following kinematic rules:

[0024]

[0025] In the formula, Indicates the first Actions selected for each time slot This represents the displacement distance per unit time step. The following boundary constraints must be satisfied:

[0026]

[0027] in, and These represent the X and Y grid boundary dimensions of the scene, respectively. and Representing the virtual center point The X and Y coordinates.

[0028] Step S2 specifically includes:

[0029] Sending drone With the receiving drone Channel power gain between :

[0030]

[0031] in, Indicates drone and The Euclidean distance between them Indicates the reference distance. This represents the path loss constant at the reference distance. This represents the path loss index.

[0032] In the Each time slot, the receiving drone In the channel The signal-to-interference-to-noise ratio is :

[0033]

[0034] In the formula, Indicates the sending drone The transmission power, This represents the power of additive white Gaussian noise. This indicates interference from other drones operating at the same frequency within the cluster. The average signal-to-interference-plus-noise ratio (SIR) of a drone swarm represents the suppression interference from external sources. ;

[0035] Interference from other drones operating at the same frequency within the cluster Represented as:

[0036]

[0037] In the formula, As an indicator function, when the drone Channel occupied The value is 1 if it is true, and 0 otherwise.

[0038] External interference Depend on An interference source is generated, represented as:

[0039]

[0040] in, Indicates the transmission power of the interference source. Indicates the source of interference To drones Channel gain;

[0041] Interference source Employing a frequency-sweeping comb jamming strategy, its first... A set of time-slot interference channels It consists of a reference frequency and comb harmonic frequencies:

[0042]

[0043]

[0044]

[0045] in, Indicates the time step of the disturbance dwell time. This represents the initial phase shift from different interference sources. Represents the total number of channels. This indicates the frequency interval of the comb interference.

[0046] Step S3 specifically includes:

[0047] According to the preset routing table For any drone in the cluster This determines its logical role in the routing link and identifies its unique predecessor node. and successor nodes ;

[0048] drones In the Local observation state vector of each time slot This vector is composed of its own physical state characteristics. Local environment interaction features and route-aware features It consists of three parts:

[0049]

[0050] Among them, route-aware features Includes the following information: predecessor node The transmit channel index used in the previous time slot Successor node The receive channel index used in the previous time slot Successor node The receiver signal-to-interference-plus-noise ratio measured in the previous time slot ,

[0051] Construct a multi-hop data transmission model and define a complete routing link. From node sequence Composed of, this link in the The condition for successful data transmission in a time slot is that each hop in the link meets the communication quality threshold:

[0052]

[0053] in, Indicates the link number The actual signal-to-interference-plus-noise ratio at the receiver. This represents the minimum SINR threshold required for reliable data transmission. This indicates that the end-to-end data transmission is considered successful only if all hops of the entire link are connected.

[0054] Step S4 specifically includes:

[0055] For any communication link In its first Effective margin of time slot link Defined as the actual signal-to-interference-plus-noise ratio at the receiver. Minimum SINR threshold required for reliable data transmission The difference:

[0056]

[0057] Among them, when When the value is 1, it indicates that the link meets the requirements for reliable transmission, and the larger the value, the higher the anti-interference margin; when When this occurs, it indicates that the link is in an interrupted state;

[0058] Definition of the first Average spectrum switching rate of time-slotted drone swarms for:

[0059]

[0060] in, This represents a collection of drones currently in the launch state. Indicates drone The transmission channel selected in the current time slot For indicator functions, The number of drones in the launch state;

[0061] Definition of the first Virtual center point of time-slotted drone swarm Relative to the target endpoint Normalized residual distance :

[0062]

[0063] in, This represents the total path distance at the start of the task.

[0064] In step S5, the UAV swarm topology control model based on potential energy field and signal-to-interference-plus-noise ratio feedback is as follows:

[0065] Composite potential field It is formed by the coupling of the target gravitational field, the communication mass field, and the boundary repulsive field, in the first... Time slot, gravitational potential energy gain based on relative distance variation Represented as:

[0066]

[0067] in, This represents the distance potential energy weighting coefficient. Represents the Euclidean distance norm.

[0068] Average signal-to-interference-plus-noise ratio introduced by drone swarm Constructing a communication quality field and communication potential energy Defined as:

[0069]

[0070] in, This represents the communication potential weighting coefficient; Represents a saturated nonlinear mapping function;

[0071] Boundary repulsive potential energy Used to restrict the virtual center point from going out of the task area. At the same time, combined with the remaining energy Imposing punishment:

[0072]

[0073] in, and These represent the out-of-bounds penalty value and the energy depletion penalty value, respectively.

[0074] Introducing gated functions Dynamically constraining gravitational potential energy, the total utility function of all UAV transmit / receive pairs Defined as:

[0075]

[0076] Among them, the gate function Represented as:

[0077]

[0078] When the average signal-to-interference-plus-noise ratio is below the safety threshold When the gating is closed, the cluster will not receive a gravitational potential energy reward even if it moves towards the destination, and will trigger an additional communication penalty. .

[0079] Step S6 specifically includes:

[0080] S61 addresses the dynamic changes in the number of neighbors and the heterogeneity of local observation dimensions in UAV swarm communication by constructing a feature extractor based on a fully connected neural network for any UAV. , its own observation vector and neighbor observation set Mapped to higher-dimensional space respectively;

[0081] S62 utilizes an attention mechanism to dynamically calculate neighbor weights, defining the query vector as originating from its own features. The key vector and value vector are derived from the features of the neighbors. Calculate attention aggregation features :

[0082]

[0083] Constructing a hybrid feature fusion vector It consists of its own features, attention aggregation features, and the features of its nearest neighbor in spatial distance. It is pieced together to enhance the ability to detect interference sources and routing nodes:

[0084]

[0085] S63, fused vector Input LSTM unit:

[0086]

[0087] The hidden state is decoupled from the output layer by multiple heads. Mapped to actions.

[0088] Step S7 specifically includes:

[0089] S71, Construct the topology state space of the drone swarm, the first Time slot input state vector This includes the normalized value of the cluster average signal-to-interference-plus-noise ratio. Normalized coordinates of the current cluster virtual center point Current topology scaling radius , Pre-planned global path completion progress index Perceived interference source distribution feature vector ,

[0090] S72, Drone Swarm Topology Action Space Cluster topology actions Perform cluster formation expansion or contraction operations; spatial translation operations. ,include Discrete displacement commands in four dimensions;

[0091] S73, establish a Q-value evaluation network with a multilayer perceptron structure, the network input being the state vector. The output is the corresponding action set. Q-values ​​for each action:

[0092]

[0093] By minimizing the mean squared error loss function between the predicted Q value and the target Q value To update the network parameters, the target Q value is determined by the potential energy field reward function defined in step S5. drive:

[0094] .

[0095] in, Indicates the action selection for the current time slot. This represents the parameters of the main network. Indicates the action selection for the next time slot. This represents the parameters of the target network.

[0096] Step S8 specifically includes:

[0097] S81 standardizes the anti-interference model for UAV swarms as a two-layer coupled partially observable Markov decision process, which is represented as a binary tuple. ,in This represents the frequency decision and power selection process. This represents the topology decision-making process of a drone swarm; the two layers communicate via state parameters. With feedback parameters Achieve dynamic coupling and mutual feedback;

[0098] S82, Design a joint optimization algorithm based on multi-agent collaborative training. The design process includes the following steps:

[0099] S821, Determine the model parameters of the optimizer Adam, including the learning rate. Discount Factor Initialize the Attention-DRQN network and experience pool for each drone. Initialize the DQN network and experience pool for the drone swarm. ;

[0100] S822 resets the interference source status and channel environment, generates an initial ring topology and virtual center point coordinates, and initializes the observation space of each UAV. and hidden state ;

[0101] S823, Obtain the set of drone position coordinates This is mapped into the environmental model to update the three-dimensional spatial distribution of the UAV swarm and the large-scale fading matrix of the channel, thus completing cross-layer state synchronization.

[0102] S824, based on current local observations The Attention-DRQN network is used to output actions. The action set is input into the environment for execution, and the reward is calculated. And obtain the next time slot observation. Calculate the average SINR feedback value of the drone swarm. ;

[0103] S825, Transmitted to DQN network, build status Using DQN network to select actions The action updates the cluster position and calculates the topological reward based on the potential energy field. And determine whether the target destination has been reached;

[0104] S826, transforms the data of the Attention-DRQN network. Store in the recycle experience replay pool Transform data from the DQN network. Store in the standard experience replay pool ,

[0105] S827, from The parameters of the Attention-DRQN network are updated using the backpropagation algorithm based on the sampled data. Using sampled data, the DQN network parameters are updated using gradient descent, and soft update operations are performed on the Target network. ;

[0106] S828, Repeat steps S823 to S827 until the maximum number of steps is reached or the drone swarm successfully reaches the destination;

[0107] S829, when the maximum number of training iterations is reached, saves the optimal network parameters and outputs the optimal collaborative path planning and resource allocation strategy for the drone swarm.

[0108] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0109] First, the anti-interference communication method for UAV swarms based on cross-layer collaboration and attention mechanism of the present invention reduces the probability of co-frequency interference and link breakage in multi-hop links by designing an Attention-DRQN network that integrates routing predecessor and successor state awareness, thereby improving the success rate of end-to-end data transmission and ensuring effective data transmission in complex electromagnetic environments.

[0110] Second, the anti-interference communication method for UAV swarms based on cross-layer collaboration and attention mechanism of the present invention uses a cross-layer collaboration strategy that drives the swarm topology through signal-to-interference-plus-noise ratio feedback to handle the strong coupling relationship between spectrum resource allocation and spatial trajectory planning. Through the dual means of "spatial avoidance" and "spectrum agility", it solves the problem of the failure of single physical layer anti-interference methods under strong interference and achieves efficient anti-interference.

[0111] Third, the anti-jamming communication method for UAV swarms based on cross-layer collaboration and attention mechanism of the present invention considers the safety gating constraint of potential energy field and energy constraint to ensure that the UAV swarm always stays above the communication safety threshold during the process of moving towards the target point. It designs a partially observable Markov decision process, so that when the UAV faces the dynamic frequency sweep of the interference source, it can still maintain the stability of decision by using historical time sequence characteristics, suppressing frequent channel switching oscillations, and further enhancing the robustness and mission survivability of the UAV swarm system. Attached Figure Description

[0112] Figure 1 This is a scenario diagram illustrating the communication of a drone swarm under complex electromagnetic environments and complex interference sources, as designed in this invention.

[0113] Figure 2 This is a time-step diagram illustrating the interaction between frequency point decision and cluster topology decision designed in this invention.

[0114] Figure 3 The diagram shows the framework of the anti-interference algorithm for UAV swarms based on the cross-layer collaboration and attention mechanism designed in this invention.

[0115] Figure 4 This is a comparison chart showing the communication link connection success rate of the algorithm of this invention and different multi-agent deep reinforcement learning algorithms.

[0116] Figure 5 This is a comparison chart of the convergence performance of the algorithm of this invention with different multi-agent deep reinforcement learning algorithms.

[0117] Figure 6 This is a schematic diagram of the anti-interference trajectory planning for UAV swarms using the algorithm of this invention. Detailed Implementation

[0118] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0119] This invention discloses a method for anti-jamming communication of UAV swarms based on cross-layer collaboration and attention mechanisms. The method includes the following steps:

[0120] Step S1: Construct a communication scenario between a cluster of drones and complex interference sources under a complex electromagnetic environment;

[0121] Step S2: Establish the UAV swarm communication link model and the frequency sweep comb interference model;

[0122] Step S3: Construct a node state awareness model and a multi-hop data transmission model based on fixed routing constraints;

[0123] Step S4: Design evaluation indicators for transmission link quality and UAV swarm topology maneuver efficiency;

[0124] Step S5: Establish a topology control model for UAV swarm based on potential energy field and signal-to-interference-plus-noise ratio feedback;

[0125] Step S6: Design a deep recurrent Q-network based on the attention mechanism to handle dynamic neighbor features, frequency decision and power selection problems;

[0126] Step S7: Design an algorithm for topology scaling and mobility control of UAV swarms based on deep Q-networks;

[0127] Step S8: Construct a cross-layer collaborative joint optimization framework, in which the UAV swarm topology control in step S5 and the UAV frequency decision and power selection in step S6 are executed alternately and mutually fed back in time steps, and the optimal anti-interference path planning and resource allocation strategy is obtained through multi-agent collaborative training.

[0128] The cross-layer collaborative anti-interference communication method for unmanned aerial vehicle (UAV) swarms of the present invention specifically includes the following steps:

[0129] In step S1, a communication scenario between a cluster of drones and complex interference sources under a complex electromagnetic environment is constructed.

[0130] like Figure 1 As shown, in the set simulation scenario, A swarm of drones A set of interference sources, starting points, and target endpoints, represented as a collection of UAVs. The set of interference sources is represented as The drone can be represented by a logical channel as The set of selectable transmit power levels represents The communication routing table for the drone swarm is as follows: This includes multiple fixed multi-hop links, and each drone uses a routing table... Determine its predecessor and successor nodes, and in the logical channel set With power set The process involves independently selecting actions to establish a communication link.

[0131] Discretize the duration of the drone swarm flight mission into Equal time slots, in the first In each time slot, a cluster topology model based on a virtual center point is established, and the cluster state is determined by the coordinates of the virtual center point. and topology scaling radius A joint decision. The _th time slot, the _th ... The location coordinates of the drone Represented as:

[0132]

[0133] in, Indicates the first The offset of the drone relative to the virtual center point is determined by the topology scaling radius. and the preset relative angle Decision; Virtual Center Point The update follows the following kinematic rules:

[0134]

[0135] in, Indicates the first Actions selected for each time slot This represents the displacement distance per unit time step. The following boundary constraints must be met:

[0136]

[0137] in, and Indicates the grid boundary size of the scene;

[0138] In step S2, the UAV swarm communication link model and the frequency-sweeping comb interference model are determined.

[0139] Sending drone With the receiving drone Channel power gain between :

[0140]

[0141] in, Indicates drone and The Euclidean distance between them Indicates the reference distance. This represents the path loss constant at the reference distance. This represents the path loss index.

[0142] In the Each time slot, the receiving drone In the channel The signal-to-interference-to-noise ratio is :

[0143]

[0144] in, Indicates the sending drone The transmission power, This represents the power of additive white Gaussian noise. This indicates interference from other drones operating at the same frequency within the cluster. The average signal-to-interference-plus-noise ratio (SIR) of a drone swarm represents the suppression interference from external sources. ;

[0145] Interference from other drones operating at the same frequency within the cluster Represented as:

[0146]

[0147] in, As an indicator function, when the drone Channel occupied The value is 1 if it is true, and 0 otherwise.

[0148] External interference Depend on An interference source is generated, represented as:

[0149]

[0150] in, Indicates the transmission power of the interference source. Indicates the source of interference To drones Channel gain;

[0151] Interference source Employing a frequency-sweeping comb jamming strategy, its first... A set of time-slot interference channels It consists of a reference frequency and comb harmonic frequencies:

[0152]

[0153]

[0154]

[0155] in, Indicates the time step of the disturbance dwell time. This represents the initial phase shift from different interference sources. Represents the total number of channels. This indicates the frequency interval of the comb interference.

[0156] In step S3, a node state-aware model and a multi-hop data transmission model based on fixed routing constraints are constructed. This specifically includes the following steps:

[0157] Step S31: Construct a route-aware UAV local observation state model.

[0158] First, according to the preset fixed routing table For any drone in the cluster This determines its logical role in the current communication link and identifies its unique predecessor node. and successor nodes Drones It needs to be aware of its own physical state, local environmental interactions, and critical routing context information to maintain link connectivity. Therefore, drones... In the Local observation state vector of each time slot Defined as:

[0159]

[0160] in, This indicates the physical state characteristics of the drone itself, including its current transmission channel, reception channel, and power level; It represents the local environmental interaction characteristics, including the distribution of perceived interference intensity; Defined as a route-aware feature vector.

[0161] Step S32: Construct a multi-hop data transmission reliability determination model.

[0162] Define a complete end-to-end routing link From node sequence Composition, data needs to be processed A hop link is required to reach the destination node from the source node. To accurately evaluate the effectiveness of anti-jamming communication, this link is used in the [missing information - likely a specific step or step]. The conditional judgment model for successful data transmission in a time slot can be expressed as:

[0163]

[0164] in, This is a link transmission status indication value; Indicates the first in the link Actual measured signal-to-interference-plus-noise ratio at the receiver end of the skip link; Defined as the minimum signal-to-interference-plus-noise ratio threshold required to ensure reliable data decoding; This indicates an indicator function that takes the value 1 if and only if the condition within the parentheses is true, otherwise it takes the value 0.

[0165] Step S4: Design evaluation indicators for transmission link quality and UAV swarm topology maneuver efficiency.

[0166] For any communication link In its first Effective margin of time slot link Defined as the actual signal-to-interference-plus-noise ratio at the receiver and the demodulation threshold. The difference:

[0167]

[0168] Among them, when When the value is 1, it indicates that the link meets the requirements for reliable transmission, and the larger the value, the higher the anti-interference margin; when When this occurs, it indicates that the link is in an interrupted state;

[0169] Definition of the first Average spectrum switching rate of time-slotted drone swarms for:

[0170]

[0171] in, This represents a collection of drones currently in the launch state. Indicates drone The transmission channel selected in the current time slot This is an indicator function.

[0172] Definition of the first Virtual center point of time-slotted drone swarm Relative to the target endpoint Normalized residual distance :

[0173]

[0174] in, This represents the total path distance at the start of the task.

[0175] Step S5: Establish a UAV swarm topology control model based on potential energy field and signal-to-interference-plus-noise ratio feedback. This specifically includes the following steps:

[0176] Step S51: Construct a target gravity and communication mass potential energy model.

[0177] Based on the mission requirements and real-time communication status of the UAV swarm, our goal is to drive swarm evolution by quantifying spatial location changes and channel quality feedback. First, we consider the Euclidean distance change between the virtual center point and the target. Time slot, gravitational potential energy gain based on relative distance variation It can be represented as:

[0178]

[0179] in, This represents the distance potential energy weighting coefficient. and These are the coordinates of the virtual center point and the target point at the current moment, respectively.

[0180] Secondly, consider the average signal-to-interference-plus-noise ratio within the cluster. Communication potential energy Defined as:

[0181]

[0182] in, This represents the communication potential weighting coefficient.

[0183] Step S52: Construct a model of boundary repulsion and gating total utility function.

[0184] Based on the security constraints and energy sustainability requirements of the mission area, our goal is to prevent the cluster from overstepping boundaries or running out of energy. Therefore, when considering the mission area boundaries... and remaining energy Under the condition of boundary repulsive potential energy It can be represented as:

[0185]

[0186] in, and These represent the out-of-bounds penalty value and the energy depletion penalty value, respectively.

[0187] Based on the security principles of cross-layer collaboration, our goal is to maximize the overall utility of the cluster by dynamically balancing task advancement and communication maintenance. Therefore, when considering communication security thresholds... Under the gating constraint, the total utility function of all UAV transmit / receive pairs can be expressed as:

[0188]

[0189] Among them, the gate function Defined as:

[0190]

[0191] When the average signal-to-interference-plus-noise ratio is below the safety threshold When the gating is closed, the cluster will not receive a gravitational potential energy reward even if it moves towards the destination, and will trigger an additional communication penalty. This forces the cluster to prioritize anti-interference maneuvers.

[0192] Step S6: Design a deep recurrent Q-network based on the attention mechanism, formulate dynamic feature aggregation and temporal memory criteria, and on the basis of maximizing link connectivity, handle the dynamic changes in the number of neighbors and the heterogeneity of local observation dimensions in UAV swarm communication, and use long short-term memory units to predict the interference pattern.

[0193] In step S61, we model the UAV spectrum decision problem as a feature extraction task under a partially observable Markov decision process.

[0194] Specifically, to address the dynamic fluctuation of the number of neighboring nodes in a drone swarm due to topology changes, a heterogeneous feature extractor based on a fully connected neural network is constructed. For any drone... Its input space is defined to contain a fixed-length self-observation vector. and the variable-length neighbor observation set Our optimization goal is to map the heterogeneous original data to a high-dimensional feature space; therefore, the feature embedding process is defined as follows:

[0195]

[0196] in, and These represent feature extraction networks for their own state and neighbor states, respectively.

[0197] Step S62: Considering maximizing the feature weights of key neighbor nodes, design a multi-head attention aggregation criterion. To accurately identify the nodes that cause the most interference to the local machine or have the closest routing correlation in the variable-length neighbor set, define a query vector. Derived from its own characteristics key vector AND value vector Derived from neighbor characteristics Aggregated features are calculated using an attention mechanism. :

[0198]

[0199] in, This is the scaling factor for the feature dimension. To further enhance the ability to perceive the nearest interference sources and key routing nodes in spatial location, a hybrid feature fusion vector is constructed. This vector consists of its own features, attention aggregation features, and the features of its nearest neighbor in spatial distance. It is pieced together:

[0200]

[0201] Step S63: Construct a timing-dependent and multi-head action output mechanism using long short-term memory units.

[0202] Given the significant temporal correlation of complex interference sources, simple instantaneous decision-making is insufficient to address them. Therefore, the fused vector... Input to LSTM cells to update hidden state and cell state Using historical memory to combat environmental uncertainty:

[0203]

[0204] The hidden state containing spatiotemporal context information is decoupled from the output layer through multi-head decoupling. This is mapped to a specific action probability distribution. For the joint decision-making based on spectrum and power, the transmit channel actions are output separately. Receive channel actions and power level operation Q value:

[0205]

[0206] Step S7: Design an algorithm for topology scaling and mobility control of UAV swarms based on deep Q-networks. The algorithm specifically includes the following steps:

[0207] S71, Construct the topology state space of the drone swarm, the first Time slot input state vector This includes the normalized value of the cluster average signal-to-interference-plus-noise ratio (SINNR) fed back from the physical layer. Normalized coordinates of the current cluster virtual center point Current topology scaling radius , Pre-planned global path completion progress index Perceived interference source distribution feature vector .

[0208] S72, Drone Swarm Topology Action Space Cluster topology actions Perform cluster formation expansion or contraction operations; spatial translation operations. ,include Discrete displacement commands in four dimensions;

[0209] S73, establish a Q-value evaluation network with a multilayer perceptron structure, the network input being the state vector. The output is the corresponding action set. Q-values ​​for each action:

[0210]

[0211] By minimizing the mean squared error loss function between the predicted Q value and the target Q value To update the network parameters, the target Q value is determined by the potential energy field reward function defined in step S5. drive:

[0212]

[0213] Step S8 integrates the topology control model designed in Step S5 with the spectrum decision model designed in Step S6 to construct a cross-layer collaborative joint optimization framework. This framework normalizes the UAV swarm anti-interference problem as a two-layer coupled partially observable Markov decision process. Furthermore, it combines a potential energy field feedback mechanism to design a multi-agent collaborative training method to obtain the optimal collaborative anti-interference strategy. Figure 3 As shown, the specific steps include:

[0214] Step S81: The cross-layer collaborative optimization model is normalized into a two-layer coupled partially observable Markov decision process. This process can be represented as coupled tuples. . in, This represents the physical layer spectrum decision-making process. The local observation space defined for step S3 includes route awareness and neighbor features; For discrete action space; This is a physical layer reward function based on link connectivity. This represents the topological maneuver decision-making process. It is a macroscopic state space that includes the cluster virtual center, the distribution of interference sources, and communication quality feedback; This refers to the macroscopic action space that includes center point translation and formation scaling; The topological reward function based on the composite potential field is defined for step S5. This represents the interlayer coupling variables, including the set of UAV position coordinates output by the topology decision and acting on the physical layer environment. And the average signal-to-interference-plus-noise ratio fed back from the physical layer and applied to the topology layer state. .

[0215] S82, Design a joint optimization algorithm based on multi-agent collaborative training. The design process includes the following steps:

[0216] S821, Determine the model parameters of the optimizer Adam, including the learning rate. Discount Factor Initialize the Attention-DRQN network and experience pool for each drone. Initialize the DQN network and experience pool for the drone swarm. ;

[0217] S822 resets the interference source status and channel environment, generates an initial ring topology and virtual center point coordinates, and initializes the observation space of each UAV. and hidden state ;

[0218] S823, Obtain the set of drone position coordinates This is mapped into the environmental model to update the three-dimensional spatial distribution of the UAV swarm and the large-scale fading matrix of the channel, thus completing cross-layer state synchronization.

[0219] S824, based on current local observations The Attention-DRQN network is used to output actions. The action set is input into the environment for execution, and the reward is calculated. And obtain the next time slot observation. Calculate the average SINR feedback value of the drone swarm. ;

[0220] S825, Transmitted to DQN network, build status Using DQN network to select actions The action updates the cluster position and calculates the topological reward based on the potential energy field. And determine whether the target destination has been reached;

[0221] S826, transforms the data of the Attention-DRQN network. Store in the recycle experience replay pool Transform data from the DQN network. Store in the standard experience replay pool .

[0222] S827, from The parameters of the Attention-DRQN network are updated using the backpropagation algorithm based on the sampled data. Using sampled data, the DQN network parameters are updated using gradient descent, and soft update operations are performed on the Target network. ;

[0223] S828, Repeat steps S823 to S827 until the maximum number of steps is reached or the drone swarm successfully reaches the destination;

[0224] S829, when the maximum number of training iterations is reached, saves the optimal network parameters and outputs the optimal collaborative path planning and resource allocation strategy for the UAV swarm.

[0225] Table 1 presents the computational complexity of different algorithms in the scenario described in this invention. This indicates the total number of drones in the drone swarm; This represents the average number of neighboring nodes within a single machine's field of view. The dimension of the UAV's local observation state vector; This represents the dimension of the hidden layer in a neural network. Represents the dimension of the action space; , , Let's represent the computational complexity of the DQN, DRQN, and Attention-DRQN algorithms, respectively:

[0226] Table 1. Computational complexity of different algorithms.

[0227] algorithm 1 Step T Steps DQN DRQN Attention-DRQN

[0228] Figure 4A comparison of the communication link connectivity success rates of the algorithm of this invention and different multi-agent deep reinforcement learning algorithms is presented. It can be observed that the Attention-DRQN algorithm exhibits the fastest convergence and stabilization speed, followed by the DRQN ​​algorithm. Although the Attention-DRQN algorithm shows some volatility in the early stages of training, it can achieve the optimal solution and reach a stable state after multiple iterations. This is mainly because the DQN algorithm cannot handle the time-varying characteristics of frequency sweep interference, leading to decision oscillations; while the DRQN ​​algorithm, although introducing LSTM memory units, cannot effectively focus on key interference sources when faced with dynamically changing neighbor numbers. In contrast, this invention dynamically extracts variable-length neighbor features through a multi-head attention mechanism, thus demonstrating superior anti-interference performance in complex dynamic electromagnetic environments.

[0229] Figure 5 A comparison of the convergence performance of the algorithm of this invention with different multi-agent deep reinforcement learning algorithms is presented. Although the DQN and DRQN ​​algorithms have certain advantages in terms of computational overhead for single-step decision-making, their convergence performance is inferior to the Attention-DRQN algorithm in situations involving broadband frequency sweeping interference and highly dynamic changes in network topology. This invention achieves an effective balance between ensuring the reliability of communication links and improving the survivability of the swarm through deep coupling of spectrum sensing and topology decision-making, providing a better solution for collaborative communication of UAV swarms in complex electromagnetic environments.

[0230] Figure 6 This is a schematic diagram illustrating the anti-jamming trajectory planning of a UAV swarm using the algorithm of this invention. Although the interference source forms a strong signal blockade zone on the direct path to the target, the swarm controlled by the cross-layer potential field can still plan a safe detour path when considering communication link connectivity constraints. This invention, by introducing a communication potential field driven by signal-to-interference-plus-noise ratio feedback, achieves a dynamic trade-off between mission advancement speed and communication security threshold, effectively avoiding link breakage caused by blind advances, and providing reliable trajectory support for swarm survival and penetration in strong interference environments.

[0231] In summary, this invention proposes an anti-jamming communication method for UAV swarms based on cross-layer collaboration and attention mechanisms. By designing an Attention-DRQN network based on a multi-head attention mechanism, variable-length neighbor features are dynamically aggregated, effectively avoiding input dimension mismatch caused by topology changes, enhancing the spectrum perception capability of UAV swarms in dynamic environments, and ensuring the connectivity of critical links. A cross-layer collaboration strategy driven by signal-to-interference-plus-noise ratio (SNR) feedback to drive the topological potential field addresses the strong coupling between spectrum decision-making and spatial maneuvering, maximizing the system's SNR and solving the problem of limited anti-jamming capability of single communication methods, achieving proactive and efficient obstacle avoidance. Considering routing constraints and energy constraints, the invention ensures that UAVs maintain the integrity of communication links during mission execution, avoiding link breakage. The optimization model is standardized as a two-layer coupled partially observable Markov decision process, and an LSTM temporal memory unit is introduced, enabling UAVs to make efficient decisions based on historical patterns even when facing frequency sweeping interference and environmental uncertainties, enhancing system robustness and stability. The designed attention-based spectrum decision-making algorithm and potential field topology control method demonstrate superior performance in comparisons with different algorithms, achieving an effective balance between communication reliability and mission progress speed, and providing a better solution for collaborative communication of UAV swarms in complex electromagnetic environments. Simulation results prove the excellent characteristics of this invention.

[0232] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented using various computer languages, such as the object-oriented programming language Python and the interpreted scripting language Matlab.

[0233] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0234] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0235] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment, causing a series of operational steps to be executed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that run on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0236] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0237] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for anti-interference communication of UAV swarms based on cross-layer collaboration and attention mechanisms, characterized in that, The cross-layer collaborative anti-interference communication method for UAV swarms includes the following steps: Step S1: Construct a communication scenario for a drone swarm under complex electromagnetic interference sources. Step S2: Construct a UAV swarm communication link model and a frequency-sweeping comb interference model; Step S3: Construct a node state awareness model and a multi-hop data transmission model based on fixed routing constraints; Step S4: Design evaluation indicators for transmission link quality and UAV swarm topology maneuver efficiency; Step S5: Construct a topology control model for UAV swarm based on potential energy field and signal-to-interference-plus-noise ratio feedback; Step S6: Design a deep recurrent Q-network based on the attention mechanism to handle dynamic neighbor features, frequency decision and power selection problems; Step S7: Design an algorithm for topology scaling and mobility control of UAV swarms based on deep Q-networks; Step S8: Construct a cross-layer collaborative joint optimization framework, in which the UAV swarm topology control in step S5 and the UAV frequency decision and power selection in step S6 are executed alternately and mutually fed back in time steps, and the optimal anti-interference path planning and resource allocation strategy is obtained through multi-agent collaborative training.

2. The anti-interference communication method for UAV swarms based on cross-layer collaboration and attention mechanisms according to claim 1, characterized in that, Step S1 specifically includes: Set up a simulation scenario, which includes... A swarm of drones A set of interference sources, starting points, and target endpoints, represented as a collection of UAVs. The set of interference sources is represented as The drone can be represented by a logical channel as Optional transmit power level set The communication routing table of the drone swarm is This includes multiple fixed multi-hop links, with each drone operating according to its routing table. Determine its predecessor and successor nodes, and in the logical channel set With power set The process involves independently selecting actions to establish a communication link; Discretize the duration of the drone swarm flight mission into Equal time slots, in the first In each time slot, a cluster topology model based on a virtual center point is established, and the cluster state is determined by the coordinates of the virtual center point. and topology scaling radius Jointly decided, the The _th time slot, the _th ... The location coordinates of the drone Represented as: ； in, Indicates the first The offset of the drone relative to the virtual center point is determined by the topology scaling radius. and the preset relative angle Decision; Virtual Center Point The update follows the following kinematic rules: ； In the formula, Indicates the first Actions selected for each time slot This represents the displacement distance per unit time step. The following boundary constraints must be satisfied: ； in, and These represent the X and Y grid boundary dimensions of the scene, respectively. and Representing the virtual center point The X and Y coordinates.

3. The anti-interference communication method for UAV swarms based on cross-layer collaboration and attention mechanisms according to claim 1, characterized in that, Step S2 specifically includes: Sending drone With the receiving drone Channel power gain between : ； in, Indicates drone and The Euclidean distance between them Indicates the reference distance. This represents the path loss constant at the reference distance. This represents the path loss index. In the Each time slot, the receiving drone In the channel The signal-to-interference-to-noise ratio is : ； In the formula, Indicates the sending drone The transmission power, This represents the power of additive white Gaussian noise. This indicates interference from other drones operating at the same frequency within the cluster. The average signal-to-interference-plus-noise ratio (SIR) of a drone swarm represents the suppression interference from external sources. ; Interference from other drones operating at the same frequency within the cluster Represented as: ； In the formula, For the indicator function, when the drone Channel occupied The value is 1 if it is true, and 0 otherwise. External interference Depend on An interference source is generated, represented as: ； in, Indicates the transmission power of the interference source. Indicates the source of interference To drones Channel gain; Interference source Employing a frequency-sweeping comb jamming strategy, its first... A set of time-slot interference channels It consists of a reference frequency and comb harmonic frequencies: ； ； ； in, Indicates the time step of the disturbance dwell time. This represents the initial phase shift from different interference sources. Represents the total number of channels. This indicates the frequency interval of the comb interference.

4. The anti-interference communication method for UAV swarms based on cross-layer collaboration and attention mechanisms according to claim 1, characterized in that, Step S3 specifically includes: According to the preset routing table For any drone in the cluster This determines its logical role in the routing link and identifies its unique predecessor node. and successor nodes ; drones In the Local observation state vector of each time slot This vector is composed of its own physical state characteristics. Local environment interaction features and route-aware features It consists of three parts: ； Among them, route-aware features Includes the following information: predecessor node The transmit channel index used in the previous time slot Successor node The receive channel index used in the previous time slot Successor node The receiver signal-to-interference-plus-noise ratio measured in the previous time slot , Construct a multi-hop data transmission model and define a complete routing link. From node sequence Composed of, this link in the The condition for successful data transmission in a time slot is that each hop in the link meets the communication quality threshold: ； in, Indicates the link number The actual signal-to-interference-plus-noise ratio at the receiver. This represents the minimum SINR threshold required for reliable data transmission. This indicates that the end-to-end data transmission is considered successful only if all hops of the entire link are connected.

5. The anti-interference communication method for UAV swarms based on cross-layer collaboration and attention mechanisms according to claim 1, characterized in that, Step S4 specifically includes: For any communication link In its first Effective margin of time slot link Defined as the actual signal-to-interference-plus-noise ratio at the receiver. Minimum SINR threshold required for reliable data transmission The difference: ； Among them, when When the value is 0, it indicates that the link meets the requirements for reliable transmission, and the larger the value, the higher the anti-interference margin; when When this occurs, it indicates that the link is interrupted; Definition of the first Average spectrum switching rate of time-slotted drone swarms for: ； in, This represents a collection of drones currently in the launch state. Indicates drone The transmission channel selected in the current time slot For indicator functions, The number of drones in the launch state; Definition of the first Virtual center point of time-slotted drone swarm Relative to the target endpoint Normalized residual distance : ； in, This represents the total path distance at the start of the task.

6. The anti-interference communication method for UAV swarms based on cross-layer collaboration and attention mechanisms according to claim 1, characterized in that, In step S5, the UAV swarm topology control model based on potential energy field and signal-to-interference-plus-noise ratio feedback is as follows: Composite potential field It is formed by the coupling of the target gravitational field, the communication mass field, and the boundary repulsive field, in the first... Time slot, gravitational potential energy gain based on relative distance variation Represented as: ； in, This represents the distance potential energy weighting coefficient. Represents the Euclidean distance norm. Average signal-to-interference-plus-noise ratio introduced by drone swarm Constructing a communication quality field and communication potential energy Defined as: ； in, This represents the communication potential weighting coefficient; Represents a saturated nonlinear mapping function; Boundary repulsive potential energy Used to restrict the virtual center point from going out of the task area. At the same time, combined with the remaining energy Imposing punishment: ； in, and These represent the out-of-bounds penalty value and the energy depletion penalty value, respectively. Introducing gating functions Dynamically constraining gravitational potential energy, the total utility function of all UAV transmit / receive pairs Defined as: ； Among them, the gate function Represented as: ； When the average signal-to-interference-plus-noise ratio is below the safety threshold When the gating is closed, the cluster will not receive a gravitational potential energy reward even if it moves towards the destination, and will trigger an additional communication penalty. .

7. The anti-interference communication method for UAV swarms based on cross-layer collaboration and attention mechanisms according to claim 1, characterized in that, Step S6 specifically includes: S61 addresses the dynamic changes in the number of neighbors and the heterogeneity of local observation dimensions in UAV swarm communication by constructing a feature extractor based on a fully connected neural network for any UAV. , its own observation vector and neighbor observation set Mapped to higher-dimensional space respectively; S62 utilizes an attention mechanism to dynamically calculate neighbor weights, defining the query vector as originating from its own features. The key vector and value vector are derived from the features of the neighbors. Calculate attention aggregation features : ； Constructing a hybrid feature fusion vector It consists of its own features, attention aggregation features, and the features of its nearest neighbor in spatial distance. It is pieced together to enhance the ability to detect interference sources and routing nodes: ； S63, fused vector Input LSTM unit: ； The hidden state is decoupled from the output layer through multi-head decoupling. Mapped to actions.

8. The anti-interference communication method for UAV swarms based on cross-layer collaboration and attention mechanism according to claim 1, characterized in that, Step S7 specifically includes: S71, Construct the topology state space of the drone swarm, the first Time slot input state vector This includes the normalized value of the cluster average signal-to-interference-plus-noise ratio. Normalized coordinates of the current cluster virtual center point Current topology scaling radius , Pre-planned global path completion progress index Perceived interference source distribution feature vector , S72, Drone Swarm Topology Action Space Cluster topology actions Perform cluster formation expansion or contraction operations; spatial translation operations. ,include Discrete displacement commands in four dimensions; S73, establish a Q-value evaluation network with a multilayer perceptron structure, the network input being the state vector. The output is the corresponding action set. Q-values ​​for each action: ； By minimizing the mean squared error loss function between the predicted Q value and the target Q value To update the network parameters, the target Q value is determined by the potential energy field reward function defined in step S5. drive: ， in, Indicates the action selection for the current time slot. This represents the parameters of the main network. Indicates the action selection for the next time slot. These represent the parameters of the target network.

9. The anti-interference communication method for UAV swarms based on cross-layer collaboration and attention mechanisms according to claim 1, characterized in that, Step S8 specifically includes: S81 standardizes the anti-interference model for UAV swarms as a two-layer coupled partially observable Markov decision process, which is represented as a binary tuple. ,in This represents the frequency decision and power selection process. This represents the topology decision-making process of a drone swarm; the two layers communicate via state parameters. With feedback parameters Achieve dynamic coupling and mutual feedback; S82, Design a joint optimization algorithm based on multi-agent collaborative training. The design process includes the following steps: S821, Determine the model parameters of the optimizer Adam, including the learning rate. Discount Factor Initialize the Attention-DRQN network and experience pool for each drone. Initialize the DQN network and experience pool for the drone swarm. ; S822 resets the interference source state and channel environment, generates an initial ring topology and virtual center point coordinates, and initializes the observation space of each UAV. and hidden state ; S823, Obtain the set of drone position coordinates This is mapped into the environmental model to update the three-dimensional spatial distribution of the UAV swarm and the large-scale fading matrix of the channel, thus completing cross-layer state synchronization. S824, based on current local observations The Attention-DRQN network is used to output actions. The action set is input into the environment for execution, and the reward is calculated. And obtain the next time slot observation. Calculate the average SINR feedback value of the drone swarm. ; S825, Transmitted to DQN network, build status Using DQN network to select actions The action updates the cluster position and calculates the topological reward based on the potential energy field. And determine whether the target destination has been reached; S826, transforms the data of the Attention-DRQN network. Store in the recycle experience replay pool Transform data from the DQN network. Store in the standard experience replay pool , S827, from The parameters of the Attention-DRQN network are updated using the backpropagation algorithm based on the sampled data. Using sampled data, the DQN network parameters are updated using gradient descent, and soft update operations are performed on the Target network. ; S828, Repeat steps S823 to S827 until the maximum number of steps is reached or the drone swarm successfully reaches the destination; S829, when the maximum number of training iterations is reached, saves the optimal network parameters and outputs the optimal collaborative path planning and resource allocation strategy for the drone swarm.

Images