Unmanned aerial vehicle network track, route and resource joint optimization method capable of resisting mobile interference

Abstract

The invention discloses an anti-mobile-interference unmanned aerial vehicle network track, route and resource joint optimization method, which comprises the following steps of: firstly, splitting an original joint decision problem into two sub-problems of unmanned aerial vehicle network channel resource allocation and track route decision; and through iterative cooperation of game theory channel allocation and multi-agent reinforcement learning trajectory routing decision, a joint optimization problem is decoupled, so that optimization of unmanned aerial vehicle network task time delay is realized under an anti-interference condition. According to the method, the RMAPPO is integrated into the RNN to capture topology time sequence characteristics, track control and route selection are covered at the same time, game theory channel allocation and recursive multi-agent reinforcement learning are combined through a decomposition type optimization framework, topology fluctuation is dynamically adapted, the coupling optimization problem is effectively solved, the anti-interference performance and the network stability are both considered, and the method is suitable for large-scale popularization and application. And real-time cooperation of interference suppression and resource allocation is realized.

Description

Technical Field

[0001] This invention belongs to the field of wireless communication technology, specifically relating to a method for joint optimization of UAV network trajectory, routing, and resources to resist mobile interference. Background Technology

[0002] Addressing the high-speed data transmission requirements of UAV (Unmanned Aerial Vehicle) sensing data acquisition networks in 6G scenarios, serving as aerial base stations or relays, and considering the network throughput and scalability constraints imposed by the inherent scarcity of spectrum resources, constructing an optimized system that balances anti-interference capabilities and efficient resource utilization has become a core objective. Due to the high mobility of UAVs, their dynamic movement leads to rapid fluctuations in network topology. Simultaneously, the broadcast nature of wireless transmission makes the network vulnerable to both co-channel interference and malicious mobile interference, further exacerbating signal quality degradation and communication latency issues. To improve spectrum efficiency, frequency reuse technology has become a key approach, effectively alleviating spectrum scarcity pressure. However, aggressive frequency reuse inevitably introduces severe interference. Furthermore, existing anti-interference strategies and resource optimization methods struggle to address the complex challenges of topology dynamics, time-varying interference, and resource coupling, necessitating an integrated joint optimization framework.

[0003] However, existing single-dimensional optimization approaches face significant limitations in dynamic UAV interference scenarios. First, existing CCI-sensing channel allocation schemes generally ignore the rapid topology reconfiguration and time-varying interference patterns caused by the high mobility of UAVs, resulting in a mismatch between channel allocation strategies and actual interference states, failing to guarantee communication stability in highly mobile scenarios. Second, existing anti-interference strategies are often fragmented. Some schemes only utilize UAV maneuverability to achieve physical-layer interference avoidance, but for static or simplified interference source model designs, they do not consider network topology fluctuations and routing connectivity disruptions caused by UAVs leaving the communication range during interference avoidance. Other schemes attempt to optimize single resources such as power and spectrum, but do not couple trajectory control and routing selection, making it difficult to cope with real-time changes in dynamic mobile interference. In addition, trajectory control, routing planning, and resource allocation are recursively coupled and dependent, forming a non-convex, NP-hard joint optimization problem. Existing exhaustive search or static planning methods are unsuitable due to excessive computational complexity, resulting in persistently high system bottleneck latency.

[0004] Currently, research on UAV network interference suppression and resource optimization mainly focuses on the following directions: (1) multi-UAV interference perception deployment optimization; (2) multi-agent reinforcement learning anti-interference resource allocation;

[0005] The paper "C.-C. Lai, A.-H. Tsai, C.-W. Ting, K.-H. Lin, J.-C. Ling, and C.-E. Tsai, 'Interference-aware deployment for maximizing user satisfaction in multi-UAV wireless networks,' IEEE Wireless Commun. Lett., vol. 12, no. 7, pp. 1189–1193, July 2023" proposes an interference-aware deployment strategy for multi-UAV networks, aiming to improve user satisfaction by optimizing the static position of UAVs to reduce interference. However, this scheme only addresses the static topology design during the network deployment phase, neglecting the dynamic movement requirements during UAV mission execution and failing to address the collaborative optimization of trajectory control and resource allocation. Therefore, it cannot cope with dynamic topology changes and time-varying interference issues caused by mobile interference. The paper "Z. Yin, Y. Lin, Y. Zhang, Y. Qian, F. Shu, and J. Li, 'Collaborative multiagent reinforcement learning aided resource allocation for UAV anti-jamming communication,' IEEE Internet of Things Journal, vol. 9, no. 23, pp. 23995–24008, 2022" employs collaborative multiagent reinforcement learning to achieve anti-jamming resource allocation for UAVs, improving anti-jamming capabilities through multiagent collaboration. However, this scheme does not integrate a UAV trajectory optimization module and does not introduce a recurrent neural network to capture the time dependency of dynamic topology, making it difficult to handle the recursive coupling relationship between trajectory, routing, and resources. In mobile interference scenarios, it is prone to optimization lag and network connectivity disruption. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a method for joint optimization of UAV network trajectory, routing, and resources to resist mobile interference. This method solves the problems of high system bottleneck latency and weak anti-interference capability caused by the impact of co-channel interference (CCI) and malicious mobile interference on UAV perception data acquisition networks under spectrum-limited environments, as well as the recursive coupling between trajectory control, routing planning, and resource allocation.

[0007] The technical solution adopted in this invention is: a method for joint optimization of UAV network trajectory, routing, and resources to resist mobile interference, the specific steps of which are as follows:

[0008] S1. Obtain information on interference sources and the UAV network mission environment, and construct a UAV network system model adapted to mobile interference scenarios;

[0009] First, collect the mission scenario information of the UAV network, that is, determine the position coordinates of each node in the UAV network and calculate the path loss of air-to-air communication and air-to-ground communication.

[0010] Then, the motion characteristics of the mobile interference sources are collected, including: the target point and the ground center location, and the spectrum resource allocation. That is, the location of the interference sources and the interference spectrum information in the environment are obtained, the location of the target point and data information are determined, and the data flow of the network is calculated.

[0011] Finally, the basic constraints of UAV data acquisition and transmission are defined, the topology of the UAV network is initialized, and the signal-to-interference-plus-noise ratio, communication rate, and end-to-end transmission delay performance of the UAV network in the initial state are calculated.

[0012] S2. For finite spectrum under disturbance, a non-cooperative game theory method is used to optimize UAV channel allocation decisions;

[0013] Based on the current drone trajectory and routing path, each drone is treated as a non-cooperative game participant. Using the binary vector of the sub-channel occupancy state as the strategy, a composite utility function integrating individual delay and system bottleneck delay is constructed. The Nash equilibrium is solved dynamically and iteratively through the best response until all drone strategies converge. The optimal channel allocation strategy of the current iteration is output to suppress the impact of co-channel interference and mobile interference.

[0014] The game consists of participants: a set of drone nodes. Strategy selection ; This is a binary indicator variable; 1 indicates that the link from UAV m to k occupies subchannel c, and 0 indicates that it does not.

[0015] S3. Based on a defined channel allocation strategy, a multi-agent reinforcement learning algorithm is applied to optimize UAV trajectory and route selection.

[0016] Using the channel allocation result output in step S2 as input, the RMAPPO algorithm is optimized using a recursive multi-agent nearest neighbor strategy. The UAV is treated as an agent, and a centralized training and distributed execution paradigm is adopted. The design incorporates observation states including UAV position, interference distribution, and delay information. Flight direction actions ("forward / backward / left / right / hover") and routing actions ("next-hop relay UAV selection") are defined. A reward function integrating delay optimization and constraint penalties is constructed. The strategy is trained using a multi-agent reinforcement learning algorithm, specifically by capturing network topology time dependencies through a recursive neural network (RNN) and combining this with a generalized advantage estimation (GAE) training strategy. The output is a UAV trajectory and routing path that satisfy safety distance and communication range constraints.

[0017] S4. Compare the changes in performance indicators. If convergence is achieved, output the optimal result; otherwise, return to step S2 to continue the loop optimization.

[0018] Calculate the system bottleneck delay after optimization in step S3 and evaluate the relative rate of change of delay. If the rate of change is lower than the preset threshold, the iteration converges and the current optimal UAV trajectory, routing path and channel allocation strategy are output. Otherwise, take the trajectory and route updated in step S3 as input, return to step S2 to re-execute the channel allocation optimization, and iterate until the convergence condition is met.

[0019] Furthermore, step S1 is specifically as follows:

[0020] S11. Collect the UAV's transmission power, position coordinates, flight speed, flight altitude, power of interference sources, interference channel information, target area of ​​the perception task, and boundary information of the mission scenario.

[0021] S12. Calculate the path loss of air-to-air communication in the UAV network;

[0022] The total available spectrum is orthogonally divided into C sub-channels, each with a bandwidth of B. Each drone can access a maximum of [number missing]. Each sub-channel can be reused by different drones. Binary indicator variables are used. Indicates whether the link from UAV m to UAV k occupies subchannel c at time slot t: Indicates that it is in use. This indicates that it is not in use.

[0023] Interference source with fixed power In the subchannel subset Continuous jamming is emitted from above. Represents the total set of sub-channels; throughout the observation period Within the system, the spectrum occupancy and transmission power of the interference source remain constant.

[0024] If the link from drone m to drone k uses any jammed sub-channel, the path loss from the interference source to the receiving drone k is... The expression is as follows:

[0025] (1);

[0026] in, This represents the carrier frequency, and v represents the speed of light. This represents the three-dimensional distance between the interference source and UAV k in time slot t. The inter-UAV link uses a free-space path loss model, and the channel gain between UAV m and UAV k is... The expression is as follows:

[0027] (2);

[0028] in, This represents the three-dimensional distance between drone m and drone k in time slot t.

[0029] S13. Calculate the air-to-air communication rate of the UAV network;

[0030] speed of inter-drone link The expression is as follows:

[0031] (3);

[0032] in, This indicates the transmit power of drone m. This represents the power of additive white Gaussian noise; This indicates coherent channel interference from other drones. Indicates the power of malicious interference; This represents the channel gain between drone m and drone k.

[0033] S14. Calculate the path loss of UAV network air-to-ground communication;

[0034] When the UAV transmits data to the ground collection center, a probabilistic line-of-sight (LOS) / non-line-of-sight (NLOS) model is used. Line-of-sight (LOS) probability. The expression is as follows:

[0035] (4);

[0036] in, h is an environment-related constant, representing the altitude of the drone above the ground. This represents the horizontal distance between the UAV m and the ground fusion center at time slot t, and , Indicates the coordinates of the ground fusion center. Denotes the two-dimensional position of drone m; the non-line-of-sight (NLOS) probability is... The path losses for line-of-sight links and non-line-of-sight links are as follows:

[0037] (5);

[0038] (6);

[0039] in, This represents the path loss of a line-of-sight link. This represents the path loss of a non-line-of-sight link. These represent the additional losses for line-of-sight (LAS) and non-LAS links, respectively. The expected path loss of the A2G channel from the UAV to the ground fusion center. The expression is as follows:

[0040] (7);

[0041] S15. Calculate the air-to-ground communication rate of the UAV network;

[0042] Aggregated uplink rate from drone to ground fusion center The expression is as follows:

[0043] (8);

[0044] in, This is a binary indicator variable representing the occupancy of subchannel c in time slot t of the link from UAV m to the ground fusion center. This indicates that the link is experiencing interference from other drones operating in the same channel. This indicates the source of interference affecting the link.

[0045] S16. Calculate network data traffic;

[0046] After collecting perception data from a designated target, the UAV forwards the data packets to the ground collection center. If the distance between some UAVs and the ground fusion center exceeds the maximum communication radius CR, then these source UAVs complete data transmission through multi-hop relays from intermediate UAVs. This indicates the transmission path connecting the source drone m and the ground fusion center.

[0047] Among them, the source drone m is Ground fusion center , Indicates the number of hops in the path. This represents the index of the i-th hop drone on the path; the set of intermediate nodes on the path is... .

[0048] use Indicates the i-th hop drone Forward to the Jump drone The amount of data. Each drone only senses one target, generating a raw data volume of... Bit; for any intermediate node Data flow must satisfy flow conservation, as expressed below:

[0049] (9);

[0050] S17. Calculate the end-to-end transmission delay of the network;

[0051] At time slot t, the drone To drones Single-hop transmission delay The expression is as follows:

[0052] (10);

[0053] in, for The previous hop node, for The next hop node. express and The communication rate.

[0054] The end-to-end delay of the data packets generated by the source drone m The expression is as follows:

[0055] (11);

[0056] Finally, regarding the drone node set System bottleneck delay at time slot t The expression is as follows:

[0057] (12).

[0058] Furthermore, step S2 is specifically as follows:

[0059] S21. Construct a composite utility function that integrates individual delay and system bottleneck delay;

[0060] When the drone trajectory is fixed, channel allocation for each time slot becomes an independent subproblem: minimizing the bottleneck latency of the current time slot system. To achieve the goal, constrain the upper limit of subchannel occupancy for each drone, i.e., at most If there are multiple sub-channels, the constraint expression is as follows:

[0061] (13);

[0062] in, This is a binary indicator variable; 1 indicates that the link from UAV m to k occupies subchannel c, and 0 indicates that it does not. Let C represent the set of drone nodes, and C represent the total number of sub-channels.

[0063] Then design the composite utility function. To balance the individual latency of the drone with the global latency of the system, the expression is as follows:

[0064] (14);

[0065] in, This represents the individual end-to-end delay of drone m at time t. This represents the system bottleneck delay at time t. These are the weighting coefficients.

[0066] S22. Solve the Nash equilibrium dynamically and iteratively through the best response until all UAV strategies converge, and output the optimal channel allocation strategy for the current iteration.

[0067] During the Nash equilibrium solution process, the drone nodes adjust their policies based on the chosen actions and the resulting benefits, following an update rule. The update rule is as follows: if an action improves the node's utility, the probability of that action being selected in the future will increase; conversely, it will decrease. The process continues until the policy reaches a pure policy state, meaning there exists at least one action whose probability of selection approaches 1, and the probabilities of other actions approach 0, or the preset maximum number of iterations is reached. If the algorithm stops and converges stably to the Nash equilibrium of the game, the set of Nash equilibrium strategies for all drone games is defined as follows: .

[0068] The specific sub-steps for solving the Nash equilibrium are as follows:

[0069] A1. Set input parameters, including: number of drones Total number of sub-channels C, total time slots T, and UAV trajectory set Maximum number of iterations ;

[0070] A2. Randomly assign sub-channels to each UAV to obtain the initial strategy. And set time slot cycle To T;

[0071] A3. Suppose that the Nash equilibrium strategy of drone m in the previous time slot is... And use it as the initial strategy for the current time slot. This accelerates convergence;

[0072] A4, Regarding Each iteration is based on a time-slot cycle. For each drone m, update the strategy up to T;

[0073] A41, Generating Actions: In Time slot, UAV m according to its current hybrid strategy Randomly select an action ;

[0074] A42. Evaluating Gains: After the game ends, evaluate the composite utility function obtained for each slice. ;

[0075] A43. Update Strategy: Choose to maximize one's own utility. sub-channels ,Right now Update strategy ,like That is, the strategy no longer changes, reaching Nash equilibrium. The Nash equilibrium strategy for the current time slot is denoted as... (t), exit the current iteration;

[0076] A5. Nash equalization strategy for optimal channel allocation across all time slots. Maximum system latency .

[0077] Furthermore, step S3 is specifically as follows:

[0078] S31. Determine the observation status of the UAV;

[0079] The observation status of each UAV m includes: its own and neighboring UAVs' real-time positions, network conflict indications and interference effects, and its own end-to-end latency. System bottleneck latency Define the optimal transmission path, or route set, for the UAV network as follows: The set of drone trajectories is .

[0080] S32. Determine the motion space;

[0081] The action space is divided into two categories, covering movement and transmission decisions:

[0082] 1) Flight direction: forward, backward, left, right, hovering;

[0083] 2) Next hop selection: Select the optimal relay node from the neighboring drones to minimize the single-hop transmission time.

[0084] S33. Construct a reward function that integrates delay optimization and constraint penalties;

[0085] S331. Determine the constraints that the reward function must follow;

[0086] 1) Safety distance restrictions between drones;

[0087] Any two drones Three-dimensional distance at time slot t The minimum safe distance must not be less than To avoid collisions, the expression is as follows:

[0088] (15);

[0089] 2) Communication radius limitation;

[0090] Transmission path Any two adjacent hops of the drone Distance at time slot t The maximum communication radius CR must not be exceeded to ensure link connectivity, as shown in the following expression:

[0091] (16);

[0092] 3) Unmanned aerial vehicle (UAV) location boundary restrictions;

[0093] The positions of all drones at time slot t It must be located within the task area; avoid deviating from the task scope. The expression is as follows:

[0094] (17);

[0095] in, and These represent the lower and upper boundaries of the task area, respectively.

[0096] 4) Target coverage limitations;

[0097] UAV m and the assigned target The square of the horizontal distance is less than the square of the coverage radius r, ensuring reliable perception of the target. The expression is as follows:

[0098] (18);

[0099] 5) Limitations on the step size of drone movement;

[0100] Set the time slot length to The drone's flight speed is constant. Drone m in The square of the positional change of adjacent time slots shall not exceed To ensure a smooth movement trajectory, the expression is as follows:

[0101] (19);

[0102] 6) The initial position is fixed;

[0103] Initial positions of all drones Preset value To ensure the initial state of the task is determined, the expression is as follows:

[0104] (20);

[0105] 7) Flow conservation;

[0106] intermediate nodes forwarding data volume Equal to the original data volume generated by itself Add all of them This is the sum of the forwarded data from the previous hop node, ensuring no data loss or redundancy. The expression is as follows:

[0107] (twenty one);

[0108] S332. Determine the reward function;

[0109] reward function To balance delay minimization with various constraints, a safety penalty term is introduced, expressed as follows:

[0110] ;

[0111] in, This indicates the penalty for violating the safe distance rule, when the constraint is met. A violation incurs a positive penalty value. These are the weighting coefficients.

[0112] S34. Training strategy through multi-agent reinforcement learning algorithm;

[0113] S341. Estimate the state value and fuse the network state. and RNN hidden state Capture time dependence:

[0114] ;

[0115] in, This represents the commentator's network parameters, where z represents the index of the time step. Indicating drone strategy, Indicates the strategy parameters, Representing computational strategies Expected value This represents the discount factor.

[0116] S342. Establish the dominance function;

[0117] Advantage function The generalized dominance estimation method (GAE) is used to reduce variance and improve estimation accuracy. The expression is as follows:

[0118]

[0119] in, Indicates the timing differential (TD) error. This represents the GAE coefficient.

[0120] S343. Calculate the total loss function. ;

[0121] The total loss function is calculated by combining the PPO pruning strategy loss, the squared TD error loss, and the entropy regularization loss. The expression is as follows:

[0122]

[0123] in, Indicates the weighting coefficient; This indicates the loss from the PPO pruning strategy. This represents the squared TD error loss. This represents the entropy regularization loss.

[0124] S344, RNN hidden state update;

[0125] The hidden state is updated based on the previous moment's state, action, and reward to capture time-series dependencies, as shown in the following expression:

[0126] (26);

[0127] in, This indicates the hidden state at the previous moment. Indicates the current observation status. Indicates the previous action, This indicates the previous reward.

[0128] S35. Based on steps S31-S34, the RMAPPO algorithm is optimized using a recursive multi-agent nearest neighbor strategy, and the UAV trajectory and routing path that meet the constraints of safe distance and communication range are output.

[0129] The specific sub-steps of the RMAPPO algorithm are as follows:

[0130] B1. Initialize the policy network for each drone. Centralized commentator network Target network, experience buffer Set the episode loop. To G;

[0131] Where G represents the total number of training rounds, and the target network parameters are initially consistent with the main network parameters.

[0132] B2. Reset the environment;

[0133] Initial drone position, channel allocation, interference source trajectory, initialization of RNN hidden state Observation status ;

[0134] B3, For For each time slot in the T-time slot cycle, a network update is performed for each UAV m;

[0135] B31, according to strategy Select Action And add Gaussian exploration noise;

[0136] B32. Perform the action;

[0137] Update drone position and observe the status of the next time slot. Update the hidden state of the RNN ;

[0138] B33. Store samples;

[0139] The sample is defined as the set of results generated by the interaction between the agent and the environment, specifically in the form of... Store the samples in the current time slot and record the total number of samples stored as . When the number of samples Training is triggered, and samples are stored in the experience buffer. ;

[0140] in, This indicates the trigger threshold.

[0141] B34, Mini-batch sampling;

[0142] from A random sample is selected from the data, and the advantage and target value are calculated, i.e., calculated using GAE. Using commentator networks to estimate target value ;

[0143] B35. Update the network;

[0144] minimize Update the commentator and strategy networks, and soft-update the target network parameters.

[0145] B4. Output the optimal drone trajectory With transmission routing .

[0146] Furthermore, step S4 is specifically as follows:

[0147] Calculate the optimized system bottleneck latency and evaluate the relative rate of change of latency; set the update threshold for the relative change as follows. Update the latency history and calculate the relative changes in latency for the three most recent latency periods. ,like If the iteration stops, then repeat steps S2 and S3; finally, the optimal UAV trajectory is output. Optimal transmission route path With optimal channel allocation strategy Otherwise, using the updated trajectory and route as input, return to step S2 to re-optimize channel allocation, and continue iterating until the convergence condition is met.

[0148] The beneficial effects of this invention are as follows: The method of this invention first decomposes the original joint decision-making problem into two sub-problems: UAV network channel resource allocation and trajectory routing decision. Then, through iterative collaboration of game theory channel allocation and multi-agent reinforcement learning trajectory routing decision, the joint optimization problem is decoupled, thereby optimizing UAV network task latency under anti-interference conditions. This invention integrates RMAPPO into RNN to capture topological temporal features, simultaneously covering trajectory control and routing selection. Through a decompositional optimization framework, it combines game theory channel allocation with recursive multi-agent reinforcement learning to dynamically adapt to topological fluctuations, effectively solving the coupling optimization problem. It balances anti-interference performance and network stability, achieving real-time collaboration between interference suppression and resource allocation. This addresses the problem of high system bottleneck latency and weak anti-interference capability caused by the recursive coupling of trajectory control, routing planning, and resource allocation in spectrum-constrained environments, where UAV perception data acquisition networks are affected by co-channel interference (CCI) and malicious mobile interference. Attached Figure Description

[0149] Figure 1 This is a flowchart of a method for joint optimization of UAV network trajectory, routing, and resources to resist mobile interference, according to the present invention.

[0150] Figure 2 This is a schematic diagram of a scenario in an embodiment of the present invention. Detailed Implementation

[0151] The method of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0152] like Figure 1 The flowchart shown is a method for joint optimization of UAV network trajectory, routing, and resources to resist mobile interference according to the present invention. The specific steps are as follows:

[0153] S1. Obtain information on interference sources and the UAV network mission environment, and construct a UAV network system model adapted to mobile interference scenarios;

[0154] The scenario in this embodiment is as follows: Figure 2 As shown, the first step is to collect the mission scenario information of the UAV network, that is, to determine the position coordinates of each node in the UAV network and calculate the path loss of air-to-air communication and air-to-ground communication.

[0155] Then, the motion characteristics of the mobile interference sources are collected, including: the target point and the ground center location, and the spectrum resource allocation. That is, the location of the interference sources and the interference spectrum information in the environment are obtained, the location of the target point and data information are determined, and the data flow of the network is calculated.

[0156] Finally, the basic constraints of UAV data acquisition and transmission are defined, the topology of the UAV network is initialized, and the signal-to-interference-plus-noise ratio, communication rate, and end-to-end transmission delay performance of the UAV network in the initial state are calculated.

[0157] S2. For finite spectrum under disturbance, a non-cooperative game theory method is used to optimize UAV channel allocation decisions;

[0158] Based on the current drone trajectory and routing path, each drone is treated as a non-cooperative game participant. Using the binary vector of the sub-channel occupancy state as the strategy, a composite utility function integrating individual delay and system bottleneck delay is constructed. The Nash equilibrium is solved dynamically and iteratively through the best response until all drone strategies converge. The optimal channel allocation strategy of the current iteration is output to suppress the impact of co-channel interference and mobile interference.

[0159] The game consists of participants: a set of drone nodes. Strategy selection ; This is a binary indicator variable; 1 indicates that the link from UAV m to k occupies subchannel c, and 0 indicates that it does not.

[0160] S3. Based on a defined channel allocation strategy, a multi-agent reinforcement learning algorithm is applied to optimize UAV trajectory and route selection.

[0161] Using the channel allocation result output in step S2 as input, the Recursive Multi-Agent Nearest Neighbor Policy Optimization (RMAPPO) algorithm is adopted. The UAV is treated as an agent, and a centralized training and distributed execution paradigm is adopted. The observation state including UAV position, interference distribution, and delay information is designed. The flight direction actions of "forward / backward / left / right / hover" and the routing action of "next-hop relay UAV selection" are defined. A reward function that integrates delay optimization and constraint penalty is constructed. The strategy is trained by multi-agent reinforcement learning algorithm, that is, by capturing the network topology time dependency through recurrent neural network (RNN) and combining it with generalized advantage estimation (GAE) training strategy to output UAV trajectory and routing path that meet the constraints of safe distance and communication range (adapting to the interference environment).

[0162] S4. Compare the changes in performance indicators. If convergence is achieved, output the optimal result; otherwise, return to step S2 to continue the loop optimization.

[0163] Calculate the system bottleneck delay after optimization in step S3 and evaluate the relative rate of change of delay. If the rate of change is lower than the preset threshold, the iteration converges and the current optimal UAV trajectory, routing path and channel allocation strategy are output. Otherwise, take the trajectory and route updated in step S3 as input, return to step S2 to re-execute the channel allocation optimization, and iterate until the convergence condition is met.

[0164] In this embodiment, step S1 is specifically as follows:

[0165] S11. Collect the UAV's transmission power, position coordinates, flight speed, flight altitude, power of interference sources, interference channel information, target area of ​​the perception task, and boundary information of the mission scenario.

[0166] S12. Calculate the path loss of air-to-air communication in the UAV network;

[0167] The total available spectrum is orthogonally divided into C sub-channels, each with a bandwidth of B. Each drone can access a maximum of [number missing]. Each sub-channel can be reused by different drones. Binary indicator variables are used. Indicates whether the link from UAV m to UAV k occupies subchannel c at time slot t: Indicates that it is in use. This indicates that it is not in use.

[0168] Interference source with fixed power In the subchannel subset Continuous jamming is emitted from above. Represents the total set of sub-channels; throughout the observation period Within the system, the spectrum occupancy and transmission power of the interference source remain constant.

[0169] If the link from drone m to drone k uses any jammed sub-channel, the path loss from the interference source to the receiving drone k is... The expression is as follows:

[0170] (1);

[0171] in, This represents the carrier frequency, and v represents the speed of light. This represents the three-dimensional distance between the interference source and UAV k in time slot t. The inter-UAV link uses a free-space path loss model, and the channel gain between UAV m and UAV k is... The expression is as follows:

[0172] (2);

[0173] in, This represents the three-dimensional distance between drone m and drone k in time slot t.

[0174] S13. Calculate the air-to-air communication rate of the UAV network;

[0175] speed of inter-drone link The expression is as follows:

[0176] (3);

[0177] in, This indicates the transmit power of drone m. This represents the power of additive white Gaussian noise; This indicates coherent channel interference from other drones. Indicates the power of malicious interference; This represents the channel gain between drone m and drone k.

[0178] S14. Calculate the path loss of UAV network air-to-ground communication;

[0179] When the UAV transmits data to the ground collection center, a probabilistic line-of-sight (LOS) / non-line-of-sight (NLOS) model is used. Line-of-sight (LOS) probability. The expression is as follows:

[0180] (4);

[0181] in, h is an environment-related constant, representing the altitude of the drone above the ground. This represents the horizontal distance between the UAV m and the ground fusion center at time slot t, and , Indicates the coordinates of the ground fusion center. Denotes the two-dimensional position of drone m; the non-line-of-sight (NLOS) probability is... The path losses for line-of-sight links and non-line-of-sight links are as follows:

[0182] (5);

[0183] (6);

[0184] in, This represents the path loss of a line-of-sight link. This represents the path loss of a non-line-of-sight link. These represent the additional losses for line-of-sight (LAS) and non-LAS links, respectively. The expected path loss of the A2G channel from the UAV to the ground fusion center. The expression is as follows:

[0185] (7);

[0186] S15. Calculate the air-to-ground communication rate of the UAV network;

[0187] Aggregated uplink rate from drone to ground fusion center The expression is as follows:

[0188] (8);

[0189] in, This is a binary indicator variable representing the occupancy of subchannel c in time slot t of the link from UAV m to the ground fusion center. This indicates that the link is experiencing interference from other drones operating in the same channel. This indicates the source of interference affecting the link.

[0190] S16. Calculate network data traffic;

[0191] After collecting perception data from a designated target, the UAV needs to forward the data packets to the ground collection center. Since the distance between some UAVs and the ground fusion center may exceed the maximum communication radius (CR), these source UAVs need to complete data transmission through multi-hop relays via intermediate UAVs. Indicates the connection source drone m Integration with the ground center (i.e.) The transmission path of ), where Indicates the number of hops in the path. This represents the index of the i-th hop drone on the path; the set of intermediate nodes on the path is... .use Indicates the i-th hop drone Forward to the Jump drone The amount of data. Each drone only senses one target, generating a raw data volume of... Bit; for any intermediate node Data flow must satisfy flow conservation, as expressed below:

[0192] (9);

[0193] S17. Calculate the end-to-end transmission delay of the network;

[0194] At time slot t, the drone To drones Single-hop transmission delay The expression is as follows:

[0195] (10);

[0196] in, for The previous hop node, for The next hop node. express and The communication rate (substituted into equation (3) for calculation, i.e.) ).

[0197] The end-to-end delay of the data packets generated by the source drone m The expression is as follows:

[0198] (11);

[0199] Finally, regarding the drone node set The system bottleneck latency at time slot t (i.e., the maximum value of all end-to-end latency of all drones). The expression is as follows:

[0200] (12).

[0201] In this embodiment, step S2 is specifically as follows:

[0202] S21. Construct a composite utility function that integrates individual delay and system bottleneck delay;

[0203] When the drone trajectory is fixed, channel allocation for each time slot becomes an independent subproblem: minimizing the bottleneck latency of the current time slot system. To achieve the goal, constrain the maximum subchannel occupancy limit for each drone (maximum). (if there are multiple sub-channels), then the constraint expression is as follows:

[0204] (13);

[0205] in, It is a binary indicator variable (1 indicates that the link from drone m to k occupies subchannel c, 0 indicates that it does not). Let C represent the set of drone nodes, and C represent the total number of sub-channels.

[0206] To address the sparsity issue when directly using system latency as a reward, a composite utility function is designed. To balance the individual latency of the drone with the global latency of the system, the expression is as follows:

[0207] (14);

[0208] in, This represents the individual end-to-end delay of drone m at time t. This represents the system bottleneck delay at time t. This is a weighting coefficient (adjusting the priority of individual and global goals).

[0209] S22. Solve the Nash equilibrium dynamically and iteratively through the best response until all UAV strategies converge, and output the optimal channel allocation strategy for the current iteration.

[0210] During the Nash equilibrium solution process, the drone nodes adjust their policies based on the chosen actions and the resulting benefits, following an update rule. The update rule is as follows: if an action improves the node's utility, the probability of that action being selected in the future will increase; conversely, it will decrease. The process continues until the policy reaches a pure policy state, meaning there exists at least one action whose probability of selection approaches 1, and the probabilities of other actions approach 0, or the preset maximum number of iterations is reached. If the algorithm stops and converges stably to the Nash equilibrium of the game, the set of Nash equilibrium strategies for all drone games is defined as follows: .

[0211] The specific sub-steps for solving the Nash equilibrium are as follows:

[0212] A1. Set input parameters, including: number of drones Total number of sub-channels C, total time slots T, and UAV trajectory set Maximum number of iterations ;

[0213] A2. Randomly assign sub-channels to each UAV to obtain the initial strategy. And set time slot cycle To T;

[0214] A3. Suppose that the Nash equilibrium strategy of drone m in the previous time slot is... And use it as the initial strategy for the current time slot. This accelerates convergence;

[0215] A4, Regarding Each iteration is based on a time-slot cycle. For each drone m, update the strategy up to T;

[0216] A41, Generating Actions: In Time slot, UAV m according to its current hybrid strategy Randomly select an action ;

[0217] A42. Evaluating Gains: After the game ends, evaluate the composite utility function obtained for each slice. ;

[0218] A43. Update Strategy: Choose to maximize one's own utility. sub-channels ,Right now Update strategy ,like That is, the strategy no longer changes, reaching Nash equilibrium. The Nash equilibrium strategy for the current time slot is denoted as... (t), exit the current iteration;

[0219] A5. Nash equalization strategy for optimal channel allocation across all time slots. Maximum system latency .

[0220] In this embodiment, step S3 is specifically as follows:

[0221] S31. Determine the observation status of the UAV;

[0222] The observation status of each UAV m includes: its own and neighboring UAVs' real-time positions, network conflict indications (spectral overlap) and interference effects (distance from the nearest interference source), and its own end-to-end delay. System bottleneck latency Define the optimal transmission path, or route set, for the UAV network as follows: The set of drone trajectories is .

[0223] S32. Determine the motion space;

[0224] The action space is divided into two categories, covering movement and transmission decisions:

[0225] 1) Flight direction: forward, backward, left, right, hover (5 discrete options in total);

[0226] 2) Next hop selection: Select the optimal relay node from the neighboring drones to minimize the single-hop transmission time.

[0227] S33. Construct a reward function that integrates delay optimization and constraint penalties;

[0228] S331. Determine the constraints that the reward function must follow;

[0229] 1) Safety distance restrictions between drones;

[0230] Any two drones Three-dimensional distance at time slot t The minimum safe distance must not be less than To avoid collisions, the expression is as follows:

[0231] (15);

[0232] 2) Communication radius limitation;

[0233] Transmission path Any two adjacent hops of the drone Distance at time slot t The maximum communication radius CR must not be exceeded to ensure link connectivity, as shown in the following expression:

[0234] (16);

[0235] 3) Unmanned aerial vehicle (UAV) location boundary restrictions;

[0236] The positions of all drones at time slot t It must be located within the task area; avoid deviating from the task scope. The expression is as follows:

[0237] (17);

[0238] in, and These represent the lower and upper boundaries of the task area, respectively.

[0239] 4) Target coverage limitations;

[0240] UAV m and the assigned target The square of the horizontal distance is less than the square of the coverage radius r, ensuring reliable perception of the target. The expression is as follows:

[0241] (18);

[0242] 5) Limitations on the step size of drone movement;

[0243] Set the time slot length to The drone's flight speed is constant. Drone m in The square of the positional change of adjacent time slots shall not exceed To ensure a smooth movement trajectory, the expression is as follows:

[0244] (19);

[0245] 6) The initial position is fixed;

[0246] Initial positions of all drones Preset value To ensure the initial state of the task is determined, the expression is as follows:

[0247] (20);

[0248] 7) Flow conservation;

[0249] intermediate nodes forwarding data volume Equal to the original data volume generated by itself Add all of them This is the sum of the forwarded data from the previous hop node, ensuring no data loss or redundancy. The expression is as follows:

[0250] (twenty one);

[0251] S332. Determine the reward function;

[0252] reward function To balance minimizing latency with various constraints, a safety penalty term is introduced, expressed as follows:

[0253] ;

[0254] in, This indicates the penalty for violating the safe distance rule, when the constraint is met. A violation incurs a positive penalty value. These are the weighting coefficients.

[0255] S34. Training strategy through multi-agent reinforcement learning algorithm;

[0256] S341. Estimate the state value and fuse the network state. and RNN hidden state (hidden state) Captures time dependencies:

[0257] ;

[0258] in, This represents the commentator's network parameters, where z represents the index of the time step. Indicating drone strategy, Indicates the strategy parameters, Representing computational strategies Expected value This represents the discount factor (balancing current and future rewards).

[0259] S342. Establish the dominance function;

[0260] Advantage function The generalized dominance estimation method (GAE) is used to reduce variance and improve estimation accuracy. The expression is as follows:

[0261]

[0262] in, This represents the timing difference (TD) error. This represents the GAE coefficient (the bias and variance tradeoff in the control advantage estimate).

[0263] S343. Calculate the total loss function. ;

[0264] The total loss function is calculated by combining the PPO pruning strategy loss, the squared TD error loss, and the entropy regularization loss (which encourages exploration and avoids local optima). The expression is as follows:

[0265]

[0266] in, Indicates the weighting coefficient; This represents the loss of the PPO pruning strategy (limiting the policy update step size). This represents the squared TD error loss (optimization value estimation). This represents the entropy regularization loss (enhancing exploratory behavior).

[0267] S344, RNN hidden state update;

[0268] The hidden state is updated based on the previous moment's state, action, and reward to capture time-series dependencies, as shown in the following expression:

[0269] (26);

[0270] in, This indicates the hidden state at the previous moment. Indicates the current observation status. Indicates the previous action, This indicates the previous reward.

[0271] S35. Based on steps S31-S34, the RMAPPO algorithm is optimized using a recursive multi-agent nearest neighbor strategy, and the UAV trajectory and routing path that meet the constraints of safe distance and communication range are output.

[0272] The specific sub-steps of the RMAPPO algorithm are as follows:

[0273] B1. Initialize the policy network for each drone. Centralized commentator network Target network, experience buffer Set the episode loop. To G;

[0274] Where G represents the total number of training rounds, and the target network parameters are initially consistent with the main network parameters.

[0275] B2. Reset the environment;

[0276] Initial drone position, channel allocation, interference source trajectory, initialization of RNN hidden state Observation status ;

[0277] B3, For For each time slot in the T-time slot cycle, a network update is performed for each UAV m;

[0278] B31, according to strategy Select Action And add Gaussian exploration noise;

[0279] B32. Perform the action;

[0280] Update drone position and observe the status of the next time slot. Update the hidden state of the RNN ;

[0281] B33. Store samples;

[0282] The sample is defined as the set of results generated by the interaction between the agent and the environment, specifically in the form of... Store the samples in the current time slot and record the total number of samples stored as . When the number of samples Training is triggered, and samples are stored in the experience buffer. ;

[0283] in, This indicates the trigger threshold.

[0284] B34, Mini-batch sampling;

[0285] from A random sample is selected from the data, and the advantage and target value are calculated, i.e., calculated using GAE. Using commentator networks to estimate target value ;

[0286] B35. Update the network;

[0287] minimize Update the commentator and strategy networks, and soft-update the target network parameters.

[0288] B4. Output the optimal drone trajectory With transmission routing .

[0289] In this embodiment, step S4 is specifically as follows:

[0290] Calculate the optimized system bottleneck latency and evaluate the relative rate of change of latency; set the update threshold for the relative change as follows. Update the latency history and calculate the relative changes in latency for the three most recent latency periods. ,like If the iteration stops, then repeat steps S2 and S3; finally, the optimal UAV trajectory is output. Optimal transmission route path With optimal channel allocation strategy Otherwise, using the updated trajectory and route as input, return to step S2 to re-optimize channel allocation, and continue iterating until the convergence condition is met.

[0291] As can be seen from the above embodiments, the method of the present invention improves the communication performance of UAV networks in mobile interference scenarios, effectively suppresses co-channel interference and malicious mobile interference, significantly reduces system bottleneck latency, and makes resource allocation and network topology more adaptable to dynamic interference environments.

[0292] In summary, the method of this invention is applied to a spectrum-constrained UAV perception network comprising multiple UAVs, a ground fusion center, and mobile interference sources. By constructing an integrated framework, it minimizes system bottleneck latency under the suppression of co-channel interference and malicious mobile interference. First, a UAV network system model adapted to mobile interference scenarios is constructed, encompassing the movement rules of UAVs and interference sources, a multi-link communication model, and a system bottleneck latency model. Constraints such as safe distance and communication range are clearly defined, transforming the coupling problem of trajectory, routing, and channel resources into a joint optimization problem, providing a basic framework for subsequent optimization. Then, an iterative framework of game theory channel allocation and multi-agent reinforcement learning trajectory routing optimization is designed. First, anti-interference allocation of channel resources is achieved through a game theory model. Then, the UAV trajectory and route are dynamically adjusted based on a recursive multi-agent reinforcement learning algorithm. Through alternating iterations of the two modules, system bottleneck latency is minimized while suppressing interference, improving the network's adaptability in dynamic scenarios.

[0293] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A method for joint optimization of UAV network trajectory, routing, and resources to resist mobile interference, the specific steps of which are as follows: S1. Obtain information on interference sources and the UAV network mission environment, and construct a UAV network system model adapted to mobile interference scenarios; First, collect the mission scenario information of the UAV network, that is, determine the position coordinates of each node in the UAV network and calculate the path loss of air-to-air communication and air-to-ground communication. Then, the motion characteristics of the moving interference source are collected, including: Target point and ground center location, spectrum resource allocation; That is, to obtain the location of the interference source and the interference spectrum information in the environment, determine the location and data information of the target point, and calculate the data flow of the network; Finally, the basic constraints of UAV data acquisition and transmission are defined, the topology of the UAV network is initialized, and the signal-to-interference-plus-noise ratio, communication rate and end-to-end transmission delay performance of the UAV network in the initial state are calculated. S2. For finite spectrum under disturbance, a non-cooperative game theory method is used to optimize UAV channel allocation decisions; Based on the current drone trajectory and routing path, each drone is treated as a non-cooperative game participant. The binary vector of the sub-channel occupancy state is used as the strategy to construct a composite utility function that integrates individual delay and system bottleneck delay. The Nash equilibrium is solved through dynamic iteration of the best response until all drone strategies converge. The optimal channel allocation strategy of the current iteration is output to suppress the effects of co-channel interference and mobile interference. The game consists of participants: a set of drone nodes. Strategy selection ; This is a binary indicator variable; 1 indicates that the link from UAV m to k occupies subchannel c, and 0 indicates that it does not. S3. Based on a defined channel allocation strategy, a multi-agent reinforcement learning algorithm is applied to optimize UAV trajectory and route selection. Using the channel allocation result output in step S2 as input, the RMAPPO algorithm is optimized using a recursive multi-agent nearest neighbor strategy. The UAV is treated as an agent, and a centralized training and distributed execution paradigm is adopted. The design incorporates observation states including UAV position, interference distribution, and delay information. Flight direction actions ("forward / backward / left / right / hover") and routing actions ("next-hop relay UAV selection") are defined. A reward function integrating delay optimization and constraint penalties is constructed. The strategy is trained using a multi-agent reinforcement learning algorithm, specifically by capturing network topology time dependencies through a recursive neural network (RNN) and combining this with a generalized advantage estimation (GAE) training strategy. The output is a UAV trajectory and routing path that satisfy safety distance and communication range constraints. S4. Compare the changes in performance indicators. If convergence is achieved, output the optimal result; otherwise, return to step S2 to continue the loop optimization. Calculate the optimized system bottleneck latency in step S3 and evaluate the relative rate of change of latency; if the rate of change is lower than the preset threshold, the iteration converges and the current optimal UAV trajectory, routing path and channel allocation strategy are output. Otherwise, using the updated trajectory and route from step S3 as input, return to step S2 to re-execute the channel allocation optimization, iterating in a loop until the convergence condition is met.

2. The method for joint optimization of UAV network trajectory, routing, and resources to resist mobile interference as described in claim 1, characterized in that, The specific steps of S1 are as follows: S11. Collect the UAV's transmission power, position coordinates, flight speed, flight altitude, power of interference sources, interference channel information, target area of ​​the sensing mission, and boundary information of the mission scenario. S12. Calculate the path loss of air-to-air communication in the UAV network; The total available spectrum is orthogonally divided into C sub-channels, each with a bandwidth of B; each drone can access a maximum of Each sub-channel can be reused by different drones; binary indicator variables are used. Indicates whether the link from UAV m to UAV k occupies subchannel c at time slot t: Indicates that it is in use. Indicates that it is not used; Interference source with fixed power In the subchannel subset Continuous jamming is emitted from above. Represents the total set of sub-channels; throughout the observation period Within the system, the spectrum occupancy and transmission power of the interference source remain constant. If the link from drone m to drone k uses any jammed sub-channel, the path loss from the interference source to the receiving drone k is... The expression is as follows: (1)； in, This represents the carrier frequency, and v represents the speed of light. The distance between the interference source and UAV k in time slot t is represented; the inter-UAV link adopts the free space path loss model, and the channel gain between UAV m and UAV k is represented. The expression is as follows: (2)； in, This represents the three-dimensional distance between drone m and drone k in time slot t; S13. Calculate the air-to-air communication rate of the UAV network; speed of inter-drone link The expression is as follows: (3)； in, This indicates the transmit power of drone m. This represents the power of additive white Gaussian noise; This indicates coherent channel interference from other drones. Indicates the power of malicious interference; This represents the channel gain between drone m and drone k; S14. Calculate the path loss of UAV network air-to-ground communication; When the UAV transmits data to the ground collection center, a probabilistic line-of-sight (LOS) / non-line-of-sight (NLOS) model is used; the probabilistic line-of-sight (LOS) model is used. The expression is as follows: (4)； in, h is an environment-related constant, representing the altitude of the drone above the ground. This represents the horizontal distance between the UAV m and the ground fusion center at time slot t, and , Indicates the coordinates of the ground fusion center. Denotes the two-dimensional position of drone m; the non-line-of-sight (NLOS) probability is... The path losses for line-of-sight links and non-line-of-sight links are as follows: (5)； (6)； in, This represents the path loss of a line-of-sight link. This represents the path loss of a non-line-of-sight link. These represent the additional losses for line-of-sight (LAS) and non-LAS links, respectively; and the expected path loss of the A2G channel from the UAV to the ground fusion center. The expression is as follows: (7)； S15. Calculate the air-to-ground communication rate of the UAV network; Aggregated uplink rate from drone to ground fusion center The expression is as follows: (8)； in, This is a binary indicator variable representing the occupancy of subchannel c in time slot t of the link from UAV m to the ground fusion center. This indicates that the link is experiencing interference from other drones operating in the same channel. This indicates the source of interference affecting the link; S16. Calculate network data traffic; After collecting perception data from a designated target, the UAV forwards the data packets to the ground collection center; however, if the distance between some UAVs and the ground fusion center exceeds the maximum communication radius CR, then these source UAVs complete data transmission through multi-hop relays from intermediate UAVs; This indicates the transmission path connecting the source drone m and the ground fusion center; Among them, the source drone m is Ground fusion center , Indicates the number of hops in the path. This represents the index of the i-th hop drone on the path; the set of intermediate nodes on the path is... ; use Indicates the i-th hop drone Forward to the Jump drone The amount of data; each drone only senses one target, generating a raw data volume of... Bit; for any intermediate node Data flow must satisfy flow conservation, as expressed below: (9)； S17. Calculate the end-to-end transmission delay of the network; At time slot t, the drone To drones Single-hop transmission delay The expression is as follows: (10)； in, for The previous hop node, for The next hop node; express and The communication rate; The end-to-end delay of the data packets generated by the source drone m The expression is as follows: (11)； Finally, regarding the drone node set System bottleneck delay at time slot t The expression is as follows: (12)。 3. The method for joint optimization of UAV network trajectory, routing, and resources to resist mobile interference as described in claim 2, characterized in that, Step S2 is as follows: S21. Construct a composite utility function that integrates individual delay and system bottleneck delay; When the drone trajectory is fixed, channel allocation for each time slot becomes an independent subproblem: minimizing the bottleneck latency of the current time slot system. To achieve the goal, constrain the upper limit of subchannel occupancy for each drone, i.e., at most If there are multiple sub-channels, the constraint expression is as follows: (13)； in, This is a binary indicator variable; 1 indicates that the link from UAV m to k occupies subchannel c, and 0 indicates that it does not. C represents the set of drone nodes, and C represents the total number of sub-channels; Then design the composite utility function. To balance the individual latency of the drone with the global latency of the system, the expression is as follows: (14)； in, This represents the individual end-to-end delay of drone m at time t. This represents the system bottleneck delay at time t. These are the weighting coefficients; S22. Solve the Nash equilibrium dynamically and iteratively through the best response until all UAV strategies converge, and output the optimal channel allocation strategy for the current iteration. In the process of solving for Nash equilibrium, the drone nodes will adjust their policies based on the selected actions and the benefits gained, following an update rule: if an action can improve the node's utility, the probability of that action being selected in the future will increase; otherwise, it will decrease. The policy reaches a pure policy state, i.e., there exists at least one action whose probability of being selected approaches 1, and the probabilities of other actions approach 0, or the iteration reaches a preset upper limit. If the algorithm stops and converges stably to the Nash equilibrium of the game, the set of Nash equilibrium strategies for all drone games is defined as follows: ; The specific sub-steps for solving the Nash equilibrium are as follows: A1. Set input parameters, including: number of drones Total number of sub-channels C, total time slots T, and UAV trajectory set Maximum number of iterations ; A2. Randomly assign sub-channels to each UAV to obtain the initial strategy. And set time slot cycle To T; A3. Suppose that the Nash equilibrium strategy of drone m in the previous time slot is... And use it as the initial strategy for the current time slot. This accelerates convergence; A4, Regarding Each iteration is based on a time-slot cycle. For each drone m, update the strategy up to T; A41, Generating Actions: In Time slot, UAV m according to its current hybrid strategy Randomly select an action ; A42. Evaluating Gains: After the game ends, evaluate the composite utility function obtained for each slice. ; A43. Update Strategy: Choose to maximize one's own utility. sub-channels ,Right now Update strategy ,like That is, the strategy no longer changes, reaching Nash equilibrium. The Nash equilibrium strategy for the current time slot is denoted as... (t), exit the current iteration; A5. Nash equalization strategy for optimal channel allocation across all time slots. Maximum system latency .

4. The method for joint optimization of UAV network trajectory, routing, and resources to resist mobile interference as described in claim 3, characterized in that, Step S3 is as follows: S31. Determine the observation status of the UAV; The observation status of each UAV m includes: its own and neighboring UAVs' real-time positions, network conflict indications and interference effects, and its own end-to-end latency. System bottleneck latency Define the optimal transmission path, or route set, for the UAV network as follows: The set of drone trajectories is ; S32. Determine the motion space; The action space is divided into two categories, covering movement and transmission decisions: 1) Flight direction: forward, backward, left, right, hovering; 2) Next-hop selection: Select the optimal relay node from neighboring drones to minimize single-hop transmission time; S33. Construct a reward function that integrates delay optimization and constraint penalties; S331. Determine the constraints that the reward function must follow; 1) Safety distance restrictions between drones; Any two drones Three-dimensional distance at time slot t The minimum safe distance must not be less than To avoid collisions, the expression is as follows: (15)； 2) Communication radius limitation; Transmission path Any two adjacent hops of the drone Distance at time slot t The maximum communication radius CR must not be exceeded to ensure link connectivity, as shown in the following expression: (16)； 3) Unmanned aerial vehicle (UAV) location boundary restrictions; The positions of all drones at time slot t It must be located within the task area; avoid deviating from the task scope. The expression is as follows: (17)； in, and These represent the lower and upper boundaries of the task area, respectively. 4) Target coverage limitations; UAV m and the assigned target The square of the horizontal distance is less than the square of the coverage radius r, ensuring reliable perception of the target. The expression is as follows: (18)； 5) Limitations on the step size of drone movement; Set the time slot length to The drone's flight speed is constant. ; Drone m in The square of the positional change of adjacent time slots shall not exceed To ensure a smooth movement trajectory, the expression is as follows: (19)； 6) The initial position is fixed; Initial positions of all drones Preset value To ensure the initial state of the task is determined, the expression is as follows: (20)； 7) Flow conservation; intermediate nodes forwarding data volume Equal to the original data volume generated by itself Add all of them This is the sum of the forwarded data from the previous hop node, ensuring no data loss or redundancy. The expression is as follows: (21)； S332. Determine the reward function; reward function To balance delay minimization with various constraints, a safety penalty term is introduced, expressed as follows: ； in, This indicates the penalty for violating the safe distance rule, when the constraint is met. A violation incurs a positive penalty value. These are the weighting coefficients; S34. Training strategy through multi-agent reinforcement learning algorithm; S341. Estimate the state value and fuse the network state. and RNN hidden state Capture time dependence: ； in, This represents the commentator's network parameters, where z represents the index of the time step. Indicating drone strategy, Indicates the strategy parameters, Representing computational strategies Expected value Indicates the discount factor; S342. Establish the dominance function; Advantage function The generalized dominance estimation method (GAE) is used to reduce variance and improve estimation accuracy. The expression is as follows: in, Indicates the timing differential (TD) error. Indicates the GAE coefficient; S343. Calculate the total loss function. ; The total loss function is calculated by combining the PPO pruning strategy loss, the squared TD error loss, and the entropy regularization loss. The expression is as follows: in, Indicates the weighting coefficient; This indicates the loss from the PPO pruning strategy. This represents the squared TD error loss. This represents the entropy regularization loss; S344, RNN hidden state update; The hidden state is updated based on the previous moment's state, action, and reward to capture time-series dependencies, as shown in the following expression: (26)； in, This indicates the hidden state at the previous moment. Indicates the current observation status. Indicates the previous action, Indicates the previous reward; S35. Based on steps S31-S34, the RMAPPO algorithm is optimized using a recursive multi-agent nearest neighbor strategy, and the UAV trajectory and routing path that meet the constraints of safe distance and communication range are output. The specific sub-steps of the RMAPPO algorithm are as follows: B1. Initialize the policy network for each drone. Centralized commentator network Target network, experience buffer Set the episode loop. To G; Where G represents the total number of training rounds, and the target network parameters are initially consistent with the main network parameters. B2. Reset the environment; Initial drone position, channel allocation, interference source trajectory, initialization of RNN hidden state Observation status ; B3, For For each time slot in the T-time slot cycle, a network update is performed for each UAV m; B31, according to strategy Select Action And add Gaussian exploration noise; B32. Perform the action; Update drone position and observe the status of the next time slot. Update the hidden state of the RNN ; B33. Store samples; The sample is defined as the set of results generated by the interaction between the agent and the environment, specifically in the form of... Store the samples in the current time slot and record the total number of samples stored as . When the number of samples Training is triggered, and samples are stored in the experience buffer. ; in, Indicates the trigger threshold; B34, Mini-batch sampling; from A random sample is selected from the data, and the advantage and target value are calculated, i.e., calculated using GAE. Using commentator networks to estimate target value ; B35. Update the network; minimize Update the commentator and policy networks, and soft-update the target network parameters; B4. Output the optimal drone trajectory With transmission routing .

5. The method for joint optimization of UAV network trajectory, routing, and resources to resist mobile interference as described in claim 4, characterized in that, Step S4 is as follows: Calculate the optimized system bottleneck latency and evaluate the relative rate of change of latency; set the update threshold for the relative change as follows. Update the latency history and calculate the relative changes in latency for the three most recent latency periods. ,like If the iteration stops, then repeat steps S2 and S3; finally, the optimal UAV trajectory is output. Optimal transmission route path With optimal channel allocation strategy ; Otherwise, using the updated trajectory and route as input, return to step S2 to re-optimize the channel allocation, and continue iterating until the convergence condition is met.

Images