Unmanned aerial vehicle route and resource optimization method based on ISAC system and federated learning

Abstract

The application discloses a method for optimizing the flight path and resources of a UAV based on an ISAC system and federated learning, and belongs to the technical field of UAV air-ground communication, joint calculation and perception. The application constructs a multi-UAV wireless communication model with the minimum energy consumption of the UAV as the target, and the communication energy consumption of the UAV, the position and speed of the UAV, the communication demand of the UAV, and the bandwidth and computing resources that can be allocated by the UAV as constraints. The application can more accurately estimate the position of the ground terminal, more quickly solve the communication demand of the dense ground terminal, and allocate deep learning tasks to multiple UAVs for calculation, fully utilize the computing resources of the UAV, and reduce the computing pressure of a single UAV. The application also jointly optimizes the bandwidth of the multiple UAVs and the ground terminal, the trajectory of the UAV, and the deep learning task calculation, so that the wireless communication rate is maximized while the energy consumption of the UAV is minimized, and the energy efficiency of the communication between the UAV and the ground terminal is improved.

Description

Technical Field

[0001] This invention belongs to the fields of UAV air-to-ground communication, joint computing and perception technology, specifically involving a method for optimizing UAV trajectory and resources based on the ISAC system and federated learning. Background Technology

[0002] In recent years, unmanned aerial vehicles (UAVs) have been widely used in various fields such as combat, surveillance, media, and rescue. Especially in the field of wireless communication, UAVs, with their superior maneuverability and high flexibility, can build efficient ground communication systems and quickly fly to designated locations. Furthermore, because UAVs fly at relatively low altitudes and are close to ground terminals, distortion caused by long-channel communication can be reduced. UAVs can also flexibly choose locations in three-dimensional space as needed and establish communication networks with ground terminals. In conclusion, in natural disasters such as earthquakes that cause extensive damage to infrastructure, wireless communication supported by UAVs can greatly improve the accuracy, reliability, and speed of disaster relief.

[0003] Despite these advantages of air-to-ground wireless communication networks built by drones, there are four unresolved technical issues.

[0004] 1. First, unlike base stations, drones are not fixed on the ground during flight and are easily affected by environmental factors such as strong winds in the air, which may lead to unstable communication links and information distortion, reducing the reliability of air-to-ground communication.

[0005] 2. Secondly, drones move in three-dimensional space, and they also move while transmitting information to ground terminals. The communication link is always affected by the drone's trajectory, making the optimal design of the drone's trajectory extremely difficult;

[0006] 3. At the same time, during information transmission and task processing, drones and ground terminals consume a lot of energy. The energy consumption of drones during flight and information transmission is affected by various factors.

[0007] 4. Finally, during communication between the UAV and the ground terminal, it is not easy to determine the precise location information of the ground terminal, which greatly limits the communication rate and stability.

[0008] Typically, the problems mentioned above are interconnected and cannot be solved simultaneously. Therefore, optimizing airspace communication systems to achieve the best results is a very important issue. Summary of the Invention

[0009] To address the problems mentioned in the background art, this invention provides a method for optimizing UAV trajectories and resources based on the ISAC system and federated learning, in order to solve the problems of low reliability of UAV ground-to-air communication, difficulty in optimizing motion trajectories, high energy consumption of UAVs, and severely limited communication speed and stability.

[0010] To achieve the above objectives, the present invention provides the following technical solution:

[0011] The method for optimizing UAV trajectories and resources based on the ISAC system and federated learning includes the following steps:

[0012] S1: Determine energy consumption constraints; determine the link limitations between the ground terminal and the UAV, the maximum value of the UAV communication power, the minimum value of the UAV communication data transmission volume, the start and end points of the UAV trajectory, the horizontal flight speed range of the UAV, and the vertical flight speed range of the UAV.

[0013] S2: Equip each drone in the drone swarm with an edge computing server, which together with the ground terminal forms a drone system, and initializes the environmental configuration information of the ground terminal and the drone system;

[0014] S3: Select a drone located in the center of the drone swarm as the central server of the ISAC system. The central server communicates with each of the other drones through a wireless link to form a federated learning architecture for learning the location of the ground terminal. Initialize the configuration information of the ISAC system and the federated learning architecture in the drone system.

[0015] S4: Obtain the initial objective function value based on the configuration information initialized in S3;

[0016] S5: Design the state space and action space of the federated learning framework. The state space is used for the federated learning framework to generate the corresponding actions of the UAV, and the action space is used for the federated learning framework to generate the resource allocation operations and trajectory planning of the UAV.

[0017] S6: Design the reward function for the federated learning framework;

[0018] S7: Under energy consumption constraints, the UAV is trained and iteratively optimized through the state space, action space and reward function. It is determined whether the objective function value or federated learning framework performance obtained at the end of training meets the expected requirements. If yes, proceed to S8; otherwise, return to S5.

[0019] S8: Outputs the optimal link and communication power between the UAV and the ground terminal, the optimal allocation of UAV computing resources, the optimal UAV trajectory, and the minimum UAV energy consumption.

[0020] Preferably, in S2, the environmental configuration information for the ground terminal and the UAV system includes: mission duration Tn, number of ground terminals, etc. Ground terminal location Number of drones Horizontal position of the drone The altitude position of the drone Number of gaps in UAV flight paths m=1,…,M; Duration of gaps in UAV flight paths drone horizontal flight speed Maximum horizontal speed of drone Vertical flight speed of drones Maximum vertical speed of drones Maximum altitude of the UAV H, constant blade power P0, hovering induced power P1, rotor blade speed Utip, and average rotor induced speed during hovering. The following parameters are considered: fuselage drag ratio d0, rotor solidity S, air density ρ, rotor disk area G, UAV transmit power P, bandwidth B, noise variable N0, environmental constants a and b, speed of light c, and losses for LoS and non-LoS links depending on the environment. and carrier frequency Constants A and C are respectively and The horizontal distance between the UAV and the ground base station in the m-th time slot The probability of the link between the UAV and the k-th ground terminal being blocked in the n-th time slot. Data transmission link path loss between base stations and drones Channel rate ;

[0021] Set an iteration number variable i, with an initial value of i=1;

[0022] The drone's trajectory is set to an elliptical path with its ends connected. The drone maintains a speed less than its maximum speed during flight. The initial position of the drone's trajectory is represented as... The position of the u-th drone in the m-th segment is represented as .

[0023] Preferably, the configuration information for initializing the ISAC system and the federated learning system in S3 includes the following steps:

[0024] S3.1: At time slot m, the distance of the UAV u to the ground terminal n for the i-th sensing is... , The calculation formula is as follows:

[0025] ;

[0026] in, It consists of the superposition of the propagation delay of the ISAC signal from UAV u to the target ground terminal n and the reflection delay of the signal from the target ground terminal n back to UAV u. Let represent the sum of all non-removable system noises present in the UAV system, where the noise has a mean of zero and a variance of . Gaussian distribution;

[0027] S3.2: In the ISAC system, the drone located at the center of the drone swarm in a horizontal position is selected as the central server to form a federated learning architecture. During the federated learning process, each drone u uses deep learning methods to perform localization perception on the ground terminal. The maximum probability of the drone finding the location of the ground terminal through deep learning is marked as a parameter by an artificial neural network (ANN). ;

[0028] Each drone u has a dataset Du for model training, in which, for all In , No. Data samples , and These are the vectors representing the actual position and the measured position of the ground terminal in the i-th time slot, respectively. It is the measured distance from the UAV u in the i-th time slot to the ground terminal. These are the inputs and corresponding outputs of the ANN model of the drone u;

[0029] For using data samples For ANN model training, the loss function in the form of Mean Squared Error (MSE) is defined as:

[0030] ;

[0031] S3.3: Define the first round of global training steps for federated learning, including:

[0032] Initialization; the central server selects the training parameters and objective for the federated learning process, and broadcasts the global model ω to the drone ensemble. ;

[0033] Local training and uploading; each drone trains its own model locally and uploads its model parameters. Uploaded to the central server;

[0034] Global model aggregation and update: The central server uses all uploaded parameters to calculate the updated weighted average model parameters, i.e. ;

[0035] let This represents the global loss function of the ANN in the central server, with ω serving as a model parameter;

[0036] Minimize the global loss function: ;

[0037] in and It is the local loss function of the ANN model in the UAV u;

[0038] Each drone's ANN model needs to be trained to minimize the corresponding loss function, i.e. Let θ represent the global relative accuracy. Each participating drone u will perform deep learning and achieve a local relative accuracy θu through multiple local iterations.

[0039] The local training energy consumption of each participating drone in one round of global iteration is

[0040] ,in, , and It is a parameter;

[0041] The upper bound Iu of the local iteration of the drone u is:

[0042] ;

[0043] in These are parameters set by the drone.

[0044] θ and θu represent the global accuracy of the learning model of the central server and the local accuracy of the learning model of each UAV u, respectively.

[0045] ;

[0046] in It is the acceptable loss function for the drone u;

[0047] ;

[0048] in It is the acceptable loss function predefined by the central server;

[0049] Both θ and θu are limited by the number of iterations, defining an upper limit on the global number of iterations for the UAV to learn the position perception of the ground terminal. :

[0050] ;

[0051] in, It is a constant.

[0052] Preferably, obtaining the initial objective function value in step S4 includes the following steps:

[0053] S4.1: Obtain the propulsion energy consumption of the u-th UAV during the m-th segment of flight: ;

[0054] S4.2: Obtain the computational energy consumption of the u-th drone in the federated learning architecture: ;

[0055] S4.3: Obtain the total energy consumption of the u-th drone through S4.1-S4.2: Where β is the weight value, substituted into the trajectory of the u-th UAV initialized in S3. Obtain the initial computing power consumption of the drone. .

[0056] Preferably, in S5, MADDPG is used as the reinforcement learning framework. MADDPG uses policy gradient as the optimization method, and the state space design and action space design are as follows:

[0057] State space design; State space This includes the drone's current status information, namely:

[0058] ;

[0059] exist middle, It is the state of GTk in time slot m, characterized by its sensing location, that is:

[0060] ;

[0061] It refers to the state of the UAV in time slot m, characterized by its position and velocity, namely:

[0062] ;

[0063] Action space design:

[0064] ;

[0065] ;

[0066] ;

[0067] ;

[0068] ;

[0069] ;

[0070] ;

[0071] ;

[0072] exist In the equation, a(m) represents the action of UAVu in time slot m, with the angle between its horizontal direction and the target value. For action, that is ; This refers to the action of UAVu in time slot m, representing the horizontal distance traveled, i.e.:

[0073] ;

[0074] ;

[0075] It represents the movement of UAVu in time slot m, indicating the vertical distance traveled. The parameter represents the bandwidth allocation between UAVu and GTk; Pu represents the communication power of UAVu; This indicates whether UAVu participates in the federated learning process and the degree of its participation.

[0076] Preferably, S6 specifically involves: designing a reward function for the reinforcement learning program, wherein the reward function is designed as a comprehensive measure of the overall benefits of the communication system, including a penalty term to limit undesirable behavior, the overall benefits including communication rate, coverage area and energy efficiency, and undesirable behavior including excessive energy consumption and channel interference;

[0077] The reward function is designed as follows:

[0078] .

[0079] Preferably, S7 specifically involves: the current allocation policy and execution action based on the MADDPG policy network, recording the current state changes of the UAV and the corresponding rewards, calculating the possible future rewards by sampling data from the MADDPG experience replay buffer, and updating the MADDPG Q network. At the same time, the policy network will also be adjusted based on these updates. When the objective function value obtained at the end of training or the performance of MADDPG meets the expected requirements, proceed to S8; otherwise, return to S5.

[0080] Compared with the prior art, the beneficial effects of the present invention are:

[0081] 1. This invention considers the construction of an air-to-ground communication system by multiple UAVs, which can make more accurate position estimation of ground terminals and can solve the communication needs of dense ground terminals more quickly.

[0082] 2. This invention distributes deep learning tasks to multiple drones for computation, making full use of the drones' computing resources and reducing the computational burden on individual drones.

[0083] 3. This invention jointly optimizes the bandwidth of multiple UAVs and ground terminals, the trajectory of UAVs, and the computation of deep learning tasks, thereby minimizing UAV energy consumption while maximizing wireless communication rate and improving the energy efficiency of communication between UAVs and ground terminals. Attached Figure Description

[0084] Figure 1 A scenario model diagram for the reinforcement learning optimization method of joint trajectory and resource allocation in this application;

[0085] Figure 2 This is a flowchart of the reinforcement learning optimization method for joint trajectory and resource allocation in this application. Detailed Implementation

[0086] To facilitate understanding of the technical content of this invention by those skilled in the art, the invention will be further described in detail below with reference to the accompanying drawings and specific examples. It should be understood that the specific examples described herein are merely illustrative and not intended to limit the scope of the invention.

[0087] like Figure 1 , Figure 2 As shown, the method for optimizing UAV trajectories and resources based on the ISAC system and federated learning includes the following steps:

[0088] S1: The drone system consists of a ground terminal and an edge computing server mounted on the drone. Each drone has certain computing power and unused computing resources, which can provide services to the entire drone system. Therefore, the drone system can break down and offload deep learning tasks to each drone to complete.

[0089] This invention jointly optimizes the communication bandwidth and power between the UAV and the ground terminal, the UAV's trajectory and speed, and the allocation of the UAV's computing resources to minimize the UAV's energy consumption. In summary, the UAV's energy consumption problem can be described as... :

[0090] : #(1)

[0091] #(1.1)

[0092] #(1.2)

[0093] #(1.3)

[0094] #(1.4);

[0095] # (1.5);

[0096] #(1.6);

[0097] exist In this document, (1.1) specifies the link restrictions of ground base stations; (1.2) specifies the maximum value of UAV communication power; (1.3) limits the minimum amount of UAV communication data transmission; (1.4) specifies the starting and ending points of UAV trajectory; (1.5) limits the horizontal flight speed of UAV; and (1.6) limits the vertical flight speed of UAV.

[0098] Obviously, the problem The objective function is a non-convex function and cannot be directly solved. This invention proposes a reinforcement learning algorithm to iteratively optimize the objective function, thereby reducing the complexity of the problem.

[0099] On the issue By breaking down the problem, this invention solves the target problem using a federated learning framework, and through iteration, the minimum energy consumption of the drone can be obtained.

[0100] according to Related to the trajectory of the drone, The problem relates to the link between the drone and the ground terminal, as well as the allocation of the drone's computing resources. Furthermore, the drone's trajectory is constrained by the link between the drone and the terminal, leading to the non-convexity of the objective problem. First, we formally expand the two energy consumption values:

[0101] #(3)

[0102] #(4)

[0103] S2: Environmental configuration information for ground terminals and UAV systems includes: mission duration T n Number of ground terminals Ground terminal location Number of drones Horizontal position of the drone The altitude position of the drone Number of gaps in UAV flight paths m=1,…,M; Duration of gaps in UAV flight paths drone horizontal flight speed Maximum horizontal speed of drone Vertical flight speed of drones Maximum vertical speed of drones The maximum altitude of the UAV is H, the constant blade power is P0, the hovering induction power is P1, and the moving blade speed is U. tip Average rotor induced velocity during hovering The following parameters are considered: fuselage drag ratio d0, rotor solidity S, air density ρ, rotor disk area G, UAV transmit power P, bandwidth B, noise variable N0, environmental constants a and b, speed of light c, and losses for LoS and non-LoS links depending on the environment. and carrier frequency Constants A and C are respectively and The horizontal distance between the UAV and the ground base station in the m-th time slot The probability of the link between the UAV and the k-th ground terminal being blocked in the n-th time slot. Data transmission link path loss between base stations and drones Channel rate ;

[0104] Set an iteration number variable i, with an initial value of i=1;

[0105] The drone's trajectory is set to an elliptical path with its ends connected. The drone maintains a speed less than its maximum speed during flight. The initial position of the drone's trajectory is represented as... The position of the u-th drone in the m-th segment is represented as .

[0106] S3: Initializing the configuration information for the drone perception and federated learning framework includes the following steps:

[0107] S3.1: At time slot m, the sensing distance of UAV u to ground terminal n is: Because the ground terminal performs multiple distance sensing operations in each time slot m, let the number of operations be Q (Q is a parameter), therefore Based on the transmission characteristics of the integrated communication and sensing system, the UAV can calculate the straight-line distance between the detected object and the UAV by receiving its own integrated communication and sensing signals and calculating the propagation delay. The formula is as follows:

[0108] ;

[0109] Where c refers to the speed of light in space. It is composed of the propagation delay of the ISAC signal from the UAV u to the target ground terminal n and the reflection delay of the signal reflected from the target ground terminal n back to the UAV u. This represents the sum of all non-removable system noises present in the system, with a mean of zero and a variance of . The Gaussian distribution.

[0110] S3.2: In the ISAC system, the drone located in the center of the drone swarm in a horizontal position is selected as the central server. With the support of the central server, each drone can accurately sense the position of the ground terminal.

[0111] The central server communicates with each drone via wireless links, forming a federated learning architecture that learns the location of the ground terminals. This federated learning design enhances the deep learning process of ground terminal location awareness in a distributed manner, thereby obtaining more accurate perception results without consuming excessive communication and computing resources from each drone.

[0112] During the federated learning process, each UAV uses deep learning to locate and perceive the ground terminal. The UAV aims to find the most probable location of the ground terminal using deep learning; the artificial neural network (ANN) parameters are labeled as follows. Each drone u has a dataset D for model training. u In D u In the middle, for all In , No. Data samples used To indicate, among which , These are vectors that include the actual and measured positions of any ground terminal in the i-th time slot; This includes the measured distance between the UAV u and each ground terminal in the i-th time slot. Clearly, These are the inputs and corresponding outputs of the ANN model of the drone u.

[0113] Assume that the dataset for each drone is obtained during the system's training phase, while the actual ground terminal location can be measured using the Global Positioning System (GPS). Then, for the data samples used... For training the ANN model, we define the loss function in the form of mean squared error (MSE) as follows:

[0114] ;

[0115] S3.3: Based on the deep learning process described above, we designed the following... Figure 1 The federated learning architecture shown;

[0116] Specifically, let Let ω represent the global loss function of the ANN in the central server, and let ω be a model parameter; then the entire model minimizes the global loss function. ,in and This is the local loss function of the ANN model in the UAV u. Each UAV's ANN model must be trained to minimize the corresponding loss function, i.e. We use θ to represent the global relative accuracy. Each participating drone u performs deep learning on its positional awareness, achieving a local relative accuracy θ through multiple local iterations. u The central server then attempts to solve the global learning problem. A round of global training in federated learning involves three steps:

[0117] Initialization: The central server selects the training parameters and target for the federated learning process. The central server also broadcasts the global model ω to the drone ensemble. .

[0118] Local training and upload: Each drone trains its own model locally and uploads its model parameters. Uploaded to the central server.

[0119] Global model aggregation and update: The central server uses all uploaded parameters to calculate the updated weighted average model parameters, i.e. ;

[0120] To perform local ANN model training on each drone, assume the onboard server's CPU frequency is F. u This frequency is configurable. Therefore, the local training energy consumption of each participating drone in one round of global iteration is... .in, It depends on the effective switching capacitor of the chip architecture; C u It is the number of CPU cycles per sample in the UAV.

[0121] The upper bound of the local iterations of the drone u is:

[0122] ;

[0123] in These are parameters set by the drone;

[0124] θ and θ u Let θ represent the global accuracy of the learning model on the central server and the local accuracy of the learning model for each UAV u, respectively; obviously, the model accuracy θ of UAV u is... u Closely related to the ANN loss function mentioned above, it can be defined as follows: ,in Let θ be the acceptable loss function for the drone u. Then, θ can be defined as... ,in It is the acceptable loss function predefined by the central server.

[0125] θ and θ uSimultaneously, due to the limitation of the number of iterations, an upper limit is defined for the global number of iterations for the UAV to learn the position perception of the ground terminal:

[0126] ;

[0127] in, It is a constant; we can observe from the upper bound of the global iteration count that when the number of iterations is fixed, very high local precision (smaller θ) is achieved. u This can improve global accuracy (smaller θ). Because θ and θ u The connection between them allows us to incentivize participating UAVs to increase the number of local iterations in each global iteration in order to obtain a smaller θ. u However, due to the SWAP constraint of the drones, each drone cannot perform too many local iterations to conserve energy. Therefore, it is necessary to set an appropriate θ for each drone u. u value.

[0128] S4: Obtain the initial objective function value based on the initialization conditions, including the following steps:

[0129] S4.1: Obtain the propulsion energy consumption of the u-th UAV during the m-th segment of flight: ;

[0130] S4.2: Obtain the computational energy consumption of the u-th drone in the federated learning architecture: ;

[0131] S4.3: Obtain the total energy consumption of the u-th drone through S4.1-S4.2: Where β is the weight value, substituted into the trajectory of the u-th UAV initialized in S3. Obtain the initial computational energy consumption of the drone. .

[0132] S5: Application of Reinforcement Learning Algorithms: This application uses MADDPG (Multiple Agent DeepDeterministic Policy Gradient) as the reinforcement learning framework. MADDPG uses policy gradient as the optimization method. The task can be decomposed into two steps:

[0133] S5.1: State space design, including the drone's current position ,speed Remaining battery power, location of ground terminal Information such as these states collectively constitute the system's state space. This information is input to the policy network to generate corresponding actions. That is:

[0134] ;

[0135] exist middle, It is the state of GTk in time slot m, characterized by its sensing location, that is:

[0136] ;

[0137] It refers to the state of the UAV in time slot m, characterized by its position and velocity, namely:

[0138] ;

[0139] S5.2: Action space design corresponds to the operations that the UAV needs to perform, namely resource allocation (bandwidth allocation, power control) and trajectory planning (flight direction, speed). MADDPG's policy network will generate the optimal resource allocation strategy and flight trajectory based on the current state.

[0140] Action space design:

[0141] ;

[0142] ;

[0143] ;

[0144] ;

[0145] ;

[0146] ;

[0147] ;

[0148] ;

[0149] exist In the equation, a(m) represents the action of UAVu in time slot m, with the angle between its horizontal direction and the target value. For action, that is ; This refers to the action of UAVu in time slot m, representing the horizontal distance traveled, i.e.:

[0150] ;

[0151] ;

[0152] It represents the movement of UAVu in time slot m, indicating the vertical distance traveled. The parameter represents the bandwidth allocation between UAVu and GTk; Pu represents the communication power of UAVu; This indicates whether UAVu participates in the federated learning process and the degree of its participation.

[0153] S6: The reward function is one of the key components of the DDPG algorithm, guiding the UAV system towards goal optimization. In UAV communication systems, the reward function is designed as a comprehensive measure of the overall effectiveness of the communication system (such as communication rate, coverage area, energy efficiency, etc.), including a penalty term to limit undesirable behaviors (such as excessive energy consumption, channel interference, etc.). The reward function is designed as follows:

[0154] .

[0155] S7: The algorithm's execution flow is as follows: First, some key components need to be initialized, including the experience replay buffer F, used to store the experience collected during training, and the time slot counter M. Next, two policy networks are initialized, namely the original policy network (…). ) and target policy network Their parameters are denoted as (θ). π )and( ), and set Simultaneously, initialize two Q-networks, one of which is the original Q-network ( The other is the target Q network (). Their parameters are ( ) )and( ), and set During initialization, the parameters of the target network also need to be set to be equal to those of the corresponding original network.

[0156] In each round of the algorithm's operation, the drone will first sense the location of each ground terminal. Subsequently, the time slot counter m is set to 1, and the system state is initialized to s(1). In the next time slot, if the task has not yet been completed, the algorithm will select an action through the policy network (π) based on the current state s(m). ,in A random function is used to increase the exploratory nature of the process. After performing this action, the system enters the next state s(m+1) and receives a reward. This set of data will be stored in the experience playback buffer F.

[0157] At the end of each round, the algorithm randomly draws a small batch of samples from the experience replay buffer F. Conduct training.

[0158] First, calculate a target value for each sample. This value consists of the current reward and the potential future reward. Then, by minimizing the loss function... Update the parameters of the original Q network. , ( The algorithm then gradually brings the parameters of the target Q-network closer to those of the original Q-network. Next, it uses the updated Q-network to guide the optimization of the policy network, enabling it to select actions more effectively. Finally, gradients are applied to the parameters of the target policy network. renew This gradually brings the parameters of the original policy network closer to those of the network.

[0159] Compare the objective function E obtained at the end of training with the pre-defined objective function value. Compare, if less than If the iteration ends, proceed to step S8; otherwise, return to step S5 to modify the state space and action space design and retrain.

[0160] S8: Achieve optimal link between the drone and the ground terminal. and communication power Optimal allocation of UAV computing resources θ, optimal UAV trajectory And the minimum energy consumption of drones.

[0161] In summary, this invention considers a reinforcement learning optimization method for joint trajectory and resource allocation in a multi-UAV system based on federated learning and integrated communication and sensing. It constructs a multi-UAV wireless communication model with the goal of minimizing UAV energy consumption and constraints on UAV communication energy consumption, UAV position and speed, UAV communication requirements, and available bandwidth and computing resources. It makes full use of multi-UAV edge computing resources, thereby minimizing UAV energy consumption while maximizing wireless communication rate and improving the communication energy efficiency between UAVs and ground terminals, which has a relatively feasible practical effect.

Claims

1. A method for optimizing UAV trajectories and resources based on the ISAC system and federated learning, characterized in that, Includes the following steps: S1: Determine energy consumption constraints; Determine the link limitations between the ground terminal and the UAV, the maximum value of the UAV's communication power, the minimum value of the amount of data transmitted by the UAV, the start and end points of the UAV's trajectory, the horizontal flight speed range of the UAV, and the vertical flight speed range of the UAV. S2: Equip each drone in the drone swarm with an edge computing server, which together with the ground terminal forms a drone system, and initializes the environmental configuration information of the ground terminal and the drone system; S3: Select a drone located in the center of the drone swarm as the central server of the ISAC system. The central server communicates with each of the other drones through a wireless link to form a federated learning architecture for learning the location of the ground terminal. Initialize the configuration information of the ISAC system and the federated learning architecture in the drone system. S4: Based on the configuration information initialized in S3, obtain the initial objective function value. The initial objective function value is the total energy consumption of the UAV. The total energy consumption of the UAV is obtained by weighted summation of the propulsion energy consumption generated by the UAV flight and the computational energy consumption generated in the federated learning architecture. S5: MADDPG is adopted as the reinforcement learning framework. The state space and action space of the federated learning framework are designed. The state space is used for the federated learning framework to generate the corresponding actions of the UAV, and the action space is used for the federated learning framework to generate the resource allocation operation and trajectory planning of the UAV. The state space includes the ground terminal's perceived position in time slot m, the UAV's position and velocity in time slot m, and the action space includes the UAV's horizontal angle, horizontal movement distance, vertical movement distance, bandwidth allocation parameters, communication power, and federated learning participation level in time slot m. S6: Design a reward function for the federated learning framework. The reward function is a comprehensive measure of the overall benefits of the communication system, including communication rate, coverage area, and energy efficiency. The reward function includes penalty terms to limit excessive energy consumption and channel interference. S7: Under energy consumption constraints, based on the MADDPG reinforcement learning framework, the UAV is trained and iteratively optimized through state space, action space, and reward function. During the training and optimization, the current state changes and corresponding rewards of the UAV are recorded. Future rewards are calculated and the Q-network and policy network are updated by sampling data from the MADDPG experience replay buffer. It is then determined whether the objective function value or the performance of the federated learning framework at the end of training meets the expected requirements. If yes, proceed to S8; otherwise, return to S5. S8: Outputs the optimal link and communication power between the UAV and the ground terminal, the optimal allocation of UAV computing resources, the optimal UAV trajectory, and the minimum UAV energy consumption. Initializing configuration information in S3 includes the following steps: S3.1: At time slot m, the l-th sensing distance of UAV u to ground terminal n is... , The calculation formula is as follows: ； in, It consists of the superposition of the propagation delay of the ISAC signal from UAV u to the target ground terminal n and the reflection delay of the signal from the target ground terminal n back to UAV u. Let represent the sum of all non-removable system noises present in the UAV system, where the noise has a mean of zero and a variance of . Gaussian distribution; S3.2: In the ISAC system, the drone located at the center of the drone swarm in a horizontal position is selected as the central server to form a federated learning architecture. During the federated learning process, each drone u uses deep learning methods to perform localization perception on the ground terminal. The maximum probability of the drone finding the location of the ground terminal through deep learning is marked as a parameter by an artificial neural network (ANN). ; Each drone u has a dataset D for model training. u In D u In the middle, for all In , No. Data samples , and These are the vectors representing the actual position and the measured position of the ground terminal in the i-th time slot, respectively. It is the measured distance from the UAV u in the i-th time slot to the ground terminal. These are the inputs and corresponding outputs of the ANN model of the drone u; For using data samples For ANN model training, the loss function in the form of Mean Squared Error (MSE) is defined as: ； S3.3: Define the first round of global training steps for federated learning, including: Initialization: The central server selects the training parameters and target for the federated learning process, and broadcasts the global model ω to the drone ensemble. ; Local training and upload: Each drone trains its own model locally and uploads its model parameters. Uploaded to the central server; Global model aggregation and update: The central server uses all uploaded parameters to calculate the updated weighted average model parameters, i.e. ,in ; let This represents the global loss function of the ANN in the central server, with ω serving as a model parameter; Minimize the global loss function: ; It is the local loss function of the ANN model in the UAV u; Each drone's ANN model is trained to minimize the corresponding loss function, i.e. ; The local training energy consumption of each participating drone in one round of global iteration is: ,in, To effectively switch capacitors, Let be the number of CPU cycles for each sample in drone u. The CPU frequency of the onboard server of the drone. The upper bound Iu of the local iteration of the drone u is: ； in These are parameters set by the drone. θ and θ u These represent the global accuracy of the learning model on the central server and the local accuracy of the learning model for each UAV u, respectively. ； in It is the acceptable loss function for the drone u; ； in It is the acceptable loss function predefined by the central server; θ and θ u Simultaneously, due to the limitation of the number of iterations, an upper limit is defined for the global number of iterations for the UAV to learn the position perception of the ground terminal. : ； in, It is a constant.

Images