Unmanned aerial vehicle track and resource optimization method based on ISAC system and federated learning

Abstract

The invention discloses an unmanned aerial vehicle track and resource optimization method based on an ISAC system and federated learning, and belongs to the technical field of air-ground communication, joint calculation and perception of unmanned aerial vehicles. A multi-unmanned aerial vehicle wireless communication model constrained by the communication energy consumption of the unmanned aerial vehicle, the position and the speed of the unmanned aerial vehicle, the communication requirement of the unmanned aerial vehicle and the distributable bandwidth and the computing resource of the unmanned aerial vehicle can perform more accurate position estimation on the ground terminal, so that the communication requirement of the dense ground terminal can be more quickly solved; according to the method, a plurality of unmanned aerial vehicles are distributed to a ground terminal, a deep learning task is allocated to the plurality of unmanned aerial vehicles for calculation, the calculation resources of the unmanned aerial vehicles are fully utilized, and the calculation pressure of a single unmanned aerial vehicle is reduced. The bandwidth of the plurality of unmanned aerial vehicles and the ground terminal, the track of the unmanned aerial vehicle and deep learning task calculation are jointly optimized; the wireless communication rate is maximized while the energy consumption of the unmanned aerial vehicle is minimized, and the communication energy efficiency of the unmanned aerial vehicle and the ground terminal is improved.

Description

Technical Field

[0001] The present invention belongs to the field of UAV air-to-ground communication, joint computing and perception technology, and specifically relates to a UAV track and resource optimization method based on an ISAC system and federated learning. Background Art

[0002] In recent years, drones have been widely used in various fields such as combat, surveillance, media, and rescue. Especially in the field of wireless communications, drones can build efficient ground communication systems and fly quickly to designated locations with their superior maneuverability and high flexibility. In addition, since drones fly at a low altitude and are close to ground terminals, the distortion caused by long channels can be reduced. Drones can also flexibly choose locations in three-dimensional space according to needs and establish communication networks with ground terminals. In summary, in natural disasters such as earthquakes where a large amount of infrastructure is destroyed, wireless communications supported by drones can greatly improve the accuracy, reliability, and speed of disaster relief.

[0003] Although the air-to-ground wireless communication network built by drones has these advantages, there are four unresolved technical problems.

[0004] 1. First, unlike base stations, drones are not fixed on the ground during flight and are easily affected by environmental factors such as strong winds in the air, which may lead to unstable communication links and information distortion, reducing the reliability of ground-to-air communications;

[0005] 2. Secondly, the drone moves in three-dimensional space and is also moving when transmitting information to the ground terminal. The communication link is always affected by the trajectory of the drone, so the optimization design of the trajectory of the drone is very difficult;

[0006] 3. At the same time, during information transmission and task processing, drones and ground terminals consume a lot of energy. The energy consumption of drones during flight and information transmission is affected by various factors;

[0007] 4. Finally, during the communication between the UAV and the ground terminal, it is not easy to determine the precise location information of the ground terminal, and the communication rate and stability are greatly limited.

[0008] Usually, the above problems are interrelated and cannot be solved at the same time. Therefore, how to optimize the airspace communication system to achieve the best effect is a very important issue. Summary of the invention

[0009] In order to solve the problems raised in the above background technology, the present invention provides a UAV trajectory and resource optimization method based on the ISAC system and federated learning to solve the problems of low reliability of ground-to-air communication of UAVs, difficulty in optimizing motion trajectories, high energy consumption of UAVs, and great limitations on communication rate and stability.

[0010] To achieve the above object, the present invention provides the following technical solutions:

[0011] The UAV trajectory and resource optimization method based on the ISAC system and federated learning includes the following steps:

[0012] S1: Determine the energy consumption constraints; determine the link limit between the ground terminal and the drone, the maximum value of the drone communication power, the minimum value of the drone communication data transmission volume, the starting point and end point of the drone trajectory, the drone's horizontal flight speed range, and the drone's vertical flight speed range;

[0013] S2: An edge computing server is installed on each drone in the drone swarm, which together with the ground terminal forms a drone system and initializes the environmental configuration information of the ground terminal and the drone system;

[0014] S3: A drone located in the center of the drone group is selected as the central server of the ISAC system. The central server communicates with each of the remaining drones through a wireless link to form a federated learning architecture for learning the location of the ground terminal and initialize the configuration information of the ISAC system and the federated learning architecture in the drone system.

[0015] S4: Obtain the initial objective function value according to the configuration information initialized in S3;

[0016] S5: Design the state space and action space of the federated learning framework. The state space is used for the federated learning framework to generate the corresponding actions of the UAV, and the action space is used for the federated learning framework to generate the resource allocation operation and trajectory planning of the UAV.

[0017] S6: Design reward function for federated learning framework;

[0018] S7: Under the energy consumption constraint, the UAV is cyclically trained and iteratively optimized through the state space, action space and reward function to determine whether the objective function value or the performance of the federated learning framework obtained at the end of the training meets the expected requirements. If yes, enter S8, otherwise return to S5;

[0019] S8: Output the optimal link and communication power between the UAV and the ground terminal, the optimal UAV computing resource allocation, the optimal UAV trajectory, and the minimum UAV energy consumption value.

[0020] Preferably, in S2, the environmental configuration information of the ground terminal and the UAV system includes: mission duration Tn, number of ground terminals , Ground terminal location 、Number of drones , the horizontal position of the drone , the altitude position of the drone , the number of gaps in the UAV flight trajectory m=1,…,M, the duration of the gap in the UAV flight trajectory Horizontal flight speed of drone 、UAV maximum horizontal speed , UAV vertical flight speed 、UAV maximum vertical speed , UAV maximum altitude H, constant blade power P0, hovering induced power P1, rotor blade speed Utip, average rotor induced speed during hovering , fuselage drag ratio d0, rotor solidity S, air density ρ, rotor disk area G, drone transmission power P, bandwidth B, noise variable N0, environmental constants a and b, speed of light c, and the loss of LoS link and non-LoS link according to different environments is and , carrier frequency , constants A and C are and , the horizontal distance between the UAV and the ground base station in the mth time slot , the blocking probability of the link between the UAV and the kth ground terminal in the nth time slot , Path loss of data transmission link between base station and drone , Channel Rate ;

[0021] Set an iteration number variable i, whose initial value is i=1;

[0022] The trajectory of the drone is required to be an elliptical trajectory connected end to end. The drone maintains a speed less than the maximum speed during flight. The starting position of the initial drone trajectory is expressed as , the position of the u-th UAV in the m-th segment is expressed as .

[0023] Preferably, initializing the configuration information of the ISAC system and the federated learning system in S3 includes the following steps:

[0024] S3.1: At time slot m, the i-th sensing distance of UAV u to ground terminal n is , The calculation formula is as follows:

[0025] ;

[0026] in, It is the superposition of the propagation delay of the ISAC signal from UAV u to the target ground terminal n and the reflection delay from the target ground terminal n to UAV u. represents the sum of various types of system noise that cannot be removed in the UAV system. The noise has a mean of zero and a variance of Gaussian distribution of

[0027] S3.2: In the ISAC system, the drone horizontally located in the center of the drone group is selected as the central server to form a federated learning architecture. During the federated learning process, each drone u uses a deep learning method to locate the ground terminal, and the artificial neural network ANN marks the maximum possibility of the drone finding the ground terminal position through deep learning as a parameter. ;

[0028] Each drone u is equipped with a dataset Du for model training. In Du, for all In , No. Data samples , and are the vectors of the actual position and measured position of the ground terminal in the i-th time slot, is the measured distance from UAV u to the ground terminal in the i-th time slot, are the input and corresponding output of the ANN model of UAV u;

[0029] For using data samples For the ANN model training, the loss function in the form of mean square error MSE is defined as:

[0030] ;

[0031] S3.3: Set a round of global training steps for federated learning, including:

[0032] Initialization; the central server selects the training parameters and objectives of the federated learning process, and the central server broadcasts the global model ω to the drone collection ;

[0033] Local training and uploading: Each drone trains its own model locally and uploads its model parameters Upload to the central server;

[0034] Global model aggregation and update: The central server uses all uploaded parameters to calculate the updated weighted average model parameters, i.e. ;

[0035] let represents the global loss function of the ANN in the central server, and ω is the model parameter;

[0036] Minimize the global loss function: ;

[0037] in and is the local loss function of the ANN model in UAV u;

[0038] Each UAV’s ANN model needs to be trained to minimize the corresponding loss function, i.e. , θ represents the global relative accuracy, and each participating drone u will perform deep learning and achieve the local relative accuracy θu through multiple local iterations;

[0039] The local training energy consumption of each UAV participating in the training in one round of global iteration is

[0040] ,in, , and is a parameter;

[0041] The upper limit Iu of the local iteration of UAV u is:

[0042] ;

[0043] in are the parameters set by drone u;

[0044] θ and θu are the global accuracy of the learning model of the central server and the local accuracy of the learning model of each drone u, respectively;

[0045] ;

[0046] in is the acceptable loss function of UAV u;

[0047] ;

[0048] in is the acceptable loss function predefined by the central server;

[0049] Both θ and θu are limited by the number of iterations, defining the upper limit of the global number of iterations of the UAV’s position perception learning of the ground terminal. :

[0050] ;

[0051] in, is a constant.

[0052] Preferably, obtaining the initial objective function value in S4 comprises the following steps:

[0053] S4.1: Obtain the propulsion energy consumption of the u-th UAV in the m-th flight segment: ;

[0054] S4.2: Get the computing energy consumption of the u-th drone in the federated learning architecture: ;

[0055] S4.3: The total energy consumption of the u-th UAV is obtained through S4.1-S4.2: , where β is the weight value, substituted into the trajectory of the u-th drone initialized by S3 , get the energy consumption of initializing the drone .

[0056] Preferably, in S5, MADDPG is used as a reinforcement learning framework, MADDPG uses policy gradient as an optimization method, and the state space design and action space design are specifically as follows:

[0057] state space design; state space Includes the current status information of the drone, namely:

[0058] ;

[0059] exist middle, is the state of GTk at time slot m, characterized by its perceived position, namely:

[0060] ;

[0061] is the state of the drone at time slot m, characterized by its position and velocity, namely:

[0062] ;

[0063] Action Space design:

[0064] ;

[0065] ;

[0066] ;

[0067] ;

[0068] ;

[0069] ;

[0070] ;

[0071] ;

[0072] exist In the figure, a(m) is the action of UAVu at time slot m, with its horizontal angle For action, that is ; is the action of UAVu at time slot m, representing the horizontal movement distance, that is:

[0073] ;

[0074] ;

[0075] is the action of UAVu at time slot m, representing the movement distance in the vertical direction; represents the parameter of bandwidth allocation between UAVu and GTk; Pu represents the communication power of UAVu; Indicates whether UAVu participates in the federated learning process and the degree of participation.

[0076] Preferably, S6 specifically includes: designing a reward function of the reinforcement learning program, wherein the reward function is designed as a comprehensive measure of the overall benefit of the communication system, including a penalty term to limit bad behavior, the overall benefit includes communication rate, coverage area and energy efficiency, and the bad behavior includes excessive energy consumption and channel interference;

[0077] The reward function is designed as:

[0078] .

[0079] Preferably, S7 specifically includes: based on the current allocation strategy and execution action of the MADDPG policy network, recording the current state change of the UAV and the corresponding reward, calculating the possible future returns through the data sampled from the MADDPG experience playback buffer, and updating the Q network of MADDPG. At the same time, the policy network will also be adjusted based on these updates. When the objective function value obtained at the end of training or the MADDPG performance meets the expected requirements, enter S8, otherwise return to S5.

[0080] Compared with the prior art, the present invention has the following beneficial effects:

[0081] 1. The present invention considers the construction of an air-to-ground communication system using multiple UAVs, which can provide more accurate position estimation for ground terminals and more quickly address the communication needs of densely packed ground terminals.

[0082] 2. The present invention distributes deep learning tasks to multiple drones for calculation, fully utilizing the computing resources of the drones and reducing the computing pressure of a single drone.

[0083] 3. The present invention jointly optimizes the bandwidth of multiple UAVs and ground terminals, the trajectory of UAVs, and the problem of deep learning task calculation, so as to minimize the energy consumption of UAVs while maximizing the wireless communication rate, thereby improving the communication energy efficiency between UAVs and ground terminals. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 A scenario model diagram of the reinforcement learning optimization method for joint trajectory and resource allocation of this application;

[0085] Figure 2 Flowchart of the reinforcement learning optimization method for joint trajectory and resource allocation of this application. DETAILED DESCRIPTION

[0086] In order to facilitate those skilled in the art to understand the technical content of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific examples. It should be understood that the specific examples described herein are only used to explain the present invention and are not used to limit the present invention.

[0087] like Figure 1 , Figure 2 As shown in FIG. 1 , the UAV trajectory and resource optimization method based on the ISAC system and federated learning includes the following steps:

[0088] S1: The UAV system consists of a ground terminal and an edge computing server mounted on a UAV. Each UAV has a certain amount of computing power and unused computing resources that can provide services for the entire UAV system. Therefore, the UAV system can split and offload deep learning tasks to each UAV to complete.

[0089] The present invention jointly optimizes the communication bandwidth and power between the UAV and the ground terminal, the trajectory and speed of the UAV, and the allocation of the computing resources of the UAV to minimize the energy consumption of the UAV. In summary, the energy consumption problem of the UAV can be described as :

[0090] : #(1);

[0091] #(1.1);

[0092] #(1.2);

[0093] #(1.3);

[0094] #(1.4);

[0095] #(1.5);

[0096] #(1.6);

[0097] exist Among them, (1.1) stipulates the connection limit of the ground base station; (1.2) stipulates the maximum value of the drone communication power; (1.3) limits the minimum value of the drone communication data transmission volume; (1.4) stipulates the starting point and end point of the drone trajectory; (1.5) limits the horizontal flight speed of the drone; (1.6) limits the vertical flight speed of the drone.

[0098] Obviously, the problem The objective function is a non-convex function and cannot be directly solved and calculated. The present invention proposes a reinforcement learning algorithm to iteratively optimize the objective function and reduce the complexity of the problem.

[0099] Questions The present invention solves the target problem through a federated learning framework, and the minimum energy consumption of the drone can be obtained after iteration.

[0100] according to Related to the trajectory of the drone, It is related to the connection between the drone and the ground terminal and the allocation of the drone's computing resources. At the same time, the drone's trajectory is also subject to the constraints of the connection between the drone and the terminal, which leads to the non-convexity of the target problem. First, the two energy consumption values ​​are formally expanded:

[0101] #(3);

[0102] #(4)

[0103] S2: Environmental configuration information of the ground terminal and the UAV system includes: Mission duration T n , the number of ground terminals , Ground terminal location 、Number of drones , the horizontal position of the drone , the altitude position of the drone , the number of gaps in the UAV flight trajectory m=1,…,M, the duration of the gap in the UAV flight trajectory Horizontal flight speed of drone 、UAV maximum horizontal speed , UAV vertical flight speed 、UAV maximum vertical speed , UAV maximum altitude H, constant blade power P0, hovering induced power P1, rotor blade speed U tip , average rotor induced speed during hovering , fuselage drag ratio d0, rotor solidity S, air density ρ, rotor disk area G, drone transmission power P, bandwidth B, noise variable N0, environmental constants a and b, speed of light c, and the loss of LoS link and non-LoS link according to different environments is and , carrier frequency , constants A and C are and , the horizontal distance between the UAV and the ground base station in the mth time slot , the blocking probability of the link between the UAV and the kth ground terminal in the nth time slot , Path loss of data transmission link between base station and drone , Channel Rate ;

[0104] Set an iteration number variable i, whose initial value is i=1;

[0105] The trajectory of the drone is required to be an elliptical trajectory connected end to end. The drone maintains a speed less than the maximum speed during flight. The starting position of the initial drone trajectory is expressed as , the position of the u-th UAV in the m-th segment is expressed as .

[0106] S3: Initializing the configuration information of the drone perception and federated learning framework includes the following steps:

[0107] S3.1: At time slot m, the sensing distance of drone u to ground terminal n is Because the ground terminal will sense the distance multiple times in each time slot m, let the number be Q times (Q is a parameter), so From the transmission properties of the integrated communication and perception system, it can be seen that the drone can calculate the straight-line distance between the detected object and the drone by receiving the integrated communication and perception signal sent by itself and calculating the propagation delay. The formula is expressed as follows:

[0108] ;

[0109] Among them, c refers to the speed of light in space, It is composed of the propagation delay of the ISAC signal from UAV u to the target ground terminal n and the reflection delay from the target ground terminal n to UAV u. It represents the sum of various types of system noise that cannot be removed in the system. The noise has a mean of zero and a variance of Gaussian distribution.

[0110] S3.2: In the ISAC system, the drone horizontally located in the center of the drone group is selected as the central server. With the support of the central server, each drone can accurately perceive the location of the ground terminal.

[0111] The central server communicates with each drone through a wireless link to form a federated learning architecture for learning the location of ground terminals. Federated learning designed in this way can enhance the deep learning process of ground terminal location perception in a distributed manner, thereby obtaining more accurate perception results without consuming too much communication and computing resources of each drone.

[0112] In the federated learning process, each drone u uses deep learning methods to locate the ground terminal. The drone uses deep learning to find the maximum possibility of the ground terminal's location. The artificial neural network (ANN) parameters are marked as ; Each drone u has a dataset D for model training u In D u For all In , No. Data samples for To indicate that , are respectively the vectors including the actual position and the measured position of any ground terminal in the i-th time slot; Includes the measured distance between UAV u and each ground terminal in the i-th time slot. Obviously, are the input and corresponding output of the ANN model of UAV u.

[0113] Assume that the data set of each drone is obtained through system operation during the training phase, and the actual ground terminal position can be measured by the global positioning system (GPS). Then, for the use of data samples For the ANN model training, we define the loss function in the form of mean squared error (MSE) as:

[0114] ;

[0115] S3.3: Based on the above deep learning process, we designed Figure 1 The federated learning architecture shown;

[0116] Specifically, let represents the global loss function of the ANN in the central server, and ω is used as a model parameter; then the entire model is to minimize the global loss function: ,in and is the local loss function of the ANN model in UAV u. Each UAV ANN model must be trained to minimize the corresponding loss function, that is, , we use θ to represent the global relative accuracy. Each participating drone u will perform deep learning on the sense of position and achieve the local relative accuracy θ through multiple local iterations. u The central server tries to solve the global learning problem. In a round of global training in federated learning, there are three steps:

[0117] Initialization: The central server selects the training parameters and objectives of the federated learning process. The central server also broadcasts the global model ω to the drone collection .

[0118] Local training and uploading: Each drone trains its own model locally and uploads its model parameters Upload to the central server.

[0119] Global model aggregation and update: The central server uses all uploaded parameters to calculate the updated weighted average model parameters, i.e. ;

[0120] In order to perform local ANN model training on each drone, assuming that the CPU frequency of the onboard server is F u , the frequency is configurable. Then, the local training energy consumption of each UAV participating in the training in one round of global iteration is .in, is the effective switching capacitance that depends on the chip architecture; C u is the number of CPU cycles per sample in the UAV.

[0121] The upper limit of the local iteration of drone u is:

[0122] ;

[0123] in are the parameters set by drone u;

[0124] θ and θ u are the global accuracy of the central server learning model and the local accuracy of the learning model of each drone u; obviously, the model accuracy θ of drone u u It is closely related to the ANN loss function mentioned above and can be defined as ,in is the acceptable loss function for drone u. Then, θ can be defined as ,in is the acceptable loss function predefined by the central server.

[0125] θ and θ uAt the same time, limited by the number of iterations, the upper limit of the global number of iterations of the drone's position perception learning of the ground terminal is defined:

[0126] ;

[0127] in, is a constant; we can observe from the upper limit of the global number of iterations that when the number of iterations is fixed, high local accuracy (smaller θ u ) can improve the global accuracy (smaller θ). Since θ and θ u We can motivate the participating UAVs to increase the number of local iterations in each global iteration to obtain a smaller θ u However, due to the SWAP constraints of the UAVs, each UAV cannot perform too many local iterations to save energy. Therefore, it is necessary to set a suitable θ for each UAV u. u value.

[0128] S4: According to the initialization conditions, the initial objective function value is obtained, including the following steps:

[0129] S4.1: Obtain the propulsion energy consumption of the u-th UAV in the m-th flight segment: ;

[0130] S4.2: Get the computing energy consumption of the u-th drone in the federated learning architecture: ;

[0131] S4.3: The total energy consumption of the u-th UAV is obtained through S4.1-S4.2: , where β is the weight value, substituted into the trajectory of the u-th drone initialized by S3 , get the energy consumption of initializing the drone .

[0132] S5: Application of reinforcement learning algorithm: This application uses MADDPG (Multiple Agent Deep Deterministic Policy Gradient) as the reinforcement learning framework. MADDPG uses policy gradient as the optimization method. The task can be decomposed into two steps:

[0133] S5.1: State space design, including the current position of the drone ,speed , remaining battery power, and the location of the ground terminal These state information together constitute the state space of the system , which is used as input by the policy network to generate corresponding actions. That is:

[0134] ;

[0135] exist middle, is the state of GTk at time slot m, characterized by its perceived position, namely:

[0136] ;

[0137] is the state of the drone at time slot m, characterized by its position and velocity, namely:

[0138] ;

[0139] S5.2: Action space design corresponds to the operations that the drone needs to perform, which is resource allocation (bandwidth allocation, power control) and trajectory planning (flight direction, speed). MADDPG's policy network will generate the optimal resource allocation strategy and flight trajectory based on the current state.

[0140] Action Space design:

[0141] ;

[0142] ;

[0143] ;

[0144] ;

[0145] ;

[0146] ;

[0147] ;

[0148] ;

[0149] exist In the figure, a(m) is the action of UAVu at time slot m, with its horizontal angle For action, that is ; is the action of UAVu at time slot m, representing the horizontal movement distance, that is:

[0150] ;

[0151] ;

[0152] is the action of UAVu at time slot m, representing the movement distance in the vertical direction; represents the parameter of bandwidth allocation between UAVu and GTk; Pu represents the communication power of UAVu; Indicates whether UAVu participates in the federated learning process and the degree of participation.

[0153] S6: The reward function is one of the keys to the DDPG algorithm, which guides the UAV system to develop in the direction of target optimization. In the UAV communication system, the reward function is designed as a comprehensive measure of the overall benefits of the communication system (such as communication rate, coverage area, energy efficiency, etc.), including penalty terms to limit bad behaviors (such as excessive energy consumption, channel interference, etc.). The reward function is designed as:

[0154] .

[0155] S7: The algorithm operation process is as follows: First, some key parts need to be initialized, including the experience playback buffer F, which is used to store the experience collected during training, and the time slot counter M. Then, two policy networks are initialized, namely the original policy network ( ) and target policy network ), their parameters are recorded as (θ π )and( ), and set ; At the same time, initialize two Q networks, one is the original Q network ( ), and the other is the target Q network ( ), their parameters are ( )and( ), and set During the initialization process, the parameters of the target network need to be set to be equal to the corresponding original network parameters.

[0156] In each round of the algorithm, the drone first senses the location of each ground terminal. . Subsequently, the time slot counter m is set to 1 and the system state is initialized to s(1). In the next time slot, if the task has not been completed, the algorithm will select an action based on the current state s(m) through the policy network (π) ,in is a random function to increase exploration. After executing this action, the system will enter the next state s(m+1) and receive a reward , this set of data will be stored in the experience replay buffer F.

[0157] At the end of each episode, the algorithm randomly draws a small batch of samples from the experience replay buffer F Conduct training.

[0158] First, a target value is calculated for each sample , which is composed of the current reward and the possible future reward. Then, by minimizing the loss function , update the parameters of the original Q network , ( ) and gradually make the parameters of the target Q network approach the parameters of the original Q network. Then, the algorithm uses the updated Q network to guide the optimization of the policy network so that the policy network can select actions more effectively. Finally, the parameters of the target policy network are also applied with gradients renew

[0159] , making it gradually approach the parameters of the original policy network.

[0160] The objective function E obtained at the end of training is compared with the set objective function value Compare, if less than , the iteration ends and executes S8; otherwise, return to step S5 to modify the state space and action space design and re-train.

[0161] S8: Get the best link between drone and ground terminal and communication power , optimal UAV computing resource allocation θ, optimal UAV trajectory , and the minimum drone energy consumption value.

[0162] In summary, the present invention considers a reinforcement learning optimization method for joint trajectory and resource allocation in a multi-UAV system based on integrated federated learning and communication perception, constructs a multi-UAV wireless communication model with the goal of minimizing UAV energy consumption, and with the UAV's communication energy consumption, the UAV's position and speed, the UAV's communication requirements, and the UAV's allocatable bandwidth and computing resources as constraints. It makes full use of multi-UAV edge computing resources, minimizes UAV energy consumption while maximizing wireless communication rate, improves the communication energy efficiency between UAVs and ground terminals, and has a relatively feasible practical effect.

Claims

1. The UAV trajectory and resource optimization method based on ISAC system and federated learning is characterized by: The following steps are involved: S1: Determine energy consumption constraints; Determine the link limit between the ground terminal and the drone, the maximum value of the drone communication power, the minimum value of the drone communication data transmission volume, the starting and ending points of the drone trajectory, the drone's horizontal flight speed range, and the drone's vertical flight speed range; S2: An edge computing server is installed on each drone in the drone swarm, which together with the ground terminal forms a drone system and initializes the environmental configuration information of the ground terminal and the drone system; S3: A drone located in the center of the drone group is selected as the central server of the ISAC system. The central server communicates with each of the remaining drones through a wireless link to form a federated learning architecture for learning the location of the ground terminal and initialize the configuration information of the ISAC system and the federated learning architecture in the drone system. S4: Obtain the initial objective function value according to the configuration information initialized in S3; S5: Design the state space and action space of the federated learning framework. The state space is used for the federated learning framework to generate the corresponding actions of the UAV, and the action space is used for the federated learning framework to generate the resource allocation operation and trajectory planning of the UAV. S6: Design reward function for federated learning framework; S7: Under the energy consumption constraint, the UAV is cyclically trained and iteratively optimized through the state space, action space and reward function to determine whether the objective function value or the performance of the federated learning framework obtained at the end of the training meets the expected requirements. If yes, enter S8, otherwise return to S5; S8: Output the optimal link and communication power between the UAV and the ground terminal, the optimal UAV computing resource allocation, the optimal UAV trajectory, and the minimum UAV energy consumption value.

2. The method for optimizing the trajectory and resources of unmanned aerial vehicles based on the ISAC system and federated learning according to claim 1, characterized in that: In S2, the environmental configuration information of the ground terminal and the UAV system includes: mission duration T n , the number of ground terminals , Ground terminal location 、Number of drones , the horizontal position of the drone , the altitude position of the drone , the number of gaps in the UAV flight trajectory m=1,…,M, the duration of the gap in the UAV flight trajectory Horizontal flight speed of drone 、UAV maximum horizontal speed , UAV vertical flight speed 、UAV maximum vertical speed , UAV maximum altitude H, constant blade power P0, hovering induced power P1, rotor blade speed U tip , average rotor induced speed during hovering , fuselage drag ratio d0, rotor solidity S, air density ρ, rotor disk area G, UAV transmission power P, bandwidth B, noise variable N0, environmental constants a and b, speed of light c, and the loss of LoS link and non-LoS link according to different environments is and , carrier frequency , constants A and C are and , the horizontal distance between the UAV and the ground base station in the mth time slot , the blocking probability of the link between the UAV and the kth ground terminal in the nth time slot , Path loss of data transmission link between base station and drone , Channel Rate ; Set an iteration number variable i, whose initial value is i=1; The trajectory of the drone is required to be an elliptical trajectory connected end to end. The drone maintains a speed less than the maximum speed during flight. The starting position of the initial drone trajectory is expressed as , the position of the u-th UAV in the m-th segment is expressed as .

3. The method for optimizing the trajectory and resources of unmanned aerial vehicles based on the ISAC system and federated learning according to claim 2, characterized in that: Initializing the configuration information of the ISAC system and the federated learning system in S3 includes the following steps: S3.1: At time slot m, the i-th sensing distance of UAV u to ground terminal n is , The calculation formula is as follows: ； in, It is the superposition of the propagation delay of the ISAC signal from UAV u to the target ground terminal n and the reflection delay from the target ground terminal n to UAV u. represents the sum of various types of system noise that cannot be removed in the UAV system. The noise has a mean of zero and a variance of Gaussian distribution of S3.2: In the ISAC system, the drone horizontally located in the center of the drone group is selected as the central server to form a federated learning architecture. During the federated learning process, each drone u uses a deep learning method to locate the ground terminal, and the maximum probability of the drone finding the ground terminal position through deep learning is marked as a parameter through the artificial neural network ANN. ; Each drone u is equipped with a dataset D for model training u , in D u For all In , No. Data samples , and are the vectors of the actual position and measured position of the ground terminal in the i-th time slot, is the measured distance from UAV u to the ground terminal in the i-th time slot, are the input and corresponding output of the ANN model of UAV u; For using data samples For the ANN model training, the loss function in the form of mean square error MSE is defined as: ； S3.3: Set a round of global training steps for federated learning, including: Initialization; the central server selects the training parameters and objectives of the federated learning process, and the central server broadcasts the global model ω to the drone collection ; Local training and uploading: Each drone trains its own model locally and uploads its model parameters Upload to the central server; Global model aggregation and update: The central server uses all uploaded parameters to calculate the updated weighted average model parameters, i.e. ; let represents the global loss function of the ANN in the central server, and ω is the model parameter; Minimize the global loss function: ; in and is the local loss function of the ANN model in UAV u; Each UAV’s ANN model needs to be trained to minimize the corresponding loss function, i.e. , θ represents the global relative accuracy. Each participating drone u will perform deep learning and achieve the local relative accuracy θ through multiple local iterations. u ; The local training energy consumption of each UAV participating in the training in one round of global iteration is: ,in, , and is a parameter; Upper limit of local iteration of drone u I u for: ； in are the parameters set by drone u; θ and θ u They are the global accuracy of the learning model of the central server and the local accuracy of the learning model of each drone u; ； in is the acceptable loss function of UAV u; ； in is the acceptable loss function predefined by the central server; θ and θ u At the same time, the upper limit of the global iteration number of the UAV’s position perception learning of the ground terminal is defined by the number of iterations. : ； in, is a constant.

4. The method for optimizing the trajectory and resources of unmanned aerial vehicles based on the ISAC system and federated learning according to claim 3, characterized in that: The step of obtaining the initial objective function value in S4 includes the following steps: S4.1: Obtain the propulsion energy consumption of the u-th UAV in the m-th flight segment: ; S4.2: Get the computing energy consumption of the u-th drone in the federated learning architecture: ; S4.3: The total energy consumption of the u-th UAV is obtained through S4.1-S4.2: , where β is the weight value, substituted into the trajectory of the u-th drone initialized by S3 , get the energy consumption of initializing the drone .

5. The method for optimizing the trajectory and resources of unmanned aerial vehicles based on the ISAC system and federated learning according to claim 4, characterized in that: In S5, MADDPG is used as the reinforcement learning framework. MADDPG uses policy gradient as the optimization method. The state space design and action space design are as follows: state space design; state space Includes the current status information of the drone, namely: ； exist middle, is the state of GTk at time slot m, characterized by its perceived position, namely: ； is the state of the drone at time slot m, characterized by its position and velocity, namely: ； Action Space design: ； ； ； ； ； ； ； ； exist In the figure, a(m) is the action of UAVu at time slot m, with its horizontal angle For action, that is ; is the action of UAVu at time slot m, representing the horizontal movement distance, that is: ； ； is the action of UAVu at time slot m, representing the movement distance in the vertical direction; Parameter representing the bandwidth allocation between UAVu and GTk; P u represents the communication power of UAVu; Indicates whether UAVu participates in the federated learning process and the degree of participation.

6. The method for optimizing the trajectory and resources of unmanned aerial vehicles based on the ISAC system and federated learning according to claim 5, characterized in that: S6 is specifically: designing a reward function for the reinforcement learning program, where the reward function is designed as a comprehensive measure of the overall benefit of the communication system, including a penalty term to limit bad behavior, where the overall benefit includes communication rate, coverage area, and energy efficiency, and bad behavior includes excessive energy consumption and channel interference; The reward function is designed as: 。 7. The method for optimizing the trajectory and resources of unmanned aerial vehicles based on the ISAC system and federated learning according to claim 1, characterized in that: S7 specifically includes: based on the current allocation strategy and execution actions of the MADDPG policy network, record the current state changes of the UAV and the corresponding rewards, calculate the possible future returns through the data sampled from the MADDPG experience replay buffer, and update the MADDPG Q network. At the same time, the policy network will also be adjusted based on these updates. When the objective function value obtained at the end of training or the MADDPG performance meets the expected requirements, enter S8, otherwise return to S5.

Images