User-centered multi-unmanned aerial vehicle fair communication dynamic deployment method

Abstract

The invention discloses a user-centered multi-unmanned aerial vehicle fair communication dynamic deployment method. The method comprises the following steps: calculating user-centered cluster fair throughput; preprocessing the relative distance between the users in the cells and the relative distance between the cells by adopting a genetic algorithm; an MADRL algorithm is used to distribute an exclusive flight action for each unmanned aerial vehicle to enable the unmanned aerial vehicle to reach a corresponding position in a specified time slot; the ground user cell selects the unmanned aerial vehicle serving the ground user cell according to the corresponding distance between each unmanned aerial vehicle and the ground user cell, and one-to-one matching of the unmanned aerial vehicles and the ground user cell is completed; according to an MADRL algorithm, bandwidth resource allocation of the corresponding unmanned aerial vehicle is realized in each user cell; repeating the steps within a specified number of time slots until a plurality of convergent actor online networks are obtained; and multi-unmanned aerial vehicle communication dynamic deployment is realized. According to the method, the problems that the flexibility of the unmanned aerial vehicle cannot be fully utilized and the fairness among a plurality of mobile users cannot be ensured in a traditional deployment strategy taking the unmanned aerial vehicle as the center are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of dynamic deployment of multiple unmanned aerial vehicles, and in particular relates to a user-centric dynamic deployment method for fair communication of multiple unmanned aerial vehicles. Background Art

[0002] With the development of the low-altitude economy, drones equipped with mobile base stations (BSs) can act as temporary aerial base stations (ABSs) to achieve rapid deployment and provide wireless coverage in designated areas. Compared with ground fixed BSs, ABSs can provide timely communications and high-quality services in the face of emergencies, and have the characteristics of high flexibility and low deployment cost.

[0003] However, as the number of users continues to increase, the challenge of effectively deploying multiple UAVs to achieve mobile user access becomes more apparent. Faced with this challenge, the traditional UAV-centric strategy involves deploying multiple fixed UAVs in a designated area, with each UAV serving only the users within its coverage area. Therefore, in this deployment scheme, it is crucial to optimize the coverage area of ​​each UAV, which can be achieved by determining the optimal flight altitude based on the UAV's transmission power. Unfortunately, the UAV-centric deployment scheme cannot take advantage of the flexibility of drones and is difficult to adapt to changes in user needs, especially when users leave the target area, and cannot provide continuous service.

[0004] Therefore, it is important to construct a user-centric multi-UAV communication network that dynamically clusters neighboring mobile users so that each UAV base station can adaptively change its position according to the time-varying user-centric cells. Summary of the invention

[0005] Purpose of the invention: The present invention provides a user-centric multi-UAV fair communication dynamic deployment method, which realizes UAV trajectory planning for ground user communication coverage by jointly optimizing UAV trajectory, user connection, communication UAV scheduling and power allocation.

[0006] Technical solution: The user-centric multi-UAV fair communication dynamic deployment method described in the present invention comprises the following steps:

[0007] (1) Based on the average path loss model of communication between drones and users and the Jain fairness index, the user-centric cell fair throughput is obtained;

[0008] (2) Modeling the optimization problem of dynamic deployment of multiple UAVs in a user-centric manner while considering fairness;

[0009] (3) Using genetic algorithm to pre-process the relative distances between users in a cell and the relative distances between cells to clarify the user-centered cell distribution in each time slot;

[0010] (4) Using the multi-agent reinforcement learning algorithm MADRL, each UAV is assigned a unique flight action based on the distribution of ground user cells so that it can reach the corresponding location within the specified time slot;

[0011] (5) The ground user cell selects the drone that serves it based on the distance between each drone and the ground user cell, completing a one-to-one matching between drones and ground cells;

[0012] (6) Based on the MADRL algorithm and user-centric cell fair throughput, bandwidth resource allocation for corresponding drones is achieved in each user cell;

[0013] (7) Repeating steps (3) to (6) within a specified number of time slots until multiple converged actor online networks are obtained, where each actor online network corresponds to a UAV;

[0014] (8) Each UAV selects its flight actions and bandwidth allocation actions according to its converged actor online network and the distribution of ground mobile user cells, thereby achieving user-centric dynamic deployment of multi-UAV communications considering fairness.

[0015] Furthermore, the implementation process of step (1) is as follows:

[0016] In the nth time slot, the three-dimensional coordinates of the jth UAV and the two-dimensional coordinates of the i-th user are expressed as and In each time slot, mobile users are divided into K user-centered clusters, where the kth n The center coordinates of a user-centered cluster are The number of users in each user-centric cluster Where I represents the total number of users, and the number of users in the kth user-centered cluster in the nth time slot is Representation: Define the time slot set N = {1,...,n,...,N}, the drone set The user-centered cell set K n ={1,...,k n ,...,K}, the set of all users I = {1,...,i,...,I}, and the set of users in different user-centric clusters in each time slot

[0017] The channel model in the considered network is based on the average path loss model, where the probability of a LOS path is given by:

[0018]

[0019] Among them, α and β are constant values ​​that depend on the environment, is the horizontal distance NLOS probability between the drone and the user, expressed as: P LOS =1-P NLOS ;

[0020] Therefore, the average path loss model is expressed as follows:

[0021]

[0022] Among them, f c and v c denote the carrier frequency and the speed of light respectively; in addition, the distance between the drone and user m is represented by l i,j Given, η LOS and η NLOS The kth ... n User-centric cells The channel gain between the user and the jth UAV It is expressed as:

[0023] Finally, the kth n The user-centric cluster The throughput per user is:

[0024]

[0025] in, Respectively represent the nth time slot and the kth time slot n The jth UAV matched by the user-centric cluster is assigned to the Bandwidth per user; Indicates whether the i-th user is assigned to the k-th user-centered cell in the n-th time slot; represents the transmission power of the jth UAV, σ 2 Represents the noise power; M ICI represents the inter-cell interference, j'∈J / J represents the j'th UAV in the nth time slot, excluding the UAV in the kth time slot. n The jth UAV matched to the cell;

[0026] Therefore, in the nth time slot, the kth n In the user-centric cluster The throughput rate per user is defined as: in, Indicates the kth time before time slot n n User-centric cells The total throughput for each user is Indicates the kth time before time slot n n The sum of the throughput of all users in a user-centric cluster;

[0027] Define the Jain fairness index to measure the kth time slot in the nth time slot n Throughput difference between users in a cell:

[0028]

[0029] Therefore, based on Jain-FI, the kth time slot of the nth time slot is n The throughput of a user-centric cell is defined as the fair throughput of the user-centric cell:

[0030]

[0031] Furthermore, the implementation process of step (2) is as follows:

[0032] For a quadcopter, the thrust of each rotor is the speed of the UAV. and acceleration Functions of , where bold indicates their vectors:

[0033]

[0034] in, and They represent the velocity direction vector and velocity magnitude of the UAV, g represents the gravity acceleration vector, and n r and m represent the number of rotors and the mass of the UAV, respectively, ρ and S FP represent the air density and the equivalent flat plate area of ​​the fuselage respectively; the propulsion power is expressed as:

[0035]

[0036] Among them, δ, c T ,ρ,c sol 、c fac , d0 represents the drag coefficient of the local blade section, the thrust coefficient based on the disc area, the air density, the rotor stability, the incremental correction factor of the induced power relative to the linear induced speed, the disc area of ​​each rotor, and the drag ratio of each rotor in the fuselage, while τ c represents the aircraft's climbing angle; the remaining energy of the jth UAV in the nth time slot is expressed as: E maxis the maximum storage energy of the UAV, Δ n is the time slot length;

[0037] Therefore, in order to realize the user-centric dynamic deployment scheme of multiple UAVs while considering fairness, the optimization problem P1 is modeled as follows:

[0038]

[0039] Among them, C1 indicates whether the jth UAV serves the kth user-centric cell in the nth time slot, C2 indicates the number of users in each user-centric cell; under the guarantee of C3, the total bandwidth allocated to users by each UAV in each user-centric cell is equal to the total bandwidth B of the UAVBS; C3 is guaranteed by C4 that the energy of the UAV is not exhausted during the mission; the motion of the UAV is decomposed into three directions: x, y, and z, and C5 and C6 reflect the maximum values ​​of acceleration and velocity in the three directions; finally, C7 is to prevent collisions between multiple UAVs; the three-dimensional matrix They represent the acceleration of the UAV in three directions, the bandwidth allocation of the UAV when serving the matched cluster, the matching between the UAV and the cell, and the matching between the user and the cell.

[0040] Furthermore, the implementation process of step (3) is as follows:

[0041] Preprocess users to form user-centered cells, and model the preprocessing problem P2 as:

[0042]

[0043] in, represents the sum of the relative distances between users in the cell, and represents the sum of relative distances between cells;

[0044] For the preprocessing problem P2, in the genetic algorithm, a chromosome is a valid solution to the preprocessing problem P2, so a complete chromosome is shown as follows: The element ch belongs to the set {1,...,i,...,I,...,I+K-1}. When the value of ch is in [1,I], it indicates the number of the ground mobile user. When the value of ch is in [I+1,I+K-1], it indicates the separation point between cells in the chromosome. According to the constraint C2 in P1, every I cells in the chromosome k After the first position, the value of the element ch will be in the range [I+1, I+K-1], and the values ​​of the elements in other positions will be in the range [1, I];

[0045] The chromosomes that meet the requirements will form the initial population pop, whose size is represented by Ω; for the population pop, the fitness function value fit(ω) of each chromosome is obtained according to the optimization objective function of P2; based on the distribution of the fitness function values ​​fit(ω) of all chromosomes in the population, the random traversal sampling (SUS) method is introduced to select chromosomes with higher fitness function values, and their number is Ω1, where Ω1<Ω; next, for the selected chromosomes, the partial matching crossover strategy completes the crossover operation of these chromosomes; if the random number is less than the mutation probability, then the elements of any two points on the chromosome are randomly exchanged; finally, the chromosomes that have undergone the above operation are combined with the Ω-Ω1 chromosomes with higher fitness function values ​​in the initial population to form a new generation of population; repeat the above steps until convergence to the optimal solution, and finally obtain the chromosome with the highest fitness function value, that is, the user clustering strategy under the current time slot.

[0046] Furthermore, the implementation process of step (4) is as follows:

[0047] The preprocessing clarifies the user-centered cell distribution in each time slot, and then optimizes the positions of multiple drones and the allocation of network resources, thereby transforming P1 into P3 problem:

[0048]

[0049] For problem P3, the MADRL algorithm is used to solve the challenge of user-centric dynamic deployment of multiple UAVs, while considering fairness. The multi-agent deep deterministic strategy MADDPG is adopted in MADRL; MADDPG involves four neural networks, including actor online network, critic online network, actor target network and critic target network, and updating these networks requires knowing the status, actions and rewards of multiple UAVs; the observation space of each agent consists of the position of all UAVs, the speed in three directions, the position of ground users and the cell clusters of ground users, which can be expressed as:

[0050]

[0051] Therefore, the state space of all agents is:

[0052] The action of each drone consists of two parts: the acceleration of the drone in three directions and the bandwidth allocation of the drone. This work is expressed as: The action set of all drones is: A={A1,...,A j};

[0053] Since each drone moves with uniform acceleration in each direction, after the actor online network outputs the action corresponding to the acceleration of the drone in three directions, the coordinates and speed updates of each drone in three directions are obtained by the following formula:

[0054]

[0055] Based on the above coordinate and speed update formula, the position and speed of the drone in the specified time slot are obtained; for this type of action, the reward function involved is first to prevent the drone speed or position from exceeding the limit, which is expressed as:

[0056]

[0057] In order to ensure that all drones do not collide, the judgment criterion is that any two drones at the same time, as long as the coordinates of the three directions of the drones are not equal at the same time, the constraint C6 in P3 is satisfied. The reward function is expressed as:

[0058]

[0059] Whether the UAV's flight can meet its energy constraints is reflected by the following reward function: j_3 =E j (n); In the above reward function, Used to adjust the size of the reward value.

[0060] Furthermore, the implementation process of step (5) is as follows:

[0061] According to the coordinates of the center of the user cell and the coordinates of each drone, the distance between different user cells and the drone is calculated. The calculation formula is described as follows:

[0062]

[0063] According to the distance formula between cells and drones, starting from the first user cell, each cell traverses the idle drones one by one until the drone closest to its center is found to complete the matching;

[0064] Design a reward function to measure the matching effect: in, Used to adjust the size of the reward value. Used to limit the distance between each UAV and its matching cell.

[0065] Furthermore, the implementation process of step (6) is as follows:

[0066] First, the users in the user cell matched with each drone are sorted from low to high according to the sum of their throughput before time slot n, and the bandwidth allocation values ​​output by their actor online network are sorted from high to low; next, the sorted cell users and the sorted bandwidth allocation values ​​are also matched one by one to achieve bandwidth resource allocation for each drone; finally, in order to improve the user-centric cell fair throughput, a reward function is designed to measure the impact of bandwidth resource allocation on the user-centric cell fair throughput. The reward function is expressed as: j_3 =CFT.

[0067] Furthermore, the implementation process of step (7) is as follows:

[0068] First, based on all reward functions, the reward for each drone is expressed as:

[0069] RE j =λ j_1 +λ j_2 +λ j_3 +λ j_4 +λ j_5

[0070] All drone rewards are expressed as:

[0071] RE={RE1,RE2,RE3,RE4,RE5}

[0072] Next, {S, A, RE, D, S'} will be stored as experience in the replay buffer, where D = {D1, ..., D j} indicates whether the training of each UAV is completed in each time slot, and S' represents the state set of all UAVs in the next time slot;

[0073] The parameters θ of the critic online network Q , the parameters θ of the actor online network μ Update using a gradient descent based method:

[0074]

[0075] Among them, the update target is ζ φ =REφ+γ(Q'(S' φ ,A'|θ μ' ),θ Q' ), while Φ and γ represent the experience size and discount factor sampled from the experience pool respectively; S φ , S' φ , A φ and RE φ Represents the data sampled from the experience pool; A' represents the actor target network in the next state S'φ The output action is recorded as θ μ' ; A indicates that the actor online network is in the current state S φ The action output under the condition of ; Q(·) is the value function of the critic online network output, Q'(·) is the value function of the critic target network output, and its parameter is denoted by θ Q '; For the parameter update of the target network, a soft update method is adopted, which is specifically given by the following formula:

[0076] θ Q' =μτθ Q +(1-τ)θ Q'

[0077] θ μ' =τθ μ +(1-τ)θ μ'

[0078] Among them, τ represents the update frequency.

[0079] Furthermore, the implementation process of step (8) is as follows:

[0080] After training, each UAV will have four converged networks; in the implementation stage, the preprocessed user-centric cell distribution is input into the actor online network of each UAV, which then outputs the operations to be performed by the UAV, ultimately achieving user-centric dynamic deployment of multi-UAV communications that takes fairness into account.

[0081] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects: First, the user-centric multi-UAV deployment scheme proposed in the present invention can not only guide multiple UAVs to adapt to the distribution of mobile users, but also effectively improve the fair throughput of the system while reducing interference between cells; in addition, compared with the traditional UAV-centric deployment scheme, the method proposed in the present invention increases the fair throughput by at least 36%, thereby ensuring the performance advantage of the user-centric multi-UAV communication network. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 is a flow chart of the present invention;

[0083] Figure 2 This is a schematic diagram of the UAV communication coverage system scenario proposed by the present invention;

[0084] Figure 3 Result graph for dynamic deployment of multiple UAVs in two consecutive time slots centered on the user;

[0085] Figure 4 A comparison diagram of the fair throughput between the deployment scheme proposed by the present invention and the traditional deployment scheme;

[0086] Figure 5 The comparison diagram of the cumulative distribution function of fair throughput of user-centric cells under different access modes;

[0087] Figure 6 Comparison plot of fair throughput for the envisioned networks under different clustering parameters. DETAILED DESCRIPTION

[0088] The present invention is further described in detail below with reference to the accompanying drawings.

[0089] like Figure 1 As shown, the present invention proposes a user-centric multi-UAV fair communication dynamic deployment method, which specifically includes the following steps:

[0090] Step 1: Based on the average path loss model of communication between drones and users and the Jain Fairness Index (Jain-FI), a user-centric cell fairness throughput model is obtained.

[0091] In the nth time slot, the 3D coordinates of the jth UAV and the 2D coordinates of the i-th user can be expressed as and In each time slot, mobile users are divided into K user-centered clusters, where the kth n The center coordinates of a user-centered cluster are expressed as The number of users in each user-centric cluster I k Can be used Where I represents the total number of users, and the number of users in the kth user-centered cluster in the nth time slot is represents the total number of users, and the number of users in the kth user-centered cluster in the nth time slot is Finally, in order to facilitate the description of the present invention, the time slot set N = {1, ..., n, ..., N} is defined, and the drone set The user-centered cell set K n ={1,...,k n ,...,K}, the set of all users I = {1,...,i,...,I}, and the set of users in different user-centric clusters in each time slot The channel model in the considered network is based on the average path loss model, where the probability of a LOS path is given by:

[0092]

[0093] Among them, α and β are constant values ​​that depend on the environment, is the horizontal distance between the drone and the user. The NLOS probability can be expressed as: P LOS =1-P NLOS .

[0094] Therefore, the average path loss equation model is expressed as follows:

[0095]

[0096] Among them, f c and v c denote the carrier frequency and the speed of light respectively. In addition, the distance between the drone and user m is given by l i,j Given, η LOS and η NLOS represents the average additional loss value of LOS and NLOS propagation of the connection. In addition, n User-centric cells The channel gain between the user and the jth UAV It is expressed as:

[0097] Finally, the kth n The user-centric cluster The throughput per user can be given by:

[0098]

[0099] in, Respectively represent the nth time slot and the kth time slot n The jth UAV matched by the user-centric cluster is assigned to the Bandwidth per user; Indicates whether the i-th user is assigned to the k-th user-centered cell in the n-th time slot; represents the transmission power of the jth UAV, σ 2 Represents the noise power; M ICI represents the inter-cell interference, which can be expressed as: j'∈J / J represents the j'th UAV in the nth time slot, excluding the UAV in the kth time slot. n The jth UAV is matched with each cell.

[0100] Therefore, in the nth time slot, the kth n The user-centric cluster The throughput rate per user is defined as: in Indicates the kth time before time slot n n User-centric cells The total throughput for each user is Indicates the kth time before time slot n n The sum of the throughput of all users in a user-centric cluster.

[0101] Then, the Jain Fairness Index (Jain-FI) is defined to measure the kth time slot in the nth time slot. n Throughput difference between users in a cell:

[0102]

[0103] Therefore, based on Jain-FI, the kth time slot of the nth time slot is n The throughput of a user-centric cell is defined as the fair throughput of the user-centric cell, as follows:

[0104] Step 2: Model the user-centric dynamic deployment optimization problem of multiple UAVs with fairness in mind.

[0105] The energy consumption of UAV mainly consists of communication-related energy and UAV propulsion-related energy. The energy consumption related to communication is much smaller than that related to propulsion, so it is often ignored. For a quad-rotor UAV, the thrust of each rotor is the UAV speed. and acceleration Functions of , where bold indicates they are vectors:

[0106]

[0107] in, and They represent the velocity direction vector and velocity magnitude of the UAV, and g represents the gravity acceleration vector. r and m represent the number of rotors and the mass of the UAV, respectively, ρ and S FP denote the air density and the equivalent flat plate area of ​​the fuselage, respectively. Therefore, the propulsion power can be expressed as:

[0108]

[0109] Among them, δ, c T ,ρ,c sol 、c fac , d0 are some related parameters in the formula, representing the drag coefficient of the local blade section, the thrust coefficient based on the disc area, the air density, the rotor stability, the incremental correction factor of the induced power relative to the linear induced speed, the disc area of ​​each rotor, and the drag ratio of each rotor in the fuselage, while τ crepresents the aircraft's climbing angle. Therefore, the remaining energy of the jth UAV in the nth time slot can be expressed as: Where E max is the maximum storage energy of the UAV, Δ n is the time slot length.

[0110] Therefore, in order to realize the user-centric dynamic deployment scheme of multiple UAVs while considering fairness, the optimization problem P1 is modeled as follows:

[0111]

[0112] Among them, C1 indicates whether the jth UAV serves the kth user-centric cell in the nth time slot, and C2 indicates the number of users in each user-centric cell. Under the guarantee of C3, the total bandwidth allocated to users by each UAV in each user-centric cell is equal to the total bandwidth B of the UAVBS. In addition, it is guaranteed by C4 that the energy of the UAV is not exhausted during the mission. In this paper, the motion of the UAV is decomposed into three directions: x, y, and z, so C5 and C6 reflect the maximum values ​​of the acceleration and velocity in these three directions. Finally, preventing collisions between multiple UAVs is reflected in C7. Three-dimensional matrix They represent the acceleration of the UAV in three directions, the bandwidth allocation of the UAV when serving the matched cluster, the matching between the UAV and the cell, and the matching between the user and the cell.

[0113] Step 3: Use a genetic algorithm to pre-process the ground mobile users to obtain suitable user cells, where the effect of the pre-processing is measured by the relative distance between users in the cell and the relative distance between cells.

[0114] Problem P1 contains discrete variables and And continuous variables and It is a non-convex optimization problem that is difficult to solve directly. First, by preprocessing users to form user-centered cells, it can be seen that the preprocessing problem P2 can be modeled as:

[0115]

[0116] in, represents the sum of the relative distances between users in the cell, and Represents the sum of the relative distances between cells.

[0117] For the preprocessing problem P2, an innovative genetic algorithm for solving the user-centric grouping problem is proposed. In fact, in the genetic algorithm, a chromosome is a valid solution to the preprocessing problem P2, so a complete chromosome is shown as follows: The element ch belongs to the set {1,...,i,...,I,...,I+K-1}. When the value of ch is in [1,I], it indicates the number of the ground mobile user, and when the value of ch is in [I+1,I+K-1], it indicates the separation point between cells in the chromosome. In fact, according to the constraint C2 in P1, every I cells in the chromosome k After the first position, the value of the element ch will be in the range [I+1, I+K-1], and the values ​​of the elements at other positions will be in the range [1,I].

[0118] Therefore, a series of chromosomes that meet the requirements will form the initial population pop, and its size can be represented by Ω. For the population pop, the fitness function value fit(ω) of each chromosome can be obtained according to the optimization objective function of P2. Based on the distribution of the fitness function values ​​fit(ω) of all chromosomes in the population, the random traversal sampling (SUS) method is introduced to select chromosomes with higher fitness function values, and the number is Ω1, where Ω1<Ω. Next, for the selected chromosomes, the partial matching crossover (PMX) strategy can complete the crossover operation of these chromosomes. If the random number is less than the mutation probability, the elements of any two points on the chromosome are randomly exchanged. Finally, the chromosomes that have undergone the above operation are combined with the chromosomes with higher fitness function values ​​of Ω-Ω1 in the initial population to form a new generation of population. Repeat the above steps until convergence to the optimal solution, and finally obtain the chromosome with the highest fitness function value, that is, the user clustering strategy under the current time slot.

[0119] Step 4: Using the multi-agent reinforcement learning (MADRL) algorithm, according to the distribution of ground user cells, each UAV is assigned an exclusive flight action so that it reaches the corresponding location within the specified time slot.

[0120] In step 3, the preprocessing clarifies the user-centric cell distribution in each time slot, so that multiple drone positions and network resource allocation can be optimized in this step, thus transforming P1 into P3 problem:

[0121]

[0122] For problem P3, the present invention adopts the proposed GA-MADRL algorithm to solve the challenge of user-centric dynamic deployment of multiple UAVs, while considering fairness. The multi-agent deep deterministic strategy (MADDPG) is adopted in MADRL. Since MADDPG involves four neural networks, including the actor online network, the critic online network, the actor target network, and the critic target network, and updating these networks requires knowing the status, actions, and rewards of multiple UAVs, it is necessary to first understand the status in the input neural network. The observation space of each agent consists of the positions of all UAVs, the speeds in three directions, the positions of ground users, and the cell clusters of ground users, which can be expressed as:

[0123]

[0124] Therefore, the state space of all agents is:

[0125] The action of each drone consists of two parts: the acceleration of the drone in three directions and the bandwidth allocation of the drone. This work is expressed as: The action set of all drones can be expressed as: A = {A1,...,A j In each actor online network, these two actions are output simultaneously, but there is a sequence in the execution process. In this step, we first discuss the action of the acceleration of the drone in three directions.

[0126] Since each drone moves with uniform acceleration in each direction, after the actor online network outputs the action corresponding to the acceleration of the drone in three directions, the coordinates and speed updates of each drone in three directions can be obtained by the following formula:

[0127]

[0128] Based on the above coordinate and speed update formula, we can get the position and speed of the drone within the specified time slot. For this type of action, the reward function involved is to prevent the drone speed or position from exceeding the limit, which can be expressed as:

[0129]

[0130] In addition, in order to ensure that all drones do not collide, the judgment criterion is that any two drones at the same time can satisfy constraint C6 in P3 as long as their coordinates in three directions are not equal at the same time. The reward function can be expressed as:

[0131]

[0132] Whether the UAV's flight can meet its energy constraints is reflected by the following reward function: j_3 =E j (n). In the above reward function, Used to adjust the size of the reward value.

[0133] Step 5: The ground user cell selects the drone to serve it according to the distance between each drone and the ground user cell, thereby completing a one-to-one matching between the drone and the ground cell.

[0134] Each drone arrives at the corresponding position according to its exclusive action. At this time, the ground mobile users have also completed the construction of the ground user cell according to step 4. Therefore, in this step, each user cell will find the drone closest to its center to complete the matching.

[0135] First, based on the coordinates of the user cell center and the coordinates of each drone, the distance between different user cells and the drone can be calculated. The calculation formula is described as follows:

[0136]

[0137] Next, according to the distance formula between the cell and the drone, starting from the first user cell, each cell traverses the idle drones one by one until the drone closest to its center is found to complete the matching. In order to measure the matching effect, a reward function is designed: in, Used to adjust the size of the reward value, and Used to limit the distance between each UAV and its matching cell.

[0138] Step 6: Based on the MADRL algorithm and user-centric cell fair throughput, bandwidth resource allocation for the corresponding drone is implemented in each user cell, including:

[0139] According to step 4, the action of each drone consists of two parts: the acceleration of the drone in three directions and the bandwidth allocation of the drone. The acceleration action of the drone in three directions is performed in step 4, so the drone bandwidth allocation action will be performed in this step.

[0140] Since the drones and user-centric cells are in a one-to-one matching relationship, and the matching problem between drones and user cells has been completed in step five, for each drone, it first sorts the users in the user cell that matches it according to the sum of their throughput before time slot n from low to high, and the bandwidth allocation values ​​output by its actor online network are sorted from high to low. Next, the sorted cell users and the sorted bandwidth allocation values ​​are also matched one-to-one, so as to realize the bandwidth resource allocation of each drone. Finally, in order to improve the fair throughput of the user-centric cell, a reward function is designed to measure the impact of bandwidth resource allocation on the fair throughput of the user-centric cell. The reward function can be expressed as: λ j_3 =CFT.

[0141] Step 7: Repeat steps 3 to 6 within the specified number of time slots as one training round. Train multiple rounds until multiple converged actor online networks are obtained, where each actor online network corresponds to a drone, including:

[0142] First, based on all the above reward functions, the reward of each drone can be expressed as: RE j =λ j_1 +λ j_2 +λ j_3 +λ j_4 +λ j_5 , and all drone rewards can be expressed as: RE = {RE1, RE2, RE3, RE4, RE5}. Next, {S, A, RE, D, S'} will be stored as experience in the replay buffer, where D = {D1, ..., D j} indicates whether the training of each UAV is completed in each time slot, and S' represents the state set of all UAVs in the next time slot.

[0143] Therefore, the parameters θ of the critic online network Q , the parameters θ of the actor online network μ The update can be performed using a gradient descent method, and the relevant update formula is expressed as:

[0144]

[0145] Among them, the update target is ζ φ =RE φ +γ(Q'(S' φ ,A'|θ μ' ),θ Q' ), while Φ and γ represent the experience size and discount factor sampled from the experience pool, respectively. In addition, S φ , S' φ , Aφ and RE φ represents the data sampled from the experience pool, and A' represents the actor target network in the next state S' φ The output action is recorded as θ μ' Similarly, A represents the actor online network in the current state S φ In addition, Q(·) is the value function of the critic online network output, Q'(·) is the value function of the critic target network output, and its parameter is denoted by θ Q '. Therefore, for the parameter update of the target network, a soft update method is adopted, which is given by the following formula:

[0146] θ Q' =τθ Q +(1-τ)θ Q'

[0147] θ μ' =τθ μ +1-τ)θ μ'

[0148] Among them, τ represents the update frequency.

[0149] Finally, after obtaining the above network-related update formula, it is stipulated that: repeating steps 3 to 6 within the set number of time slots is a training round, and "episode" is defined as the number of training rounds. In each training round, ground users are preprocessed to form a user-centered cell. Then, each drone starts to explore according to the action value output by the actor online network. At this time, in order to increase the exploratory nature, the action of each drone can be updated as follows: A j =A j |θ μ +Λ, where Λ represents the action value obtained by Gaussian noise. Subsequently, the drones update their positions and velocities. Subsequently, each user-centered cell is matched with the nearest drone that is not assigned to other clusters. Then each drone completes the allocation of bandwidth resources in the user cell that matches it. After the action is executed, {S, A, RE, D, S'} is stored in the experience pool, where the size of the data stored in the experience pool is denoted as Γ, and the size of the minimum training data of the experience pool is set to Network training occurs every fixed number of times, with a frequency set to When the training conditions are met, And i_episode% And the current time slot is the initial time slot of the current training round, where i_episode represents the current number of trainings. The 4 networks of each drone will randomly extract data from the experience pool and train according to the network update formula. If the drone violates C3 or C5 in P3 or moves out of bounds, the action execution of the current time slot will end, resulting in a transition to the next training round. Finally, when the training phase ends, each drone will obtain 4 converged networks, including the actor's online network.

[0150] Step 8: Each UAV selects its flight action and bandwidth allocation action according to its converged actor online network and the distribution of ground mobile user cells, thus achieving user-centric dynamic deployment of multi-UAV communication considering fairness.

[0151] After training, each drone will have four converged networks. In the implementation phase, the preprocessed user-centric cell distribution is input into the actor online network of each drone, which then outputs the actions to be performed by the drone, ultimately achieving a user-centric multi-drone communication dynamic deployment that considers fairness.

[0152] Figure 2 The present invention provides a scenario for realizing user-centric dynamic deployment of multi-UAV communications taking fairness into consideration under the conditions of satisfying constraints such as UAV flight trajectory, user scheduling, and energy consumption. Figure 3 The distribution of mobile users and the positions of drones in two consecutive time slots randomly selected during the mission cycle are shown, where the length of each time slot is set to 5 seconds. Figure 3 It can be seen that as the user moves, the cell centered on the user changes, and the position of each drone is adjusted accordingly. Therefore, the proposed scheme can guide multiple drones to adapt to the distribution of mobile users. Figure 4 The figure is a comparison of the fair throughput of the proposed deployment scheme compared with the traditional deployment scheme. Compared with the traditional one-drone-centric deployment scheme, the present invention proposes a user-centric fair and dynamic deployment scheme, which can achieve fair on-demand deployment as the ground users move. Therefore, under the condition of different numbers of users, the fairness throughput obtained by this scheme is better than the fairness throughput obtained by the traditional fixed base station deployment scheme. Figure 5The cumulative distribution function comparison diagram of fair throughput of user-centric cells under different access methods, including GA-MADRL combined with fairness, FDMA method and GA-MADRL without fairness. GA-MADRL combined with fairness is based on Jain-FI and adopts an adaptive bandwidth allocation strategy to provide fair services to users in user-centric cells. In contrast, the FDMA method only allocates bandwidth evenly, while GA-MADRL without fairness does not consider Jain-FI when serving users. Therefore, the access method adopted in this paper can achieve higher user-centric cell fair throughput. Figure 6 is a comparison of the fair throughput of the envisioned network under different clustering parameters, where The value of indicates whether the relative distance between users in the cell centered on the user is taken into account, and The value of indicates whether the relative distances between user-centric cells are taken into account. As the number of mobile users increases, the interference between different user-centric cells also increases, which reduces the fair throughput of the system. Therefore, compared with considering only a single factor, the appropriate and It can effectively reduce the interference between clusters and thus improve the fair throughput of the system.

[0153] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A user-centric multi-UAV fair communication dynamic deployment method, characterized in that: The following steps are involved: (1) Based on the average path loss model of communication between drones and users and the Jain fairness index, the user-centric cell fair throughput is obtained; (2) Modeling the optimization problem of dynamic deployment of multiple UAVs in a user-centric manner while considering fairness; (3) Using genetic algorithm to pre-process the relative distances between users in a cell and the relative distances between cells to clarify the user-centered cell distribution in each time slot; (4) Using the multi-agent reinforcement learning algorithm MADRL, each UAV is assigned a unique flight action based on the distribution of ground user cells so that it can reach the corresponding location within the specified time slot; (5) The ground user cell selects the drone that serves it based on the distance between each drone and the ground user cell, completing a one-to-one matching between drones and ground cells; (6) Based on the MADRL algorithm and user-centric cell fair throughput, bandwidth resource allocation for corresponding drones is achieved in each user cell; (7) Repeating steps (3) to (6) within a specified number of time slots until multiple converged actor online networks are obtained, where each actor online network corresponds to a UAV; (8) Each UAV selects its flight actions and bandwidth allocation actions according to its converged actor online network and the distribution of ground mobile user cells, thereby achieving user-centric dynamic deployment of multi-UAV communications considering fairness.

2. According to claim 1, a user-centric multi-UAV fair communication dynamic deployment method is characterized in that: The implementation process of step (1) is as follows: In the nth time slot, the three-dimensional coordinates of the jth UAV and the two-dimensional coordinates of the i-th user are expressed as and In each time slot, mobile users are divided into K user-centered clusters, where the kth n The center coordinates of a user-centered cluster are The number of users in each user-centric cluster Where I represents the total number of users, and the number of users in the kth user-centered cluster in the nth time slot is Representation: Define the time slot set N = {1,...,n,...,N}, the drone set The user-centered cell set K n ={1,...,k n ,...,K}, the set of all users I = {1,...,i,...,I}, and the set of users in different user-centric clusters in each time slot The channel model in the considered network is based on the average path loss model, where the probability of a LOS path is given by: Among them, α and β are constant values ​​that depend on the environment, is the horizontal distance NLOS probability between the drone and the user, expressed as: P LOS =1-P NLOS ; Therefore, the average path loss model is expressed as follows: Among them, f c and v c denote the carrier frequency and the speed of light respectively; in addition, the distance between the drone and user m is represented by l i,j Given, η LOS and η NLOS The kth ... n User-centric cells The channel gain between the user and the jth UAV It is expressed as: Finally, the kth n The user-centric cluster The throughput per user is: in, Respectively represent the nth time slot and the kth time slot n The jth UAV matched by the user-centric cluster is assigned to the Bandwidth per user; Indicates whether the i-th user is assigned to the k-th user-centered cell in the n-th time slot; represents the transmission power of the jth UAV, σ 2 Represents noise power; M ICI represents the inter-cell interference, j'∈J / J represents the j'th UAV in the nth time slot, excluding the UAV in the kth time slot. n The jth UAV matched to the cell; Therefore, in the nth time slot, the kth n The user-centric cluster The throughput rate per user is defined as: in, Indicates the kth time before time slot n n User-centric cells The total throughput for each user is Indicates the kth time before time slot n n The sum of the throughput of all users in a user-centric cluster; Define the Jain fairness index to measure the kth time slot in the nth time slot n Throughput difference between users in a cell: Therefore, based on Jain-FI, the kth time slot of the nth time slot is n The throughput of a user-centric cell is defined as the fair throughput of the user-centric cell:

3. According to claim 1, a user-centric multi-UAV fair communication dynamic deployment method is characterized in that: The implementation process of step (2) is as follows: For a quadcopter, the thrust of each rotor is the speed of the UAV. and acceleration Functions of , where bold indicates their vectors: in, and They represent the velocity direction vector and velocity magnitude of the UAV, g represents the gravity acceleration vector, and n r and m represent the number of rotors and the mass of the UAV, respectively, ρ and S FP represent the air density and the equivalent flat plate area of ​​the fuselage respectively; the propulsion power is expressed as: Among them, δ, c T ,ρ,c sol 、c fac , d0 represents the drag coefficient of the local blade section, the thrust coefficient based on the disc area, the air density, the rotor stability, the incremental correction factor of the induced power relative to the linear induced speed, the disc area of ​​each rotor, and the drag ratio of each rotor in the fuselage, while τ c represents the aircraft's climbing angle; the remaining energy of the jth UAV in the nth time slot is expressed as: E max is the maximum storage energy of the UAV, Δ n is the time slot length; Therefore, in order to realize the user-centric dynamic deployment scheme of multiple UAVs while considering fairness, the optimization problem P1 is modeled as follows: P1: s.t.C1: C2: C3: C4: C5: C6: C7: Among them, C1 indicates whether the jth UAV serves the kth user-centric cell in the nth time slot, C2 indicates the number of users in each user-centric cell; under the guarantee of C3, the total bandwidth allocated to users by each UAV in each user-centric cell is equal to the total bandwidth B of the UAVBS; C3 is guaranteed by C4 that the energy of the UAV is not exhausted during the mission; the motion of the UAV is decomposed into three directions: x, y, and z, and C5 and C6 reflect the maximum values ​​of acceleration and velocity in the three directions; finally, C7 is to prevent collisions between multiple UAVs; the three-dimensional matrix They represent the acceleration of the UAV in three directions, the bandwidth allocation of the UAV when serving the matched cluster, the matching between the UAV and the cell, and the matching between the user and the cell.

4. According to the user-centric multi-UAV fair communication dynamic deployment method of claim 1, it is characterized in that: The implementation process of step (3) is as follows: Preprocess users to form user-centered cells, and model the preprocessing problem P2 as: P2: s.t.C1: in, represents the sum of the relative distances between users in the cell, and represents the sum of relative distances between cells; For the preprocessing problem P2, in the genetic algorithm, a chromosome is a valid solution to the preprocessing problem P2, so a complete chromosome is shown as follows: The element ch belongs to the set {1,...,i,...,I,...,I+K-1}. When the value of ch is in [1,I], it indicates the number of the ground mobile user. When the value of ch is in [I+1,I+K-1], it indicates the separation point between cells in the chromosome. According to the constraint C2 in P1, every I cells in the chromosome k After the first position, the value of the element ch will be in the range [I+1, I+K-1], and the values ​​of the elements in other positions will be in the range [1, I]; The chromosomes that meet the requirements will form the initial population pop, whose size is represented by Ω; for the population pop, the fitness function value fit(ω) of each chromosome is obtained according to the optimization objective function of P2; based on the distribution of the fitness function values ​​fit(ω) of all chromosomes in the population, the random traversal sampling (SUS) method is introduced to select chromosomes with higher fitness function values, and their number is Ω1, where Ω1<Ω; next, for the selected chromosomes, the partial matching crossover strategy completes the crossover operation of these chromosomes; if the random number is less than the mutation probability, then the elements of any two points on the chromosome are randomly exchanged; finally, the chromosomes that have undergone the above operation are combined with the Ω-Ω1 chromosomes with higher fitness function values ​​in the initial population to form a new generation of population; repeat the above steps until convergence to the optimal solution, and finally obtain the chromosome with the highest fitness function value, that is, the user clustering strategy under the current time slot.

5. According to claim 1, a user-centric multi-UAV fair communication dynamic deployment method is characterized in that: The implementation process of step (4) is as follows: The preprocessing clarifies the user-centered cell distribution in each time slot, and then optimizes the positions of multiple drones and the allocation of network resources, thereby transforming P1 into P3 problem: P3: s.t.C1: C2: C3: C4: C5: C6: For problem P3, the MADRL algorithm is used to solve the challenge of user-centric dynamic deployment of multiple UAVs, while considering fairness. The multi-agent deep deterministic strategy MADDPG is adopted in MADRL; MADDPG involves four neural networks, including actor online network, critic online network, actor target network and critic target network, and updating these networks requires knowing the status, actions and rewards of multiple UAVs; the observation space of each agent consists of the position of all UAVs, the speed in three directions, the position of ground users and the cell clusters of ground users, which can be expressed as: Therefore, the state space of all agents is: The action of each drone consists of two parts: the acceleration of the drone in three directions and the bandwidth allocation of the drone. This work is expressed as: The action set of all drones is: A={A1,...,A j }; Since each drone moves with uniform acceleration in each direction, after the actor online network outputs the action corresponding to the acceleration of the drone in three directions, the coordinates and speed updates of each drone in three directions are obtained by the following formula: Based on the above coordinate and speed update formula, the position and speed of the drone in the specified time slot are obtained; for this type of action, the reward function involved is first to prevent the drone speed or position from exceeding the limit, which is expressed as: In order to ensure that all drones do not collide, the judgment criterion is that any two drones at the same time, as long as the coordinates of the three directions of the drones are not equal at the same time, the constraint C6 in P3 is satisfied. The reward function is expressed as: Whether the UAV's flight can meet its energy constraints is reflected by the following reward function: j_3 =E j (n); In the above reward function, Used to adjust the size of the reward value.

6. The user-centric multi-UAV fair communication dynamic deployment method according to claim 1 is characterized in that: The implementation process of step (5) is as follows: According to the coordinates of the center of the user cell and the coordinates of each drone, the distance between different user cells and the drone is calculated. The calculation formula is described as follows: According to the distance formula between cells and drones, starting from the first user cell, each cell traverses the idle drones one by one until the drone closest to its center is found to complete the matching; Design a reward function to measure the matching effect: in, Used to adjust the size of the reward value. Used to limit the distance between each UAV and its matching cell.

7. The user-centric multi-UAV fair communication dynamic deployment method according to claim 1 is characterized in that: The implementation process of step (6) is as follows: First, the users in the user cell matched with each drone are sorted from low to high according to the sum of their throughput before time slot n, and the bandwidth allocation values ​​output by their actor online network are sorted from high to low; Next, the sorted cell users and the sorted bandwidth allocation values ​​are also matched one by one to achieve bandwidth resource allocation for each drone. Finally, in order to improve the user-centric cell fair throughput, a reward function is designed to measure the impact of bandwidth resource allocation on the user-centric cell fair throughput. The reward function is expressed as: j_3 =CFT.

8. The user-centric multi-UAV fair communication dynamic deployment method according to claim 1 is characterized in that: The implementation process of step (7) is as follows: First, based on all reward functions, the reward for each drone is expressed as: RE j =λ j_1 +λ j_2 +λ j_3 +λ j_4 +λ j_5 All drone rewards are expressed as: RE={RE1,RE2,RE3,RE4,RE5} Next, {S, A, RE, D, S'} will be stored as experience in the replay buffer, where D = {D1, ..., D j } indicates whether the training of each UAV is completed in each time slot, and S' represents the state set of all UAVs in the next time slot; The parameters θ of the critic online network Q , the parameters θ of the actor online network μ Update using a gradient descent based method: Among them, the update target is ζ φ =RE φ +γ(Q'(S' φ ,A'θ μ' ),θ Q' ), while Φ and γ represent the experience size and discount factor sampled from the experience pool respectively; S φ , S' φ , A φ and RE φ Represents the data sampled from the experience pool; A' represents the actor target network in the next state S' φ The output action is recorded as θ μ' ; A indicates that the actor online network is in the current state S φ The action output under the condition of ; Q(·) is the value function of the critic online network output, Q'(·) is the value function of the critic target network output, and its parameter is denoted by θ Q '; For the parameter update of the target network, a soft update method is adopted, which is specifically given by the following formula: i Q' =tθ Q +(1-τ)θ Q' i μ' =tθ μ +(1-τ)θ μ' Among them, τ represents the update frequency.

9. The user-centric multi-UAV fair communication dynamic deployment method according to claim 1 is characterized in that: The implementation process of step (8) is as follows: After training, each UAV will have four converged networks; in the implementation stage, the preprocessed user-centric cell distribution is input into the actor online network of each UAV, which then outputs the operations to be performed by the UAV, ultimately achieving user-centric dynamic deployment of multi-UAV communications that takes fairness into account.

Images