Channel allocation method for ground communication assisted by multiple unmanned aerial vehicle base stations

Abstract

The invention discloses a channel allocation method for multi-unmanned aerial vehicle base station assisted ground communication, which utilizes a Markov game framework to carry out modeling on trajectory optimization and channel allocation decision of an unmanned aerial vehicle assisted communication system, and proposes a multi-intelligent reinforcement learning model based on event-driven Transform. An event-driven monitoring strategy optimization mechanism aims to learn an optimal event-driven listening strategy based on a current channel state, unnecessary information listening and processing of a plurality of unmanned aerial vehicle base stations are effectively reduced, and a Transform network extracts useful information and modes from a large amount of observation information. The evaluation network of the unmanned aerial vehicle base station can more accurately evaluate the state information, so that the dynamic and intelligent channel allocation of the plurality of unmanned aerial vehicle base stations is realized. The method provided by the invention shows good strategy convergence performance.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a channel allocation method for multi-UAV base station assisted ground communication. Background Art

[0002] In drone-assisted communication, reasonable channel allocation is a key factor in improving transmission rate and realizing drone-assisted efficient communication. Selecting a channel with high availability for information transmission is a method to ensure smooth communication and improve spectrum utilization. When multiple drone base stations are used to provide communication coverage in an area, since the position of the drone is constantly changing and the electromagnetic environment of each drone is different, how to select a suitable communication channel in an interference environment becomes a key issue for achieving high-quality drone-assisted communication. In fact, the joint decision of channel allocation of multiple drone base stations constitutes a Markov game, and the solution of the Nash equilibrium of the auxiliary communication game can ensure that each multi-drone base station in the game achieves the optimal decision. However, the existing Nash equilibrium solution method is mostly to use multi-constraint polynomials to solve the analytical solution or use convex optimization theory to solve the equilibrium of the game when prior experience is known. The present invention is aimed at the channel allocation decision problem of multi-drone assisted communication, and adopts a model-free reinforcement learning method to enable multiple drone base stations to optimize the movement trajectory and channel allocation strategy through interactive learning with the environment, which provides a new idea for the development of future wireless communication technology. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes a channel allocation method for multi-UAV base station assisted ground communication, which more effectively solves the problem of channel allocation for multiple UAV assisted communications.

[0004] In order to achieve the above object, the technical solution of the present invention is as follows:

[0005] A channel allocation method for multi-UAV base station assisted ground communication comprises the following steps:

[0006] Obtain information of M UAV base stations and N ground users in the UAV-assisted communication system, and initialize the joint strategy of several UAV base stations;

[0007] Construct a multi-intelligence reinforcement learning model based on event-driven Transformer, which includes states, drone base stations, actions and rewards. The actions include movement vectors, vectors for selecting whether to pair with each ground user, channel selection vectors, and event-driven monitoring strategy outputs. The state of the drone base station in each time slot includes the location of the ground user paired with the current drone base station and the pairing and channel selection information of other drone base stations.

[0008] Input the state observation data of the drone base station in the previous time slot into the multi-intelligence reinforcement learning model and calculate the state value function V(s,a);

[0009] Calculate and compare event-driven conditions. If the conditions are met, trigger the event-driven monitoring strategy to monitor the broadcasts of other drone base stations. Use the location of the ground user paired with the current drone base station and the pairing and channel selection information of other drone base stations obtained through monitoring as the input of the Transformer model, output V(q,κ,μ), and calculate the advantage function based on V(q,κ,μ) If the condition is not met, the advantage function is calculated based on the position of the ground user paired with the current drone base station

[0010] Based on the advantage function, the action data corresponding to the state observation data of the previous time slot is determined. The UAV base station interacts with the simulation environment according to the action data, calculates the output of the Critic network, that is, the Q value, and feeds the Q value back to the Actor network of the UAV base station to assist the UAV base station in updating its strategy.

[0011] Preferably, the joint strategy for initializing several drone base stations includes setting that each ground user in the drone-assisted communication system can only be paired with one drone base station, and each drone base station will serve at least one ground user; in each time slot, multiple drone base stations independently decide to pair with ground users in turn, and select a channel from the shared channel, and broadcast the pairing information and channel selection to other drone base stations; several drone base stations will design trajectories in real time according to the location information of the paired ground users, and decide whether to listen to the broadcasts of other drone base stations based on an event-driven strategy.

[0012] Preferably, an over-listening penalty operator ε'(t) is added to the reward mechanism of the multi-intelligence reinforcement learning model, and the formula is as follows:

[0013] ε'(t)=c1*Onehot event-driven (t) (13)

[0014] Among them, Onehot event-driven is a one-hot vector indicating whether the monitoring event occurs, and c1 is the penalty coefficient.

[0015] Preferably, the optimization objectives of the M drone base stations are defined as:

[0016]

[0017]

[0018] p m,n (t) = Ptotal (19)

[0019]

[0020] b m,n (t)≥b min ,p m,n (t)≥p min ,n∈N (21)

[0021]

[0022] In the formula, p m,n (t) and b m,n (t) are the transmission power and bandwidth resources allocated to ground user n by UAV base station m; b min is the minimum divisible bandwidth; p min is the minimum divisible power.

[0023] Preferably, the strategy is updated with the following formula:

[0024]

[0025] In the formula, C is the penalty coefficient, γ is the discount factor, is the approximate objective function, is the KL divergence between the current policy and the candidate policy.

[0026] Based on the above technical scheme, the beneficial effects of the present invention are as follows: the present invention aims at the channel allocation strategy optimization problem of multiple UAV base stations in the UAV-assisted communication scenario, and uses the Markov game framework to model the trajectory optimization and channel allocation decision of the multi-UAV assisted communication system; the present invention proposes a multi-agent reinforcement learning strategy optimization method based on event-driven Transformer. The event-driven monitoring strategy optimization mechanism aims to learn the optimal event-driven listening strategy based on the current channel state, effectively reducing unnecessary information listening and processing of multiple UAV base stations. The Transformer network extracts useful information and patterns from a large amount of state information, so that the evaluation network of the UAV base station can more accurately evaluate the state information, thereby realizing dynamic and intelligent channel allocation of multiple UAV base stations. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a schematic diagram of a drone-assisted communication system in one embodiment;

[0028] Figure 2 The invention is a timing decision process of allocating channels of several UAV base stations in a channel allocation method for multi-UAV base station assisted ground communication in an embodiment;

[0029] Figure 3 is a block diagram of a multi-intelligence reinforcement learning model based on an event-driven Transformer in an embodiment;

[0030] Figure 4 is the trajectory design and channel selection result of three UAV base stations in one embodiment;

[0031] Figure 5 Schematic diagram of average reward curves of three drone base stations in an embodiment, wherein (a) is a schematic diagram of average reward curves of drone base station 1 using HAPPO, HATRPO, TransformerMATR and EDGT-MARL respectively; (b) is a schematic diagram of average reward curves of drone base station 2 using HAPPO, HATRPO, TransformerMATR and EDGT-MARL respectively; (c) is a schematic diagram of average reward curves of drone base station 3 using HAPPO, HATRPO, TransformerMATR and EDGT-MARL respectively;

[0032] Figure 6 1 is a box plot of the average rewards of three drone base stations in an embodiment, wherein (a) is a box plot of the average rewards of drone base station 1 using HAPPO, HATRPO, TransformerMATR and EDGT-MARL respectively; (b) is a box plot of the average rewards of drone base station 2 using HAPPO, HATRPO, Transformer MATR and EDGT-MARL respectively; (c) is a box plot of the average rewards of drone base station 3 using HAPPO, HATRPO, TransformerMATR and EDGT-MARL respectively. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0034] like Figures 1 to 3 As shown, this embodiment provides a channel allocation method for multi-UAV base station assisted ground communication, which is applied to a UAV assisted communication system. The schematic diagram of the UAV assisted communication system is shown in FIG. Figure 1 As shown. First, the channel allocation problem of multiple UAV-assisted communications is modeled. In this system, M UAV base stations provide wireless communication services for N ground users, and multiple UAV base stations share L channels. The set of UAV base stations is defined as M = {1, 2, ..., M}, the set of ground users is defined as N = {1, 2, ..., N}, and the set of shared channels is defined as L = {1, 2, ..., L}.

[0035] In a dynamic assisted communication environment of multiple UAV base stations, each UAV base station makes sequential decisions by interacting with the unknown environment. This joint decision-making process is usually called a random game, or Markov game.

[0036] Therefore, the auxiliary communication of multiple UAV base stations is modeled as a Markov game and interacts with the environment according to the following process: at time t, based on the current state s t ∈S, drone base station i adopts strategy π i . Joint action of multiple drone base stationsπ=(π 1 ,...,π n ) corresponds to the joint strategy Then, the drone gets the reward r t = r(s t ,a t ), and according to the probability P(s t+1 |s t ,a t )Transfer to the next state s t+1 .

[0037] In this UAV-assisted communication system, all ground users move randomly on the two-dimensional ground, and each UAV base station flies in three-dimensional space and uses frequency division multiplexing technology to provide wireless services to ground users.

[0038] Step 1: Initialize the joint strategy of multiple drone base stations Specifically: In each time slot, multiple drone base stations first independently decide to pair with ground users in turn, and select a channel from L shared channels, and then broadcast the pairing information and channel selection to other drone base stations. After that, they will design trajectories in real time according to the location information of the paired ground users, and decide whether to listen to the broadcasts of other drone base stations based on event-driven strategies.

[0039] In order to improve the communication quality of the system and avoid wasting resources, we assume that each ground user can only be paired with one UAV base station, and each UAV base station will serve at least one ground user.

[0040] In the assisted communication model, the coordinates of the UAV and the ground user are expressed as and in Among them, x m ,x n ∈(-100,100) and z m ∈(0,100).

[0041] First, the receiving power and channel loss of ground users are modeled as shown in formulas (1)-(3):

[0042] The air-to-ground communication model based on statistical probability is adopted. Furthermore, to simplify the calculation, it is assumed that there is no line of sight between the UAV base station and the ground user, and the line-of-sight link (LoS) is uniformly adopted. Therefore, at time t, the total path loss from the UAV base station m to the ground user n in the LoS link can be described as:

[0043]

[0044] in, is the instantaneous path loss from UAV base station m to ground user n in the LoS link, is the additional loss of free space transmission.

[0045]

[0046] In the formula, f c is the carrier frequency, is the straight-line distance from the drone base station m to the ground user n, and c is the speed of light. m,n (t) is the power allocated to ground user n by UAV base station m at time t.

[0047] Therefore, the received signal power available at ground user n is:

[0048]

[0049] Then, the motion model of the drone base station is constructed, as shown in formulas (4)-(5):

[0050] Considering the flight safety of the drone base station, its minimum flight altitude is set to 5 meters.

[0051] In the Cartesian coordinate system, the velocity of the drone base station m is expressed as:

[0052]

[0053] Then the position update equation of the drone base station m is:

[0054]

[0055] Then, the pairing and channel selection model of the drone base station is described as shown in formulas (6)-(7):

[0056] Ground users move randomly at a speed lower than that of the drone base station. At each time t, the ground user will be paired by a drone base station and provide communication services. In addition, each drone base station must serve at least one ground user at time t.

[0057] Using binary variables σ m,n(t)∈{0,1} represents the connection relationship between the drone base station m and the ground user n. When the ground user n is paired with the drone base station m, σ m,n Set to 1 if yes, otherwise set to 0.

[0058]

[0059] Defining binary variables To control the channel selection of the drone base station. When the drone base station m selects channel l, is 1 if the value is true, otherwise it is 0.

[0060]

[0061] In the UAV-assisted communication system, the action of the UAV base station a m ∈A m Including the movement vector v m , the vector vector1m that selects whether to pair with each ground user, the channel selection vector onehot2m, and the event-driven monitoring strategy output event-driven policy , that is, a m =v m ,vector1m,onehot2m,event-driven policy The dimension of vector1m is the number of ground users N, and the dimension of onehot2m is the number of shared channels L. Event-driven strategy output event-driven policy Is a continuous value used as an event-driven gating threshold.

[0062] For the drone base station m, the total power allocated is P total , according to the total number of paired ground users, the total bandwidth B total Evenly distribute, formula (8) and formula (9) give the mathematical description of power and bandwidth resource allocation:

[0063] P m,n (t) = P total (8)

[0064]

[0065] where s m (t) represents the number of ground users paired with the UAV base station m in time slot t.

[0066] Finally, a description of the communication rate and co-channel interference is given, as shown in equations (10) and (11).

[0067] Define n0 as the power spectral density, B m,n(t) is the bandwidth resource from UAV base station m to ground user n. Based on Shannon channel capacity theory, the communication rate of the link from UAV base station m to ground user n is:

[0068]

[0069] In the formula, is the co-channel interference generated by all other UAV base stations except UAV base station m on channel l to ground user n in time slot t, which is described as follows:

[0070]

[0071] In the formula, h i,n (t) is the interference path loss from UAV base station i to ground user n on the LoS link, which is calculated in the same way as the path loss between UAV base station m and ground user n; P i is the interference power of UAV base station i.

[0072] In order to maximize the throughput of the multi-UAV assisted communication system while reducing co-channel interference, each UAV base station first needs to optimize the pairing strategy and channel strategy, and then optimize the flight trajectory and event-driven listening strategy. max As the time period of an episode, t is used as the time interval for determination:

[0073] t∈T={1,2,...,T max} (12)

[0074] Figure 2 The time series decision process of the multi-UAV assisted communication network is given. At each time slot t, each UAV base station first needs to decide which ground users to pair with and allocate channels to the paired ground users. Specifically, the UAV base stations will choose to pair with ground users in turn according to their priority. For example, in an environment with three UAV base stations, UAV base station 1 has the highest priority and is the first to pair with a user; UAV base station 3 has the lowest priority and is the last to pair with a user.

[0075] In this paper, in order to reduce interference and improve communication quality, each ground user will be paired with only one drone base station, and each drone base station will serve at least one ground user to avoid wasting resources. Ground users that have been paired with previous drone base stations will no longer be paired with other drone base stations, thus avoiding repeated pairing. At the same time, each drone base station will select an unoccupied or less occupied channel based on the current channel conditions to avoid co-channel interference. The drone base station will then broadcast pairing and channel selection information to all other drone base stations. Finally, each drone base station will adjust its trajectory in real time based on the current location of the paired ground user.

[0076] Step 2: Calculate the state value function V(s,a): For each UAV base station, the state information in each time slot includes the location of the paired ground user and the pairing and channel selection information of other UAV base stations.

[0077] In order to avoid too many monitoring events, an excessive monitoring penalty operator ε'(t) is added to the reward mechanism to reduce the communication overhead of the drone base station.

[0078] ε'(t)=c1*Onehot event-driven (t) (13)

[0079] Among them, Onehot event-driven Is a one-hot vector indicating whether the listening event occurs. event-driven If Onehot is 1, a monitoring event occurs, and the drone base station will monitor the broadcast information of all other drone base stations to assist its own decision-making; event-driven If it is 0, no monitoring event will occur, and the drone base station will make decisions based only on its own local state information. c1 is the penalty coefficient.

[0080] The optimization objectives of several drone base stations are defined as:

[0081]

[0082] p m,n (t) = P total (19)

[0083]

[0084] b m,n (t)≥b min ,p m,n (t)≥p min ,n∈N (21)

[0085]

[0086] In the formula, p m,n (t) and b m,n (t) are the transmission power and bandwidth resources allocated to ground user n by UAV base station m; b min is the minimum divisible bandwidth; p min is the minimum divisible power.

[0087] After modeling and optimizing the problem description, this paper proposes a new event-driven transformer multi-agent reinforcement learning framework (Event-Driven Transformer MARL, EDT-MARL) to solve the trajectory design and channel allocation problems in multi-UAV assisted communication. The event-driven monitoring strategy learns the optimal monitoring time based on the current channel state, so that the UAV base station can adaptively adjust the threshold of the monitoring trigger to decide whether to monitor global information. This reduces unnecessary information monitoring of the UAV base station and improves efficiency.

[0088] In addition, in order to avoid excessive event-driven monitoring, a new reward mechanism is designed to incorporate the excessive monitoring penalty term into the reward function of the drone base station in a weighted manner to reduce its communication overhead. The drone base station uses the proposed method to optimize its own trajectory design and channel allocation strategy in the communication network to maximize the cumulative reward. In addition, under the guarantee of the monotonic strategy improvement of the proposed method, the drone base station gradually achieves joint strategy convergence.

[0089] Most communication scenarios using multi-agent reinforcement learning rely on wireless communication to share observations. As the number of agents and data dimensionality increase, communication performance will be affected by packet loss and communication latency. In addition, the bandwidth of communication networks in the real world is limited. Therefore, minimizing the communication overhead between agents is crucial. To help achieve this, we have conducted a lot of research on related work, among which event-driven communication is one of the most effective methods. A new event-driven listening strategy optimization method is proposed to reduce unnecessary information listening by drone base stations.

[0090] Step 3: Calculate and compare event-driven trigger conditions Onehot trig Specifically, the action network of each drone base station will output an adaptive event-driven policy value event-driven policy , as the threshold for gating, such as Figure 3 Then, the event-driven strategy value is compared with the output value of the pre-evaluation network. For comparison, if event-driven policy Greater than If the value is high, the broadcast of other drone base stations will be listened to, otherwise.

[0091] Step 4: If the event-driven monitoring strategy is triggered, encode the monitored state information o(s,a), calculate the transformer value function V(q,κ,μ), and then calculate the advantage function

[0092] The specific calculation is as follows:

[0093]

[0094] Where q is the query; κ and μ form a key-value pair; d κ is the column of the q and κ matrices, i.e. the vector dimension.

[0095]

[0096] Step 5: Otherwise, the event-driven monitoring strategy is not triggered, which enables the UAV base station to learn the event-driven monitoring strategy from historical interaction data, thereby improving the communication efficiency of the multi-UAV assisted communication system.

[0097] Then, a lightweight Transformer network is introduced to extract useful information and patterns from a large amount of observation data, so that the evaluation network of the drone base station can more accurately evaluate the state information to improve its own decision-making. Compared with the classic Transformer network architecture, the lightweight Transformer network used in the present invention removes the residual and normalization layers, thereby reducing the computational complexity. Figure 3 As shown in Figure 1, the input data of the lightweight Transformer network is the status information of the drone base station.

[0098] The Transformer was originally designed for machine translation tasks and is an encoder-decoder structure where the encoder maps an input sequence of tokens to a latent representation and then the decoder generates the desired output sequence in an autoregressive manner, where at each step, the Transformer takes as input all previously generated tokens. The attention mechanism with residual connections is one of the most important components of the Transformer, a technique used to build deep neural networks. Residual connections help alleviate the vanishing and exploding gradient problem in deep network training while simplifying the network training process.

[0099] In the present invention, when the event-driven listening condition is activated, the drone base station needs to collect broadcast information from others to help itself make decisions. However, when the number of drone base stations is large, the dimension of information that needs to be processed is also exponential. Therefore, the present invention uses a lightweight Transformer model and embeds it into the Critic network of the drone base station, with the aim of extracting the most valuable information from the massive interactive information, while assisting the decision output of the drone base station Actor network. The lightweight Transformer model used in the present invention specifically includes the input data

[0100] Figure 3 The framework of event-driven Transformer multi-agent reinforcement learning is given. Figure 3 As shown in the figure, based on the centralized training distributed execution (CTDE) structure, an event-driven monitoring optimization strategy is introduced as one of the output strategies of the Actor network. In this way, each drone base station can listen to the broadcast information of other drone base stations to learn the optimal event-driven monitoring strategy.

[0101] Each drone base station will pre-process the status information through the Transformer network before inputting it into the evaluation network.

[0102] Step 6: Calculate the output of the Critic network, i.e. the Q value: The output Q value is fed back to the Actor network of the drone base station to assist in updating the strategy of the drone base station.

[0103] Step 7: Update strategy Where C is the penalty coefficient, γ is the discount factor, is the approximate objective function, It is the KL divergence between the current strategy and the candidate strategy, which is used to measure the gap between the two strategies.

[0104] experiment

[0105] Specific experimental results: Figure 4 The moving trajectories and channel selection results of three UAV base stations are given. Ground user 1 (3,2|3,1) means that ground user 1 is paired with UAV base station 3 through channel 2 at the beginning, and paired with UAV base station 3 through channel 1 at the end, and so on. Figure 5 The average reward curves of three drone base stations are given; Figure 6 The average reward box plots of the three drone base stations are given; Table 1 gives the final rewards of the three drone base stations. It can be seen that compared with the benchmark algorithms such as Heterogeneous-Agent Proximal Policy Optimization (HAPPO), Heterogeneous-Agent Trust Region Policy Optimization (HATRPO), and Transformer Multi-agent Trust Region (Transformer MATR), the method proposed in the present invention shows better policy convergence performance.

[0106] Table 1 Final rewards of three drone base stations

[0107]

[0108] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0109] The above is only a preferred implementation of a channel allocation method for multi-UAV base station assisted ground communication disclosed in the present invention, and is not intended to limit the protection scope of the embodiments of this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of this specification shall be included in the protection scope of the embodiments of this specification.

Claims

1. A channel allocation method for multi-UAV base station assisted ground communication, characterized in that: The steps include: Obtain information of M UAV base stations and N ground users in the UAV-assisted communication system, and initialize the joint strategy of several UAV base stations; Construct a multi-intelligence reinforcement learning model based on event-driven Transformer, which includes states, drone base stations, actions, and rewards. The actions include movement vectors, vectors for selecting whether to pair with each ground user, channel selection vectors, and event-driven monitoring strategy outputs. The state of the UAV base station in each time slot includes the location of the ground user currently paired with the UAV base station and the pairing and channel selection information of other UAV base stations; Input the state observation data of the drone base station in the previous time slot into the multi-intelligence reinforcement learning model and calculate the state value function V(s,a); Calculate and compare event-driven conditions. If the conditions are met, trigger the event-driven monitoring strategy to monitor the broadcasts of other drone base stations. Use the location of the ground user paired with the current drone base station and the pairing and channel selection information of other drone base stations obtained through monitoring as the input of the Transformer model, output V(q,κ,μ), and calculate the advantage function Aπ based on V(q,κ,μ) k (V(q,κ,μ)); If the condition is not met, the advantage function A is calculated based on the position of the ground user paired with the current drone base station πk (V); Based on the advantage function, the action data corresponding to the state observation data of the previous time slot is determined. The UAV base station interacts with the simulation environment according to the action data, calculates the output of the Critic network, that is, the Q value, and feeds the Q value back to the Actor network of the UAV base station to assist the UAV base station in updating its strategy.

2. According to claim 1, a channel allocation method for multi-UAV base station assisted ground communication is characterized in that: The joint strategy of initializing several drone base stations includes assuming that each ground user in the drone-assisted communication system can only be paired with one drone base station, and each drone base station will serve at least one ground user; in each time slot, multiple drone base stations independently decide to pair with ground users in turn, and select a channel from the shared channel, and broadcast the pairing information and channel selection to other drone base stations; Several drone base stations will design trajectories in real time based on the location information of paired ground users, and decide whether to listen to the broadcasts of other drone base stations based on event-driven strategies.

3. The channel allocation method for multi-UAV base station assisted ground communication according to claim 1 is characterized in that: The over-listening penalty operator ε,(t) is added to the reward mechanism of the multi-intelligence reinforcement learning model, and the formula is as follows: ε,(t)=c1*Onehot event-driven (t) (13) Among them, Onehot event-driven is a one-hot vector indicating whether the monitoring event occurs, and c1 is the penalty coefficient.

4. The channel allocation method for multi-UAV base station assisted ground communication according to claim 1 is characterized in that: The optimization objectives of the M drone base stations are defined as: p m,n (t)=P total (19) b m,n (t)≥b min ,p m,n (t)≥p min ,n∈N (21) In the formula, p m,n (t) and b m,n (t) are the transmission power and bandwidth resources allocated to ground user n by UAV base station m; b min is the minimum divisible bandwidth; p min is the minimum divisible power.

5. The channel allocation method for multi-UAV base station assisted ground communication according to claim 1 is characterized in that: The strategy is updated with the following formula: In the formula, C is the penalty coefficient, γ is the discount factor, is the approximate objective function, is the KL divergence between the current policy and the candidate policy.

Images