Air-to-ground hybrid communication method considering space, time domain and frequency domain parameter optimization

Abstract

The invention discloses an air-to-ground hybrid communication method considering space, time domain and frequency domain parameter optimization. The method comprises the following steps: establishing a UAV track and channel model; a communication link rule is established based on a hybrid communication architecture: each time slot is divided into an uplink and downlink transmission stage based on TDD, the duration is divided according to an uplink and downlink ratio, a system total frequency bandwidth is divided into sub-frequency bandwidths of each UE according to a sub-bandwidth division ratio based on FDMA, and a communication decision is determined according to an uplink and downlink data transmission rate. The uplink and downlink data transmission rate is obtained based on an uplink and downlink ratio, a plurality of sub-bandwidth division ratios and a channel model; based on a communication decision, taking a UAV track, an uplink and downlink ratio and a division ratio of a plurality of sub-bandwidths as optimization parameters, taking the total number of UE covered by communication as an optimization target, and adding constraint conditions to establish a problem model; and establishing a reinforcement learning model and introducing a strategy optimization algorithm to optimize optimization parameters so as to realize an optimization target. The optimal performance of the communication system can be obtained.

Description

Technical Field

[0001] The present invention belongs to the field of communication, and in particular relates to an air-to-ground hybrid communication method considering optimization of space, time domain and frequency domain parameters. Background Art

[0002] In recent years, the rapid development of drones and communication technology has provided a solid technical foundation for the low-altitude economy. In emergency rescue, the rapid and accurate transmission of information is the key to the entire communication. Damage to the communication infrastructure caused by disasters such as earthquakes and rainstorms can lead to communication paralysis in the entire disaster area. In this case, drones equipped with airborne communication base station systems can be quickly deployed to establish temporary communication networks, restore the communication capabilities of the disaster area, and ensure smooth rescue and command and dispatch.

[0003] Liu et al. in the paper "Liu Y, Liu K, Han J, Zhu L, Xiao Z, Xia X, Resource Allocation and 3D Placement for UAV-Enabled Energy-Efficient IoT Communications, IEEE Internet of Things Journal, 2020, PP(99):1-1" studied the optimal aerial positioning and resource allocation of drone base stations in the Internet of Things (IoT) network, with the goal of minimizing the total transmission power of IoT devices. They used the K-Means algorithm for clustering, the improved HD4M algorithm for subchannel allocation, and finally used the alternating iterative method to jointly optimize the IoT device transmission power and the drone height.

[0004] Hattab et al. also optimized network resources in the paper “Hattab G, Cabric D, Energy-Efficient Massive IoT Shared Spectrum Access over UAV-enabled Cellular Networks, IEEE Transactions on Communications, 2020, PP(99):1-1”. They proposed a time division duplex transmission protocol to provide shared spectrum access and improve the average resource allocation between IoT devices and users (UEs) through a random geometry protocol, but the UE will be subject to additional interference from IoT devices. Therefore, they optimized the nominal transmission power of IoT devices to maximize their energy efficiency while limiting interference to UEs.

[0005] In the IoT network, Park et al. proposed a power splitting (PS) strategy in the document "Park G, Lee K, Optimization of the Trajectory, Transmit Power, and Power Splitting Ratio for Maximizing the Available Energy of a UAV Aided SWIPT System, Sensors, 2022, 22(23): 9081" to jointly optimize the trajectory, transmission power and power ratio of the UAV to maximize the average spectrum efficiency and ensure the minimum average available energy for ground user equipment.

[0006] The above methods optimize the drone space, transmission power, time domain and frequency domain to varying degrees and combinations, but ignore the simultaneous optimization of time domain and frequency domain parameters, which leaves room for system performance optimization.

[0007] However, at present, due to the limitations of UAV (Unmanned Aerial Vehicle) trajectories, time domain resources and frequency domain resources, the number of ground nodes for effective communication cannot achieve full coverage. Summary of the invention

[0008] In order to solve the above problems existing in the prior art, the present invention provides an air-to-ground hybrid communication method, device and electronic device considering the optimization of space, time domain and frequency domain parameters. The technical problem to be solved by the present invention is achieved by the following technical solutions:

[0009] In a first aspect, an embodiment of the present invention provides an air-to-ground hybrid communication method considering space, time domain and frequency domain parameter optimization, the method comprising:

[0010] For scenarios with multiple UAVs and user nodes UE, a UAV trajectory model is established, and a channel model between UAV and UE is established based on line-of-sight links and non-line-of-sight links;

[0011] Based on the designed hybrid communication architecture, a communication link rule is established, including: each time slot is divided into uplink and downlink transmission phases based on TDD technology, and the transmission duration is divided according to the uplink and downlink ratio; based on FDMA technology, the total frequency bandwidth of the system in each time slot is divided into sub-frequency bandwidths of each UE according to multiple sub-bandwidth division ratios matching the number of UEs; the communication decision between the UE and the UAV in each time slot is determined according to the uplink and downlink data transmission rates, and the uplink and downlink data transmission rates are calculated based on the uplink and downlink ratios, the multiple sub-bandwidth division ratios and the channel model;

[0012] Based on the communication decision, the UAV trajectory, uplink and downlink ratio, and multiple sub-bandwidth division ratios are used as optimization parameters, the total number of UEs covered by the communication is used as the optimization target, and multiple constraints are added to establish a problem model;

[0013] A reinforcement learning model is established, and a preset strategy optimization algorithm is introduced to optimize the optimization parameters of the problem model to achieve the optimization goal.

[0014] In one embodiment of the present invention, the UAV trajectory model is expressed as:

[0015]

[0016] In the scenario, there are J UAVs deployed above the ground, represented by There are I UEs deployed on the ground, represented by The flight cycle of the UAV is divided into T equal time periods, expressed as Each time period is a time slot, which is represented by duration τ, and τ is small enough; the jth UAV is represented by UAV j , whose coordinates in time slot t are expressed as and Respectively represent UAV j The horizontal and vertical coordinates of the time slot t, h j Represents UAV j Flight altitude; Represents UAV j The initial coordinates of Represents UAV j Flight distance in time slot t; Represents UAV j The flight angle at time slot t has a value range of [0,2π];

[0017] In the UAV trajectory model, the UAV flight coordinate constraint is expressed as:

[0018]

[0019] Among them, l max Indicates the maximum flight distance of the drone in a time slot.

[0020] In one embodiment of the present invention, establishing a channel model between the UAV and the UE based on the line-of-sight link and the non-line-of-sight link includes:

[0021] Determine the UAV in time slot t j and UE i The line-of-sight connection probability and the non-line-of-sight connection probability are expressed as:

[0022]

[0023] Among them, LoS and NLoS represent line-of-sight and non-line-of-sight respectively; UE i represents the i-th UE; represents the UAV in time slot t j and UE i The probability of line-of-sight connection between represents the UAV in time slot t j and UE i The probability of non-line-of-sight connection between α and β; α and β are environmental constants; represents the UAV in time slot t j and UE i The elevation angle between

[0024] Determine the UAV in time slot t j and UE i The average LoS path loss and the average NLoS path loss between are expressed as:

[0025]

[0026] in, and They represent the UAVs in time slot t. j and UE i The average LoS path loss and the average NLoS path loss between them; f represents the UAV j and UE i The channel transmission frequency, in Hz; represents the UAV in time slot t j and UE i The distance between them; c represents the speed of light; η LoS and η NLoS They represent the additional loss on the LoS path and the additional loss on the NLoS path respectively;

[0027] According to the UAV in time slot t j and UE i The probability of line-of-sight connection and non-line-of-sight connection between UAVs in time slot t j and UE i The average LoS path loss and the average NLoS path loss between them are used to determine the UAV j and UE i The average path loss between is expressed as:

[0028]

[0029] in, is the UAV in time slot t jand UE i The average path loss between is used to characterize the channel status.

[0030] In one embodiment of the present invention, in the communication link rule,

[0031] UAV in time slot t j and UE i The calculation formula for the uplink data transmission rate is:

[0032]

[0033] UAV in time slot t j and UE i The calculation formula for the downlink data transmission rate is:

[0034]

[0035] in, is the UAV in time slot t j and UE i Uplink data transmission rate; is the UAV in time slot t j and UE i Downlink data transmission rate; k t is the uplink and downlink ratio in time slot t, k t τ represents the transmission duration of the uplink transmission phase in time slot t, (1-k t )τ represents the transmission duration of the downlink transmission phase in time slot t; w represents the size of the total frequency bandwidth of the system, and the sub-frequency bandwidth of I UE divided is expressed as a i is the sub-bandwidth allocation ratio of the i-th UE; P ij Indicates UE i To UAV j The transmission power; represents the UAV in time slot t j and UE i The average path loss between ji Represents UAV j and UE i The transmission power; represents the UAV in time slot t j and UE i The average path loss between σ0 represents the power spectral density of the additive white Gaussian noise at the receiving end.

[0036] In one embodiment of the present invention, in the communication link rule, the communication decision is expressed as:

[0037]

[0038] in, Indicates UE in time slot t i communication decisions, Indicates UE i The maximum uplink data transmission rate to all UAVs; Indicates UE i The maximum downlink data transmission rate with all UAVs; c iu,min Indicates the minimum value required for the uplink data transmission rate; c id,min Indicates the minimum value required for the downlink data transmission rate; if Indicates UE i Establish a communication link with the UAV, within the communication coverage; if Indicates UE i No communication link has been established with the UAV and it is not within the communication coverage area.

[0039] In one embodiment of the present invention, the expression of the problem model is:

[0040]

[0041] Among them, OP represents the problem model; max represents finding the maximum value; A set representing the sub-bandwidth division ratio; represents a collection of UAV trajectories;

[0042] represents the set of uplink and downlink ratios; the multiple items in st represent constraints, where (a) represents UE i Is it covered? (b) is the constraint satisfied by the sub-bandwidth division ratio; (c) to (f) are the constraints of the drone trajectory, (c) limits the drone flight angle θ, (d) limits the drone flight distance l within the cycle; (e) limits the drone coordinates in the next time slot; (f) limits the drone flight area; x t and t represents the horizontal and vertical coordinates of the UAV at time slot t; L and H represent the length and width of the UAV’s flight area.

[0043] In one embodiment of the present invention, the reinforcement learning model includes an MDP model, and the preset policy optimization algorithm includes a PPO algorithm.

[0044] In one embodiment of the present invention, the process of establishing the MDP model includes:

[0045] According to the UE i and UAVs jThe uplink data transmission rate, downlink data transmission rate and channel state are used to construct the state space, which is expressed as:

[0046]

[0047] According to the UAV flight angle, UAV flight distance, uplink and downlink ratio, and sub-bandwidth division ratio in time slot t, the action space is constructed and expressed as:

[0048]

[0049] Set the reward function to:

[0050]

[0051] Among them, N t N is the total number of UEs covered by communication in time slot t; t is the total number of UEs covered by communication in time slot t-1.

[0052] In a second aspect, an embodiment of the present invention provides an air-to-ground hybrid communication device considering space, time domain and frequency domain parameter optimization, the device comprising:

[0053] The UAV trajectory model and channel model building module is used to build the UAV trajectory model for the scenario containing multiple UAVs and user nodes UE, and to build the channel model between the UAV and the UE based on the line-of-sight link and the non-line-of-sight link;

[0054] The communication link rule establishment module is used to establish the communication link rules based on the designed hybrid communication architecture, including: each time slot is divided into uplink and downlink transmission phases based on the TDD technology, and the transmission duration is divided according to the uplink and downlink ratio; based on the FDMA technology, the total frequency bandwidth of the system in each time slot is divided into the sub-frequency bandwidth of each UE according to a plurality of sub-bandwidth division ratios matching the number of UEs; the communication decision between the UE and the UAV in each time slot is determined according to the uplink and downlink data transmission rates, and the uplink and downlink data transmission rates are calculated based on the uplink and downlink ratios, the plurality of sub-bandwidth division ratios and the channel model;

[0055] A problem model building module is used to build a problem model based on the communication decision, using the UAV trajectory, the uplink and downlink ratio, and the ratio of multiple sub-bandwidth divisions as optimization parameters, using the total number of UEs covered by the communication as the optimization target, and adding multiple constraints;

[0056] The parameter optimization module is used to establish a reinforcement learning model and introduce a preset strategy optimization algorithm to optimize the optimization parameters of the problem model to achieve the optimization goal.

[0057] In a third aspect, an embodiment of the present invention provides an air-to-ground communication system, comprising multiple unmanned aerial vehicles (UAVs) and user nodes (UEs). The air-to-ground communication system adopts the air-to-ground hybrid communication method considering the optimization of space, time domain and frequency domain parameters as described in the first aspect to achieve air-to-ground communication.

[0058] Beneficial effects of the present invention:

[0059] In the air-to-ground hybrid communication method considering the optimization of space, time domain and frequency domain parameters provided in the embodiment of the present invention, based on the air-to-ground communication system, a UAV trajectory model and a channel model are established, and a hybrid communication architecture is proposed, which defines the communication link rules between ground user nodes and UAVs, considers the optimization of parameters in space, time domain and frequency domain, takes UAV trajectory, uplink and downlink ratio, and multiple sub-bandwidth division ratios as optimization parameters, takes the total number of UEs covered by communication as the optimization target, and adds multiple constraints to establish a problem model. By establishing a reinforcement learning model and using a strategy optimization algorithm to optimize the optimization parameters, the optimization target is achieved and the optimal performance of the communication system is obtained. The present invention increases the number of UE nodes effectively covered by UAV network communication, and provides a new method for efficient operation of scenarios such as emergency communications. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A flow chart of an air-to-ground hybrid communication method considering space, time domain and frequency domain parameter optimization provided by an embodiment of the present invention;

[0061] Figure 2 A schematic diagram of a scenario including multiple unmanned aerial vehicles (UAVs) and user nodes (UEs) according to an embodiment of the present invention;

[0062] Figure 3 A schematic diagram of the structure of a hybrid communication architecture provided by an embodiment of the present invention;

[0063] Figure 4 A schematic diagram of the structure of an air-to-ground hybrid communication device considering space, time domain and frequency domain parameter optimization provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0065] In order to optimize the performance of the communication system by optimizing the UAV running trajectory, uplink and downlink time slot ratio and bandwidth allocation parameters during the communication between multiple ground nodes and UAV, the embodiments of the present invention provide an air-to-ground hybrid communication method, device and air-to-ground communication system considering the optimization of space, time domain and frequency domain parameters, which can be used in the field of social public safety, such as emergency communication.

[0066] It should be noted that the execution subject of the air-to-ground hybrid communication method considering space, time domain and frequency domain parameter optimization provided by the embodiment of the present invention can be an air-to-ground hybrid communication device considering space, time domain and frequency domain parameter optimization, and the device can be run in an electronic device. Among them, the electronic device can be a server, a cloud server, a drone edge computing device, etc. of the back-end control center, but of course it is not limited to this.

[0067] In a first aspect, an embodiment of the present invention provides an air-to-ground hybrid communication method considering optimization of space, time domain and frequency domain parameters, such as Figure 1 As shown, the method may include the following steps:

[0068] S1, for scenarios with multiple UAVs and user nodes UE, establish a UAV trajectory model, and establish a channel model between UAV and UE based on line-of-sight links and non-line-of-sight links;

[0069] Figure 2 is a schematic diagram of a scene containing multiple UAVs and user nodes UE; in the scene, J UAVs are deployed above the ground, represented by The purpose is to use UAV as an aerial base station to quickly restore communication; there is I UE deployed on the ground, and the UE can be the communication device of the user in the disaster area, which is expressed as The flight cycle of the UAV is divided into T equal time periods, expressed as Each time period is a time slot, represented by duration τ. Assuming τ is small enough and equal in all time periods, the position of the UAV remains relatively constant in each time period. The jth UAV is represented by UAV j , whose coordinates in time slot t are expressed as and Respectively represent UAV j The horizontal and vertical coordinates of the time slot t, h j Represents UAV j The flight altitude is constant. Represents UAV j The initial coordinates of Represents UAV j Flight distance in time slot t; Represents UAV j The flight angle at time slot t is in the range of [0,2π]; the maximum flight speed of the drone is v max , then the maximum flight distance of the UAV in each time slot t can be expressed as l max =v max τ.

[0070] Therefore, the horizontal and vertical coordinates of the UAV in time slot t, that is, the UAV trajectory model can be expressed as:

[0071]

[0072] Among them, the UAV flight coordinate constraint is expressed as:

[0073]

[0074] Among them, l max Indicates the maximum flight distance of the drone in a time slot.

[0075] In the embodiment of the present invention, the channel model covers two types: line-of-sight (LoS) and non-line-of-sight (NLoS). Specifically, the channel model between the UAV and the UE is established based on the line-of-sight link and the non-line-of-sight link, including the following steps:

[0076] 1) Determine the UAV in time slot t j and UE i The line-of-sight connection probability and the non-line-of-sight connection probability are expressed as:

[0077]

[0078] Among them, LoS and NLoS represent line-of-sight and non-line-of-sight respectively; UE i represents the i-th UE; represents the UAV in time slot t j and UE i The probability of line-of-sight connection between represents the UAV in time slot t j and UE i The probability of non-line-of-sight connection between α and β; α and β are environmental constants; represents the UAV in time slot t j and UE i The elevation angle between

[0079] 2) Determine the UAV in time slot t j and UE i The average LoS path loss and the average NLoS path loss between are expressed as:

[0080]

[0081] in, and They represent the UAVs in time slot t. j and UE i The average LoS path loss and the average NLoS path loss between them; f represents the UAV j and UE iThe channel transmission frequency, in Hz; represents the UAV in time slot t j and UE i The distance between them; c represents the speed of light; η LoS and η NLoS They represent the additional loss on the LoS path and the additional loss on the NLoS path, respectively, LoS and η NLoS is a known value;

[0082] 3) According to the UAV in time slot t j and UE i The probability of line-of-sight connection and non-line-of-sight connection between UAVs in time slot t j and UE i The average LoS path loss and the average NLoS path loss between them are used to determine the UAV j and UE i The average path loss between is expressed as:

[0083]

[0084] in, is the UAV in time slot t j and UE i The average path loss between is used to characterize the channel status.

[0085] S2, based on the designed hybrid communication architecture, establish communication link rules, including: each time slot is divided into uplink and downlink transmission phases based on TDD technology, and the transmission duration is divided according to the uplink and downlink ratio; based on FDMA technology, the total frequency bandwidth of the system in each time slot is divided into sub-frequency bandwidths of each UE according to multiple sub-bandwidth division ratios matching the number of UEs; the communication decision between the UE and the UAV in each time slot is determined according to the uplink and downlink data transmission rates, and the uplink and downlink data transmission rates are calculated based on the uplink and downlink ratios, the multiple sub-bandwidth division ratios and the channel model;

[0086] In the embodiment of the present invention, a hybrid communication architecture is designed, referred to as HybridComm, which is divided into two layers. The upper layer adopts TDD technology to realize full-duplex communication; the lower layer adopts FDMA to realize multiple access of multiple nodes on the ground. The specific block diagram is shown in Figure 3. The communication link rules are established according to the hybrid communication architecture.

[0087] Specifically, the full-duplex communication function is implemented through TDD technology. In TDD technology, each time slot is divided into uplink and downlink signal transmission stages. Figure 3 Take time slot t as an example, its duration is τ, and the user UE in time slot t iThe uplink signal transmission phase to UAV has a transmission duration of kτ, and the UAV to UE i In the downlink signal transmission phase, the transmission duration is (1-k)τ, the uplink-downlink ratio k is between 0 and 1, and the uplink-downlink ratio k of time slot t is dynamically adjusted according to the amount of uplink and downlink data in time slot t-1. This part is implemented by establishing a problem model and optimizing the parameters using reinforcement learning. Please refer to S3 to S4 below for details.

[0088] The ground multi-node multiple access function is implemented through FDMA technology. In FDMA technology, simultaneous communication of multiple users is achieved by dividing the available spectrum into multiple independent frequency channels. Specifically, the total frequency bandwidth of the system is divided according to multiple sub-bandwidth division ratios that match the number of UEs. Each user is assigned a fixed sub-frequency bandwidth that can be used continuously during a call to ensure that the signals between different users do not interfere with each other. FDMA supports parallel communication, allowing multiple users to transmit data simultaneously, and can reuse the same frequency in different geographical areas to improve spectrum utilization. Therefore, the total frequency bandwidth of the entire system is divided into I non-overlapping sub-frequency bandwidths, expressed as Among them, w represents the size of the total frequency bandwidth of the system, a i It represents the sub-bandwidth allocation ratio of the i-th UE, and the sum of all sub-bandwidth allocation ratios is 1.

[0089] Due to factors such as distance and environment, when the UAV establishes communication links with different UEs, the network throughput varies, which directly affects the quality of network communication. Indicates that if UE i If the maximum transmission rate between J UAVs is greater than or equal to a certain threshold, a communication link is established with the UAV with the maximum transmission rate, which is expressed as If UE i If the maximum transmission rate between J UAVs is less than a certain threshold, no communication link will be established, which is expressed as

[0090] According to the relevant settings of the hybrid communication architecture described above, the UAV in time slot t j and UE i The calculation formula for the uplink data transmission rate is:

[0091]

[0092] UAV in time slot t j and UE i The calculation formula for the downlink data transmission rate is:

[0093]

[0094] in, is the UAV in time slot t j and UE i Uplink data transmission rate; is the UAV in time slot t j and UE i Downlink data transmission rate; k t is the uplink and downlink ratio in time slot t, k t τ represents the transmission duration of the uplink transmission phase in time slot t, (1-k t )τ represents the transmission duration of the downlink transmission phase in time slot t; w represents the size of the total frequency bandwidth of the system, and the sub-frequency bandwidth of I UE divided is expressed as a i is the sub-bandwidth allocation ratio of the i-th UE; P ij Indicates UE i To UAV j The transmission power; represents the UAV in time slot t j and UE i The average path loss between ji Represents UAV j and UE i The transmission power; represents the UAV in time slot t j and UE i The average path loss between σ0 represents the power spectral density of the additive white Gaussian noise at the receiving end.

[0095] The communication decision is expressed as:

[0096]

[0097] in, Indicates UE in time slot t i communication decisions, Indicates UE i The maximum uplink data transmission rate to all UAVs; Indicates UE i The maximum downlink data transmission rate with all UAVs; c iu,min Indicates the minimum value required for the uplink data transmission rate; c id,min Indicates the minimum value required for the downlink data transmission rate; if Indicates UE i Establish a communication link with the UAV, within the communication coverage; if Indicates UE i No communication link has been established with the UAV and it is not within the communication coverage area.

[0098] S3, based on the communication decision, using the UAV trajectory, uplink and downlink ratio, and multiple sub-bandwidth division ratios as optimization parameters, using the total number of UEs covered by the communication as the optimization target, and adding multiple constraints to establish a problem model;

[0099] The present invention optimizes the total number of UEs covered by the communication, which is a key indicator that depends on the data transmission rate between the UAV and the UE. Achieving maximum UE coverage requires a holistic approach, and the present invention optimizes several key parameters simultaneously, including the UAV trajectory, the uplink and downlink ratio of the uplink and downlink in TDD technology, and the sub-bandwidth division ratio in FDMA technology.

[0100] Specifically, according to the above steps, UE i To UAV j The transmission rate between them is obtained to obtain the function of the total number of UEs covered by the communication, and it is used as the objective function to construct the problem model, and the constraints such as the value range of the communication decision, the sum of all sub-bandwidth division ratios, the flight angle range of the UAV, the flight distance range within the UAV time slot, the value range of the horizontal and vertical coordinates within the UAV time slot, and the flight range of the UAV during the entire cycle are considered.

[0101] The expression of the problem model is:

[0102]

[0103] Among them, OP represents the problem model; max represents finding the maximum value; A set representing the sub-bandwidth division ratio; represents a collection of UAV trajectories;

[0104] represents the set of uplink and downlink ratios; the multiple items in st represent constraints, where (a) represents UE i Is it covered? (b) is the constraint satisfied by the sub-bandwidth division ratio; (c) to (f) are the constraints of the drone trajectory, (c) limits the drone flight angle θ, (d) limits the drone flight distance l within the cycle; (e) limits the drone coordinates in the next time slot; (f) limits the drone flight area; x t and t represents the horizontal and vertical coordinates of the UAV at time slot t; L and H represent the length and width of the UAV’s flight area.

[0105] S4, establishing a reinforcement learning model, and introducing a preset strategy optimization algorithm to optimize the optimization parameters of the problem model to achieve the optimization goal.

[0106] The embodiment of the present invention can use the existing reinforcement learning algorithm to construct a reinforcement learning model, and introduce relevant policy optimization algorithms to optimize the above optimization parameters. For example, in an optional implementation, the reinforcement learning model may include an MDP (Markov Decision Process) model, and the preset policy optimization algorithm may include a PPO (Proximal Policy Optimization) algorithm.

[0107] Based on this, the specific process of establishing the MDP model may include:

[0108] 1) According to the UE in time slot t i and UAVs j The uplink data transmission rate, downlink data transmission rate and channel state are used to construct the state space, which is expressed as:

[0109]

[0110] 2) According to the UAV flight angle, UAV flight distance, uplink and downlink ratio, and sub-bandwidth division ratio in time slot t, the action space is constructed and expressed as:

[0111]

[0112] 3) Set the reward function to:

[0113]

[0114] Among them, the reward is an indicator to measure the effect of the action. The flight position, uplink and downlink ratio, and sub-bandwidth division ratio of the drone in each period are all random. When designing the reward and punishment function, in order to quickly converge and find the optimal solution, the present invention gives rewards when the coverage rate increases in adjacent periods, and gives penalties when it decreases, and gives rewards when the coverage rate remains unchanged, and gives penalties when it increases. In addition, in order to ensure uninterrupted communication of ground rescue equipment, the same penalty is imposed when there is no coverage. N t N is the total number of UEs covered by communication in time slot t; t is the total number of UEs covered by communication in time slot t-1.

[0115] After the MDP model is established, the PPO algorithm is introduced to optimize the optimization parameters of the problem model. The PPO algorithm is a prominent policy gradient method widely used in DRL (Deep Reinforcement Learning), which aims to enhance training stability by limiting the difference between the updated policy and its previous version.

[0116] In the embodiment of the present invention, the process of introducing the PPO algorithm to optimize the optimization parameters of the problem model includes:

[0117] Step 1: Input parameters, including the policy network parameter set θ π 、Value network parameter set φ V ;

[0118] Step 2: For each value of K, execute steps 3 to 7, where K represents the number of training cycles of the PPO algorithm and is a positive integer.

[0119] Step 3, by running the policy π in the environment K =π(θ K ) to collect the trajectory set Among them, the input state space and the output action space are the mean and standard deviation of each dimension. Each action can be sampled from a multidimensional Gaussian distribution. The trajectory set refers to the interaction between the agent and the environment under the guidance of the current strategy, collecting a series of state, action, reward, and next state data, τ i ={(s 1 ,a 1 ,r 1 ,s 2 ),(s 2 ,a 2 ,r 2 ,s 2+1 ),...,(s T ,a T ,r T ,s T+1 )};

[0120] Regarding states, actions, and rewards, please combine the previous state space, action space, reward function, and the existing PPO algorithm to understand them. I will not explain them in detail here.

[0121] Step 4: Calculate the cumulative return

[0122] Among them, γ is the discount factor;

[0123] Step 5: Based on the current value function Calculate odds estimate The calculation formula is: in, represents the cumulative return after selecting action a in state s; where δ t is the time difference error at time t, expressed as s t+1 is the state value at time t+1, s t is the state value at time t; λ is the GAE parameter;

[0124] Step 6, update the policy by maximizing the PPO-Clip objective, expressed as:

[0125]

[0126] Among them, π θ (a t |s t ) represents the state s under the current strategy t Take action a t probability; Indicates the state s under the old strategy t Take action a t probability; is the advantage function estimate; ε is the hyperparameter of the clipping range; Represents the probability ratio of the policy being pruned to avoid excessive policy updates that affect stability.

[0127] Step 7, fit the value function through mean square error regression, expressed as:

[0128]

[0129] Among them, V φ (s t ) is the state s t The value function of .

[0130] In general, the current strategy Next, we collect the states, actions, rewards, and next state values ​​in the MDP model in time slot t, and use the value network To calculate the advantage estimate for each state-action pair Finally, the policy network parameter set θ is completed π and the value network parameter set φ V value.

[0131] Through the above steps, the optimization parameters of each time slot can be continuously optimized within a flight cycle to achieve the optimization goal.

[0132] In the air-to-ground hybrid communication method considering the optimization of space, time domain and frequency domain parameters provided in the embodiment of the present invention, based on the air-to-ground communication system, a UAV trajectory model and a channel model are established, and a hybrid communication architecture is proposed, which defines the communication link rules between ground user nodes and UAVs, considers the optimization of parameters in space, time domain and frequency domain, takes UAV trajectory, uplink and downlink ratio, and multiple sub-bandwidth division ratios as optimization parameters, takes the total number of UEs covered by communication as the optimization target, and adds multiple constraints to establish a problem model. By establishing a reinforcement learning model and using a strategy optimization algorithm to optimize the optimization parameters, the optimization target is achieved and the optimal performance of the communication system is obtained. The present invention increases the number of UE nodes effectively covered by UAV network communication, and provides a new method for efficient operation of scenarios such as emergency communications.

[0133] In a second aspect, corresponding to the above method embodiment, the embodiment of the present invention further provides an air-to-ground hybrid communication device considering the optimization of space, time domain and frequency domain parameters, such as Figure 4 As shown, the device comprises:

[0134] The UAV trajectory model and channel model building module is used to build the UAV trajectory model for the scenario containing multiple UAVs and user nodes UE, and to build the channel model between the UAV and the UE based on the line-of-sight link and the non-line-of-sight link;

[0135] The communication link rule establishment module is used to establish the communication link rules based on the designed hybrid communication architecture, including: each time slot is divided into uplink and downlink transmission phases based on the TDD technology, and the transmission duration is divided according to the uplink and downlink ratio; based on the FDMA technology, the total frequency bandwidth of the system in each time slot is divided into the sub-frequency bandwidth of each UE according to a plurality of sub-bandwidth division ratios matching the number of UEs; the communication decision between the UE and the UAV in each time slot is determined according to the uplink and downlink data transmission rates, and the uplink and downlink data transmission rates are calculated based on the uplink and downlink ratios, the plurality of sub-bandwidth division ratios and the channel model;

[0136] A problem model building module is used to build a problem model based on the communication decision, using the UAV trajectory, the uplink and downlink ratio, and the ratio of multiple sub-bandwidth divisions as optimization parameters, using the total number of UEs covered by the communication as the optimization target, and adding multiple constraints;

[0137] The parameter optimization module is used to establish a reinforcement learning model and introduce a preset strategy optimization algorithm to optimize the optimization parameters of the problem model to achieve the optimization goal.

[0138] For the specific processing process of each module of the device, please refer to the relevant content of the first aspect and will not be repeated here.

[0139] The air-to-ground hybrid communication device provided by the embodiment of the present invention, which takes into account the optimization of space, time domain and frequency domain parameters, can increase the number of UE nodes effectively covered by UAV network communication, and provide a new solution for the efficient operation of scenarios such as emergency communications.

[0140] In the third aspect, corresponding to the above method embodiment, the embodiment of the present invention further provides an air-to-ground communication system, including multiple unmanned aerial vehicles (UAVs) and user nodes (UEs), wherein the air-to-ground communication system adopts the air-to-ground hybrid communication method considering the optimization of space, time domain and frequency domain parameters described in the first aspect to realize air-to-ground communication. For the specific communication method, please refer to the content of the first aspect, which will not be described in detail here.

[0141] The air-to-ground communication system provided by the embodiment of the present invention realizes air-to-ground communication by adopting the proposed air-to-ground hybrid communication method considering the optimization of space, time domain and frequency domain parameters, which can obtain the optimal performance of the system and increase the number of UE nodes effectively covered by UAV network communication.

[0142] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification.

[0143] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0144] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. An air-to-ground hybrid communication method considering space, time domain and frequency domain parameter optimization, characterized in that: include: For scenarios with multiple UAVs and user nodes UE, a UAV trajectory model is established, and a channel model between UAV and UE is established based on line-of-sight links and non-line-of-sight links; Based on the designed hybrid communication architecture, a communication link rule is established, including: each time slot is divided into uplink and downlink transmission phases based on TDD technology, and the transmission duration is divided according to the uplink and downlink ratio; based on FDMA technology, the total frequency bandwidth of the system in each time slot is divided into sub-frequency bandwidths of each UE according to multiple sub-bandwidth division ratios matching the number of UEs; the communication decision between the UE and the UAV in each time slot is determined according to the uplink and downlink data transmission rates, and the uplink and downlink data transmission rates are calculated based on the uplink and downlink ratios, the multiple sub-bandwidth division ratios and the channel model; Based on the communication decision, the UAV trajectory, uplink and downlink ratio, and multiple sub-bandwidth division ratios are used as optimization parameters, the total number of UEs covered by the communication is used as the optimization target, and multiple constraints are added to establish a problem model; A reinforcement learning model is established, and a preset strategy optimization algorithm is introduced to optimize the optimization parameters of the problem model to achieve the optimization goal.

2. The air-to-ground hybrid communication method considering space, time domain and frequency domain parameter optimization according to claim 1, characterized in that: The UAV trajectory model is expressed as: In the scenario, there are J UAVs deployed above the ground, represented by There are I UEs deployed on the ground, represented by The flight cycle of the UAV is divided into T equal time periods, expressed as Each time period is a time slot, which is represented by duration τ, and τ is small enough; the jth UAV is represented by UAV j , whose coordinates in time slot t are expressed as and Respectively represent UAV j The horizontal and vertical coordinates of the time slot t, h j Represents UAV j Flight altitude; Represents UAV j The initial coordinates of Represents UAV j Flight distance in time slot t; Represents UAV j The flight angle at time slot t has a value range of [0,2π]; In the UAV trajectory model, the UAV flight coordinate constraint is expressed as: Among them, l max Indicates the maximum flight distance of the drone in a time slot.

3. The air-to-ground hybrid communication method considering space, time domain and frequency domain parameter optimization according to claim 2, characterized in that: The establishing of a channel model between the UAV and the UE based on the line-of-sight link and the non-line-of-sight link includes: Determine the UAV in time slot t j and UE i The line-of-sight connection probability and the non-line-of-sight connection probability are expressed as: Among them, LoS and NLoS represent line-of-sight and non-line-of-sight respectively; UE i represents the i-th UE; represents the UAV in time slot t j and UE i The probability of line-of-sight connection between represents the UAV in time slot t j and UE i The probability of non-line-of-sight connection between α and β; α and β are environmental constants; represents the UAV in time slot t j and UE i The elevation angle between Determine the UAV in time slot t j and UE i The average LoS path loss and the average NLoS path loss between are expressed as: in, and They represent the UAVs in time slot t. j and UE i The average LoS path loss and the average NLoS path loss between them; f represents the UAV j and UE i The channel transmission frequency, in Hz; represents the UAV in time slot t j and UE i The distance between them; c represents the speed of light; η LoS and η NLoS They represent the additional loss on the LoS path and the additional loss on the NLoS path respectively; According to the UAV in time slot t j and UE i The probability of line-of-sight connection and non-line-of-sight connection between UAVs in time slot t j and UE i The average LoS path loss and the average NLoS path loss between them are used to determine the UAV j and UE i The average path loss between is expressed as: in, is the UAV in time slot t j and UE i The average path loss between is used to characterize the channel status.

4. The air-to-ground hybrid communication method considering space, time domain and frequency domain parameter optimization according to claim 3, characterized in that: In the communication link rules, UAV in time slot t j and UE i The calculation formula for the uplink data transmission rate is: UAV in time slot t j and UE i The calculation formula for the downlink data transmission rate is: in, is the UAV in time slot t j and UE i Uplink data transmission rate; is the UAV in time slot t j and UE i Downlink data transmission rate; k t is the uplink and downlink ratio in time slot t, k t τ represents the transmission duration of the uplink transmission phase in time slot t, (1-k t )τ represents the transmission duration of the downlink transmission phase in time slot t; w represents the size of the total frequency bandwidth of the system, and the sub-frequency bandwidth of I UE divided is expressed as a i is the sub-bandwidth allocation ratio of the i-th UE; P ij Indicates UE i To UAV j The transmission power; represents the UAV in time slot t j and UE i The average path loss between ji Represents UAV j and UE i The transmission power; represents the UAV in time slot t j and UE i The average path loss between σ0 represents the power spectral density of the additive white Gaussian noise at the receiving end.

5. The air-to-ground hybrid communication method considering space, time domain and frequency domain parameter optimization according to claim 4, characterized in that: In the communication link rule, the communication decision is expressed as: in, Indicates UE in time slot t i communication decisions, Indicates UE i The maximum uplink data transmission rate to all UAVs; Indicates UE i The maximum downlink data transmission rate with all UAVs; c iu,min Indicates the minimum value required for the uplink data transmission rate; c id,min Indicates the minimum value required for the downlink data transmission rate; if Indicates UE i Establish a communication link with the UAV and within the communication coverage; if Indicates UE i No communication link has been established with the UAV and it is not within the communication coverage area.

6. The air-to-ground hybrid communication method considering space, time domain and frequency domain parameter optimization according to claim 5, characterized in that: The expression of the problem model is: Among them, OP represents the problem model; max represents finding the maximum value; A set representing the sub-bandwidth division ratio; represents a collection of UAV trajectories; represents the set of uplink and downlink ratios; the multiple items in st represent constraints, where (a) represents UE i Is it covered? (b) is the constraint satisfied by the sub-bandwidth division ratio; (c) to (f) are the constraints of the drone trajectory, (c) limits the drone flight angle θ, (d) limits the drone flight distance l within the cycle; (e) limits the drone coordinates in the next time slot; (f) limits the drone flight area; x t and t represents the horizontal and vertical coordinates of the UAV at time slot t; L and H represent the length and width of the UAV’s flight area.

7. The air-to-ground hybrid communication method considering space, time domain and frequency domain parameter optimization according to claim 6, characterized in that: The reinforcement learning model includes an MDP model, and the preset strategy optimization algorithm includes a PPO algorithm.

8. The air-to-ground hybrid communication method considering space, time domain and frequency domain parameter optimization according to claim 7, characterized in that: The process of establishing an MDP model includes: According to the UE i and UAVs j The uplink data transmission rate, downlink data transmission rate and channel state are used to construct the state space, which is expressed as: According to the UAV flight angle, UAV flight distance, uplink and downlink ratio, and sub-bandwidth division ratio in time slot t, the action space is constructed and expressed as: Set the reward function to: Among them, N t N is the total number of UEs covered by communication in time slot t; t is the total number of UEs covered by communication in time slot t-1.

9. An air-to-ground hybrid communication device considering space, time domain and frequency domain parameter optimization, characterized in that: include: The UAV trajectory model and channel model building module is used to build the UAV trajectory model for the scenario containing multiple UAVs and user nodes UE, and to build the channel model between the UAV and the UE based on the line-of-sight link and the non-line-of-sight link; The communication link rule establishment module is used to establish the communication link rules based on the designed hybrid communication architecture, including: each time slot is divided into uplink and downlink transmission phases based on the TDD technology, and the transmission duration is divided according to the uplink and downlink ratio; based on the FDMA technology, the total frequency bandwidth of the system in each time slot is divided into the sub-frequency bandwidth of each UE according to a plurality of sub-bandwidth division ratios matching the number of UEs; the communication decision between the UE and the UAV in each time slot is determined according to the uplink and downlink data transmission rates, and the uplink and downlink data transmission rates are calculated based on the uplink and downlink ratios, the plurality of sub-bandwidth division ratios and the channel model; A problem model building module is used to build a problem model based on the communication decision, using the UAV trajectory, the uplink and downlink ratio, and the ratio of multiple sub-bandwidth divisions as optimization parameters, using the total number of UEs covered by the communication as the optimization target, and adding multiple constraints; The parameter optimization module is used to establish a reinforcement learning model and introduce a preset strategy optimization algorithm to optimize the optimization parameters of the problem model to achieve the optimization goal.

10. An air-to-ground communication system, characterized in that: The air-to-ground communication system comprises a plurality of unmanned aerial vehicles (UAVs) and user nodes (UEs). The air-to-ground communication system realizes air-to-ground communication by adopting the air-to-ground hybrid communication method considering the optimization of space, time domain and frequency domain parameters as described in any one of claims 1 to 8.

Images