Solar-powered UAV data transmission method, readable storage medium and device

Abstract

The present invention discloses a solar-powered UAV data transmission method, a readable storage medium and a device, including: inputting the acquired user location information of the UAV service area, the channel information between the target UAV and the user, and the real-time power of the target UAV into a pre-trained UAV data transmission model, and obtaining the optimal transmission power and the optimal flight path for allocating the maximum downlink speed of the user to the UAV at the minimum battery capacity decay rate. The present invention controls the target UAV to transmit data by allocating the optimal transmission power and the optimal flight path for the maximum downlink speed of the user to the UAV at the minimum battery capacity decay rate, fully considering the impact of battery aging on UAV communication, making the UAV battery work within a reasonable SoC range, and improving the battery life while ensuring the maximum downlink speed of the UAV data transmission.

Description

Technical Field

[0001] The invention relates to a solar-powered unmanned aerial vehicle data transmission method, a readable storage medium and a device, and belongs to the technical field of unmanned aerial vehicle communication. Background Art

[0002] At present, in terms of energy constraints for drones, most people only consider how to efficiently utilize the limited onboard energy of drones, and then perform trajectory optimization and wireless resource allocation. The constraints on drone energy consumption are mainly concentrated on: the energy consumption of drones during flight cannot exceed a certain threshold or the drone battery capacity must be greater than 0 before the service ends. Existing optimization technology solutions ignore the fact that improper charging and discharging of batteries will accelerate the aging process of batteries, resulting in a decrease in battery capacity. In addition, rapid battery aging will also increase the cost of replacing batteries. Summary of the invention

[0003] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a solar drone data transmission method, a readable storage medium and a device, which fully consider the impact of battery aging on drone communication, enable the drone battery to operate within a reasonable SoC range, and improve battery life while ensuring the maximum downlink speed of drone data transmission.

[0004] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0005] In one aspect, the present invention provides a solar-powered UAV data transmission method, comprising:

[0006] Obtain the user location information of the drone service area, the channel information between the target drone and the user, and the real-time power of the target drone;

[0007] The user location information of the drone service area, the channel information between the target drone and the user, and the real-time power of the target drone are input into the pre-trained drone data transmission model to obtain the optimal transmission power and optimal flight path that enables the drone to allocate the maximum downlink speed to the user at the minimum battery capacity decay rate;

[0008] According to the optimal transmission power and the optimal flight path, the target UAV is controlled to transmit data.

[0009] Among them, the training of the pre-trained drone data transmission model includes:

[0010] Based on the user's location information, the drone's battery information, and the channel information between the drone and the user, a working environment model between the drone and the user is constructed;

[0011] Based on the working environment model, the optimal transmission power and optimal flight path for allocating the maximum downlink speed of users under the minimum battery capacity decay rate of the UAV are taken as the optimization goals to determine and train the UAV data transmission model.

[0012] Furthermore, the working environment model between the drone and the user is constructed according to the user location information, the drone power information, and the channel information between the drone and the user, including:

[0013] The working environment model includes a drone network model, a drone channel model and a communication energy model;

[0014] The UAV communication network model is established by the following formula:

[0015] ;

[0016] ;

[0017] Among them: the total time of the drone performing the communication task T Divide into N equal time slots, each of which is ; Indicates that the drone is n The position of the time slot, Indicates that the drone is n time slot x Axis coordinates, Indicates that the drone is n time slot y Axis coordinates, Indicates that the drone is n time slot z Axis coordinates; Indicates that the drone is n The horizontal speed of each time slot is Indicates that the drone is n The horizontal angle of each time slot is Indicates that the drone is n The vertical speed of each time slot;

[0018] Indicates that the drone is time slot x Axis coordinates, Indicates that the drone is time slot y Axis coordinates, Indicates that the drone is time slot z Axis coordinates;

[0019] There are a total ofK terrestrial users, Indicates k User in n The position of the time slot, Indicates k User in n time slot x Axis coordinates, Indicates k User in n time slot y Axis coordinates;

[0020] The UAV channel model is established by the following formula:

[0021] ;

[0022] in: represents the average path loss from the user to the UAV; Indicates the distance from the user to the drone LoS Path loss, , d Indicates the distance from the user to the drone, Indicates the carrier frequency, c represents the speed of light, express LoS Additional attenuation factor caused by the link; Represents the user and the drone LoS Communication probability, , a , b All represent parameters that depend on the environment. Indicates the elevation angle between the user and the drone; Indicates the distance from the user to the drone NLoS Path loss, , express NLoS Additional attenuation factor caused by the link; Represents the user and the drone NLoS Communication probability, ; Indicated in n Time slot users k The instantaneous transmission rate obtained is Indicates user k The communication bandwidth Indicated in n Time slot users k The received signal-to-noise ratio at , Indicates n time slots are allocated to users k The transmission power, Indicated in n The channel gain between the drone and the user in each time slot is represents the noise power, , Indicates n time slot drone to user k The average path loss of

[0023] The communication energy model is established by the following formula:

[0024] ;

[0025] in: Represents the solar power generation power, represents the energy conversion efficiency, represents the area of ​​the drone's solar panels, represents the average solar radiation on Earth, represents the atmospheric transmittance, represents the atmospheric extinction coefficient, represents the proportional height of the Earth, z Indicates the flight altitude of the drone;

[0026] Indicates the UAV communication power consumption; Indicates n time slots are allocated to users k The transmission power, K Indicates that there are a total of K terrestrial users, k Indicates k Users; Indicates the UAV flight power consumption, Indicates the profile power of the drone blades in hovering state. , represents the contour drag coefficient, Indicates the air density, s Indicates rotor strength, represents the rotor disc area, represents the blade angular velocity, R represents the rotor radius, It represents the average rotor induced speed when the drone is hovering. represents the tip speed of the UAV's rotor blades, represents the induced power, , Indicates the incremental correction factor for the induced power, Indicates the weight of the drone, Represents the drag ratio of the drone body.

[0027] Furthermore, based on the working environment model, the optimal transmission power and optimal flight path for allocating the maximum downlink speed of the user to the drone at the minimum battery capacity decay rate are taken as the optimization goal, and the drone data transmission model is determined and trained, including:

[0028] Based on the working environment model, the optimization objective is to allocate the maximum downlink speed of the user under the minimum battery capacity decay rate, and the constraint conditions of the optimization objective are constructed;

[0029] Based on Markov decision making, the optimization objective is mapped to an observable virtual space;

[0030] Determine and train a UAV data transmission model based on an optimization objective mapped into an observable virtual space.

[0031] Furthermore, based on the working environment model, the optimization objective is to allocate the maximum downlink speed of the user under the minimum battery capacity decay rate of the UAV, and the constraint conditions of the optimization objective are constructed, including:

[0032] Establish the first constraint:

[0033] ;

[0034] ;

[0035] ;

[0036] ;

[0037] in: D Indicates the range of the drone service area. Indicates the maximum speed of the drone in the horizontal direction. Indicates the maximum speed of the drone in the vertical direction, Indicates that the drone is n time slot x Axis coordinates, Indicates that the drone is n time slot y Axis coordinates, Indicates that the drone is n time slot z Axis coordinates; Indicates that the drone is n The horizontal speed of each time slot is Indicates that the drone is n The horizontal angle of each time slot is Indicates that the drone is n The vertical speed of the time slot is represents pi;

[0038] Create the second constraint:

[0039] ;

[0040] ;

[0041] in: Indicates n time slots are allocated to users k The transmission power, K Indicates that there are a total of K terrestrial users, k Indicates k Users, Indicates the maximum transmission power of the drone;

[0042] Create the third constraint:

[0043] ;

[0044] in: Indicates user k The communication bandwidth Indicated in n Time slot users k The received signal-to-noise ratio at Indicates the user's minimum QoS requirement, QoS indicates the user's minimum service quality;

[0045] Establish the fourth constraint:

[0046] ;

[0047] ;

[0048] ;

[0049] ;

[0050] ;

[0051] in: Indicated in n The remaining battery power of the drone in each time slot, Indicates the initial power of the drone. Indicates the maximum remaining power of the drone; SoC indicates the battery power status, SoH indicates the battery health status, SoC ( t ) indicates that the drone is at time t Battery state of charge, SoH( t ) indicates that the drone is at time t Battery health status;

[0052] Z Represents the test function of SoC: ,when hour, The value of ,otherwise The value of , Represents a constant greater than 1, Indicates the drone battery discharge coefficient, Indicates i The solar charging capacity of the drone in each time slot, Indicates i The power consumption of the drone flight in each time slot is Indicates i Power consumption of drone communication in each time slot; Indicated in The remaining battery power of the drone in each time slot; Indicates that the drone is at time Battery health status, Indicates that the drone is at time battery charge status.

[0053] Furthermore, the Markov decision-making method maps the optimization target to an observable virtual space, including:

[0054] Convert the optimization objective into a tuple:

[0055] ;

[0056] in: represents the state space, represents the action space, represents the reward function, represents the state transition probability, represents the discount factor;

[0057] Time slot n When , the state space is expressed as ; in: Indicates that the drone is n The location of the time slot;

[0058] There are a total of K terrestrial users, Indicates k User in n The location of the time slot; Indicated in n The remaining battery power of the drone in each time slot;

[0059] In the past, drones n -The maximum downlink speed obtained in 1 time slot, , Indicates that the drone is i The downlink speed of the time slot; n When there are time slots, the action space is expressed as ;

[0060] ;

[0061] in: Indicates that the drone is n The horizontal speed of each time slot is Indicates that the drone is n The horizontal angle of each time slot is Indicates that the drone is n The vertical speed of the time slot is Indicates n time slots are allocated to users K The transmission power of

[0062] No. n When there are time slots, the reward function is expressed as ; ;

[0063] in: It means the penalty for the drone flying out of the service area. Representation and The corresponding weight coefficient; Indicates the penalty for violating the user service quality. Representation and The corresponding weight coefficient; represents the penalty for violating the battery aging constraint, Representation and The corresponding weight coefficient; Indicates that the user is n The sum of the downlink rates in time slots, , Indicated in n Time slot users k The instantaneous transmission rate obtained is Representation and The corresponding weight coefficient; Indicates the first reward, , In the past, drones n -The maximum downlink speed obtained in 1 time slot, Representation and The corresponding weight coefficient; Indicates the second reward, , , Indicates that the drone is n The distance moved by each time slot, Indicates the first time slot, Representation and The corresponding weight coefficient.

[0064] Furthermore, determining and training the drone data transmission model according to the optimization target mapped into the observable virtual space includes:

[0065] Use SoftActor-Critic network to train the UAV data transmission model;

[0066] Among them, the SoftActor-Critic network includes an Actor network, two main Q networks, two target Q networks, and an experience replay pool;

[0067] The Actor network is used to sample the current state of the drone and generate the current action;

[0068] The experience replay pool stores training samples generated based on the current action;

[0069] The target Q network updates the parameters of the main Q network using a soft update method.

[0070] Furthermore, the method of using the SoftActor-Critic network to train the drone data transmission model includes the following steps:

[0071] 1.1. Get time slot n Current state of the drone data transmission model ;

[0072] 1.2. According to the current status , determine the drone action under the current strategy , Execute actions Corresponding rewards ;

[0073] 1.3. Flag to determine whether to terminate the drone action ;

[0074] 1.4. Deposit into the experience pool and update the current status ;

[0075] 1.5. Sampling from the Experience Pool training samples , use the sampled training samples to update the network parameters of the Actor network and the main Q network;

[0076] 1.6. When the total duration of the drone communication mission is reached or the battery is not working within the limited SoC range, stop updating the network parameters, otherwise execute step 1.1.

[0077] Furthermore, the updating of network parameters of the Actor network, the target Q network and the main Q network using the sampled training samples includes:

[0078] The network parameters of the main Q network are updated using the following formula:

[0079] ;

[0080] Use the following formula to update the network parameters of the Actor network:

[0081] ;

[0082] in: represents the loss function used to update the network parameters of the main Q network; Represents the loss function used to update the network parameters of the Actor network; Indicates j The output value of the target Q network; represents the temperature coefficient; Indicates i The state value of the training sample, Indicates i The action value of each training sample; Re-parameterized sampling is obtained i The action value of each training sample; Represents the network parameters of the Actor network; represents the network parameters of the target Q network; Indicates that the drone is in status Make the following move The probability distribution of

[0083] The target Q network updates the main Q network parameters using soft update:

[0084] ;

[0085] represents the network parameters of the main Q network; represents the soft update coefficient;

[0086] Indicates the TD Target value. ;

[0087] Indicates i+The state value of a training sample, Indicates i+ Action value of 1 training sample; represents the discount factor, Indicates i The reward for each training sample.

[0088] On the other hand, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the solar-powered UAV data transmission method as described above is implemented.

[0089] The present invention also provides a computer device, comprising:

[0090] A memory for storing instructions;

[0091] The processor is used to execute the instructions so that the device implements the solar drone data transmission method as described above.

[0092] Compared with the prior art, the present invention has the following beneficial effects:

[0093] 1. The present invention inputs the user location information of the drone service area, the channel information between the target drone and the user, and the real-time power of the target drone into a pre-trained drone data transmission model, and obtains the optimal transmission power and optimal flight path that enables the drone to allocate the maximum downlink speed to the user at the minimum battery capacity decay rate. It fully considers the impact of battery aging on drone communication, makes the drone battery work within a reasonable SoC range, ensures the maximum downlink speed of drone data transmission, and improves the battery life, solving the problem that the current optimization of drone power does not consider battery aging, resulting in too fast decay of drone life.

[0094] 2. The present invention takes into account the dynamic scenario of ground user movement. Aiming at the shortcomings of traditional convex optimization methods, the optimization problem is transformed into a Markov decision process. The state space, action space and reward function are designed for the scenario. The SoftActor-Critic network is used to train the UAV data transmission model, realizing the joint optimization of UAV transmission power control and UAV flight trajectory. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] Figure 1 is a flow chart of a solar-powered UAV data transmission method provided by an embodiment of the present invention;

[0096] Figure 2 It is a training flow chart of a drone data transmission model in a solar drone data transmission method provided by an embodiment of the present invention;

[0097] Figure 3This is a schematic diagram of a drone service area in a solar drone data transmission method provided by an embodiment of the present invention;

[0098] Figure 4 It is a schematic diagram of a Soft Actor-Critic (SAC) network structure in a solar-powered UAV data transmission method provided by an embodiment of the present invention;

[0099] Figure 5 It is a schematic diagram of the effect of a solar-powered UAV data transmission method provided by an embodiment of the present invention on the data throughput of the UAV with or without battery aging constraints;

[0100] Figure 6 It is a schematic diagram of the effect of a solar-powered UAV data transmission method provided by an embodiment of the present invention on the remaining power of a UAV with or without battery aging constraints. DETAILED DESCRIPTION

[0101] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention. Example

[0102] like Figure 1 A solar-powered drone data transmission method is shown, comprising:

[0103] Obtain the user location information of the drone service area, the channel information between the target drone and the user, and the real-time power of the target drone;

[0104] The user location information of the drone service area, the channel information between the target drone and the user, and the real-time power of the target drone are input into the pre-trained drone data transmission model to obtain the optimal transmission power and optimal flight path that enables the drone to allocate the maximum downlink speed to the user at the minimum battery capacity decay rate;

[0105] Control the target drone to transmit data based on the optimal transmission power and optimal flight path;

[0106] like Figure 2 As shown, the training of the pre-trained drone data transmission model includes:

[0107] According to the user's location information, the drone's battery information, and the channel information between the drone and the user, a working environment model between the drone and the user is constructed. Specifically:

[0108] like Figure 3 As shown, the working environment model includes the UAV network model, the UAV channel model and the communication energy model;

[0109] The UAV communication network model is established by the following formula:

[0110] ;

[0111] ;

[0112] Among them: the total time of the drone performing the communication task T Divide into N equal time slots, each of which is ; Indicates that the drone is n The position of the time slot, Indicates that the drone is n time slot x Axis coordinates, Indicates that the drone is n time slot y Axis coordinates, Indicates that the drone is n time slot z Axis coordinates; Indicates that the drone is n The horizontal speed of each time slot is Indicates that the drone is n The horizontal angle of each time slot is Indicates that the drone is n The vertical speed of each time slot; Indicates that the drone is time slot x Axis coordinates, Indicates that the drone is time slot y Axis coordinates, Indicates that the drone is time slot z Axis coordinates;

[0113] There are a total of K terrestrial users, Indicates k User in n The position of the time slot, Indicates k User in n time slot x Axis coordinates, Indicates k User in n time slot y Axis coordinates;

[0114] The UAV channel model is established by the following formula:

[0115] ;

[0116] in: represents the average path loss from user to no one;

[0117] LoS ( Line of Sight )and NLoS ( Non Line of Sight ) are the two main types of propagation environments in wireless communications; LoS ( Line of Sight )‌ indicates line-of-sight propagation, which means that there are no obstacles between the transmitter and the receiver, and the signal propagates in a straight line; NLoS ( Non Line of Sight ) indicates non-line-of-sight propagation, which means that there are obstacles blocking the signal between the transmitter and the receiver, and the signal needs to be propagated through reflection, diffraction, etc.

[0118] Indicates the distance from the user to the drone LoS Path loss, , d Indicates the distance from the user to the drone, Indicates the carrier frequency, c represents the speed of light, express LoS Additional attenuation factor caused by the link; Represents the user and the drone LoS Communication probability, , a , b All represent parameters that depend on the environment. Indicates the elevation angle between the user and the drone; Indicates the distance from the user to the drone NLoS Path loss, , express NLoS Additional attenuation factor caused by the link; Represents the user and the drone NLoS Communication probability, ;

[0119] Indicated in n Time slot users k The instantaneous transmission rate obtained is Indicates user k The communication bandwidth Indicated in n Time slot users k The received signal-to-noise ratio at , Indicates n time slots are allocated to users k The transmission power, Indicated in n The channel gain between the drone and the user in each time slot is represents the noise power, , Indicates n time slot drone to user k The average path loss of

[0120] The communication energy model is established by the following formula:

[0121] ;

[0122] in: Represents the solar power generation power, represents the energy conversion efficiency, represents the area of ​​the drone's solar panels, represents the average solar radiation on Earth, represents the atmospheric transmittance, represents the atmospheric extinction coefficient, represents the proportional height of the Earth, z Indicates the flight altitude of the drone;

[0123] Indicates the UAV communication power consumption; Indicates the UAV flight power consumption, Indicates the profile power of the drone blades in hovering state. , represents the contour drag coefficient, Indicates the air density, s Indicates rotor strength, represents the rotor disc area, represents the blade angular velocity, R represents the rotor radius, It represents the average rotor induced speed when the drone is hovering. represents the tip speed of the UAV's rotor blades, represents the induced power, , Indicates the incremental correction factor for the induced power, Indicates the weight of the drone, Represents the drag ratio of the drone body.

[0124] Based on the working environment model, the optimal transmission power and optimal flight path for allocating the maximum downlink speed of the user under the minimum battery capacity decay rate are taken as the optimization goal, and the UAV data transmission model is determined and trained. Specifically:

[0125] Based on the working environment model, the optimization goal is to allocate the maximum downlink speed of the user under the minimum battery capacity decay rate, and the constraints of the optimization goal are constructed, including:

[0126] Establish the first constraint:

[0127] ;

[0128] ;

[0129] ;

[0130] ;

[0131] in: D Indicates the range of the drone service area. Indicates the maximum speed of the drone in the horizontal direction. Indicates the maximum speed of the drone in the vertical direction, Indicates that the drone is n time slot x Axis coordinates, Indicates that the drone is n time slot y Axis coordinates, Indicates that the drone is n time slot z Axis coordinates; Indicates that the drone is n The horizontal speed of each time slot is Indicates that the drone is n The horizontal angle of each time slot is Indicates that the drone is n The vertical speed of the time slot is represents pi;

[0132] Create the second constraint:

[0133] ;

[0134] ;

[0135] in: Indicates n time slots are allocated to users k The transmission power, K Indicates that there are a total of K terrestrial users, k Indicates k Users, Indicates the maximum transmission power of the drone;

[0136] Create the third constraint:

[0137] ;

[0138] in: Indicates user k The communication bandwidth Indicated in n Time slot users k The received signal-to-noise ratio at Indicates the user's minimum QoS requirement, QoS indicates the user's minimum service quality;

[0139] Establish the fourth constraint:

[0140] ;

[0141] ;

[0142] ;

[0143] ;

[0144] ;

[0145] in: Indicated in n The remaining battery power of the drone in each time slot, Indicates the initial power of the drone. Indicates the maximum remaining power of the drone; SoC indicates the battery power status, SoH indicates the battery health status, SoC ( t ) indicates that the drone is at time t Battery state of charge, SoH( t ) indicates that the drone is at time t Battery health status;

[0146] Z Represents the test function of SoC: ,when hour, The value of ,otherwise The value of , Represents a constant greater than 1, Indicates the drone battery discharge coefficient, Indicates i The solar charging capacity of the drone in each time slot, Indicates i The power consumption of the drone flight in each time slot is Indicates iPower consumption of drone communication in each time slot;

[0147] Indicated in The remaining battery power of the drone in each time slot; Indicates that the drone is at time Battery health status, Indicates that the drone is at time Battery charge status;

[0148] Based on Markov decision making, the target mapping value is optimized to an observable virtual space, including:

[0149] Convert the optimization objective into a tuple: ;

[0150] in: represents the state space, represents the action space, represents the reward function, represents the state transition probability, represents the discount factor;

[0151] Time slot n When , the state space is expressed as ; ;

[0152] in: Indicates that the drone is n The location of the time slot;

[0153] There are a total of K terrestrial users, Indicates k User in n The location of the time slot;

[0154] Indicated in n The remaining battery power of the drone in each time slot;

[0155] In the past, drones n -The maximum downlink speed obtained in 1 time slot, , Indicates that the drone is i Downlink speed of each time slot;

[0156] No. i When there are time slots, the action space is expressed as ; ;

[0157] in: Indicates that the drone is nThe horizontal speed of each time slot is Indicates that the drone is n The horizontal angle of each time slot is Indicates that the drone is n The vertical speed of the time slot is Indicates n time slots are allocated to users K The transmission power of

[0158] No. n When there are time slots, the reward function is expressed as ; ;

[0159] in: It means the penalty for the drone flying out of the service area. Representation and The corresponding weight coefficient; Indicates the penalty for violating the user service quality. Representation and The corresponding weight coefficient; represents the penalty for violating the battery aging constraint, Representation and The corresponding weight coefficient; Indicates that the user is n The sum of the downlink rates in time slots, , Indicated in n Time slot users k The instantaneous transmission rate obtained is Representation and The corresponding weight coefficient; Indicates the first reward, , In the past, drones n -The maximum downlink speed obtained in 1 time slot, Representation and The corresponding weight coefficient; Indicates the second reward, , , Indicates that the drone is n The distance moved by each time slot, Indicates the first time slot, Representation and The corresponding weight coefficient.

[0160] Determine and train the drone data transmission model based on the optimization objectives mapped into the observable virtual space, specifically:

[0161] like Figure 4As shown, a SoftActor-Critic network is used to train a UAV data transmission model; wherein: the SoftActor-Critic network includes an Actor network, a Critic network and an experience replay pool; the Critic network includes a first main Q network, a second main Q network, a first target Q network, a second target Q network and an experience replay pool; the Actor network is used to sample the current state of the UAV and generate the current action; the experience replay pool stores training samples generated based on the current action; the target Q network updates the parameters of the main Q network in a soft update manner, and the use of two main Q networks and two target Q networks can reduce the overestimation of the Q value and increase the stability of training; the SoftActor-Critic network is used to train a UAV data transmission model, including the following steps:

[0162] 1.1. Get time slot n Current state of the drone data transmission model ;

[0163] 1.2. According to the current status , determine the drone action under the current strategy , Execute actions Corresponding rewards ;

[0164] 1.3. Flag to determine whether to terminate the drone action ;

[0165] 1.4. Deposit into the experience pool and update the current status ;

[0166] 1.5. Sampling from the Experience Pool training samples , use the sampled training samples to update the network parameters of the Actor network and the main Q network, including:

[0167] The network parameters of the main Q network are updated using the following formula:

[0168] ;

[0169] Use the following formula to update the network parameters of the Actor network:

[0170] ;

[0171] in: represents the loss function used to update the network parameters of the main Q network; Represents the loss function used to update the network parameters of the Actor network; Indicates jThe output value of the target Q network; represents the temperature coefficient; Indicates i The state value of the training sample, Indicates i The action value of each training sample; Represents the reparameterized sampling to obtain i The action value of each training sample; Represents the network parameters of the Actor network; represents the network parameters of the target Q network; Represents the probability distribution of the drone taking actions under state .

[0172] The target Q network updates the main Q network parameters using soft update:

[0173] ;

[0174] represents the network parameters of the main Q network; represents the soft update coefficient;

[0175] Indicates the TD Target value. ;

[0176] Indicates i+ The state value of a training sample, Indicates i+ Action value of 1 training sample; represents the discount factor, Indicates i The reward for each training sample.

[0177] 1.6. When the total duration of the drone communication mission is reached or the battery is not working within the limited SoC range, stop updating the network parameters, otherwise execute step 1.1.

[0178] The simulation parameters are set as follows: the flight area of ​​the drone is a 1km×1km cell, the total duration of the drone's communication mission is T=60s, the maximum transmit power is 30dBm, the total bandwidth of the system communication is 5MHz, there are 5 users on the ground doing random movements, each user is allocated a bandwidth of 1MHz, the user's minimum service quality requirement is 5Mbit / s, the drone's flight starting point is (100, 100, 200), the drone's minimum flight altitude is 100m, the maximum flight altitude is 300m, the maximum horizontal speed is 50m / s, and the maximum vertical speed is 6m / s;

[0179] like Figure 5 and Figure 6As shown in the figure, under different user distributions, after adding the battery aging constraint, the system throughput obtained by user distribution 1, user distribution 2, user distribution 3 and user distribution 4 is reduced. This is because the battery aging constraint limits the discharge current of the drone, so that the drone cannot fly to the served user at a faster speed. Although a higher system throughput is obtained without the battery aging constraint, this performance improvement is at the expense of the battery health state. When the drone is not constrained by battery aging, the remaining energy of user distribution 1, user distribution 2, user distribution 3 and user distribution 4 is less than the remaining energy with the battery aging constraint. According to the relationship between the battery health state SoH and the power state SoC, it can be seen that within the time, the more the battery power is consumed, the faster the battery health state will decline. Therefore, adding the battery aging constraint can effectively manage the drone battery health state.

[0180] Example 2

[0181] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the following method:

[0182] Obtain the user location information of the drone service area, the channel information between the target drone and the user, and the real-time power of the target drone;

[0183] The user location information of the drone service area, the channel information between the target drone and the user, and the real-time power of the target drone are input into the pre-trained drone data transmission model to obtain the optimal transmission power and optimal flight path that enables the drone to allocate the maximum downlink speed to the user at the minimum battery capacity decay rate;

[0184] According to the optimal transmission power and the optimal flight path, the target UAV is controlled to transmit data.

[0185] Among them, the training of the pre-trained drone data transmission model includes:

[0186] Based on the user's location information, the drone's battery information, and the channel information between the drone and the user, a working environment model between the drone and the user is constructed;

[0187] Based on the working environment model, the optimal transmission power and optimal flight path for allocating the maximum downlink speed of the user under the minimum battery capacity decay rate are taken as the optimization goal to determine the UAV data transmission model;

[0188] Train the UAV data transmission model to obtain a trained UAV data transmission model.

[0189] Example 3

[0190] A computer device comprising:

[0191] A memory for storing instructions;

[0192] The processor is used to execute instructions so that the device performs the steps of implementing the following method:

[0193] Obtain the user location information of the drone service area, the channel information between the target drone and the user, and the real-time power of the target drone;

[0194] The user location information of the drone service area, the channel information between the target drone and the user, and the real-time power of the target drone are input into the pre-trained drone data transmission model to obtain the optimal transmission power and optimal flight path that enables the drone to allocate the maximum downlink speed to the user at the minimum battery capacity decay rate;

[0195] According to the optimal transmission power and the optimal flight path, the target UAV is controlled to transmit data.

[0196] Among them, the training of the pre-trained drone data transmission model includes:

[0197] Based on the user's location information, the drone's battery information, and the channel information between the drone and the user, a working environment model between the drone and the user is constructed;

[0198] Based on the working environment model, the optimal transmission power and optimal flight path for allocating the maximum downlink speed of the user under the minimum battery capacity decay rate are taken as the optimization goal to determine the UAV data transmission model;

[0199] Train the UAV data transmission model to obtain a trained UAV data transmission model.

[0200] In the description of the present invention, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", etc. may explicitly or implicitly include one or more of the features.

[0201] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0202] The present application is described with reference to the flowcharts of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process in the flowchart can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0203] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A function specified in a process or multiple processes.

[0204] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 The steps of a specified function in a process or multiple processes.

[0205] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A solar-powered drone data transmission method, characterized in that: include: Obtain the user location information of the drone service area, the channel information between the target drone and the user, and the real-time power of the target drone; The user location information of the drone service area, the channel information between the target drone and the user, and the real-time power of the target drone are input into the pre-trained drone data transmission model to obtain the optimal transmission power and optimal flight path that enables the drone to allocate the maximum downlink speed to the user at the minimum battery capacity decay rate; According to the optimal transmission power and the optimal flight path, the target UAV is controlled to transmit data; wherein the training of the pre-trained UAV data transmission model includes: Based on the user's location information, the drone's battery information, and the channel information between the drone and the user, a working environment model between the drone and the user is constructed; Based on the working environment model, the optimal transmission power and optimal flight path for allocating the maximum downlink speed of users under the minimum battery capacity decay rate of the UAV are taken as the optimization goals to determine and train the UAV data transmission model.

2. The solar-powered drone data transmission method according to claim 1, characterized in that: The method of constructing a working environment model between the drone and the user based on the user location information, the drone power information, and the channel information between the drone and the user includes: The working environment model includes a drone network model, a drone channel model and a communication energy model; The UAV communication network model is established by the following formula: ； ； Among them: the total time of the drone performing the communication task T Divide into N equal time slots, each of which is ; Indicates that the drone is n The position of the time slot, Indicates that the drone is n time slot x Axis coordinates, Indicates that the drone is n time slot y Axis coordinates, Indicates that the drone is n time slot z Axis coordinates; Indicates that the drone is n The horizontal speed of each time slot is Indicates that the drone is n The horizontal angle of each time slot is Indicates that the drone is n The vertical speed of each time slot; Indicates that the drone is time slot x Axis coordinates, Indicates that the drone is time slot y Axis coordinates, Indicates that the drone is time slot z Axis coordinates; There are a total of K terrestrial users, Indicates k User in n The position of the time slot, Indicates k User in n time slot x Axis coordinates, Indicates k User in n time slot y Axis coordinates; The UAV channel model is established by the following formula: ； in: represents the average path loss from user to UAV; Indicates the distance from the user to the drone LoS Path loss; Represents the user and the drone LoS Probability of communication; Indicates the distance from the user to the drone NLoS Path loss; Represents the user and the drone NLoS Probability of communication; Indicated in n Time slot users k The instantaneous transmission rate obtained is Indicates user k The communication bandwidth Indicated in n Time slot users k The received signal-to-noise ratio at The communication energy model is established by the following formula: ； in: Represents the solar power generation power, represents the energy conversion efficiency, represents the area of ​​the drone's solar panels, represents the average solar radiation on Earth, represents the atmospheric transmittance, represents the atmospheric extinction coefficient, represents the proportional height of the Earth, z Indicates the flight altitude of the drone; represents the UAV communication power consumption, Indicates n time slots are allocated to users k The transmission power, K Indicates that there are a total of K terrestrial users, k Indicates k Users; Indicates the UAV flight power consumption, Indicates the profile power of the drone blades in hovering state. Indicates the air density, s Indicates rotor strength, represents the rotor disc area, It represents the average rotor induced speed when the drone is hovering. represents the tip speed of the UAV's rotor blades, represents the induced power, Indicates the weight of the drone, Represents the drag ratio of the drone body.

3. The solar-powered drone data transmission method according to claim 1, characterized in that: Based on the working environment model, the optimal transmission power and optimal flight path for allocating the maximum downlink speed of the user to the UAV at the minimum battery capacity decay rate are taken as the optimization goal, and the UAV data transmission model is determined and trained, including: Based on the working environment model, the optimization objective is to allocate the maximum downlink speed of the user under the minimum battery capacity decay rate, and the constraint conditions of the optimization objective are constructed; Based on Markov decision making, the optimization objective is mapped to an observable virtual space; Determine and train a UAV data transmission model based on an optimization objective mapped into an observable virtual space.

4. The solar-powered drone data transmission method according to claim 3, characterized in that: Based on the working environment model, the optimization objective is to allocate the maximum downlink speed of the user under the minimum battery capacity decay rate of the UAV, and the constraint conditions of the optimization objective are constructed, including: Establish the first constraint: ； ； ； ； in: D Indicates the range of the drone service area. Indicates the maximum speed of the drone in the horizontal direction. Indicates the maximum speed of the drone in the vertical direction, Indicates that the drone is n time slot x Axis coordinates, Indicates that the drone is n time slot y Axis coordinates, Indicates that the drone is n time slot z Axis coordinates; Indicates that the drone is n The horizontal speed of each time slot is Indicates that the drone is n The horizontal angle of each time slot is Indicates that the drone is n The vertical speed of the time slot is represents pi; Create the second constraint: ； ； in: Indicates n time slots are allocated to users k The transmission power, K Indicates that there are a total of K terrestrial users, k Indicates k Users, Indicates the maximum transmission power of the drone; Create the third constraint: ； in: Indicates user k The communication bandwidth Indicated in n Time slot users k The received signal-to-noise ratio at Indicates the user's minimum QoS requirement, QoS indicates the user's minimum service quality; Establish the fourth constraint: ； ； ； ； ； in: Indicated in n The remaining battery power of the drone in each time slot, Indicates the initial power of the drone. Indicates the maximum remaining power of the drone; SoC indicates the battery power status, SoH indicates the battery health status, SoC ( t ) indicates that the drone is at time t Battery state of charge, SoH( t ) indicates that the drone is at time t Battery health status; Z represents the test function of SoC, Indicates the drone battery discharge coefficient, Indicates i The solar charging capacity of the drone in each time slot, Indicates i The power consumption of the drone flight in each time slot is Indicates i Power consumption of drone communication in each time slot; Indicated in The remaining battery power of the drone in each time slot; Indicates that the drone is at time Battery health status, Indicates that the drone is at time battery charge status.

5. The solar-powered drone data transmission method according to claim 3, characterized in that: The Markov decision-making method maps the optimization target to an observable virtual space, including: Convert the optimization objective into a tuple: ； in: represents the state space, represents the action space, represents the reward function, represents the state transition probability, represents the discount factor; No. n When there are time slots, the state space is expressed as ; ； in: Indicates that the drone is n The location of the time slot; There are a total of K terrestrial users, Indicates k User in n The location of the time slot; Indicated in n The remaining battery power of the drone in each time slot; In the past, drones n -The maximum downlink speed obtained in 1 time slot, n When there are time slots, the action space is expressed as ; ； in: Indicates that the drone is n The horizontal speed of each time slot is Indicates that the drone is n The horizontal angle of each time slot is Indicates that the drone is n The vertical speed of the time slot is Indicates n time slots are allocated to users K The transmission power of No. n When there are time slots, the reward function is expressed as ; ； in: It means the penalty for the drone flying out of the service area. Representation and The corresponding weight coefficient; Indicates the penalty for violating the user service quality. Representation and The corresponding weight coefficient; represents the penalty for violating the battery aging constraint, Representation and The corresponding weight coefficient; Indicates that the user is n The sum of the downlink rates in time slots, Representation and The corresponding weight coefficient; Indicates the first reward, Representation and The corresponding weight coefficient; Indicates the second reward, Representation and The corresponding weight coefficient.

6. The solar-powered drone data transmission method according to claim 3, characterized in that: The method of determining and training a drone data transmission model according to an optimization target mapped into an observable virtual space includes: Use SoftActor-Critic network to train the UAV data transmission model; Among them, the SoftActor-Critic network includes an Actor network, two main Q networks, two target Q networks, and an experience replay pool; The Actor network is used to sample the current state of the drone and generate the current action; The experience replay pool stores training samples generated based on the current action; The target Q network updates the parameters of the main Q network using a soft update method.

7. The solar-powered drone data transmission method according to claim 6, characterized in that: The method of using the SoftActor-Critic network to train the drone data transmission model includes the following steps: 1.

1. Get time slot n Current state of the drone data transmission model ; 1.

2. According to the current status , determine the drone action under the current strategy , Execute actions Corresponding rewards ; 1.

3. Flag to determine whether to terminate the drone action ; 1.

4. Deposit into the experience pool and update the current status ; 1.

5. Sampling from the Experience Pool training samples , use the sampled training samples to update the network parameters of the Actor network and the main Q network; 1.

6. When the total duration of the drone communication mission is reached or the battery is not working within the limited SoC range, stop updating the network parameters, otherwise execute step 1.

1.

8. The solar-powered drone data transmission method according to claim 7, characterized in that: The updating of the network parameters of the Actor network, the target Q network and the main Q network using the sampled training samples includes: The network parameters of the main Q network are updated using the following formula: ； Use the following formula to update the network parameters of the Actor network: ； in: represents the loss function used to update the network parameters of the main Q network; Represents the loss function used to update the network parameters of the Actor network; Indicates j The output value of the target Q network; represents the temperature coefficient; Indicates i The state value of the training sample, Indicates i The action value of each training sample; Re-parameterized sampling is obtained i The action value of each training sample; Represents the network parameters of the Actor network; represents the network parameters of the target Q network; Indicates that the drone is in status Make the following move The probability distribution of The target Q network updates the main Q network parameters using soft update: ； represents the network parameters of the main Q network; represents the soft update coefficient; Indicates the TD Target value. ; Indicates i+ The state value of a training sample, Indicates i+ Action value of 1 training sample; represents the discount factor, Indicates i The reward for each training sample.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the solar-powered UAV data transmission method according to any one of claims 1 to 8 is implemented.

10. A computer device, characterized in that: include: A memory for storing instructions; A processor is used to execute the instructions so that the device implements the solar drone data transmission method as described in any one of claims 1 to 8.

Images