Energy efficiency oriented multi-unmanned aerial vehicle assisted power transfer optimization method

Abstract

The application discloses an energy-efficiency-oriented multi-unmanned aerial vehicle (UAV) assisted energy transfer optimization method, which is applied to a relay UAV in a downlink communication system. The method comprises the following steps: determining relay communication parameters of each UAV in the downlink communication system, so that an energy efficiency value is maximum, wherein the relay communication parameters comprise a power splitting ratio, a time slot allocation ratio and a transmission power; the energy efficiency value represents a ratio of a sum of information throughputs of all links in a unit time slot to a sum of consumed energies of all UAVs; signals transmitted by a base station to the UAVs in a wireless energy transfer mode are received; for each UAV, energy and information are separated from the signals based on the relay communication parameters corresponding to the UAV, energy is supplemented, information is transmitted to a predetermined user cluster, and the user cluster comprises at least one user terminal in a coverage range. The energy efficiency of the UAVs can be significantly improved, and the global performance of the UAVs in the downlink communication system is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer technology, and particularly relates to a multi-unmanned aerial vehicle (UAV) assisted energy transfer optimization method oriented to energy efficiency. BACKGROUND

[0002] Unmanned aerial vehicles (UAVs) have become an ideal platform for constructing emergency communication networks due to their high mobility, rapid deployment, and low cost. By deploying UAV relay nodes, communication obstacles caused by complex terrains (such as mountains and building obstructions) can be effectively overcome, and flexible line-of-sight (LoS) communication links can be provided for ground users.

[0003] The limited battery energy of UAVs is a core challenge to achieve persistent and stable communication services. Simultaneous wireless information and power transfer (SWIPT) technology can simultaneously perform information transmission and wireless energy transmission using the same radio frequency signal, which can ensure communication quality and alleviate the endurance problem of UAVs, providing an effective way to solve the energy limitation of UAVs. However, this approach is heavily dependent on the quality of the energy transfer scheme.

[0004] Currently, the optimization design of the energy transfer scheme for UAVs often unilaterally pursues the maximization of transmission rate or the extension of endurance time. This single-index optimization method fails to fully consider and effectively balance the inherent coupling relationship between communication service quality and system energy consumption, resulting in poor overall performance of UAVs in the communication system. SUMMARY

[0005] In view of this, the purpose of the present application is to provide a multi-unmanned aerial vehicle (UAV) assisted energy transfer optimization method oriented to energy efficiency.

[0006] To achieve the above purpose, the present application adopts the following technical solutions:

[0007] The application provides an energy efficiency-oriented multi-unmanned aerial vehicle (UAV) assisted energy transfer optimization method, which is applied to a relay UAV in a downlink communication system. The method comprises the following steps: determining relay communication parameters of each UAV in the downlink communication system, so that an energy efficiency value is maximum, wherein the relay communication parameters comprise a power splitting ratio, a time slot allocation ratio and a transmission power; the energy efficiency value represents a ratio of a sum of information throughputs of all links in a unit time slot to a sum of energies consumed by all UAVs; a signal transmitted by a base station to the UAVs in a wireless energy transfer manner is received; for each UAV, energy and information are separated from the signal based on the relay communication parameters corresponding to the UAV, the energy is supplemented, and the information is transmitted to a predetermined user cluster, wherein the user cluster comprises at least one user terminal in a coverage range.

[0008] In some embodiments, the energy efficiency value is determined by the following method: determining energy consumed by the UAVs in a unit time slot based on energy consumed by the UAVs for communication with the user terminals in the user cluster in the unit time slot, energy consumed by the UAVs for maintaining a hovering state in the unit time slot and energy captured by the UAVs from the signal in the unit time slot; determining information throughputs of the UAVs for communication with the user terminals in the unit time slot based on a downlink signal transmission rate of the UAVs in the unit time slot and a downlink signal reception rate of each user terminal in the user cluster corresponding to the UAV; and taking a ratio of a sum of the information throughputs of all the UAVs for communication with the user terminals in the unit time slot to a sum of the energies consumed by all the UAVs in the unit time slot as the energy efficiency value.

[0009] In some embodiments, the sum of the energies consumed by all the UAVs in the unit time slot is determined based on the following formula:

[0010]

[0011] In the formula, E total represents the sum of the energies consumed by all the UAVs in the unit time slot, M represents a number of the UAVs, represents energy consumed by the UAV m for communication with the user terminals in the user cluster in the unit time slot, represents energy consumed by the UAV m for maintaining a hovering state in the unit time slot, represents energy captured by the UAV m from the signal in the unit time slot.

[0012] In some embodiments, the sum of the information throughputs of all the UAVs for communication with the user terminals in the unit time slot is determined based on the following formula:

[0013]

[0014] In the formula, R totalLet M represent the sum of the information throughput of all UAVs communicating with the user terminal within a unit time slot, where M represents the number of UAVs, τ represents the time slot allocation ratio, and l represents the length of a unit time slot. This represents the downlink signal transmission rate of the drone m. This represents the downlink signal reception rate of UAV m when it transmits a signal to user terminal n through subchannel k in time slot t; K represents the number of subchannels; N represents the downlink signal reception rate of UAV m. m This represents the number of user terminals within the user cluster corresponding to drone m.

[0015] In some embodiments, determining the relay communication parameters of each UAV in the downlink communication system includes: determining constraints, including numerical constraints on the relay communication parameters, energy constraints on each UAV, and link rate constraints, wherein the numerical constraints include: the power splitting ratio being in a first value range, the time slot allocation ratio being in a second value range, and the transmission power being in a third value range; the energy constraints include: the energy captured by the UAV in a time slot is not less than the energy consumed; the link rate constraints include: the downlink information transmission rate of the UAV is not less than a first threshold, and the downlink information reception rate of the user terminal is not less than a second threshold; constructing the state space of a Markov chain based on the captured energy, consumed energy, downlink information transmission rate, and downlink information reception rate; constructing the action space of the Markov chain based on the power splitting ratio, time slot allocation ratio, and transmission power; constructing the reward function of the Markov chain based on the energy efficiency value, energy constraints, and link rate constraints; and determining the relay communication parameters based on the Markov chain such that the relay communication parameters satisfy the constraints and the energy efficiency value is maximized.

[0016] In some embodiments, a reward function for a Markov chain is constructed based on an energy efficiency value, an energy constraint, and a link rate constraint, including: using the energy efficiency value as a first reward term; determining a second reward term based on the energy constraint, wherein the second reward term is positively correlated with the number of drones that satisfy the energy constraint; determining a third reward term based on the link rate constraint, wherein the third reward term is positively correlated with the number of links that satisfy the link rate constraint; and determining a reward function based on a weighted sum of the first, second, and third reward terms.

[0017] In some embodiments, the reward function is determined by the following formula:

[0018] r(t)=w1ξ EE (t)+w2r E (t)+w3r R (t)

[0019] In the formula, w1, w2, and w3 represent weights; ξ EE (t) represents the energy efficiency value; rE (t) represents a second reward term; r R (t) represents a third reward term; wherein,

[0020] r E (t) = delta M

[0021]

[0022] In the formula, delta and lambda represent positive rewards, and M represents the number of unmanned aerial vehicles satisfying the energy constraint condition; represents the downlink signal receiving rate of the unmanned aerial vehicle m when sending signals to the user terminal n through the subchannel k at the time slot t; represents the downlink signal transmission rate of the unmanned aerial vehicle m; K represents the number of subchannels; N m represents the number of user terminals in the user cluster corresponding to the unmanned aerial vehicle m.

[0023] In some embodiments, the relay communication parameters are determined based on a Markov chain, including: solving the Markov chain by using a deep reinforcement learning algorithm to obtain the relay communication parameters.

[0024] In some embodiments, based on the relay communication parameters, the energy and information are separated from the signal, the energy is supplemented with the power, and the information is sent to the predetermined user cluster, including: for any time slot, the time slot is divided into a first interval and a second interval based on a time slot allocation ratio; in the first interval, the energy and information are separated from the signal based on a power splitting ratio, and the energy is transmitted to the battery; in the second interval, the information is sent to the user cluster according to the transmission power.

[0025] In some embodiments, before determining the relay communication parameters of each unmanned aerial vehicle in the downlink communication system, the method further includes: clustering all user terminals, determining a user cluster corresponding to each unmanned aerial vehicle, and allocating a subchannel to each user terminal in the user cluster, different user terminals corresponding to different subchannels.

[0026] The energy efficiency-oriented multi-unmanned aerial vehicle assisted energy transfer optimization method provided by the application has the following beneficial effects:

[0027] The method provided by the application adopts a multi-unmanned aerial vehicle assisted relay transmission network, improves the cooperative coverage gain; through the dynamic energy management strategy of wireless energy transfer, the unmanned aerial vehicle can capture energy from the signal sent by the base station to cover the energy consumption of continuous flight and communication, and realizes long-time stable communication service; and the power splitting ratio, time slot allocation ratio and transmission power when the energy efficiency value is maximum are adopted for signal splitting, energy capture and signal sending and other operations, which can significantly improve the energy efficiency of the unmanned aerial vehicle and help improve the overall performance of the unmanned aerial vehicle in the downlink communication system. BRIEF DESCRIPTION OF DRAWINGS

[0028] The above and other objects, features and advantages of the present application will become more apparent from the following description when taken in conjunction with the accompanying drawings, in which:

[0029] Figure 1 is a flowchart of an embodiment of the energy-efficiency-oriented multi-UAV assisted energy-carried transmission optimization method according to the present application;

[0030] Figure 2 is a schematic diagram of the correspondence between the UAVs and the user clusters in an embodiment of the energy-efficiency-oriented multi-UAV assisted energy-carried transmission optimization method according to the present application;

[0031] Figure 3 is a flowchart of the determination of the energy efficiency value in an embodiment of the energy-efficiency-oriented multi-UAV assisted energy-carried transmission optimization method according to the present application;

[0032] Figure 4 is a schematic diagram of the principle of the relay communication in an embodiment of the energy-efficiency-oriented multi-UAV assisted energy-carried transmission optimization method according to the present application;

[0033] Figure 5 is a flowchart of the determination of the relay communication parameters in an embodiment of the energy-efficiency-oriented multi-UAV assisted energy-carried transmission optimization method according to the present application;

[0034] Figure 6 is a flowchart of the determination of the reward function in an embodiment of the energy-efficiency-oriented multi-UAV assisted energy-carried transmission optimization method according to the present application;

[0035] Figure 7a is a schematic diagram of the convergence of the determination of the reward curve in an embodiment of the energy-efficiency-oriented multi-UAV assisted energy-carried transmission optimization method according to the present application;

[0036] Figure 7b is a schematic diagram of the convergence of the determination of the energy efficiency in an embodiment of the energy-efficiency-oriented multi-UAV assisted energy-carried transmission optimization method according to the present application;

[0037] Figure 8 is a schematic diagram of the structure of an electronic device suitable for implementing an embodiment of the present application. DETAILED DESCRIPTION

[0038] The present application is described below based on embodiments, and it should be understood by those of ordinary skill in the art that the drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0039] "comprise," "containing" etc. in the entire specification and claims should be construed to be inclusive, rather than exclusive or exhaustive; that is, in a meaning equivalent to "including, but not limited to."

[0040] Figure 1 A flow chart of one embodiment of the energy efficiency oriented multi-UAV assisted energy transfer optimization method according to the present application is shown. The flow includes the following steps:

[0041] Step 110, determining the relay communication parameters of each UAV in the downlink communication system, so that the energy efficiency value is maximum.

[0042] Wherein, the relay communication parameters include power splitting ratio, time slot allocation ratio and transmission power; the energy efficiency value represents the ratio of the sum of information throughputs of all links in a unit time slot to the sum of energy consumed by all UAVs.

[0043] In this embodiment, the downlink communication system can include a base station for transmitting signals, a plurality of UAVs as relay nodes, and a user end for receiving signals. The UAVs as relay nodes can receive signals transmitted by the base station, and separate energy for supplementing their own power and information for forwarding to the user end from the signals.

[0044] The base station can provide communication services for a plurality of user ends. According to the coverage range of each UAV, all user ends can be divided into a plurality of user clusters, and each UAV can provide communication services for the user cluster within the coverage range. The user end can include an IoT (Internet of Things) device, and can also include a mobile communication device such as a mobile phone.

[0045] In some optional embodiments of this embodiment, before determining the relay communication parameters of each UAV in the downlink communication system, it can further include: step 100, clustering all user ends, determining the user cluster corresponding to each UAV, and assigning a subchannel to each user end in the user cluster. Wherein, different user ends correspond to different subchannels.

[0046] In combination Figure 2 For example, the K-means algorithm can be used to cluster the user ends, a plurality of user ends located within the coverage range of a UAV are determined as the user cluster of the UAV, and a plurality of user clusters as shown in the figure are obtained. Figure 2 Each user cluster corresponds to a UAV.

[0047] In this embodiment, by clustering the user ends, the user cluster corresponding to each UAV is obtained, and an independent subchannel is assigned to each user end, which helps to improve the communication quality of the UAV.

[0048] Step 120: Use the drone to receive the signal transmitted by the base station in a wireless power transmission mode.

[0049] Step 130: For each UAV, based on the relay communication parameters corresponding to that UAV, separate energy and information from the signal, use the energy to replenish the power, and send the information to the corresponding user cluster.

[0050] The user cluster includes at least one user terminal within the coverage area.

[0051] In some optional embodiments of this example, step 130 may further include: for any time slot, dividing the time slot into a first interval and a second interval based on the time slot allocation ratio; in the first interval, separating energy and information from the signal based on the power split ratio, and transmitting the energy to the battery; in the second interval, transmitting the information to the user cluster according to the transmission power.

[0052] As an example, the execution entity of the energy efficiency-oriented multi-UAV assisted energy transmission optimization method in this embodiment can be a server of the downlink communication system. The server can communicate with the UAV to obtain the UAV's parameters (such as position parameters, performance parameters, etc.), determine the UAV's relay communication parameters based on the parameters, and then send the relay communication parameters to the UAV to control the UAV to operate according to the relay communication parameters.

[0053] The energy efficiency-oriented multi-UAV assisted power transmission optimization method in this embodiment adopts a multi-UAV assisted relay transmission network to improve cooperative coverage gain. Through the dynamic energy management strategy of wireless power transmission, the UAV can capture energy from the signal sent by the base station to cover the energy consumption of continuous flight and communication, thus realizing long-term stable communication service. Furthermore, by using the power splitting ratio, time slot allocation ratio and transmission power when the energy efficiency value is maximized, the operation of signal splitting, energy capture and signal transmission can significantly improve the energy efficiency of the UAV and help improve the overall performance of the UAV in the downlink communication system.

[0054] The following is for reference. Figure 3 ,exist Figure 3 In the illustrated embodiment, the energy efficiency value can be determined by the following steps.

[0055] Step 310: Based on the energy consumption of the UAV communicating with the user terminal in the user cluster within a unit time slot, the energy consumption of the UAV maintaining a hovering state within a unit time slot, and the energy captured by the UAV from the signal within a unit time slot, determine the energy consumed by the UAV within a unit time slot.

[0056] Step 320, determining the information throughput of the UAV in a unit time slot for communication with the user terminal based on the downlink signal transmission rate of the UAV in the unit time slot and the downlink signal reception rate of each user terminal in the user cluster corresponding to the UAV.

[0057] Step 330, taking the ratio of the sum of the information throughput of all UAVs in a unit time slot for communication with the user terminal and the sum of the energy consumed by all UAVs in a unit time slot as the energy efficiency value.

[0058] In the embodiment, the energy efficiency value can be determined according to the information throughput of all UAVs in a unit time slot for communication with the user terminal and the energy consumed, thereby representing the global performance of the UAV in the downlink communication system, and the energy constraint can be provided for the optimization problem of multi-UAV assisted energy transfer, and the relay communication parameters of the UAV determined in this way can improve the global performance of the UAV in the downlink communication system.

[0059] In some optional embodiments of the embodiment, the sum of the energy consumed by all UAVs in a unit time slot can be determined based on the following formula (1):

[0060]

[0061] In the formula, E total represents the sum of the energy consumed by all UAVs in a unit time slot, M represents the number of UAVs, represents the energy consumed by the UAV m in a unit time slot for communication with the user terminal in the user cluster, represents the energy consumed by the UAV m in a unit time slot for hovering, represents the energy captured by the UAV m in a unit time slot from the signal.

[0062] In some optional embodiments of the embodiment, the sum of the information throughput of all UAVs in a unit time slot for communication with the user terminal is determined based on the following formula (2):

[0063]

[0064] In the formula, R total represents the sum of the information throughput of all UAVs in a unit time slot for communication with the user terminal, M represents the number of UAVs, τ represents the time slot allocation ratio, and l represents the length of a unit time slot, represents the downlink signal transmission rate of the UAV m, represents the downlink signal reception rate of the UAV m when sending a signal to the user terminal n through the subchannel k at time slot t, and K represents the number of subchannels; N m represents the number of user terminals in the user cluster corresponding to the UAV m.

[0065] Based on equations (1) and (2), the energy efficiency value ξ can be obtained. EE As shown in equation (3).

[0066]

[0067] The following is combined with Figure 4 The calculation methods for each parameter in the above formula are illustrated by example. For example... Figure 4 As shown, assume the total service time of the drone is T. o , will T o The system is divided into T equal-length time slots, each with a time slot length of l, forming a time slot set T = {1, 2, ..., T}. Each time slot is further divided into two intervals according to a time slot allocation ratio τ. In the first interval, the base station transmits information and energy to the UAV via power-carrying transmission for a duration of τl. The UAV splits the received signal into two parts: energy and information. The power splitting ratio of UAV m is 0 < ρ. m <1, ρ m Part of the signal is used for information decoding, 1-ρ m Part of the signal is used for energy capture. In the second interval, the UAV forwards the decoded information to the user terminal for a duration of (1-τ)l.

[0068] During the process of the UAV receiving signals from the base station, ignoring the noise generated during the information decoding process and only considering channel noise, the signal-to-noise ratio (SNR) of the signal received by the UAV m for the information decoding part is:

[0069]

[0070] Among them, P 0，m It is the transmission power of the base station to the drone m. Let represent the additive white Gaussian noise power of the channel from the base station to the UAV. The downlink signal transmission rate of UAV m is:

[0071]

[0072] in, α0 represents the unit path loss. and Let m represent the distance from the base station to the drone m and the distance from the drone m to the user terminal n, respectively.

[0073] According to the linear energy capture model, the energy captured by UAV m is: Where η represents the energy conversion efficiency.

[0074] The corresponding signal to interference plus noise ratio (SINR) is:

[0075]

[0076] wherein, is the transmission power of the UAV m allocated to the user terminal n, denotes that the UAV m serves the user terminal n through the subchannel k, is the power of the out-of-cluster interference signal, is the additive white Gaussian noise power of the channel from the UAV m to the user terminal n. The downlink signal reception rate of the user terminal n is:

[0077]

[0078] wherein,

[0079] The total energy consumption of the UAV m in communication with the user terminals in the user cluster is: wherein, N m is the number of user terminals in the user cluster served by the UAV m. The energy consumed by the UAV m in maintaining the hovering state in the time slot t is: wherein, denotes the hovering power of the UAV m.

[0080] Based on the above parameters, the equations (1) and (2) can be solved, and the energy efficiency value can be obtained through the equation (3).

[0081] Figure 5 An embodiment of the energy efficiency-oriented multi-UAV assisted energy transfer optimization method of the present application is shown in the flowchart for determining the relay communication parameters, as shown in Figure 5 The flowchart can include the following steps.

[0082] Step 510, determining the constraint conditions.

[0083] The constraint conditions include the numerical constraint conditions of the relay communication parameters, the energy constraint conditions of each UAV and the link rate constraint conditions, wherein the numerical constraint conditions include that the power splitting ratio is located in a first value interval, the time slot allocation ratio is located in a second value interval, and the transmission power is located in a third value interval; the energy constraint conditions include that the energy captured by the UAV in a time slot is not less than the energy consumed; and the link rate constraint conditions include that the downlink information transmission rate of the UAV is not less than a first threshold value, and the downlink information reception rate of the user terminal is not less than a second threshold value.

[0084] Step 520, constructing a state space of the Markov chain based on the captured energy, the consumed energy, the downlink information transmission rate and the downlink information reception rate.

[0085] Step 530, constructing an action space of the Markov chain based on the power splitting ratio, the time slot allocation ratio and the transmission power.

[0086] Step 540, constructing a reward function of the Markov chain based on the energy efficiency value, the energy constraint condition and the link rate constraint condition.

[0087] Step 550, determining the relay communication parameter based on the Markov chain, so that the relay communication parameter satisfies the constraint condition and the energy efficiency value is maximum.

[0088] As an example, the constraint condition can be expressed as:

[0089]

[0090] C2: 0 < τ < 1

[0091]

[0092] wherein the power splitting ratio ρ = {ρ m | m ∈ M}, the transmission power M represents the number of unmanned aerial vehicles, N represents the number of user terminals in the user cluster corresponding to the unmanned aerial vehicle m, ρ m represents the power splitting ratio of the unmanned aerial vehicle m, τ represents the time slot allocation ratio, represents the transmission power of the unmanned aerial vehicle m when transmitting information to the user terminal n, represents the energy consumed by the unmanned aerial vehicle m when communicating with the user terminals in the user cluster in a unit time slot, represents the energy captured by the unmanned aerial vehicle m from the signal in a unit time slot, represents the downlink signal transmission rate of the unmanned aerial vehicle m, represents the downlink signal reception rate of the unmanned aerial vehicle m when transmitting a signal to the user terminal n through the subchannel k. The first value interval is (0, 1), the second value interval is (0, 1), and the third value interval is The first threshold value is R Umin , and the second threshold value is R Dmin . As an example, the first threshold value is not less than 10 bps / Hz, which can be 10 bps / Hz, 15 bps / Hz, 20 bps / Hz, etc. The second threshold value is not less than 0.8 bps / Hz, which can be 0.8 bps / Hz, 1 bps / Hz, 1.5 bps / Hz, etc.

[0093] C1, C2, and C3 are the numerical constraints for the relay communication parameters. C4 is the energy constraint for the UAV, which ensures that the energy captured by the UAV from the signal can cover the energy consumed by relay communication and hovering, allowing the UAV to operate continuously. C5 and C6 are the link rate constraints for the UAV, which ensure the communication quality of the UAV.

[0094] Based on the above constraints, the state space of a Markov chain can be represented as: The action space can be represented as: A = {ρ m ,τ,P U}

[0095] In this embodiment, the problem of maximizing energy efficiency can be combined with the problem of optimizing UAV relay communication parameters, and modeled as a Markov chain. By solving the Markov chain, the relay communication parameters obtained satisfy the constraints and maximize the energy efficiency. When the UAV runs with these relay communication parameters, it can balance communication quality and system energy consumption, thereby improving the overall performance of the UAV in the downlink communication system.

[0096] In some embodiments, step 530 above can be achieved by... Figure 6 The process shown constructs the reward function for the Markov chain, as follows: Figure 6 As shown, the process may include the following steps.

[0097] Step 610: Use energy efficiency value as the first reward item.

[0098] Step 620: Determine the second reward item based on the energy constraint.

[0099] The second reward item is positively correlated with the number of drones that meet the energy constraint conditions.

[0100] Step 630: Determine the third reward item based on the link rate constraint.

[0101] The third reward item is positively correlated with the number of links that meet the link rate constraint.

[0102] Continue to combine Figure 5 The examples shown in the embodiments are for illustrative purposes only. The second reward is positively correlated with the number of drones that meet C4; that is, the more drones that meet C4 out of all drones, the larger the value of the second reward. The third reward is positively correlated with the number of drones that meet C5 and C6; that is, the more drones that meet C5 and C6 out of all drones, the larger the value of the third reward.

[0103] Step 640: Determine the reward function based on the weighted sum of the first reward item, the second reward item, and the third reward item.

[0104] In the embodiment, the reward function is constructed by fusing energy efficiency, energy constraint and link transmission rate constraint, which can drive the strategy to optimize in the direction of maximizing energy efficiency, thereby achieving global energy efficiency optimization of the multi-UAV system under complex constraints.

[0105] In some optional embodiments of the embodiment, the reward function can be determined by the following formula (4):

[0106] r(t)=w1ξ EE (t)+w2r E (t)+w3r R (t) (4)

[0107] In the formula, w1, w2, w3 represent weights; ξ EE (t) represents the energy efficiency value; r E (t) represents the second reward term; r R (t) represents the third reward term; wherein,

[0108] r E (t)=δM

[0109]

[0110] In the formula, δ and λ represent positive rewards, and M represents the number of UAVs satisfying the energy constraint condition; represents the downlink signal receiving rate of the UAV m when transmitting a signal to the user terminal n through the subchannel k at the time slot t; represents the downlink signal transmission rate of the UAV m; K represents the number of subchannels; N m represents the number of user terminals in the user cluster corresponding to the UAV m.

[0111] In some embodiments, in order to more accurately and efficiently solve the high-dimensional nonlinear joint optimization problem represented by the above Markov chain, the above step 550 can use a deep reinforcement learning algorithm to solve the Markov chain to obtain the relay communication parameters.

[0112] As an example, the SAC (Soft Actor-Critic) algorithm in the deep reinforcement learning algorithm can be used to solve the Markov chain. For example, a multi-layer perception can be used to build the strategy network (Actor Network) and the value network (Critic Network) of the SAC algorithm, each multi-layer perception can include two hidden layers, each hidden layer can include 128 neurons, and a ReLu activation function is used. Then, the strategy network and the value network can be trained in the following manner.

[0113] First, the agent (i.e., policy network and value network, also called model) is initialized. Assume the experience replay pool size D, the policy network parameters The value network parameters θ1, θ2, and the target network parameters are set, and the parameters of the policy and value networks are completely copied to the corresponding target networks

[0114] Then, the following main training loop is performed: from the first episode, initialize the action ρ m , τ, P U , and observe the initial state s0 of the environment. At each time step of the current episode, the agent interacts with the environment once. The current environment state s t is input to the policy network, and the action a t is output by the policy network; the agent performs the action a t and interacts with the environment, obtains the reward r t and reaches the next state s t+1 . Then, the interaction data of the agent and the environment <s t , a t , r t , s t+1 > is stored as an experience tuple in the experience replay pool. A batch size B of experience tuples is randomly sampled from the experience replay pool for training. The Q value y of the target network is calculated, i.e., the true value of the action performed this time is equal to the immediate reward plus the long-term value of the best future action after discounting. The two value networks are updated by gradient descent, so that the Q(s, a) output by them is as close as possible to the Q value y just calculated.

[0115] Here, the loss function of the value network can be Then, the policy network is updated again, so that the policy network learns to output actions that can make the value network give higher Q values. Then, the target network is updated softly, so that the target network parameters slowly track the parameters of the main network at a small rate κ. The loss function of the policy network can be

[0116] Finally, the current state s t is updated to s t+1 , and the next time step is prepared until the maximum time step of the current episode is reached, and the episode is ended. The next episode loop is started until the maximum number of episodes is reached, and the training is ended, obtaining the trained agent. The reward curve output after training is shown in Figure 7a , and the convergence result of energy efficiency is shown in Figure 7b . As can be seen from Figure 7a and Figure 7b , the SAC algorithm is convergent.

[0117] The trained agent learns the optimal strategy autonomously by interacting with the environment, and since the energy efficiency value, energy constraint condition and link rate constraint condition are integrated in the reward function, the strategy can be driven to optimize in the direction of maximizing energy efficiency; in the deployment process, the strategy network and value network can dynamically determine the relay communication parameters of the unmanned aerial vehicle according to the real-time channel state and user end position, so as to realize the global energy efficiency optimization of the unmanned aerial vehicle in the downlink communication system under complex constraints.

[0118] Reference is made below to Figure 8 , which shows a structural diagram of an electronic device 800 suitable for implementing embodiments of the present application. The terminal device in embodiments of the present application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet PCs), and the like, as well as fixed terminals such as digital TVs, desktop computers, and the like. Figure 8 The terminal device shown is merely an example and should not impose any limitation on the functions and use range of embodiments of the present application.

[0119] As shown in Figure 8 , the electronic device 800 can include a processing device (such as a central processor, a graphics processor, etc.) 801, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 802 or loaded into a random access memory (RAM) 803 from a storage device 808. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0120] In general, the following devices can be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; storage devices 808 including, for example, a magnetic tape, a hard disk, and the like; and communication devices 809. The communication devices 809 can allow the electronic device 800 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it should be understood that all of the shown devices are not required to be implemented or possessed. More or fewer devices can be alternatively implemented or possessed. Figure 8 Each block shown in the middle can represent one device or, as needed, multiple devices.

[0121] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program in accordance with embodiments of the present application. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication device 809, or installed from the storage device 808, or installed from the ROM 802. When the computer program is executed by the processing device 801, the above-described functions defined in the methods of embodiments of the present application are performed. It should be noted that the computer readable medium of embodiments of the present application can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In embodiments of the present application, the computer readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In embodiments of the present application, the computer readable signal medium can include a computer readable program code carried in a baseband or as a part of a carrier wave, in which the computer readable program code is embodied. Such a propagated computer readable signal medium can take many forms, including but not limited to, electro-magnetic, optical or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can be used to carry or transmit a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to, wire, cable, RF (radio frequency), etc., or any suitable combination thereof.

[0122] The computer readable medium can be included in the electronic device; or can exist independently of the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to determine, for each unmanned aerial vehicle in a downlink communication system, a relay communication parameter, so that an energy efficiency value is maximum, the relay communication parameter including a power split ratio, a time slot allocation ratio and a transmission power, the energy efficiency value representing a ratio of a sum of information throughputs of all links in a unit time slot to a sum of energies consumed by all unmanned aerial vehicles, receive, by the unmanned aerial vehicle, a signal transmitted by a base station in a wireless power transfer manner, separate, for each unmanned aerial vehicle, energy and information from the signal based on the relay communication parameter corresponding to the unmanned aerial vehicle, replenish the energy and transmit the information to a predetermined user cluster including at least one user terminal in a coverage range.

[0123] Computer program code for carrying out operations of embodiments of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0124] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0125] Those skilled in the art can easily understand that the above-mentioned preferred embodiments can be combined and superimposed freely without conflict.

[0126] The above description is merely preferred embodiments of the present application, but not for limiting the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for optimizing energy-efficiency multi-UAV-assisted power transmission, applied to relay UAVs in downlink communication systems, characterized in that... The method includes: The relay communication parameters for each UAV in the downlink communication system are determined to maximize the energy efficiency value. The relay communication parameters include the power split ratio, the time slot allocation ratio, and the transmission power. The energy efficiency value represents the ratio of the sum of the information throughput of all links within a unit time slot to the sum of the energy consumed by all UAVs. Using drones to receive signals transmitted by base stations via wireless power transmission; For each of the aforementioned drones, based on the relay communication parameters corresponding to that drone, energy and information are separated from the signal, the energy is used to replenish the battery, and the information is sent to the corresponding user cluster, the user cluster including at least one user terminal within the coverage area; The energy efficiency value is determined in the following manner: The energy consumed by the UAV in a unit time slot is determined based on the energy consumption of the UAV communicating with the user terminal in the user cluster within a unit time slot, the energy consumption of the UAV maintaining a hovering state within a unit time slot, and the energy captured by the UAV from the signal within a unit time slot. Based on the downlink signal transmission rate of the UAV within a unit time slot and the downlink signal reception rate of each user terminal in the user cluster corresponding to the UAV, the information throughput of the UAV communicating with the user terminal within a unit time slot is determined. The energy efficiency value is the ratio of the sum of the information throughput of all the UAVs communicating with the user terminal in a unit time slot to the sum of the energy consumed by all the UAVs in a unit time slot.

2. The method according to claim 1, characterized in that, The sum of energy consumed by all the aforementioned drones per unit time slot is determined based on the following formula: ； In the formula, E total This represents the sum of energy consumed by all the aforementioned drones within a unit time slot, where M represents the number of drones. Indicates drone m The energy consumed in communicating with user terminals within a user cluster per unit time slot. Indicates drone m The energy consumed to maintain a hovering state within a unit time slot. Indicates drone m Energy captured from the signal within a unit time slot.

3. The method according to claim 2, characterized in that, The sum of the information throughput of all the aforementioned UAVs communicating with the user terminal within a unit time slot is determined based on the following formula: ； In the formula, R total This represents the sum of the information throughput of all the aforementioned drones communicating with the user terminal within a unit time slot, where M represents the number of drones. This indicates the time slot allocation ratio. l Indicates the length of a unit time slot. This represents the downlink signal transmission rate of the drone m. Indicates drone m via sub-channel k To the user terminal n Downlink signal reception rate when transmitting signals K Indicates the number of sub-channels; N m Indicates drone m The number of client terminals within the corresponding user cluster.

4. The method according to claim 3, characterized in that, Determining the relay communication parameters for each UAV in the downlink communication system includes: The constraints are determined, including numerical constraints on the relay communication parameters, energy constraints on each UAV, and link rate constraints. The numerical constraints include: the power splitting ratio being within a first value range, the time slot allocation ratio being within a second value range, and the transmission power being within a third value range. The energy constraints include: the energy captured by the UAV in a time slot is not less than the energy consumed. The link rate constraints include: the downlink information transmission rate of the UAV is not less than a first threshold, and the downlink information reception rate of the user terminal is not less than a second threshold. Based on the captured energy, the consumed energy, the downlink information transmission rate, and the downlink information reception rate, the state space of the Markov chain is constructed; Based on the power splitting ratio, the time slot allocation ratio, and the transmission power, the action space of the Markov chain is constructed. Based on the energy efficiency value, the energy constraint, and the link rate constraint, the reward function of the Markov chain is constructed. Based on the Markov chain, the relay communication parameters are determined such that the relay communication parameters satisfy the constraints and the energy efficiency value is maximized.

5. The method according to claim 4, characterized in that, Based on the energy efficiency value, the energy constraint, and the link rate constraint, the reward function of the Markov chain is constructed, including: The energy efficiency value will be used as the first reward item. Based on the energy constraint, a second reward item is determined, which is positively correlated with the number of drones that meet the energy constraint. Based on the link rate constraint, a third reward item is determined, which is positively correlated with the number of links that satisfy the link rate constraint. The reward function is determined based on the weighted sum of the first reward item, the second reward item, and the third reward item.

6. The method according to claim 5, characterized in that, The reward function is determined by the following formula: ； In the formula, Indicates weight; This represents the energy efficiency value; This indicates the second reward item; This refers to the third reward item; where, ； ； In the formula, and Indicates a positive reward. M This represents the number of drones that satisfy the energy constraint condition; Indicates drone m In the time slot t via sub-channel k To the user terminal n Downlink signal reception rate when transmitting signals; This represents the downlink signal transmission rate of the drone m; K Indicates the number of sub-channels; N m Indicates drone m The number of client terminals within the corresponding user cluster.

7. The method according to claim 4, characterized in that, Based on the Markov chain, the relay communication parameters are determined, including: The Markov chain is solved using a deep reinforcement learning algorithm to obtain the relay communication parameters.

8. The method according to claim 1, characterized in that, Based on the relay communication parameters, energy and information are separated from the signal, the energy is used to replenish the battery, and the information is sent to a predetermined user cluster, including: For any given time slot, based on the time slot allocation ratio, the time slot is divided into a first interval and a second interval; In the first interval, based on the power splitting ratio, the energy and the information are separated from the signal, and the energy is transferred to the battery; In the second interval, the information is sent to the user cluster according to the transmission power.

9. The method according to claim 1, characterized in that, Before determining the relay communication parameters for each UAV in the downlink communication system, the following steps are also included: All user terminals are clustered to determine the user cluster corresponding to each UAV, and a sub-channel is assigned to each user terminal in the user cluster, with different user terminals corresponding to different sub-channels.

Images