Multi-unmanned aerial vehicle 3D hovering position joint optimization method and device and unmanned aerial vehicle base station

Abstract

The invention discloses a multi-unmanned aerial vehicle 3D hovering position joint optimization method and device and an unmanned aerial vehicle base station. The method comprises the following steps:firstly, acquiring state information of a heterogeneous network where unmanned aerial vehicles are located; inputting the state information to a deep reinforcement learning network constructed in advance, determining hovering positions at next moment through a current strategy, and obtaining reward function values of the hovering positions of the unmanned aerial vehicles at the current moment from the environment; determining a gradient strategy algorithm based on different strategy depths, calculating an updated gradient, and synchronously updating strategy parameters of the plurality of unmanned aerial vehicles; according to the updated gradient, iteratively executing the steps from obtaining state information to synchronously updating the strategy parameters of the plurality of unmanned aerial vehicles, so that an objective strategy function is gradually converged until an optimal strategy is obtained. The device comprises a state acquisition unit, a reward unit, a gradient updating unit and a training unit. The method can be executed by a processor of the unmanned aerial vehicle base station. According to the method, the device and the unmanned aerial vehicle base station in the invention, autonomous learning of the multi-unmanned aerial vehicle in the environment can be realized, and dynamic and non-stable environment changes can be adapted to.

Description

technical field [0001] The invention relates to the technical field of wireless communication, in particular to a joint optimization method and device for multi-UAV 3D hovering positions and a UAV base station. Background technique [0002] The multi-UAV hovering position optimization technology is an indispensable key technology in the UAV communication system, and the wireless communication system is developing into a diversified and heterogeneous form. In a heterogeneous network, macro base stations, small base stations, and UAV base stations exist at the same time. The hovering position of the UAV base station determines the communication rate between the UAV and the ground communication terminal, as well as the interference noise to other base stations in the communication system, and indirectly affects the communication load of the ground base station. [0003] The current method for multi-UAV hovering position optimization is mainly a game equilibrium method. For ex...

AI Technical Summary

Problems solved by technology

[0004] However, when this existing solution uses the game equilibrium method for optimization, it usually has very strong assumptions about the wireless communication environment, such as the following assumptions: 1) The ground communication terminal is stationary, so the optimized control of the UAV is only for a real network 2) The UAV and the ground communication terminal only establish a one-to-one static link; 3) Assume that the UAV only moves in the vertical direction and remains stationary in the horizontal direction, and the joint estimation of the two-dimensional state takes fixed value; 4) single base station type, etc.

The above assumptions greatly limit the operating mechanism of the UAV system, and cannot be applied to heterogeneous networks of multi-base station types, which is far from the real scene.

[0005] In summary, the UAV hovering position optimization method in the prior art cannot be applied to the actual communication environment due to many restrictive factors

Images