Unmanned aerial vehicle flight control method based on imitation learning and reinforcement learning algorithms

Abstract

The invention discloses an unmanned aerial vehicle flight control method based on imitation learning and reinforcement learning algorithms. The method comprises the following steps: creating an unmanned aerial vehicle flight simulation environment simulator; defining a basic action set of flight; classifying the trajectory data according to the flight basic actions; for each flight action, learning mapping network parameters from a flight basic action to an original action by utilizing imitation learning; counting the minimum continuous action number of each basic action; constructing an upper-layer reinforcement learning network, and adding the minimum continuous action number as punishment p of aircraft action inconsistency; in the simulator, obtaining current observation information andawards, and selecting corresponding flight basic actions by using a pDQN algorithm; inputting the state information of the aircraft into an imitation learning neural network corresponding to the flight basic action, and outputting the original action of the simulator; inputting the obtained original action into a simulator to obtain observation and awards of the next moment; and performing training by using a pDQN algorithm until the strategy network on the upper layer converges.

Description

technical field [0001] The invention relates to a UAV flight control method based on imitation learning and reinforcement learning algorithms in a complex and rapidly changing environment, and belongs to the technical field of UAV control. Background technique [0002] The problem of autonomous flight of UAVs in unknown environments has always been one of the main difficulties in autonomous flight of UAVs. The traditional UAV flight mainly adopts a rule-based control method. First, list the possible impacts of the environment, and then Let experts in relevant fields formulate flight rules. However, this method generally only achieves ideal results in relatively simple environments. In a complex and changing environment, due to many influencing factors and frequent changes in the surrounding environmental scenes, there will be a large number of sample data that have not appeared in the training data. Flight rules tend to be less effective or even non-existent in such situat...

AI Technical Summary

Problems solved by technology

[0005] Purpose of the invention: In view of the problem of autonomous flight of drones in the prior art, rule control cannot handle complex and changing environments, and the reinforcement learning algorithm has too large action exploration space, sparse rewards, and the learned flight strategy is not stable enough and fluctuates greatly. The invention provides a UAV flight control method based on imitation learning and reinforcement learning algorithms

Images