Unmanned aerial vehicle concealing approach method of employing priority random sampling strategy-based Double DQN

Abstract

The invention discloses an unmanned aerial vehicle concealing approach method of employing a priority random sampling strategy-based Double DQN. The unmanned aerial vehicle concealing approach methodcomprises the steps of firstly, establishing a both-side air combat situation diagram of concealing approach and establishing a dominant area and an exposed area in a concealing approach process through the diagram; secondly, establishing state space of an unmanned aerial vehicle and converting the state space into feature space and speed limit-based unmanned aerial vehicle action space; thirdly,building a priority random sampling strategy-based double-depth Q learning network; fourthly, establishing a target potential function reward according to the relative positions of friend or foe bothsides in the dominant area and the exposed area, establishing an obstacle reward according to the distance between the unmanned aerial vehicle and an obstacle, and superposing the target potential function reward and the obstacle reward as the total reward for concealing approach training of a Double DQN neural network; and finally inputting a current feature sequence of the unmanned aerial vehicle into the trained Q target neural network in the Double DQN to obtain an optimal concealing approach strategy of the unmanned aerial vehicle. According to the method disclosed by the invention, the problem of model-free concealing approach of the unmanned aerial vehicle is mainly solved.

Description

technical field [0001] The invention belongs to the field of unmanned aerial vehicle air combat decision-making, in particular to a Double DQN unmanned aerial vehicle concealed enemy contact method based on a priority random sampling strategy. [0002] technical background [0003] As the air combat environment becomes more and more complex and unknown, the new generation of unmanned aerial vehicles puts more emphasis on the characteristics of low detectability, high maneuverability, networked operations and stealth penetration trajectory optimization. Using the strategy of concealed contact with the enemy, the UAV can quickly reach the dominant area, forming attackable conditions, while avoiding the exposed area within the attack range of the enemy aircraft. Therefore, covert engagement plays a vital role in the combat and survivability of UAVs. Due to the high real-time requirements of air combat, it is difficult to complete the timely and accurate control of UAVs by the r...

AI Technical Summary

Problems solved by technology

Due to the high real-time requirements of air combat, it is difficult to complete the timely and accurate control of UAVs by the remote control method currently used by ground stations. Therefore, the intelligence level of UAVs is improved so that UAVs can independently perceive the battlefield environment and automatically Generating control commands to complete the maneuver selection in air combat is the main research direction of current UAV air combat

[0004] Reinforcement learning is a learning algorithm that uses a "trial and error" method to interact with the environment. Due to the uncertainty and complexity of the air combat environment, traditional reinforcement learning cannot solve the "dimension disaster" faced by high-dimensional continuous state space policy learning. Therefore, the DQN algorithm that combines deep learning and reinforcement learning is produced, and the neural network of deep learning is used to fit the action value function to solve this problem

However, the DQN algorithm uses the same neural network for action selection and action evaluation, which is prone to overfitting, resulting in the selected action not being the optimal solution but the suboptimal solution of the current state.

Images