Multi-unmanned aerial vehicle cooperative attack and defense confrontation method based on strategy set MADDPG

Abstract

The invention discloses a multi-unmanned aerial vehicle cooperative attack and defense confrontation method based on a strategy set MADDPG. The method comprises the following steps: firstly, constructing a multi-unmanned aerial vehicle cooperative attack and defense confrontation combat mission environment; secondly, establishing a combined state space and a combined action space of the multi-unmanned aerial vehicle system; then, designing a reward function based on a group target for the multi-unmanned aerial vehicle attack and defense confrontation problem, wherein the reward function comprises two parts, namely, an attack-party unmanned aerial vehicle and a defense-party unmanned aerial vehicle; thirdly, training the network based on the strategy set MADDPG; and finally, realizing a multi-unmanned aerial vehicle cooperative attack and defense confrontation decision by using the trained network model. According to the method, the MADDPG algorithm is improved, the reward function based on a group target is designed, learning of a cooperative attack and defense strategy is guided, the problem that a single agent in a multi-unmanned aerial vehicle system has a strong over-fitting strategy for competitors of the agent is effectively solved by using a strategy set, the attack and defense efficiency is higher, therefore, the unmanned aerial vehicle is endowed with a better cooperative attack and defense confrontation capability.

Description

technical field [0001] The invention belongs to the technical field of air combat decision-making, and specifically relates to a multi-unmanned aerial vehicle cooperative offensive and defensive confrontation method based on a strategy set MADDPG. [0002] technical background [0003] With the continuous advancement of computer technology, reinforcement learning algorithms have been widely used in the research of UAV air combat maneuver decision-making. In a multi-UAV system, while one UAV is learning, other UAVs are also learning, and the actions they perform affect the environment, making the current learning environment change, which will make the multi-UAV system not If the MDP model is satisfied, then reinforcement learning cannot be directly applied to multi-UAV systems. In order to solve this problem, some algorithms use the reinforcement learning method of state prediction to solve the multi-UAV intelligent decision-making problem. Each UAV first uses the past state...

AI Technical Summary

Problems solved by technology

Among them, the most widely used is Deep Deterministic Policy Gradient (DDPG) and its improved algorithm. This algorithm uses the policy network to directly output actions and can cope with the output of continuous actions, but there is an important problem, because each The strategy of the agent is updated and iterated, resulting in the environment being dynamically unstable for a specific agent

Images