Unmanned aerial vehicle cluster collaborative learning method based on multi-agent reinforcement learning

Abstract

The invention discloses an unmanned aerial vehicle cluster collaborative learning method based on multi-agent reinforcement learning. The method comprises the steps of constructing an environment simulator based on aerodynamics; enabling each unmanned aerial vehicle to acquire and maintain a local observation value of the unmanned aerial vehicle; enabling each unmanned aerial vehicle to acquire and maintain a suggested observation value from a teammate as a student; providing a guidance value for other unmanned aerial vehicles as a teacher; executing an action strategy based on the local observation value of the machine and the suggested observation value obtained from the teammate, obtaining a reward and transferring to the next state; performing training based on the MADDPG thought untilthe value network and the strategy network converge; carrying out the execution stage in a distributed mode, that is, each unmanned aerial vehicle executes an action strategy based on the local observation value and the suggested observation value provided by the teammate. Complementation of observation values between unmanned aerial vehicle clusters can be achieved on the premise of relatively low cost, independent autonomous decision of the unmanned aerial vehicles is achieved, and the communication problem caused by a 'master-slave' structure is solved.

Description

technical field [0001] The invention relates to a multi-agent reinforcement learning-based cooperative learning method for unmanned aerial vehicle clusters, which belongs to the technical field of unmanned aerial vehicle cluster cooperation. Background technique [0002] With the advancement of science and technology and the improvement of UAV technology, UAV swarms are becoming more and more important in daily life, and are widely used in daily transportation, disaster rescue, military games and other fields. In the existing drone cluster cooperation, the drone cluster is mainly controlled in a "master-slave" manner, that is, the "master" drone is mainly responsible for processing the data obtained by each drone and distributing instructions to the participants. The "slave" drone. This type of method has strict requirements on the communication channel. If the communication channel is interfered or maliciously attacked, the information obtained by the drone cluster of the ...

AI Technical Summary

Problems solved by technology

However, in a MADDPG-based UAV cluster, a single UAV information sensor (camera, etc.) has a limited field of vision, cannot effectively obtain global information, and cannot achieve true autonomous decision-making.

Images