A multi-agent reinforcement learning unmanned ship formation collision avoidance method based on policy sequential update

Abstract

The application provides a multi-agent reinforcement learning unmanned ship formation collision avoidance method based on policy sequential update, and belongs to the technical field of unmanned ship control. In the model training stage, a multi-unmanned ship formation cooperative decision environment is constructed, and a leader-follower topological structure is established. A local observation vector is constructed for each ship. A multi-agent reinforcement learning decision model including a policy network and a value evaluation network is constructed. A collision avoidance guiding mechanism based on a vector field histogram is introduced. The expected heading is dynamically switched according to the minimum obstacle distance, and the observation vector is integrated. When avoiding collision, the target point of the follower is switched from the formation expected point to the global target point of the leader. A policy sequential update mechanism is adopted. The parameters of each unmanned ship are updated in a preset order. In the online execution stage, the trained policy network is deployed, and each ship independently outputs an action. When the original leader fails, the follower closest to the global target point of the original leader is selected as a new leader.

Images