Trajectory Tracking Control Method for Autonomous Underwater Vehicle Based on Deep Reinforcement Learning

Abstract

The invention provides an autonomous underwater vehicle (AUV) trajectory tracking control method based on deep reinforcement learning, belonging to the field of deep reinforcement learning and intelligent control. The autonomous underwater vehicle trajectory tracking control method based on deep reinforcement learning includes the steps: defining an AUV trajectory tracking control problem; establishing a Markov decision-making process model of the AUV trajectory tracking problem; constructing a hybrid policy-evaluation network which consists of multiple policy networks and evaluation networks;and finally, solving the target policy of AUV trajectory tracking control by the constructed hybrid policy-evaluation network, for the multiple evaluation networks, evaluating the performance of eachevaluation network by defining an expected Bellman absolute error and updating only one evaluation network with the lowest performance at each time step, and for the multiple policy networks, randomly selecting one policy network at each time step and using a deterministic policy gradient to update, so that the finally learned policy is the mean value of all the policy networks. The autonomous underwater vehicle trajectory tracking control method based on deep reinforcement learning is not easy to be influenced by the bad AUV historical tracking trajectory, and has high precision.

Description

technical field [0001] The invention belongs to the field of deep reinforcement learning and intelligent control, and relates to an autonomous underwater vehicle (AUV) trajectory tracking control method based on deep reinforcement learning. Background technique [0002] The development of deep-sea submarine science is highly dependent on deep-sea exploration technology and equipment. Due to the complex deep-sea environment and extreme conditions, currently, deep-sea operating autonomous underwater vehicles are mainly used to replace or assist humans in deep-sea detection, observation and sampling. For mission scenarios where humans cannot reach the on-site operations, such as marine resource exploration, seabed survey, and oceanographic mapping, ensuring the autonomy and controllability of AUV underwater movement is the most basic and important functional requirement. premise of the task. However, many offshore applications of AUVs (such as trajectory tracking control, targ...

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to propose a deep reinforcement learning-based AUV trajectory tracking control method, which uses a hybrid policy-evaluation network structure, and uses multiple quasi-Q learning and deterministic policy gradients to train the evaluation network and policy respectively Network, to overcome the problems of low control accuracy, inability to achieve continuous control and unstable learning process in previous methods based on reinforcement learning, and realize high-precision AUV trajectory tracking control and stable learning

Images