A self-floating control method for underwater vehicles based on demonstration data reinforcement learning technology

Abstract

The invention relates to an autonomous floating control method for underwater vehicles based on demonstration data reinforcement learning technology, which belongs to the technical field of marine equipment control. Autonomous floating control method of underwater vehicle based on data deep reinforcement learning technology. Based on the DDPG algorithm, the present invention realizes the floating control of the continuous action space underwater vehicle, and uses demonstration data in the training process to accelerate the convergence of the algorithm, and at the same time delays the update of the actor network in the algorithm framework, effectively eliminating the reinforcement learning algorithm There is an overestimation problem in .

Description

technical field [0001] The invention relates to an autonomous floating control method of an underwater vehicle based on demonstration data reinforcement learning technology, and belongs to the technical field of marine equipment control. Background technique [0002] Underwater vehicle is a kind of marine equipment with flexible maneuverability, strong maneuverability, low cost and convenient carrying. Marine environmental monitoring and other fields. With the increasing scarcity of land resources, the ocean has gradually become an extremely important strategic resource base for all countries. Therefore, the research and development of marine equipment technology has received more and more attention. Due to its unique advantages, irreplaceable role and broad application prospects, the development of underwater vehicles has been highly valued and supported. [0003] However, if the underwater vehicle works in a complex and unknown marine environment, if it encounters its ow...

AI Technical Summary

Problems solved by technology

[0006] In the existing research and invention patents, although there is an idea of ​​using reinforcement learning algorithm framework to control underwater vehicles, there are also some obvious defects: First, some methods are based on the Q-learning algorithm in reinforcement learning. The algorithm is only suitable for the case where the action space is discrete, and does not meet the continuous change of the actuator during the actual use of the underwater vehicle (Q.L.Zhang et al., "Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle", IEEE ACCESS , vol.8, pp.24258-24268, Jan.2020); second, the reinforcement learning algorithm including Q function estimation has the problem of overestimating the cumulative reward; third, the existing research on the underwater vehicle The two-dimensional simplification of the motion space greatly reduces the dynamic parameters that should be considered, and the actual underwater vehicle moves in three-dimensional space. In such a state space, the traditional reinforcement learning algorithm will be difficult to converge; finally, based on The randomness in the early training process of the reinforcement learning motion control method will bring great risks to the controlled object, and the integration of demonstration data into the training process can solve this problem well

Images