A control method of underwater robot based on reinforcement learning and its tracking control method

Abstract

The invention discloses an underwater robot control method based on reinforcement learning and a control method of tracking with an underwater robot, belonging to the technical field of underwater robot control. According to the methods provided by the invention, a control center gives desired trajectory information of the underwater robot, and sends the information to the underwater robot; according to a probability density function of uncertain parameters in an underwater robot model, sampling points are respectively selected for the uncertain parameters, and order reduction is executed foran original dynamical model based on the sampling points; the underwater robot performs interaction with surrounding environment so as to learn environment information, a one step cost function is calculated at different states and value updating is executed, weight of a value function corresponding to a control strategy is solved through a least square method, improvement of the control strategyis executed through a gradient descent method, loop iteration is executed for two processes of value updating and strategy improvement until convergence is realized, and consequently, an optimal control strategy of current position tracking desired trajectory is obtained; and steps above are repeated for obtaining the optimal control strategy for the rest desired trajectory tracking, and finally,a tracking task is completed.

Description

technical field [0001] The invention relates to the field of underwater robot control, in particular to an underwater robot control method based on reinforcement learning and a tracking control method thereof. Background technique [0002] With the application of marine resources more and more widely, underwater robots have also received more attention. An important application of underwater robots in the ocean is position tracking, but the underwater environment is complex and changeable, making it difficult to obtain the model parameters of underwater robots and difficult to control them. [0003] Searching in the prior art found that the patent application with publication number CN106708069A designed a coordinated planning and control method for an underwater mobile working robot. The method includes planning the current desired speed and state in real time through dynamic tracking of the differentiator, using iterative task priority method to transform the task plannin...

AI Technical Summary

Problems solved by technology

However, this invention does not take into account the influence of the uncertainty in the underwater environment on the underwater robot. In the marine environment, the underwater robot will be subject to various disturbances during operation, such as the force of surge, swing and heave. Influence, if these uncertain factors are not taken into account in the algorithm, the ideal effect will not be achieved in actual operation

Images