Neural network reinforcement learning control method of autonomous underwater robot

Abstract

The invention provides a neural network reinforcement learning control method of an autonomous underwater robot. The neural network reinforcement learning control method of the autonomous underwater robot comprises the steps that current pose information of an autonomous underwater vehicle (AUV) is obtained; quantity of a state is calculated, the state is input into a reinforcement learning neuralnetwork to calculate a Q value in a forward propagation mode, and parameters of a controller are calculated by selecting an action A; the control parameters and control deviation are input into the controller, and control output is calculated; the autonomous robot performs thrust allocation according to executing mechanism arrangement; and a reward value is calculated through control response, reinforcement learning iteration is carried out, and reinforcement learning neural network parameters are updated. According to the neural network reinforcement learning control method of the autonomousunderwater robot, a reinforcement learning thought and a traditional control method are combined, so that the AUV judges the self motion performance in navigation, the self controller performance isadjusted online according to experiences generated in the motion, a complex environment is adapted faster through self-learning, and thus, better control precision and control stability are obtained.

Description

technical field [0001] The invention belongs to the technical field of autonomous underwater robot control, in particular to an autonomous underwater robot neural network reinforcement learning control method. Background technique [0002] Autonomous Underwater Vehicle (AUV), as a member of the unmanned carrier system, AUV is an underwater vehicle with autonomous navigation function. By carrying various sensor equipment and communication systems, the position and attitude information of the unmanned vehicle can be obtained, and the control system of the AUV is formed with the power actuator. At the same time, AUV, as an underwater vehicle, can carry the required equipment according to the tasks to be performed to achieve various purposes such as detection and military. To evaluate the performance of an AUV, it is often judged by the speed, accuracy and stability of its task execution. AUVs with good performance can usually achieve higher efficiency in task execution. [00...

AI Technical Summary

Problems solved by technology

[0005] Traditional controllers often have two main methods. One requires people to continuously adjust parameters offline based on experience, which takes a long time and has poor adaptability to environmental changes.

The other is to identify the model parameters of the AUV, and then design the controller. This method is more dependent on the accuracy of the identification model, and it is more difficult to identify external disturbances.

Images