Unmanned ship reinforcement learning controller structure with data drive and design method thereof

Abstract

The invention discloses an unmanned ship reinforcement learning controller structure with data drive and a design method thereof. The controller structure comprises an unknown information extraction module, a prediction model generation module, a reward function module and a moving horizon optimization module. The unmanned ship reinforcement learning controller structure with data drive and the design method thereof are based on the data drive with no need for accurate mathematical modeling for a controlled unmanned ship. The controller only employs the unknown information extraction module tocollect control input and output state data information of the unmanned ship and extract a dynamics unknown function, the prediction model generation module reconstructs the extraction information toobtain a prediction model, and the controller does not depend on accurate manual modeling of the unmanned ship. The unmanned ship reinforcement learning controller structure with data drive and the design method thereof do not need to design different controllers for the two levels of the kinematics and the dynamics. Through a prediction model and a set reward function, the control input is subjected to moving horizon optimization to achieve an optimal control effect. The unmanned ship reinforcement learning controller structure with data drive and the design method thereof can be suitable for an all-drive unmanned ship and an under-actuated unmanned ship.

Description

technical field [0001] The invention relates to the technical field of unmanned ship motion control, in particular to a data-driven unmanned ship reinforcement learning controller structure and design method. Background technique [0002] Intelligent unmanned ships are the trend of ship development. In the 21st century, with the rapid development of new concepts and new technologies such as big data and artificial intelligence, the level of ship intelligence has also been continuously improved, and the development and application of intelligent unmanned ships have also had technological support. In the key technical field of intelligent unmanned ships, motion control technology is the premise and basis for realizing autonomous navigation of unmanned ships. [0003] Aiming at the problem of unmanned ship motion control, there are already some feasible technical solutions. For example, patent CN107015562A proposes a control method for underactuated surface ships that meets t...

AI Technical Summary

Problems solved by technology

The controller design method based on the unmanned ship model has the advantages of interpretability and intuition, and has been widely studied and applied. However, for complex and changeable environments or complex controlled objects, the precise unmanned ship mathematical model The establishment and setting of controller parameters will be very cumbersome and not conducive to the adjustment of parameters

In addition, traditional controllers have fixed parameters and do not have adaptive learning capabilities. Even if there are some adaptive parameter adjustment methods, they are often limited by the expressiveness and robustness of artificially established unmanned ship models and environmental models. poor

Images