Underwater vehicle docking control method based on reinforcement learning

Abstract

The invention relates to an underwater vehicle docking control method based on reinforcement learning, belongs to the technical field of ocean control experiments, and introduces a reliable boundary updated by new and old strategies based on a PPO algorithm framework in deep reinforcement learning to improve the stability of agent learning. Meanwhile, a self-adaptive rollback cutting mechanism is adopted, the rollback strength is adjusted in a self-adaptive mode according to the situation that task successful completion experience is collected, and therefore the upper limit and the lower limit of new and old strategy updating are adjusted, and an intelligent agent is encouraged to explore in the initial training stage and converge stably in the later training stage. In the aspect of simulation training, a docking training environment considering ocean current and ocean wave interference is constructed, the training environment is used for intelligent body learning, and the anti-interference capability of the underwater vehicle is greatly improved.

Description

technical field [0001] The invention relates to an underwater vehicle docking control method based on reinforcement learning, and belongs to the technical field of marine control experiments. Background technique [0002] As a special marine survey equipment, underwater vehicles have been widely used in many marine engineering fields such as seabed topographic mapping, marine resource exploration, shipwreck monument investigation, oil and gas pipeline maintenance, life science monitoring, etc. indispensable means. However, due to the need to ensure the flexibility of the underwater vehicle itself and to carry corresponding equipment, the limited energy carried by the underwater vehicle limits its long-term cruise capability, and it is inevitable to replenish the energy regularly. In order to prevent underwater vehicles from relying on surface ships for energy supplementation, so that they have fully automatic long-term operation capabilities, such as the Chinese patent docu...

AI Technical Summary

Problems solved by technology

[0006] In view of the deficiencies in the prior art, in order to solve the problem of underwater vehicle docking control, the present invention is based on the PPO algorithm framework in deep reinforcement learning, and proposes an underwater vehicle docking control method based on adaptive reliable boundary rollback clipping reinforcement learning

Images