Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Reinforcement learning based TDOF (three degree of freedom) AUV (autonomous underwater vehicle) control method

A technology of an underwater vehicle and a control method, which is applied in the directions of adaptive control, general control system, control/regulation system, etc., to achieve the effect of improving control efficiency and increasing convergence speed

Active Publication Date: 2019-12-20
ZHEJIANG UNIV OF TECH
View PDF6 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

But at present, the application of this method in the control of small AUVs is still in the exploratory stage

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Reinforcement learning based TDOF (three degree of freedom) AUV (autonomous underwater vehicle) control method
  • Reinforcement learning based TDOF (three degree of freedom) AUV (autonomous underwater vehicle) control method
  • Reinforcement learning based TDOF (three degree of freedom) AUV (autonomous underwater vehicle) control method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0056] The present invention will be further elaborated below by means of drawings and specific embodiments.

[0057] refer to Figure 1 ~ Figure 4 , a three-degree-of-freedom autonomous underwater vehicle control method based on reinforcement learning, including the following steps:

[0058] Step 1) Initialize the experience playback pool;

[0059] Step 2) Obtain the state s of the AUV at the current moment from the processor, and input it to the main processor;

[0060] Step 3) output action a by the main processing, and send to the slave processor;

[0061] Step 4) Control the propeller thrust from the processing control motor drive module to move, and re-read the state s' after the AUV moves;

[0062] Step 5) The main processor calculates the return r after the action, and the sample priority P;

[0063] Step 6) Store (s, a, r, s', P) in the experience pool;

[0064] Step 7) Extract a batch of samples from the experience pool according to the sample priority for train...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

A reinforcement learning based TDOF (three degree of freedom) AUV (autonomous underwater vehicle) control method comprises steps as follows: 1) position state parameters of an AUV control system modelare defined; 2) an expression of the AUV is defined; 3) a strategy and evaluation network structure is constructed; 4) a target and current value network parameter updating mode is designed; 5) an exploration and utilization strategy of AUV movement is designed; 6) an experience replay strategy based on priority is designed; 7) an experience replay pond sample storage structure with priority is constructed; 8) a return function is defined; and 9) an AUV motion control target strategy is solved. The convergence rate of the control effect is increased, the control efficiency is further improved, and particularly, the method has certain development prospects in the field of underwater vehicles.

Description

technical field [0001] The invention relates to a three-degree-of-freedom autonomous underwater vehicle control method based on reinforcement learning, and belongs to the technical field of underwater robot control. Background technique [0002] The depletion of the earth's resources has prompted people to turn their attention to the resource-rich ocean, and the development value of underwater resources has become increasingly prominent, thus further promoting the development of underwater robots. In recent years, the hot spot of most countries in the world has been placed on the development of autonomous underwater robots. Autonomous underwater vehicle (AUV) is a new tool for effectively performing underwater operations. Portable small AUV has a wide range of application prospects in hydrological monitoring and underwater resource exploration in waters such as offshore, lakes and reservoirs, and has the advantages of small size, modularization, ease of use, and low cost. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G05B13/04
CPCG05B13/042
Inventor 洪榛刘涛安曼李严陈博俞立
Owner ZHEJIANG UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com