Cleaning robot optimal path planning method based on approximate model multi-step optimization

An optimal path planning, cleaning robot technology, applied in instruments, two-dimensional position/channel control, vehicle position/route/altitude control, etc., can solve the problems of accelerating strategy and algorithm convergence, hindering strategy acquisition, etc. Obstacle avoidance and finding the optimal path, increasing the representation ability, promoting the effect of policy and algorithm convergence

Active Publication Date: 2018-11-06
CHANGSHU INSTITUTE OF TECHNOLOGY
View PDF10 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

When the learned model is accurate enough, it can accelerate the convergence of strategies and algorithms. When the learne...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Cleaning robot optimal path planning method based on approximate model multi-step optimization
  • Cleaning robot optimal path planning method based on approximate model multi-step optimization
  • Cleaning robot optimal path planning method based on approximate model multi-step optimization

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0037] The present invention will be further described below in conjunction with the examples, but it is not intended to limit the present invention.

[0038] please combine figure 1 As shown, the optimal path planning method for a cleaning robot based on multi-step optimization of an approximate model involved in this embodiment includes the following steps:

[0039] Step 1), initialize the model, set the environmental state space X as the limit value of the horizontal and vertical coordinates of the two rooms, and the action in the action space U is a fixed value for the robot to move along the [-π, +π] angle;

[0040] Step 2), initialize the hyperparameters, set the discount rate γ=0.9, the decay factor λ=0.9, the number of episodes=200, the exploration variance of the Gaussian function is 0.2, the maximum time step included in each episode is 200, and the learning of the value function The learning rate is 0.6, the learning rate of the policy is 0.6, the learning rate of ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a cleaning robot optimal path planning method based on approximate model multi-step optimization. The method comprises steps: a model, hyperparameters and an environment are initialized; an exploration strategy is selected; the model is updated with the current sample; a value function, a strategy and the current state are updated; whether the current plot is ended is judged and further, the current sampling trajectory is used to update a trajectory pool; a reconstructed sampling trajectory is used to update the trajectory pool; all trajectories in the trajectory pool are used to update the model; the model is adopted for planning; whether to reach the maximum plot number is judged; and finally, according to a learnt optimal strategy, the optimal cleaning robot planning path is acquired. Through adopting the sampling trajectory and the single sample to update the model, the model learning precision is improved; the model is then used for planning, the learning speeds of the value function, the strategy and the whole algorithm are improved; and the sample utilization efficiency is also improved, and thus, the optimal cleaning robot planning path is acquired by adopting the fewer samples in the shorter time.

Description

technical field [0001] The invention relates to a cleaning robot path planning method, in particular to a cleaning robot optimal path planning method based on approximate model multi-step optimization. Background technique [0002] The autonomous path planning of cleaning robots is a common problem in the field of control. In this problem, all possible states of the robot can be modeled as a state space, all actions that can occur in it can be modeled as an action space, and the next possible state reached after the current state has an action can be modeled as a transition function, and will reach the next possible state. The immediate reward a state obtains is modeled as a reward function, which transforms the problem into a Markov decision process. The conventional idea to solve this problem is to use discrete reinforcement learning methods, such as Q-learning and SARSA algorithm to solve, such as direct discrete state space and action space, that is, divide the state sp...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G05D1/02
CPCG05D1/0221
Inventor 钟珊龚声蓉董瑞志姚宇峰马帅
Owner CHANGSHU INSTITUTE OF TECHNOLOGY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap