Optimal path planning method for cleaning robot based on multi-step optimization of approximate model

Abstract

The invention discloses a cleaning robot optimal path planning method based on approximate model multi-step optimization. The method comprises steps: a model, hyperparameters and an environment are initialized; an exploration strategy is selected; the model is updated with the current sample; a value function, a strategy and the current state are updated; whether the current plot is ended is judged and further, the current sampling trajectory is used to update a trajectory pool; a reconstructed sampling trajectory is used to update the trajectory pool; all trajectories in the trajectory pool are used to update the model; the model is adopted for planning; whether to reach the maximum plot number is judged; and finally, according to a learnt optimal strategy, the optimal cleaning robot planning path is acquired. Through adopting the sampling trajectory and the single sample to update the model, the model learning precision is improved; the model is then used for planning, the learning speeds of the value function, the strategy and the whole algorithm are improved; and the sample utilization efficiency is also improved, and thus, the optimal cleaning robot planning path is acquired by adopting the fewer samples in the shorter time.

Description

technical field [0001] The invention relates to a path planning method for a cleaning robot, in particular to an optimal path planning method for a cleaning robot based on multi-step optimization of an approximate model. Background technique [0002] The problem of autonomous path planning for cleaning robots is a common problem in the field of control. This problem can model all possible states of the robot as a state space, model all actions that can occur as an action space, and model the next possible state reached after the current state takes an action as a transition function, and will reach the next The immediate reward obtained by a state is modeled as a reward function, which converts the problem into a Markov decision process. The conventional way to solve this problem is to use discrete reinforcement learning methods, such as Q-learning and SARSA algorithm to solve it, such as directly discretizing the state space and action space, that is, dividing the state sp...

AI Technical Summary

Problems solved by technology

When the learned model is accurate enough, it can accelerate the convergence of strategies and algorithms. When the learned model is not accurate enough, using the model for planning will hinder the acquisition of the optimal solution of the strategy.

Images