Reinforcement learning method based on bidirectional model

Abstract

A reinforcement learning method based on a bidirectional model is used for robot control, and is characterized by comprising a forward model, a reverse model, a forward strategy and a reverse strategy, tracks are bidirectionally generated from a certain real state, and iteration is continuously performed in three stages: a data collection stage, a model learning stage and a strategy optimization stage until an algorithm converges. The method has the advantages that compared with a traditional forward model, the bidirectional model is smaller in model accumulative error under the condition thatvirtual tracks with the same length are generated, and in a further simulation control experiment, compared with a previous model-based method, the method is better in sampling efficiency and progressive performance.

Description

technical field [0001] The invention relates to the field of reinforcement learning methods, in particular to research on model cumulative errors in model-based reinforcement learning. Background technique [0002] Reinforcement learning can be divided into model-free reinforcement learning and model-based reinforcement learning according to whether the environment is modeled. Among them, the model-free reinforcement learning directly trains a policy function or value function by sampling the data obtained in the real environment, while the model-based reinforcement learning first learns a model to fit the state change function through the real data obtained by interacting with the environment. The model is then used to generate simulated trajectories to optimize a policy or controller. Although model-free reinforcement learning has achieved very good results on many tasks, the achievement of these results often requires a large amount of data interacting with the environme...

AI Technical Summary

Problems solved by technology

[0007] But in general, these studies are limited to the forward model, and in order to avoid the impact of too large model cumulative error, a certain compromise needs to be made in terms of the length of the generated trajectory and applicability.

Images