Hierarchical reinforcement learning training method and device based on imitation learning

Abstract

The invention discloses a hierarchical reinforcement learning training method and device based on imitation learning, and an electronic device. The method comprises the steps of obtaining teaching data of a human expert; conducting pre-training based on imitation learning by using the teaching data, and determining an initial strategy; and based on the initial strategy, performing retraining basedon reinforcement learning, and determining a training model. The teaching data is used for pre-training and re-training, priori knowledge and strategies are effectively utilized, the search space isreduced, and the training efficiency is improved.

Description

technical field [0001] The present application relates to the field of machine learning, and more specifically, relates to a hierarchical reinforcement learning training method, device and electronic equipment based on imitation learning. Background technique [0002] Reinforcement learning is one of the most concerned research directions in the field of artificial intelligence in recent years, and has achieved many dazzling results in many fields. An agent learns to behave in an environment by performing certain actions and observing the rewards or outcomes obtained from those actions. However, an important shortcoming of reinforcement learning is the curse of dimensionality. When the dimension of the system state increases, the number of parameters to be trained will increase exponentially, which will consume a large amount of computing and storage resources. [0003] Based on the inherent hierarchical and compositional nature of real-world tasks, hierarchical reinforceme...

AI Technical Summary

Problems solved by technology

[0006] However, in the past, only the teaching data of imitation learning was used to pre-train the parameters of the hierarchical reinforcement learning model. The imitation learning and hierarchical reinforcement learning were divided into two different steps, and the teaching data was not applied to the hierarchical reinforcement learning step. Intrinsic connections in the training process are ignored

Images