Trust region strategy optimization method and device based on post-event experience and related equipment

Abstract

The invention discloses a trust region strategy optimization method and device based on post-event experience and related equipment, and the method comprises the steps: S100, taking an arrived targetpoint in experience data as a virtual target point, and generating virtual post-event experience data; S200, filtering the virtual target based on a post-event target filtering algorithm to obtain corresponding training data; s300, based on the virtual experience data, correcting the distribution deviation of the virtual empirical data and the original empirical data through weighted importance sampling; s400, correcting the distribution deviation between the virtual experience data and the original empirical data based on weighted importance sampling so as to estimate an inter-strategy KL divergence value; and S500, correcting the strategy gradient direction through the KL divergence, and calculating and updating the strategy step length through the maximum KL divergence step length. According to the method, an intelligent agent can complete an effective exploration process on the environment and tasks based on a small amount of interaction data and a simply designed reward function,and behavior strategies are efficiently learned and updated.

Description

technical field [0001] The invention belongs to the field of machine learning intelligent robots, and in particular relates to a trust domain policy optimization method, device and related equipment based on post-event experience. Background technique [0002] With the rapid development of artificial intelligence technology, it has emerged in many industries through intelligent and automated information processing. However, the current mainstream deep learning methods in the field of artificial intelligence mostly rely on large-scale human-labeled data. How to obtain data and complete the learning process through the autonomous interaction between robots or agents and the environment is a major difficulty in the field of artificial intelligence. As an important branch technology in the field of artificial intelligence, reinforcement learning can help robots explore and learn in the process of autonomous interaction with the environment. However, reinforcement learning is cu...

AI Technical Summary

Problems solved by technology

However, reinforcement learning is currently facing many problems such as slow learning speed, difficult reward function design, and low exploration efficiency, so it is difficult to be applied in actual complex tasks.

Images