A method and system for generating state data for reinforcement learning

Abstract

The invention discloses a state data generation method and system for reinforcement learning, and the method comprises the steps: obtaining all first state data of an intelligent agent in a first learning stage, and obtaining second state data in the first state data, wherein the distance between the second state data and a learning target accords with a preset step number range; training the variational automatic encoder by using the first state data to obtain a trained encoder of the variational automatic encoder, and sampling to obtain a plurality of first potential variables; inputting second state data into the trained encoder to obtain a plurality of second potential variables; selecting a third potential variable meeting a preset condition from the first potential variable and the second potential variable; and inputting the third potential variable into a decoder of the variational automatic encoder to generate initial state data of the second learning stage. By generating thenew state data, the agent starts to explore from the new state data instead of the original state data at a certain probability, the exploration time is shortened, the reward frequency is increased, and the method is suitable for any reinforcement learning method.

Description

technical field [0001] This application relates to the field of deep reinforcement learning, in particular to a method and system for generating state data for reinforcement learning. Background technique [0002] Deep Reinforcement Learning (DRL) has achieved remarkable success in continuous decision-making tasks such as Go and robotic arms. In reinforcement learning (Reinforcement Learning, RL), the agent (Agent) observes the environment (Environment) state (State), selects the action with the greatest reward (Reward) expectation, and receives the feedback given by the environment. Training by temporal difference or policy gradient, or an Actor-Critic Algorithm combining the two. However, in real-world applications, a common problem is that the rewards (Reward) are sparse. Some tasks can only be rewarded when the goal is completed, and the rewards are zero in other cases. And the target is difficult to explore through traditional exploration strategies, which brings grea...

AI Technical Summary

Problems solved by technology

However, the introduction of internal rewards deviates from the original goal, and it is easy to cause instability in the training process

Images