Reinforcement learning model optimization method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention discloses a reinforcement learning model optimization method and device, a storage medium and electronic equipment, and relates to the field of deep reinforcement learning. The method comprises: obtaining a variational reasoning network based on historical data generated by interaction between an actor reviewer network and an environment, and training the actor reviewer network through the variational reasoning network; generating an initial actor double-reviewer network based on the trained actor reviewer network; and replacing a dominant function in the initial actor double-commentator network with a generalized dominant function to obtain an actor double-commentator network corresponding to the initial actor double-commentator network. By adopting the embodiment of the invention, the utilization rate of historical data can be improved, the generalization of the deep reinforcement learning model is improved, and the training time is reduced.

Description

technical field [0001] The present application relates to the field of deep reinforcement learning, in particular to an optimization method, device, storage medium and electronic equipment for a reinforcement learning model. Background technique [0002] Reinforcement learning refers to an unsupervised active learning method in which the agent (Agent) accumulates frequent interactions with the environment to obtain rewards, with the goal of maximizing the ultimate benefit. The quality of the actions of the agent is completely determined by the signal of the environmental feedback, and this is used as a way to guide the reinforcement learning model, and through exploration to obtain strategies that can obtain greater value. An excellent reinforcement learning algorithm can well balance exploration (Exploration) and development (Exploitation), and better explore the environment after maximizing the development of known feedback information. [0003] Although the direction of ...

AI Technical Summary

Problems solved by technology

[0003] Although the direction of the optimization method for the reinforcement learning model of the agent is novel and has attracted much attention, it also has the problem of low utilization efficiency of sampling experience and the problem of too much dependence on the modeling of the environment due to its characteristics.

The former problem is because the historical data provided to the ActorCritic network for updating is usually replaced once the on-orbit algorithm is updated, causing a lot of problems of insufficient utilization of experience, and interacting with the environment to generate a large amount of data, only after The simple random sampling method cannot quickly and accurately enable the agent to learn effective information

The latter problem is due to the lack of generalization ability of the existing deep reinforcement learning, which leads to the agent's excellent performance in one environment, but it is difficult to reproduce in another environment.

Images