Multi-objective reinforcement learning method and device based on Pareto optimization

Abstract

The invention relates to the field of reinforcement learning, and discloses a multi-target reinforcement learning method and device based on Pareto optimization, and the method comprises the steps: solving a multi-target reinforcement learning problem through employing a summarization mode, and calculating a Q value of each sub-target for each strategy; performing non-dominated sorting on the sub-target Q values by using a Pareto dominating theory to obtain a Pareto leading edge set; randomly selecting an action from the Pareto frontier set to interact with an environment; generating a multi-target DQN algorithm based on a Pareto frontier set, and training a target network by using the DQN algorithm to generate a strategy network; and updating the expectation of the Pareto frontier sub-target Q value according to the policy network, and approaching a set of all Pareto optimal deterministic policies by directly popularizing the deep Q network to multiple targets, thereby showing better performance, rapid convergence and relatively better stability, and providing more diversified solutions.

Description

technical field [0001] The present application relates to the field of reinforcement learning, in particular to a multi-objective reinforcement learning method and device based on Pareto optimization. Background technique [0002] Reinforcement learning (RL) is a framework that learns an optimal action policy for an agent based on task-related future rewards received from the agent's environment. While most RL algorithms are structured to achieve a specific goal, many real-world applications are inherently characterized by the presence of multiple, potentially conflicting goals, such as economic systems, healthcare, transportation, management, natural resources, mission planning and robot control. [0003] As a generalization of standard reinforcement learning methods, multi-objective reinforcement learning (MORL) addresses the need for trade-offs between competing objectives. In MORL, the reward signal is a vector for each agent, where each element represents a maximized ...

AI Technical Summary

Problems solved by technology

[0007] The main purpose of the present invention is to provide a multi-objective reinforcement learning method and device based on Pareto optimization, which aims to solve the problem that the existing technology cannot achieve fast convergence and relatively good stability in deep learning

Images