Automatic Control Method Based on Multi-objective Reinforcement Learning Algorithm Using Gradients

Abstract

The invention relates to the technical field of automatic control, in particular to an automatic control method based on a gradient-based multi-objective reinforcement learning algorithm. The present invention uses the known gradient information in the reinforcement learning algorithm to update the function parameters. Compared with the single-strategy multi-objective reinforcement learning algorithm, this algorithm can obtain the Pareto frontier solution set, and can select different parameters according to different needs during actual control. Control strategy; compared with the convex hull method in the multi-strategy multi-objective reinforcement learning algorithm, this algorithm does not depend on the linear weighting of the target return, and can obtain the control strategy of the non-convex area in the frontier solution; compared with the multi-strategy multi-objective reinforcement The multi-parameter method in the learning algorithm, this algorithm can solve all Pareto frontier solutions at one time, and the algorithm efficiency is high; compared with the method of combining multi-objective optimization algorithm in the multi-strategy and multi-objective reinforcement learning algorithm, this algorithm uses Gradient information speeds up the convergence speed of the algorithm.

Description

technical field [0001] The invention relates to the technical field of automatic control, in particular to an automatic control method based on a gradient-based multi-objective reinforcement learning algorithm. Background technique [0002] In order to solve the continuous time-domain decision-making problem of robot automatic control, reinforcement learning is an effective tool, but in the actual process, many automatic control problems are multi-objective problems. It is difficult to find the correct mapping relationship between the target and the state, and learn the correct value function and strategy for the single-objective reinforcement learning algorithm, so multi-objective reinforcement learning is required. At present, multi-objective reinforcement learning is mainly divided into two categories, one is single-strategy methods, and the other is multi-strategy methods. The single-strategy method can only obtain a single strategy, and cannot obtain multiple Pareto fr...

AI Technical Summary

Problems solved by technology

Multi-strategy methods are divided into the following categories: convex hull method, variable parameter method, and reinforcement learning algorithm combined with multi-objective optimization algorithm. Among them, the convex hull method uses the weighted summation of gradients, so it is difficult to obtain the non-convex area. Strategy; the variable parameter method is repeated execution but the strategy method, the algorithm efficiency is low; the reinforcement learning algorithm combined with the multi-objective optimization algorithm fails to use the known gradient information in the reinforcement learning algorithm, and the convergence speed is slow

Images