Method for selecting reward function in adversarial imitation learning

Abstract

The invention provides a method for selecting reward functions in adversarial imitation learning. The method comprises the following steps: constructing a strategy network with a parameter theta, a discrimination network with a parameter w and at least two reward functions; obtaining teaching data under an expert strategy and storing the teaching data into an expert data buffer containing an expert track, wherein the input of the control strategy network is a state returned by the simulation environment, and the output is a decision action; enabling the discrimination network to update parameters by using the state action pair under the expert strategy and the state action pair of the strategy network; in the award calculation stage, judging that the input of the network is a state actionpair of the strategy network, and judging that the output value is an award value obtained by award function calculation; selecting a reward function of the current task according to the sizes of theperformance indexes of different reward functions; and storing the parameters of the strategy network corresponding to the selected reward function. And the intelligent agent learns under the guidanceof different reward functions, and then selects an optimal reward function in a specific task scene according to the performance evaluation index.

Description

technical field [0001] The invention relates to the technical field of reward function selection, in particular to a method for selecting a reward function in adversarial imitation learning. Background technique [0002] In recent years, as deep learning has made major breakthroughs in the fields of image recognition, speech technology, and natural language processing, deep reinforcement learning, which combines deep neural networks and reinforcement learning, has also achieved results in large-scale strategy optimization problems such as Go and StarCraft. surpassed human performance. One of the bottlenecks of reinforcement learning is that when faced with practical control problems such as autonomous driving and robots, it takes time and effort to design a reasonable reward function based on expert experience. Data-driven imitation learning provides a way to solve this problem. It does not need to manually design reward functions, and can learn strategies comparable to exp...

AI Technical Summary

Problems solved by technology

[0005] In order to solve the problem that the reward function form used in the imitation learning algorithm based on generative confrontation network is not optimal in all tasks in the prior art, and therefore cannot achieve optimal performance in multiple tasks, the present invention provides a A Method of Reward Function Selection in Adversarial Imitation Learning

Images