A multi-agent cross-modal depth deterministic strategy gradient training method based on image input

Abstract

The invention relates to a multi-agent cross-modal depth deterministic strategy gradient training method based on image input. Firstly, a mechanical arm training environment in a simulation platform is constructed; Then two director intelligent bodies and a student intelligent body which are input by utilizing different modalities are constructed; Secondly, based on a depth deterministic strategygradient algorithm, an actor module and a critic module of a director and an actor module of a learner are trained, and finally a cross-modal depth reinforcement learning mechanical arm training algorithm based on image input is achieved; When the overall training is finished; a mob actor network can be used only; high-dimensional image input is received; the action capable of completing the taskis output; Moreover, the method is very suitable for being migrated to a real environment, and since the real environment cannot provide full-state modal information and the image modal information isrelatively easy to obtain, after an actor network of a mob is trained, the demand of the full-state modal information can be abandoned, and a relatively good output strategy can be obtained by directly utilizing image input.

Description

technical field [0001] The invention belongs to the reinforcement learning algorithm in the field of artificial intelligence and robot, and more specifically relates to a multi-agent cross-modal depth deterministic strategy gradient training method based on image input. Background technique [0002] In recent years, due to the rapid increase of computing resources and the development of deep learning networks, the use of large amounts of data to train supervised learning algorithm models has achieved very good results in many fields. At present, there are two main categories of methods for applying learning-based methods in the field of robot control: self-supervised learning and reinforcement learning. The method of self-supervised learning is to let the robot directly or indirectly collect task data and label it, and then rely on a large amount of labeled training data to complete the training of the deep neural network. [0003] Compared with the method of self-supervise...

AI Technical Summary

Problems solved by technology

But when the explored environment is too huge, for example, when using a 1920x1024x4 color-depth image as the input of the actor, the convergence of the critic cannot be guaranteed.

Excessively large exploration space causes the actor network to greatly reduce the exploration efficiency, and the reduction of exploration efficiency will lead to the inability to obtain effective training samples. Since the actor and critic in DDPG share the same set of training samples, this leads to the critic's training being explored by the actor. Efficiency impact

[0009]The second disadvantage is that the gradient provided by the critic to the actor for training is a weak guide, which does not guarantee that the direction of each gradient update is correct, which will As a result, in the huge exploration space, the samples explored by the actor cannot train the critic well, and the critic in turn uses the wrong gradient to guide the learning of the actor, resulting in worse and worse training effects. Such tasks cannot be completed using existing technologies

Images