Multi-agent reinforcement learning training method with high sample efficiency

Abstract

The invention discloses a multi-agent reinforcement learning training method with high sample efficiency, and the method comprises the following steps: (1) constructing a multi-agent system which is controlled by a multi-agent reinforcement learning model; (2) collecting a plurality of training samples and storing the training samples in a container; (3) extracting a training sample, and performing data preprocessing on the extracted training sample; (4) training the model by adopting a strategy delay updating mode, and when the executor network, the evaluator network and the target network of the model are updated, firstly updating the evaluator network for n * d times, then updating the executor network for n times, and finally updating the target network for n times; (5) training a strategy function and an evaluation function in the model by using the maximum action entropy and the objective function of function smoothing; and (6) after training is completed, using a multi-agent system for application. By utilizing the method, the problems of low sample efficiency and high training cost of the existing multi-agent reinforcement learning algorithm in a real task can be solved.

Description

technical field [0001] The invention belongs to the technical field of multi-agent reinforcement learning, and in particular relates to a multi-agent reinforcement learning training method with high sample efficiency. Background technique [0002] Facing increasingly complex and large-scale swarm control tasks in the real world, integrated single-agent solutions are increasingly facing resource and condition constraints. A multi-agent system is a system composed of multiple relatively simple interactive agents in the same environment. This system is often used to solve complex problems that are difficult to solve for independent agents and single-layer systems. Compared with independent agents or single-layer systems , the multi-agent system effectively improves the robustness, reliability and scalability of the whole system. With the development of emerging technologies such as the Internet and smart devices, more and more new task scenarios can be modeled as multi-agent s...

AI Technical Summary

Problems solved by technology

[0007] The invention provides a multi-agent reinforcement learning training method with high sample efficiency, which can solve the problems of low sample efficiency and high training cost faced by existing multi-agent reinforcement learning algorithms in real tasks

Images