A Multi-Core Processor-Single Graphics Processor Deep Reinforcement Learning Acceleration Method

Abstract

The present invention proposes a multi-core processor-single graphics processor deep reinforcement learning acceleration method, which can establish a deep reinforcement learning framework based on the PPO algorithm on the CPU+GPU platform, and at the same time invents an accelerated collection sampling and A pipelined approach to inference. The multi-environment process simulation process is realized on a multi-core CPU, and multiple environment processes are arranged on each core. At the same time, the CPU plays the role of controlling the data. Implement the neural network model inference process on the GPU. The CPU and the GPU respectively store the memory space for the action network and the evaluation network parameters. In the process of interaction between the environment and the agent, each time information is stored in the memory of the CPU’s experience information pool. Truncate parameters and filter conditions, and extract batch-scale data from its own experience information pool memory for training. Through load balancing, when the simulation time of the CPU single-core stacking environment process is the same as the neural network model reasoning time, the present invention can realize the pipeline structure of parallel execution of environment sampling and action reasoning: through parallel half CPU simulator and half GPU reasoning running, and prepare for the next half-CPU simulator and half-GPU inference process during data transmission, which can further accelerate the speed of reinforcement learning, reaching an overall training speed nearly twice that of traditional methods.

Description

technical field [0001] The invention belongs to the field of computers, in particular to a deep reinforcement learning acceleration method based on a multi-core central processing unit (CPU)-single graphics processing unit (GPU) platform. Background technique [0002] Deep reinforcement learning (DRL, deep reinforcement learning) is the product of the combination of deep learning and reinforcement learning. It integrates the powerful understanding ability of deep learning on perception problems such as vision and the decision-making ability of reinforcement learning, and realizes end-to-end learning. . The emergence of deep reinforcement learning has made reinforcement learning technology truly practical, able to solve complex problems in real-world scenarios, and has been widely used in various engineering fields, such as industrial manufacturing, robot positioning recognition, game games, etc. [0003] Literature published in 2013 (Mnih V, Kavukcuoglu K, Silver D, et al. ...

AI Technical Summary

Problems solved by technology

Literature (Liang Xingxing, Feng Yanghe, Ma Yang, Cheng Guangquan, Huang Jincai, Wang Qi, Zhou Yuzhen, Liu Zhong. A review of multi-agent deep reinforcement learning [J / OL]. Acta Automatica Sinica: 2019.05) proposed the DRL multi-environment (Agent ) communication method and influencing parameters, this document does not involve how to realize it on hardware, and the acceleration that can be achieved by the same algorithm in different hardware environments is different

Images