Hardware architecture for accelerating Deep Q-Network algorithm and design space exploration method of hardware architecture

Abstract

The invention discloses a hardware architecture for accelerating a Deep Q-Network algorithm and a design space exploration method of the hardware architecture. The hardware architecture comprises: a universal processor module, which is responsible for interacting with an external environment and realizing calculation of a reward function, and is also responsible for maintaining a Deep Q-Network algorithm experience pool; an external DDR memory, which is responsible for storage of an experience pool of the Deep Q-Network algorithm; an AXI bus interface, which is of a universal AXI bus interfacestructure and is responsible for achieving transmission and feedback of control signals and data signals between the universal processor and the FPGA programmable logic module; a Target Q module, which is responsible for realizing forward reasoning calculation of a Target Q network; and a Current Q module, which is responsible for realizing forward reasoning and back propagation of the Current Qnetwork. According to the invention, under the condition of highly optimizing the FPGA hardware architecture, the real-time calculation of the Deep Q-Network algorithm is realized.

Description

technical field [0001] The invention belongs to the technical field of artificial intelligence; in particular, it relates to a hardware framework for accelerating a Deep Q-Network algorithm and a design space exploration method thereof. Background technique [0002] Deep reinforcement learning is an emerging artificial intelligence technology, which evolved from the combination of traditional reinforcement learning algorithms and deep learning algorithms, and is mainly used in the fields of robot control, automatic driving, search and recommendation, etc. Representative algorithms in the field of study. However, since the Deep Q-Network algorithm includes two types of calculations, the forward reasoning and the backpropagation of the neural network, when the neural network is large in scale, there are problems of large storage resource consumption and high computational complexity. However, the current research and implementation of deep reinforcement learning usually use a...

AI Technical Summary

Problems solved by technology

However, since the Deep Q-Network algorithm includes two types of calculations, the forward reasoning and the backpropagation of the neural network, when the neural network is large in scale, there are problems of large storage resource consumption and high computational complexity.

However, the current research and implementation of deep reinforcement learning usually uses a large-scale image processing unit GPU board server, which is difficult to apply in edge computing scenarios with limited hardware resources and power consumption, and its applicability is not high; The FPGA hardware computing architecture is not optimized, nor is the design space explored

Images