Systolic Array Based Sparse Neural Network Processor

Abstract

The invention provides a sparse neural network processor based on a systolic array, which includes a storage unit, a control unit, a sparse matrix operation array, a calculation unit, and a bus array. The storage unit is used to store weights, gradients, features, and instruction sequences for scheduling data streams. According to the control of the instruction sequence, the control unit takes out the data required for the training and reasoning process from the storage unit, converts it into a sparse matrix operation format, and sends it to the sparse matrix operation array. The sparse matrix operation array includes multiple processing units connected in a systolic array to complete the sparse matrix operation. The computing unit is used to complete element-wise operations such as nonlinear activation functions. The bus array uses internal data transfer to send the same data segment to different rows of the systolic array to reduce storage overhead. This processor makes full use of the sparsity of weights and features to improve the speed and power consumption ratio of neural network training and reasoning, and has the advantages of high concurrency and low bandwidth requirements.

Description

technical field [0001] The present invention relates to neural network technology and computer architecture, in particular to a computer architecture used for sparse neural network computing. Background technique [0002] In recent years, deep learning has gradually made more and more remarkable achievements in fields such as image recognition and speech processing. However, with the continuous increase of network depth, the computing power and memory access bandwidth required in the process of deep neural network training and inference are gradually difficult to be satisfied by traditional computing platforms. Therefore, the industry and academia have proposed various domain-specific architectures (domain specified architectures) applied to neural networks to meet this demand. Among them, the systolic array architecture has attracted great attention from the industry and academia due to its characteristics of high concurrency and low bandwidth requirements. [0003] On th...

AI Technical Summary

Problems solved by technology

However, it is worth noting that due to the irregularity of the sparse patterns of neural network weights, gradients, and features, this sparsity of neural networks is often difficult to be effectively utilized, especially on highly regular computing platforms such as systolic arrays.

Images