Convolutional neural network accelerator based on feature map sparsity

Abstract

The invention discloses a convolutional neural network accelerator based on feature map sparsity. The convolutional neural network accelerator comprises an input feature map coding module, a weight coding module, a data flow control module, a sparse matrix calculation module and a bus. The bus is respectively connected with the data flow control module, the input feature map coding module, the weight coding module and the sparse matrix calculation module; the input feature map encoding module encodes the feature map according to the 0 element in the feature map; the weight coding module provides corresponding weight data for the sparse matrix calculation module according to coding information input into the feature map coding module; the data flow control module controls the working modes of other modules according to the register information; and the sparse matrix calculation module performs convolution calculation by using data provided by the input feature map coding module and the weight coding module. According to the method, the accelerator can be switched to use the sparsity in the weight, the method can be flexibly applied, and the sparse weight is supported.

Description

technical field [0001] The invention relates to the field of convolutional neural network hardware accelerators, and belongs to the technical field of integrated circuit hardware acceleration, in particular to a convolutional neural network accelerator based on feature map sparsity. Background technique [0002] In recent years, artificial intelligence technology has developed rapidly, and deep neural networks have made major breakthroughs in natural language processing and computer vision. With the popularization of mobile devices and the emergence of the Internet of Things, the demand for deploying neural networks on mobile devices or Internet of Things devices is increasing day by day. However, the large amount of data movement and computational complexity in the algorithm pose a huge challenge to the power consumption and performance of terminal equipment, hindering the deployment of CNN algorithms in the fields of smart phones, smart cars, and smart homes. [0003] At ...

AI Technical Summary

Problems solved by technology

However, a large amount of data movement and computational complexity in the algorithm pose a huge challenge to the power consumption and performance of the terminal device, which hinders the application deployment of the CNN algorithm in the fields of smart phones, smart cars, and smart homes.

[0003] At present, there have been many methods for hardware acceleration of CNN algorithms, and the design is very good in terms of flexibility and multiplier utilization efficiency, but these designs cannot break through the computational power requirements of the convolutional layer, or guide the model in Sparse the weights during training, then use the sparsity of the weights for calculations, and then sparsely guide the model requires additional training time, which is not conducive to the direct deployment of the model

[0004] In the paper "An Efficient Hardware Accelerator for Structured SparseConvolutional Neural Networks on FPGAs", the use of the 0 element of the weight is realized through a large-scale lookup table, but the method used in the paper needs to consume a lot of lookup table resources on the one hand, and on the other hand requires In the training phase of the neural network model, additionally guide the generation of 0 elements of the model weight, which is not conducive to the direct deployment of the model

At the same time, in the current mainstream neural network models, a large number of Relu activation functions (f(x)=MAX(0,x)) are used, which makes a large number of 0 elements appear in the feature map. The current method does not use these 0 elements.

Images