Zero Coefficient Skipping Convolution Neural Network Engine

Abstract

A convolution engine, such as a convolution neural network, operates efficiently with respect to sparse kernels by implementing zero skipping. An input tile is loaded and accumulated sums are calculated for the input tile for non-zero coefficients by shifting the tile according to a row and column index of the coefficient in the kernel. Each coefficient is applied individually to tile and the result written to an accumulation buffer before moving to the next non-zero coefficient. A 3D or 4D convolution may be implemented in this manner with separate regions of the accumulation buffer storing accumulated sums for different indexes along one dimension. Images are completely processed and results for each image are stored in the accumulation buffer before moving to the next image.

Description

RELATED APPLICATION

[0001] This application claims the benefit of U.S. Provisional Application Ser. No. 62 / 373,518 entitled ZERO COEFFICIENT SKIPPING CONVOLUTION NEURAL NETWORK ENGINE and filed Aug. 11, 2016, which is hereby incorporated herein by reference in its entirety.BACKGROUNDField of the Invention

[0002] This invention relates to systems and methods for performing matrix convolution, such as for use in implementing a convolution neural network.Background of the Invention

[0003] Many machine learning applications using Convolutional Neural Networks (CNN) require very high computation and memory bandwidth. One way to reduce the requirement is to zero prune the coefficients and skip the computation when a coefficient is zero. These existing software and hardware optimization techniques are based on matrix multiplications. One example is the Sparse Matrix Multiplication technique described in Sparse Convolutional Neural Networks (Baoyuan Liu, Min Wangl, Hassan Forooshl, Marshall Tappe...

Images