Neural Network Accelerator Zero-Skipping for Sparse CNN Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial neural network (ANN) processing systems face inefficiencies in computational speed and power consumption, particularly in mobile devices and autonomous driving applications, due to the need for extensive computations and the handling of zero weights during convolution operations.
Innovation Solution
A hierarchically sparse convolution neural network (CNN) model with structured pruning and zero-skipping methods is employed, maintaining the ANN's shape and size while distributing zeros in a structured manner, utilizing a convolution accelerator (CA) hardware to skip zero processing and reduce bandwidth and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional ANN processing is used, then computation accuracy is maintained, but computational speed is slow and power consumption is high
Solution Approach 1:
The patent extracts and removes zero weights from the neural network computation. By identifying and eliminating zero-weight operations from the computational graph, the system processes only non-zero weights, directly improving computational speed while maintaining accuracy. This is achieved through sparse computation techniques that skip multiplications and additions involving zero values.
Solution Approach 2:
The patent segments the neural network computation into sparse and dense regions. By dividing the weight matrix into non-zero and zero elements, the system can apply optimized computation paths for each segment, processing non-zero weights efficiently while skipping zero-weight operations, thereby improving overall computational speed.
2Use of energy by moving object
If traditional ANN processing is used, then all weights are processed, but power consumption is high
Solution Approach 1:
The patent extracts zero weights from the computation pipeline to eliminate unnecessary energy consumption. By identifying zero-weight positions in advance and excluding them from the computation graph, the system avoids performing multiplications and additions that would consume power, directly reducing overall power consumption while maintaining computational accuracy.
Solution Approach 2:
The patent applies partial action by processing only the necessary subset of weights (non-zero weights) rather than all weights. This selective processing approach reduces the total number of computational operations, thereby reducing power consumption without compromising the accuracy of the neural network inference.
3Quantity of substance
If traditional ANN processing is used, then complete computation is performed, but bandwidth consumption is high
Solution Approach 1:
The patent extracts and removes zero-weight data from the computation flow. By identifying zero weights beforehand and excluding them from data transmission and processing pipelines, the system reduces the volume of data that needs to be moved through the system, thereby reducing bandwidth consumption and improving data utilization efficiency.
4Productivity
If structured pruning is applied to increase zero weights, then computational efficiency improves, but hardware complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-processing the neural network to identify and mark zero-weight positions before the actual computation. This pre-marking allows the hardware to efficiently skip zero-weight operations during inference without requiring complex real-time detection logic, thereby improving computation efficiency while keeping hardware complexity manageable.
Data Source
AI summary
Systems and methods for structured-pruning, zero-skipping and accelerated processing of an artificial neural network (ANN) are described. The ANN may include one or more convolution layers. 2D channels in filters of the convolution layers comprise fully pruned channels (FPCs), each containing only zero weights, and mixed channels (MCs), each containing at least one non-zero weight. At least a portion of the MCs satisfy a limited zero sequence (LZS) condition limiting the number and location of zeroes in the MC. The LZS condition may be based on a number of weights that a zero-skipping circuit of a computing system for processing the ANN is configured to evaluate and skip in a single cycle. Thus, when processing the structurally-pruned ANN using the zero-skipping method, the computing system may avoid processing zero weights. This may allow speeding up the ANN processing and reducing the power required for processing.