Neural Network Convolution Sparsity Exploitation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processing technologies face inefficiencies in execution time and power dissipation due to the need to compute zero values during convolutional operations in Convolutional Neural Networks (CNNs).
Innovation Solution
The method involves exploiting Input Feature Map (IFM) and kernel sparsity by fetching and processing IFM and kernel tiles, performing convolutional operations exclusively or in parallel, and skipping zero value computations using front-end cores, which generate Output Feature Map (OFM) tiles by synchronizing Processing Elements (PEs) and accumulating results efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional convolutional operations are performed on all values including zeros, then complete feature map processing is achieved, but execution time and power dissipation increase due to unnecessary computations
Solution Approach 1:
The patent extracts and removes zero values from the input feature map and kernel data before performing convolutional operations. By separating non-zero elements and processing only those, the system eliminates unnecessary computations while maintaining the integrity of the neural network processing function.
Solution Approach 2:
The patent implements skipping mechanisms that allow the convolutional operation to bypass zero values entirely. The system skips directly to non-zero elements, performing multiply-accumulate operations only on relevant data points, thereby reducing execution time without compromising output accuracy.
2Reliability
If zero values are computed during convolutional operations, then complete mathematical accuracy is maintained, but power dissipation increases due to unnecessary computations
Solution Approach 1:
The patent extracts and removes zero values from the input feature map and kernel data before performing convolutional operations. By separating non-zero elements and processing only those, the system eliminates unnecessary computations while maintaining the integrity of the neural network processing function.
Solution Approach 2:
The patent implements skipping mechanisms that allow the convolutional operation to bypass zero values entirely. The system skips directly to non-zero elements, performing multiply-accumulate operations only on relevant data points, thereby reducing execution time without compromising output accuracy.
3Productivity
If all IFM and kernel tiles are processed including zero values, then complete feature transformation is achieved, but processing time increases due to sparsity exploitation opportunities
Solution Approach 1:
The patent segments the input feature map and kernel data into non-zero and zero components. By organizing data into sparse representations that highlight only non-zero elements, the system enables efficient processing of only relevant portions while maintaining complete feature transformation capability.
Solution Approach 2:
The patent implements skipping mechanisms that allow the convolutional operation to bypass zero values entirely. The system skips directly to non-zero elements, performing multiply-accumulate operations only on relevant data points, thereby reducing execution time without compromising output accuracy.
Data Source
AI summary
A method and an apparatus for processing layers in a neural network fetch Input Feature Map (IFM) tiles of an IFM tensor and kernel tiles of a kernel tensor, perform a convolutional operation on the IFM tiles and the kernel tiles by exploiting IFM sparsity and kernel sparsity, and generate a plurality of OFM tiles corresponding to the IFM tiles.