Sparse CNN Convolution via Block Multiply Accumulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face high computational complexity and power consumption issues, making them challenging to execute on low-power embedded devices, especially for applications requiring high-resolution image processing like Advanced Driver Assistance Systems (ADAS).
Innovation Solution
The implementation of a block multiply accumulation (BMA) operation in CNNs, which reduces complexity by eliminating multiplications with zero-value coefficients and performing convolutions as block operations, along with sparsification techniques to force more coefficients to zero, thereby reducing the number of multiply-accumulate operations (MACs).
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full frame CNN applications are executed on low power embedded devices, then high resolution image processing capability is achieved, but power consumption exceeds single digit watts
Solution Approach 1:
The patent extracts and removes zero-value coefficients from the convolutional filter weights, creating a sparse representation of the filter. This extraction eliminates unnecessary multiplication operations while preserving the essential computational functionality, thereby reducing power consumption without sacrificing image processing resolution
Solution Approach 2:
The patent changes the sparsity parameter of the filter weights from dense (all values present) to sparse (only non-zero values retained). This parameter transformation reduces the number of active compute operations from full frame to reduced complexity convolution, enabling execution within single digit watts while maintaining processing capability
2Measurement precision
If standard convolution operations are performed on high resolution images, then processing accuracy is maintained, but computational complexity becomes too high for embedded devices
Solution Approach 1:
The patent extracts only the non-zero coefficients from the convolutional filter, removing redundant computational elements. This extraction maintains processing accuracy by preserving only the essential weight values while dramatically reducing computational complexity for embedded devices
Solution Approach 2:
The patent applies partial convolution by only computing operations for non-zero coefficients rather than performing complete full-frame convolution. This partial action approach maintains sufficient processing accuracy for high resolution images while reducing computational complexity to manageable levels for embedded hardware
3Productivity
If more coefficients are forced to zero through sparsification, then number of multiply-accumulate operations is reduced, but filter precision may be compromised
Solution Approach 1:
The patent employs feedback mechanisms during training that monitor and adjust the sparsification process. This feedback ensures that coefficients are set to zero only when confidence is high, preventing over-sparsification that would compromise filter precision while maximizing the reduction of multiply-accumulate operations
Solution Approach 2:
The patent performs preliminary training with relaxed sparsity constraints to establish a baseline filter precision, then progressively applies stronger sparsification pressure. This preliminary action ensures that the filter maintains sufficient precision before aggressive sparsification reduces the number of MAC operations
Data Source
AI summary
A method for convolution in a convolutional neural network (CNN) is provided that includes accessing a coefficient value of a filter corresponding to an input feature map of a convolution layer of the CNN, and performing a block multiply accumulation operation on a block of data elements of the input feature map, the block of data elements corresponding to the coefficient value, wherein, for each data element of the block of data elements, a value of the data element is multiplied by the coefficient value and a result of the multiply is added to a corresponding data element in a corresponding output block of data elements comprised in an output feature map.