Sparse CNN Inference Using Quantized Sparsity Masks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) with great depths require significant computational power and energy, making it challenging to run them in a low-cost and energy-efficient manner.
Innovation Solution
Implement quantization and sparse convolution techniques to reduce the overhead of CNNs by determining a sparsity mask for output feature maps, indicating non-zero entries, and computing only those values, using a highly quantized version of the network without pre-training or external knowledge.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a convolutional neural network with great depths is used, then model accuracy and capability are improved, but computational power requirements and energy consumption increase significantly
Solution Approach 1:
The patent extracts and utilizes the sparsity property inherent in CNN computations by identifying zero-value outputs and excluding them from subsequent processing. A sparsity mask is generated to indicate which output positions contain non-zero values, allowing the system to take out and process only the necessary computations, thereby reducing energy consumption while maintaining model accuracy.
Solution Approach 2:
Instead of computing all output values in convolution operations, the patent applies partial action by computing only the non-zero values indicated by the sparsity mask. This partial computation approach significantly reduces the number of operations required, lowering energy consumption while preserving the essential information needed for accurate inference.
2Reliability
If a convolutional neural network with great depths is used, then model accuracy and capability are improved, but computational overhead increases
Solution Approach 1:
The patent extracts the sparsity pattern from CNN computations and uses it to eliminate redundant calculations. By generating a sparsity mask that identifies zero-value positions, the system extracts only the necessary computational paths, thereby reducing computational overhead while maintaining model accuracy.
Solution Approach 2:
The patent performs preliminary action by generating the sparsity mask before executing the full convolution operation. This pre-computation of the sparsity pattern allows subsequent operations to skip unnecessary calculations, significantly reducing computational overhead while preserving model accuracy.
3Use of energy by moving object
If quantization and sparse convolution are applied, then computational overhead and energy consumption are reduced, but processing speed may be affected
Solution Approach 1:
The patent performs preliminary action by pre-computing the sparsity mask before the main convolution operation. This allows the system to identify and exclude zero-value positions in advance, so that during inference, only necessary computations are performed, thereby maintaining high processing speed while reducing energy consumption.
Solution Approach 2:
The patent ensures continuity of useful action by maintaining the sparsity mask throughout the inference process, allowing continuous skipping of zero-value computations. This continuous optimization maintains high inference speed while consistently reducing energy consumption across all processing stages.
Data Source
AI summary
Various implementations of the subject matter as described herein relate to a sparse convolutional neural network. In some implementations, a computer-implemented method comprises: quantizing an input feature map to obtain a quantized input feature map; determining, based on the quantized input feature map, a sparsity mask for an output feature map through a quantized version of a convolutional neural network, the sparsity mask indicating positions of non-zero entries in the output feature map; and determining, based on the input feature map, the non-zero entries indicated by the sparsity mask in the output feature map through the convolutional neural network.