Accelerating CNN Convolution by Skipping Zero-Valued Inputs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional convolution operations in CNNs require extensive computational resources and time due to redundant multiply-by-zero operations, especially after rectified linear unit (ReLU) layers that nullify negative values, leading to inefficient processing.
Innovation Solution
The system employs circuitry that identifies non-zero input values, computes products with kernel coefficients, and stores them in buffers, allowing for parallel computation and summation of output values without redundant operations, thereby reducing processing time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional convolution operations are performed with all input values including zeros, then complete computational accuracy is maintained, but processing time and computational resources are excessively consumed
Solution Approach 1:
The patent extracts and identifies zero-valued input elements before the convolution operation, separating them from non-zero elements. This allows the system to exclude zero elements from subsequent multiplication operations, eliminating redundant computations while maintaining computational accuracy for non-zero elements.
Solution Approach 2:
The patent applies partial action by performing convolution operations only on non-zero input elements rather than all elements. The system identifies the subset of non-zero elements and applies the convolution kernel exclusively to these elements, reducing the total number of operations required while preserving the accuracy of the final result.
2Reliability
If all multiplication operations are performed including multiply-by-zero operations, then computational completeness is ensured, but computational efficiency deteriorates
Solution Approach 1:
The patent extracts zero-valued elements from the input data before processing, creating a separated representation of zero and non-zero elements. This extraction enables the convolution operation to skip multiplication with zero elements entirely, improving computational efficiency while maintaining reliability through proper handling of the extracted zero elements.
Solution Approach 2:
The system performs partial multiplication operations only on non-zero input elements combined with the convolution kernel. By applying the operation partially rather than completely to all elements, the system achieves the same reliable result with significantly reduced computational effort, as zero multiplications are eliminated.
3Speed
If the system processes only non-zero input values, then processing speed is improved, but computational complexity increases due to zero identification
Solution Approach 1:
The patent applies preliminary action by identifying and marking zero-valued input elements before the main convolution operation begins. This preliminary identification step, performed once before processing, enables subsequent operations to quickly skip zero elements without repeated checking, improving processing speed while managing complexity through a single-pass preprocessing approach.
Data Source
AI summary
Described embodiments include a system that includes one or more buffers and circuitry. The circuitry is configured to process a plurality of input values, by identifying each of the input values that is not zero-valued, and, for each value of the identified input values, computing respective products of coefficients of a kernel with the value and storing at least some of the respective products in the buffers. The circuitry is further configured to compute a plurality of output values, by retrieving respective sets of stored values from the buffers, at least some of the retrieved sets including one or more of the products, and summing the retrieved sets. The circuitry is further configured to output the computed output values. Other embodiments are also described.