Sparse CNN Accelerator Eliminating Zero Multiplications
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face challenges in energy efficiency and computational requirements, particularly in mobile platforms like autonomous vehicles and cameras, due to the large computation needs and energy consumption during inference processes, especially with deeper networks and larger input sets.
Innovation Solution
A sparse convolutional neural network (SCNN) accelerator that processes compressed-sparse data by eliminating multiplications with zero operands, using a compact representation of weights and activations, and optimizing dataflow to reduce energy consumption and improve throughput, leveraging sparsity in weights and activations to fit more data in on-chip RAM and reduce DRAM accesses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If dense convolutional neural networks are used for inference, then model accuracy is maintained, but energy consumption and computational requirements increase significantly
Solution Approach 1:
The patent extracts and processes only the non-zero elements from the convolutional neural network computations. By identifying and processing only the significant (non-zero) weights and activations, the system eliminates redundant computations with zero operands, thereby reducing energy consumption while preserving the essential computational contributions that maintain model accuracy.
Solution Approach 2:
The patent changes the computational parameter from processing all elements (dense) to processing only non-zero elements (sparse). This parameter change in the data representation and processing approach allows the system to reduce the number of operations performed, directly lowering energy consumption while maintaining the accuracy-critical computations.
2Measurement precision
If deeper neural networks with larger input sets are deployed, then model capability and accuracy improve, but computation requirements become prohibitively large
Solution Approach 1:
The patent extracts only the necessary computational operations by identifying non-zero elements in deeper network layers. This extraction approach allows the system to handle complex, deep architectures with larger input sets by processing only the significant computational contributions, thereby managing computation requirements for advanced models.
Solution Approach 2:
The patent segments the computational workload by dividing it into individual non-zero element operations rather than processing entire dense matrices or layers. This segmentation allows the system to break down the prohibitively large computation requirements of deep networks into manageable units that can be processed efficiently.
3Quantity of substance
If compressed-sparse data format is used, then memory storage efficiency improves, but data processing complexity increases
Solution Approach 1:
The patent implements self-service by having the processing element automatically identify and handle non-zero elements within the compressed-sparse format. The system's own processing mechanism is designed to naturally work with the compressed representation, extracting and computing only the necessary elements without requiring additional complex decompression or conversion steps, thereby managing processing complexity while achieving memory efficiency.
Data Source
AI summary
A method, computer program product, and system perform computations using a sparse convolutional neural network accelerator. Compressed-sparse data is received for input to a processing element, wherein the compressed-sparse data encodes non-zero elements and corresponding multi-dimensional positions. The non-zero elements are processed in parallel by the processing element to produce a plurality of result values. The corresponding multi-dimensional positions are processed in parallel by the processing element to produce destination addresses for each result value in the plurality of result values. Each result value is transmitted to a destination accumulator associated with the destination address for the result value.