Sparse Neural Network Accelerator Zero-Operand Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep convolutional neural networks (DCNs) face increased sparsity in multiplication operands due to techniques that reduce computational overhead, leading to inefficient use of computation units in hardware accelerators.
Innovation Solution
A hardware accelerator design that determines whether operands are non-zero before processing, avoiding multiplication by zero-valued operands and utilizing queues to maximize computation unit utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If weight pruning, ReLU activation, or dropout techniques are applied to reduce computational overhead, then classification quality is improved, but sparsity of multiplication operands increases
Solution Approach 1:
The patent extracts and removes zero-valued operands from the computation pipeline before they reach the MAR units. By detecting zero operands and excluding them from processing, the system eliminates wasted computation while preserving the benefits of sparsity-inducing techniques like weight pruning and dropout.
Solution Approach 2:
The patent performs preliminary detection of zero operands before multiplication occurs. The zero-detection logic operates in advance of the MAR unit processing, preventing useless computations from being initiated and enabling the system to adapt dynamically to sparse input patterns.
2Loss of energy
If sparsity of multiplication operands increases, then computational overhead is reduced, but utilization of computation units deteriorates
Solution Approach 1:
The patent introduces dynamic control logic that adapts the computation pipeline based on the sparsity pattern of input operands. The system dynamically routes non-zero operand pairs to MAR units while skipping zero operands, allowing computation unit utilization to respond automatically to varying sparsity levels without fixed configuration.
Solution Approach 2:
The patent introduces an intermediary zero-detection and filtering mechanism between the operand storage and MAR units. This intermediary component analyzes operand values and selectively forwards only non-zero pairs to computation units, acting as a mediator that reconciles the conflict between sparsity and utilization.
3Manufacturing precision
If all operand pairs are loaded to MAR unit for processing, then computation completeness is maintained, but energy consumption increases due to multiplying by zero
Solution Approach 1:
The patent extracts zero-valued operands from the processing stream and removes them before they can consume energy in multiplication operations. This extraction occurs at the input stage of the MAR unit, ensuring that no energy is wasted on multiplying by zero while all necessary non-zero computations are preserved.
Solution Approach 2:
The patent converts the harmful effect of sparsity (which normally causes wasted energy on zero multiplications) into a benefit by using the presence of zeros as a signal to skip computation. The zero-detection mechanism transforms what would be energy-wasting operations into opportunities for computational savings.
Data Source
AI summary
A method, a computer-readable medium, and an apparatus for a sparse neural network are provided. The apparatus may include a hardware accelerator. The apparatus may determine, for each pair of operands to be processed by a MAR unit, whether both operands of the pair are non-zero. The apparatus may prevent a pair of operands to be processed by the MAR unit from being loaded to a multiplier of the MAR unit when an operand of the pair of operands is zero. The apparatus may place the pair of operands into one of a plurality of queues when both operands of the pair of operands are non-zero.