Dual Sparsity ANN Acceleration via Redundant Operation Skipping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies, including CPUs, GPUs, and FPGAs, face challenges in efficiently processing artificial neural networks (ANNs) due to high computational loads, power consumption, and programming complexity, particularly in handling massive data movement and integration with AI software environments.
Innovation Solution
A system and method that utilize arithmetic units, accumulation units, and processing units to identify and skip irrelevant operations by labeling and selecting only relevant pairs of input values and weights, thereby reducing redundant computations and optimizing hardware resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If GPUs are used to process large ANNs, then throughput capacity is improved, but power consumption increases excessively
Solution Approach 1:
The patent extracts and processes only the non-zero elements from sparse weight matrices and activation vectors, removing redundant zero-value computations entirely. This selective processing reduces the effective computational load from billions of operations to a fraction thereof, achieving high throughput without proportionally increasing power consumption
Solution Approach 2:
Instead of performing complete dense matrix multiplications, the patent applies partial action by computing only the necessary non-zero contributions to the output. This partial computation approach maintains productivity while significantly reducing energy expenditure compared to full GPU-based dense processing
2Productivity
If FPGAs are used to compute ANNs, then parallel processing capability is improved, but programming complexity increases
Solution Approach 1:
The patent develops a universal sparse computation kernel that can be deployed across different FPGA architectures without requiring architecture-specific programming. The same sparse matrix multiplication algorithm and data structures work across various FPGA platforms, reducing programming complexity while maintaining parallel processing advantages
Solution Approach 2:
The patent uses configurable parameters to adapt the sparse computation engine to different ANN sizes, sparsity levels, and hardware resources. By changing parameters rather than rewriting code, the same FPGA implementation can handle various workloads, simplifying the programming model while preserving parallel processing capability
3Reliability
If dense matrix operations are performed in ANNs, then computational completeness is improved, but computational load increases
Solution Approach 1:
The patent extracts only the meaningful non-zero computations from dense matrix operations, removing redundant zero multiplications and additions. This extraction maintains computational completeness for the actual data while eliminating the excessive computational load introduced by processing zero values
Solution Approach 2:
The patent applies partial action by performing computations only where needed (non-zero elements) rather than executing complete dense matrix multiplications. This approach maintains the reliability of results for actual data while reducing the overall computational load to necessary operations only
Data Source
AI summary
Systems and methods for accelerating computation of an artificial neural network (ANN) are provided. An example method comprises receiving, by processing units coupled with arithmetic units and accumulation units, a first plurality of first values and a second plurality of second values associated with one or more neurons of the ANN, generating, by the processing units, a plurality of pairs, wherein each pair of the plurality of pairs has a first value of the first plurality and a second value of the second plurality and the first value and the second value satisfy criteria, performing, by the arithmetic units, mathematical operations on pairs of the plurality of pairs to obtain results; accumulating, by the accumulation units, the results to obtain accumulated results, and determining, by the processing units and based on the accumulated results, an output of the neurons.