Dual-Sparsity Neuron Computation for Faster ANN Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing technologies, such as CPUs, GPUs, and FPGAs, face challenges in efficiently processing artificial neural networks (ANNs) due to high power consumption, programming complexity, and inefficient handling of massive data movement, particularly in large-scale industrial applications, without effectively addressing the parallel nature and redundancy of neuron computations.
Innovation Solution
A system comprising arithmetic and accumulation units that identify and perform mathematical operations only on relevant pairs of input values and weights, using identifiers to assign accumulation units for result accumulation, thereby optimizing computations by skipping irrelevant operations and reducing hardware requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If fully dense neural network layers are used, then computational accuracy is maintained, but computational performance and energy efficiency deteriorate
Solution Approach 1:
The patent applies parameter changes by transforming neural network weights and activations from dense format to sparse format, changing the density parameter from 1.0 to a lower value. This allows the same computational accuracy to be maintained while significantly improving computational performance and energy efficiency through sparsity-aware operations
Solution Approach 2:
The patent implements partial action by applying sparsity transformations selectively to specific neural network layers rather than all layers. The system identifies candidate layers and applies dual sparsity transformations only where beneficial, maintaining accuracy while improving performance in a targeted manner
2Productivity
If sparsity transformations are applied to neural network layers, then computational performance is improved, but implementation complexity increases
Solution Approach 1:
The patent segments the neural network into different layer types (convolutional, fully-connected, transpose convolutional) and applies appropriate sparsity transformations to each. This segmentation allows complex sparsity operations to be broken down into manageable, layer-specific implementations
Solution Approach 2:
The patent introduces intermediary data structures (sparse weight matrices, sparse activation vectors, and their dense equivalents) that facilitate the transformation between dense and sparse representations. These intermediaries simplify the implementation of sparsity-aware operations while maintaining compatibility with standard neural network frameworks
3Use of energy by moving object
If dual sparsity transformations are implemented, then energy consumption is reduced, but training and inference time increases
Solution Approach 1:
The patent performs preliminary action by pre-computing and storing sparse weight matrices and identifying non-zero elements before the forward pass. This preprocessing step reduces the computational burden during inference, allowing energy-efficient sparse operations to be executed faster despite the initial transformation time
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods for accelerating computation of an artificial neural network (ANN) are provided. An example method comprises receiving, by processing units coupled with arithmetic units and accumulation units, a first plurality of first values and a second plurality of second values associated with one or more neurons of the ANN, generating, by the processing units, a plurality of pairs, wherein each pair of the plurality of pairs has a first value of the first plurality and a second value of the second plurality and the first value and the second value satisfy criteria, performing, by the arithmetic units, mathematical operations on pairs of the plurality of pairs to obtain results; accumulating, by the accumulation units, the results to obtain accumulated results, and determining, by the processing units and based on the accumulated results, an output of the neurons.