Sparse Neural Network Architecture Bypassing Zero Computations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network hardware accelerators are inefficient in computing sparse neural networks as they cannot bypass the computation of zero values in dataflow, limiting the benefits of sparsity in neural network architectures.
Innovation Solution
An efficient hardware architecture that bypasses the computation of zero values in dataflow by using processing engines (PEs) to store and reuse computed input neurons in local SRAM, and employs a decomposition technique to compute dense networks by generating intermediate neurons for additional input, thereby optimizing computations in sparse neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If current neural network hardware accelerators are used to compute sparse neural networks, then the computation can be performed, but the computation of zero values cannot be bypassed resulting in wasted energy and reduced efficiency
Solution Approach 1:
The patent extracts and removes zero-value computations from the dataflow by implementing a sparsity-aware architecture that identifies and bypasses zero multiplications. The system separates non-zero computations from zero computations, executing only the necessary non-zero operations while skipping zero operations entirely, thereby eliminating wasted energy on futile calculations.
Solution Approach 2:
The patent implements dynamic sparsity handling where the hardware architecture adapts its computation path based on the actual sparsity pattern of the input data. The system dynamically routes data through computation paths that are activated only when non-zero values are present, allowing the architecture to optimize its energy consumption based on the actual computational needs rather than following a fixed computation schedule.
2Quantity of substance
If sparse neural networks are used to reduce matrix coefficients, then the number of coefficients is reduced, but existing hardware accelerators cannot effectively utilize this sparsity
Solution Approach 1:
The patent applies local quality by implementing different processing characteristics for different parts of the dataflow based on their sparsity patterns. The architecture identifies regions with high sparsity and applies optimized computation paths specifically tailored for those regions, while maintaining standard computation paths for dense regions. This localized optimization allows the system to fully exploit the sparsity benefits where they exist without compromising the processing of dense regions.
Solution Approach 2:
The patent changes the computational parameters by adjusting the activation thresholds and computation routing decisions based on the observed sparsity patterns. The system monitors the density of incoming data and dynamically adjusts its computation strategy, switching between sparse-optimized paths and standard paths based on the actual parameter characteristics of the data being processed.
Data Source
AI summary
A system and method for computing a sparse neural network having a plurality of output layers, each of which has a neuron value. Processing engines (PEs) each have a local memory for storing neurons for use with different weight values in a following cycle. A multiplexer selects between the input neuron or the output of the memory. Output from the multiplexor is received along with a weight input to a multiplier whose output is directed to an integrator. A decomposition technique performs a network computation through the use of intermediate neurons when the input neuron is larger than the local memory capacity, and provides data reuse by reusing neurons stored in local memory. Neural systems can be implemented using a neural index to address each of multiple PEs and a parallel-serial first-in-first-out (FIFO) to serially store values in main memory.