Dynamic Sparse Neural Network Execution via Predictable Output Neuron Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network execution techniques require costly offline sparsity calculations and are often one-size-fits-all, leading to reduced accuracy and idle processing cycles due to the use of input or output sparsity rather than their combination.
Innovation Solution
A system and method for dynamically executing sparse neural networks by reducing and expanding input and output dimensions using a predictable output neuron map, optimizing sparsity for each layer to improve efficiency and reduce latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If offline sparsity calculations are performed, then sparsity optimization is achieved, but computational cost increases
Solution Approach 1:
The patent performs sparsity calculations offline before neural network execution, pre-determining which neurons and connections should be pruned. This preliminary action separates the optimization phase from the execution phase, allowing the neural network to run efficiently without performing costly sparsity calculations during inference.
Solution Approach 2:
The patent introduces dynamic sparsity adjustment that allows the sparsity pattern to be modified based on input characteristics. Different input samples can trigger different sparsity levels, enabling the system to adapt between computational efficiency and accuracy requirements in real-time during execution.
2Productivity
If input sparsity or output sparsity is used individually, then implementation simplicity is maintained, but processing cycles become idle reducing productivity
Solution Approach 1:
The patent combines both input sparsity and output sparsity techniques into a unified framework. By merging these two approaches, the system eliminates idle processing cycles that occur when using only one technique, as the combination allows continuous productive work throughout the neural network execution pipeline.
Solution Approach 2:
The patent creates a universal sparsity framework that can handle both input and output sparsity through the same architectural mechanisms. This multi-functional approach uses shared components for both types of sparsity, managing complexity while enabling combined operation of multiple sparsity techniques.
3Measurement precision
If one-size-fits-all sparsity techniques are applied, then implementation ease is improved, but accuracy decreases for certain inputs
Solution Approach 1:
The patent applies different sparsity patterns to different regions of the neural network based on local input characteristics. Rather than applying a uniform sparsity technique globally, the system identifies specific input samples that benefit from higher or lower sparsity levels and applies optimized sparsity patterns locally to those regions, preserving accuracy for critical inputs while maintaining efficiency elsewhere.
Solution Approach 2:
The patent implements dynamic sparsity adjustment that adapts the sparsity pattern based on the specific characteristics of each input sample. This dynamic approach allows the system to switch between different sparsity configurations during execution, optimizing accuracy for inputs that require it while maintaining high efficiency for inputs that tolerate greater sparsity.
Data Source
AI summary
Systems and methods for dynamically executing sparse neural networks are provided. In one implementation, a system for providing dynamic sparsity in a neural network may include at least one memory storing instructions and at least one processor configured to execute the instructions to: reduce an input vector and a set of weights of the neural network, execute an input layer of the neural network using the reduced input vector and set of weights to generate a reduced output vector; expand the reduced output vector to a full output vector using first predictable output neurons (PONs); using a PON map, reduce a dimension of the full output vector; execute subsequent layers of the neural network using the reduced full output vector to produce a second reduced output vector; and expand the second reduced output vector to a second full output vector using second PONs.