Neural Network Bit Vector Sparsity Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Typical neural networks face inefficiencies in processing complex input data due to high operational requirements, which is exacerbated in low-power, high-performance embedded systems like smartphones, leading to suboptimal performance in real-time data analysis.
Innovation Solution
The proposed solution involves generating a bit vector based on zero and non-zero input activations and weights, merging these bit vectors into the data, sorting, and performing operations only on non-zero values, thereby reducing unnecessary computations and enhancing processing speed and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If typical neural network devices process complex input data using conventional methods, then they can perform neural network operations, but they require a large amount of operations and consume excessive power, leading to inefficient performance on resource-constrained systems
Solution Approach 1:
The patent extracts and identifies zero-value activations from the input data stream, separating them from non-zero activations. This extraction allows the system to eliminate unnecessary operations involving zero values, reducing both computational operations and power consumption while maintaining processing speed for meaningful data.
Solution Approach 2:
Instead of performing operations on all input activations, the patent applies partial action by selectively processing only non-zero activations. The bit vector mechanism enables the system to perform exactly the necessary operations (on non-zero values) without excessive computation on zero values, optimizing the balance between processing completeness and resource consumption.
2Productivity
If typical neural network devices perform operations on all input activations including zeros, then they maintain simple processing logic, but they execute unnecessary operations that reduce processing efficiency and increase latency
Solution Approach 1:
The patent performs preliminary action by generating bit vectors that identify zero and non-zero activations before the main computational operations. This pre-processing step allows the system to skip unnecessary operations on zero values, significantly reducing processing latency and increasing operations per second without adding complex runtime decision-making.
Solution Approach 2:
The bit vector serves as an intermediary data structure that mediates between the input activations and the computational units. This intermediary enables efficient routing of data, allowing the system to quickly identify and process only non-zero activations, thereby increasing operational throughput and reducing overall processing time.
3Productivity
If neural network devices process all input data uniformly, then they maintain simple hardware architecture, but they cannot efficiently exploit the sparsity pattern of neural network data, leading to suboptimal performance on embedded systems
Solution Approach 1:
The patent applies local quality by treating different activations differently based on their zero or non-zero status. The bit vector mechanism creates local differentiation in the data stream, allowing computational units to apply different processing paths to different data elements, thereby exploiting sparsity patterns without requiring globally complex architectural changes.
Solution Approach 2:
The patent changes the parameter representation of activations by introducing bit vectors that encode zero/non-zero status. This parameter transformation enables the system to efficiently recognize and handle sparse data patterns, improving processing efficiency through a relatively simple addition of bit vector generation and masking logic to the hardware architecture.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A processor-implemented neural network method includes: generating a bit vector based on whether each of a plurality of input activations within a neural network is 0; merging the bit vector into the input activations such that bit values within the neural network included in the bit vector are most significant bits (MSBs) of multi bit expressions of the input activations; merging the bit vector into weights such that the bit values included in the bit vector are MSBs of multi bit expressions of the weights; sorting the input activations and the weights based on bits corresponding to the MSBs; and implementing the neural network, including performing operations between the sorted input activations and the sorted weights.