Neural Network Decompression Pipeline for Mixed Compression Schemes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks require large memory and bandwidth for processing large quantities of data, leading to increased hardware requirements and power consumption, especially when executing repeated manipulations of activation and weight values.
Innovation Solution
A neural network accelerator with a decompression pipeline that uses zero value and shared value compression schemes to reduce data volume, allowing for on-the-fly decompression and compression, and selecting the appropriate compression scheme based on operating mode or data type to optimize performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If large quantities of neural network data are stored and transmitted, then processing capability is maintained, but hardware requirements and power consumption increase
Solution Approach 1:
The patent extracts and removes redundant information from neural network data by identifying and eliminating duplicate weight values and activation values. By taking out only the essential unique data and storing frequency information separately, the system reduces the volume of data that needs to be stored and transmitted, thereby lowering power consumption while maintaining processing capability.
Solution Approach 2:
Instead of storing each individual data value and its frequency separately, the patent inverts the approach by storing unique values once and using frequency counters to represent how many times each value occurs. This inversion allows the system to reconstruct the original data distribution without transmitting or storing redundant copies, reducing hardware requirements and energy usage.
2Quantity of substance
If data compression is applied to reduce data volume, then bandwidth usage and memory requirements decrease, but data processing complexity increases
Solution Approach 1:
The patent segments the data processing into distinct phases: compression phase where frequency information is extracted and stored, and decompression phase where data is reconstructed using frequency counters. By dividing the process into manageable segments with clear interfaces, the system reduces overall complexity despite the added compression functionality.
Solution Approach 2:
The patent introduces frequency counters as intermediary structures that mediate between the compressed data representation and the original data format. These counters act as a bridge, allowing the system to work with compact representations during storage and transmission while automatically reconstructing the full data set during processing, thus managing complexity through an intermediary layer.
Data Source
AI summary
Described herein is a neural network accelerator (NNA) with a decompression unit that can be configured to perform multiple types of decompression. The decompression may include a separate subunit for each decompression type. The subunits can be coupled to form a pipeline in which partially decompressed results generated by one subunit are input for further decompression by another subunit. Depending on which types of compression were applied to incoming data, any number of the subunits may be used to produce a decompressed output. In some embodiments, the decompression unit is configured to decompress data that has been compressed using a zero value compression scheme, a shared value compression scheme, or both. The NNA can also include a compression unit implemented in a manner similar to that of the decompression unit.