Neural Network Data Compression Pipeline for Memory and Bandwidth Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks require large memory and bandwidth for processing and storing large quantities of data, leading to increased hardware requirements and power consumption, particularly due to the repeated manipulation of activation and weight values.
Innovation Solution
A neural network accelerator with a decompression and compression pipeline that utilizes zero value and shared value compression schemes to reduce data volume, implemented in a data move engine with decompression and compression units, allowing for on-the-fly data processing and storage optimization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If large quantities of neural network data are stored and transmitted, then processing capability is maintained, but hardware requirements and power consumption increase
Solution Approach 1:
The patent extracts and removes redundant information from neural network data by identifying and eliminating duplicate weight values and activation values. By taking out only the essential unique values and their positions, the system reduces data volume for storage and transmission while maintaining the ability to reconstruct the original data for processing, thus lowering power consumption without sacrificing processing capability
Solution Approach 2:
The patent changes the representation parameters of neural network data by encoding weight and activation values in a compressed format. Instead of storing full precision values for all neurons, the system transforms the data into a compact representation that records only unique values and their positions, reducing memory requirements and bandwidth usage while preserving processing functionality
2Productivity
If large quantities of neural network data are stored and transmitted, then processing capability is maintained, but hardware requirements increase
Solution Approach 1:
The patent extracts and removes redundant information from neural network data by identifying and eliminating duplicate weight values and activation values. By taking out only the essential unique values and their positions, the system reduces data volume for storage and transmission while maintaining the ability to reconstruct the original data for processing, thus lowering power consumption without sacrificing processing capability
Solution Approach 2:
The patent changes the representation parameters of neural network data by encoding weight and activation values in a compressed format. Instead of storing full precision values for all neurons, the system transforms the data into a compact representation that records only unique values and their positions, reducing memory requirements and bandwidth usage while preserving processing functionality
3Quantity of substance
If data compression is applied to reduce data volume, then memory and bandwidth needs are reduced, but data processing complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing the compressed representation of neural network data before processing. The compression algorithm identifies unique values and their positions in advance, creating a compact data structure that can be efficiently stored and transmitted. This preliminary compression step reduces the data volume that needs to be handled during actual neural network processing, offsetting the initial compression complexity with ongoing processing efficiency gains
Data Source
AI summary
Described herein is a neural network accelerator (NNA) with a decompression unit that can be configured to perform multiple types of decompression. The decompression may include a separate subunit for each decompression type. The subunits can be coupled to form a pipeline in which partially decompressed results generated by one subunit are input for further decompression by another subunit. Depending on which types of compression were applied to incoming data, any number of the subunits may be used to produce a decompressed output. In some embodiments, the decompression unit is configured to decompress data that has been compressed using a zero value compression scheme, a shared value compression scheme, or both. The NNA can also include a compression unit implemented in a manner similar to that of the decompression unit.