Inline Decompression for Sparse Neural Network Data Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks with many zeros, such as those using rectified linear unit (ReLU) activation functions or pruned recurrent neural networks, face challenges in efficiently compressing and decompressing data due to the high sparsity of weights, which affects memory usage and bandwidth in neural network inference accelerators.
Innovation Solution
The proposed solution involves a compression engine that stacks compresses data by removing zero words from uncompressed data blocks, creating a compressed data block and a map metadata to map non-zero words, allowing for efficient decompression by a decompression engine that recovers the original data using the metadata.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is compressed by removing zero words, then memory efficiency and bandwidth are improved, but device complexity increases due to the need for compression and decompression engines with metadata mapping
Solution Approach 1:
The data block is segmented into individual words that can be independently processed. The compression engine processes each word separately, determining whether to retain it or replace it with a zero placeholder based on sparsity criteria. This segmentation enables efficient compression by focusing only on non-zero elements while maintaining the ability to reconstruct the original data through the decompression engine using metadata mapping.
2Quantity of substance
If compression is applied to reduce zero words, then bandwidth requirements are reduced, but processing time increases due to the additional compression and decompression steps
Solution Approach 1:
The compression engine performs preliminary action by identifying and marking non-zero words during the compression phase, storing their positions in metadata. This preliminary identification allows the decompression engine to efficiently reconstruct the original data by only placing non-zero values at their correct positions, rather than processing every word. The preliminary action of creating the metadata map during compression enables faster decompression despite the additional processing step.
Data Source
AI summary
Stack compression refers to compression of data in one or more dimensions. For uncompressed data blocks that are very sparse, i.e., data blocks that contain many zeros, stack compression can be effective. In stack compression, uncompressed data block is compressed into compressed data block by removing one or more zero words from the uncompressed data block. A map metadata that maps the zero words of the uncompressed data block is generated during compression. With the use of the map metadata, the compressed data block can be decompressed to restore the uncompressed data block.