Multi-Stage Sparse Data Compression for Faster Memory Transactions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern digital systems face significant overhead in storing and processing large volumes of data due to the presence of numerous zero values, which can be addressed by supporting sparsity in hardware to achieve performance speed-up, particularly in AI applications where computation of zero values is often ignored.
Innovation Solution
A hierarchical compression technique is employed to store and retrieve data efficiently by using a multi-stage compression algorithm that includes bit masks to specify positions of non-zero values, allowing for faster memory transactions and reduced memory bandwidth, enabling processors to operate on compressed data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in full uncompressed form in memory, then all values (including zeros) are available for processing, but memory bandwidth and read/write times are significantly reduced due to the large volume of data
Solution Approach 1:
The patent extracts only the non-zero values from the dataset and stores them in memory along with their position information (bit masks). This eliminates the storage and transmission of redundant zero values, significantly reducing memory bandwidth requirements and read/write times while maintaining the ability to reconstruct the full dataset when needed.
Solution Approach 2:
The patent segments the dataset into non-zero values and their position indicators (bit masks), storing them separately. This segmentation allows the system to store and process only the essential data elements, reducing memory bandwidth consumption while enabling efficient reconstruction of the complete dataset through the decompression circuit.
2Loss of time
If compressed data is stored in memory to reduce memory bandwidth, then memory read and write times are reduced, but additional decompression processing is required
Solution Approach 1:
The patent introduces a decompression circuit as an intermediary component that automatically reconstructs the full dataset from the compressed stored form. This circuit uses the stored bit masks to place non-zero values in their correct positions, providing the full dataset to the processor without requiring complex software-based decompression algorithms.
3Productivity
If sparsity is supported in hardware to achieve performance speed-up, then computation of zero values is skipped, but hardware complexity increases to support sparsity operations
Solution Approach 1:
The patent performs preliminary compression of the dataset before storage, organizing non-zero values and their positions in advance. This preliminary action enables the hardware to directly utilize the compressed format for sparse computations without requiring additional complexity during the computation phase, as the sparsity structure is already established in the stored data.
Data Source
AI summary
Embodiments of the present disclosure include a digital circuit and method for multi-stage compression. Digital data values are compressed using a multi-stage compression algorithm and stored in a memory. A decompression circuit receives the values and performs a partial decompression. The partially compressed values are provided to a processor, which performs the final decompression. In one embodiment, a vector of N length compressed values are decompressed using a first bit mask into two N length sets having non-zero values. The two N length sets are further decompressed using two M length bit masks into M length sparse vectors, each having non-zero values.