Sparse Tensor Decoding With On-the-Fly ZVC Bitmap Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computing systems face challenges in efficiently processing and storing large multi-dimensional data structures like tensors due to the significant memory and bandwidth requirements of deep neural networks (DNNs) and convolutional neural networks (CNNs), particularly because they often contain many zero-value elements that are not efficiently handled by traditional compression methods.
Innovation Solution
An in-line sparsity-aware tensor data distribution system that decodes zero-value-compression (ZVC) data vectors, allowing for flexible tensor data processing without storing uncompressed data through the on-chip memory hierarchy, by reconstructing sparsity bitmaps on the fly and using programmable schedules to manage non-zero elements, thereby reducing data movement and improving energy efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional compression methods are used to store tensor data, then storage space is reduced, but zero-value elements are not efficiently handled leading to wasted memory and bandwidth
Solution Approach 1:
The patent segments the tensor data storage by separating zero-value elements from non-zero elements. It uses a sparsity bitmap to track zero positions and a zero-value-compression data vector to store only non-zero elements, thereby efficiently utilizing storage space while maintaining data accessibility.
Solution Approach 2:
The patent introduces a new dimension to the storage structure by adding a sparsity bitmap layer that operates at the bit level. This bitmap layer provides metadata about zero-value positions, enabling the system to efficiently navigate and compress the underlying tensor data without losing structural information.
2Reliability
If uncompressed tensor data is stored to maintain full data integrity, then data accuracy is preserved, but memory consumption and bandwidth requirements increase significantly
Solution Approach 1:
The patent creates a compressed representation of the tensor data that copies only the essential non-zero elements into a zero-value-compression data vector. The sparsity bitmap serves as a metadata copy that preserves the structural information needed to reconstruct the original tensor, thereby maintaining data integrity with reduced memory consumption.
3Quantity of substance
If zero-value-compression data vectors are used to reduce memory usage, then storage efficiency improves, but decoding complexity increases
Solution Approach 1:
The patent performs preliminary organization of data during the encoding phase by separating zero and non-zero elements and creating the sparsity bitmap. This preliminary action simplifies the decoding process, as the decoder only needs to read the compact representation and reconstruct the tensor without performing complex analysis or transformations.
Data Source
AI summary
Methods, systems, articles of manufacture, and apparatus are disclosed to decode zero-value-compression data vectors. An example apparatus includes: a buffer monitor to monitor a buffer for a header including a value indicative of compressed data; a data controller to, when the buffer includes compressed data, determine a first value of a sparse select signal based on (1) a select signal and (2) a first position in a sparsity bitmap, the first value of the sparse select signal corresponding to a processing element that is to process a portion of the compressed data; and a write controller to, when the buffer includes compressed data, determine a second value of a write enable signal based on (1) the select signal and (2) a second position in the sparsity bitmap, the second value of the write enable signal corresponding to the processing element that is to process the portion of the compressed data.