Sparse Vector Decoding With On-the-Fly Bitmap Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing DNN and CNN processing systems face inefficiencies due to the large memory and bandwidth requirements of storing and processing sparse vectors, leading to increased energy consumption and reduced performance, particularly in mobile devices with size, temperature, and power constraints.
Innovation Solution
An in-line sparsity-aware tensor data distribution system that decodes zero-value-compression data vectors, reconstructs sparsity bitmaps on the fly, and distributes non-zero elements to processing elements (PEs) based on a programmable schedule, reducing data movement and energy expenditure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If sparse vectors are stored and processed in traditional DNN/CNN systems, then complete data representation is achieved, but memory and bandwidth requirements increase significantly
Solution Approach 1:
The patent segments sparse vectors into two components: sparsity bitmaps (indicating zero/non-zero positions) and compressed data vectors (containing only non-zero values). This segmentation allows the system to store and process only essential information, reducing memory and bandwidth requirements while maintaining complete data representation through the combination of both components.
Solution Approach 2:
The patent extracts zero-value elements from sparse vectors, removing unnecessary data that consumes memory and bandwidth. By taking out only the essential non-zero elements and their position indicators, the system achieves efficient storage and processing without losing any meaningful information needed for DNN/CNN operations.
2Measurement precision
If sparse vectors are stored and processed in traditional systems, then data accuracy is maintained, but energy consumption increases
Solution Approach 1:
By segmenting sparse vectors into sparsity bitmaps and compressed data vectors, the system reduces the total data volume that needs to be transferred and processed. This segmentation maintains data accuracy because the bitmap preserves exact position information while the compressed vector contains only meaningful values, thereby reducing energy consumption without sacrificing precision.
Solution Approach 2:
The patent changes the representation parameters of sparse vectors from dense floating-point format to a hybrid format combining bitmaps (1-bit per element) and compressed vectors. This parameter change reduces the amount of data movement and processing operations required, directly lowering energy consumption while maintaining the precision needed for accurate DNN/CNN computations.
3Loss of energy
If zero-value-compression data vectors are decoded, then data movement is minimized, but decoding complexity increases
Solution Approach 1:
The patent applies preliminary compression during the data preparation phase, organizing sparse vectors into sparsity bitmaps and compressed data vectors before processing. This preliminary action eliminates the need for complex runtime decoding operations, as the data is already in an optimized format that can be directly utilized by DNN/CNN systems, thereby minimizing data movement energy without requiring complex decoding mechanisms during execution.
Data Source
AI summary
Methods, systems, articles of manufacture, and apparatus are disclosed to decode zero-value-compression data vectors. An example apparatus includes: a buffer monitor to monitor a buffer for a header including a value indicative of compressed data; a data controller to, when the buffer includes compressed data, determine a first value of a sparse select signal based on (1) a select signal and (2) a first position in a sparsity bitmap, the first value of the sparse select signal corresponding to a processing element that is to process a portion of the compressed data; and a write controller to, when the buffer includes compressed data, determine a second value of a write enable signal based on (1) the select signal and (2) a second position in the sparsity bitmap, the second value of the write enable signal corresponding to the processing element that is to process the portion of the compressed data.