Deep Learning Memory Compression With Adaptive Bit Width Packing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning networks face challenges in memory compression due to their specific access patterns and value distributions, which differ from general-purpose systems, leading to inefficiencies in energy consumption and memory usage.
Innovation Solution
A method for on-chip memory compression that adjusts data width based on the highest magnitude value within a group, using a bit width register to store the necessary bits for each value, allowing for efficient storage and decompression of data streams in deep learning networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If general-purpose compression approaches are used to compress memory data, then memory compression is achieved, but the system cannot efficiently handle the specific access patterns and value distributions of deep learning workloads
Solution Approach 1:
The patent applies local quality by making the compression approach specific to deep learning workloads rather than using general-purpose methods. The system is designed with specialized compression techniques that adapt to the particular access patterns and value distributions found in neural network operations, treating deep learning data differently from general-purpose data.
Solution Approach 2:
The patent changes parameters by adjusting compression strategies based on the specific characteristics of deep learning data. The system monitors access patterns and value distributions to dynamically modify compression parameters, optimizing for the unique properties of neural network workloads rather than using fixed general-purpose compression settings.
2Ease of manufacture
If fixed bit width is used for all data values, then storage simplicity is maintained, but memory space is wasted when many values have smaller magnitudes
Solution Approach 1:
The patent applies dynamics by making the bit width dynamic rather than fixed. The system adjusts the number of bits used to represent data values based on the actual magnitude of values in the dataset. This allows the storage format to adapt to the data distribution, using fewer bits for small values and more bits when larger magnitudes are encountered, thereby reducing overall memory footprint while maintaining simplicity.
3Quantity of substance
If variable bit width compression is implemented, then memory footprint is reduced, but decompression complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing the bit width information alongside the compressed data. During compression, the system determines the appropriate bit width for each value or group of values and stores this metadata in advance. This preliminary preparation simplifies the decompression process, as the bit width information is already available and does not require complex analysis during decompression.
Data Source
AI summary
A system and method for memory compression for deep learning networks. The method includes: compacting an input data stream by identifying a bit width necessary to accommodate the value from the input data stream with the highest magnitude; storing a least significant bits of the input data stream in a first memory store, the number of bits equal to the bit width, wherein if the value requires more bits than those currently left unused in the first memory store, the remaining bits are written into a second memory store; and outputting the value of the first memory store, as a consecutive part of a compressed data stream, with an associated width of the data in the first memory store when the first memory store becomes full and copying the value of the second memory store to the first memory store; and decompressing the compressed data stream.