Hardware Entropy Compression for Memory Bandwidth Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression and decompression techniques often implemented in software fail to effectively address the memory bandwidth bottleneck in processing systems, particularly when handling machine learning workloads, and assume a minimum sparsity in data that may not be applicable to all neural networks.
Innovation Solution
A hardware-based system that uses entropy encoding to compress and decompress data by generating a probability distribution of codewords and storing them in a code table, allowing for efficient transfer of data between processors and memory devices, and includes a decompressor device to process compressed data and send it to processors, while a compressor device compresses data for storage in memory devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data compression is implemented in software, then flexibility in handling different data types is improved, but memory bandwidth bottleneck and processing time increase
Solution Approach 1:
The patent replaces software-based compression/decompression with hardware-based entropy encoding/decoding circuits. This substitution of mechanical/software systems with dedicated hardware circuits resolves the contradiction by providing both the flexibility needed for different data types and the high-speed processing required to avoid memory bandwidth bottlenecks.
Solution Approach 2:
The patent introduces entropy encoder and decoder as intermediary hardware components between memory and processors. These intermediaries handle the compression and decompression tasks in hardware, allowing flexible adaptation to different neural network data types while maintaining high processing speeds that prevent memory bandwidth bottlenecks.
2Quantity of substance
If compression is applied to all data, then storage efficiency is improved, but computational expense increases due to processing incompressible data
Solution Approach 1:
The patent applies different processing treatments to different portions of data based on their compressibility characteristics. The entropy encoder automatically adapts to the local properties of neural network data (weights, biases, activations) and applies compression only where beneficial, avoiding the computational expense of compressing already-incompressible data while maintaining storage efficiency for compressible portions.
3Speed
If hardware-based compression is used, then processing speed is improved, but device complexity increases
Solution Approach 1:
The patent implements entropy encoding in hardware by changing the operational parameters of the compression system. The entropy encoder and decoder are designed with configurable parameters that can be adjusted based on the specific neural network workload, allowing high processing speeds while managing hardware complexity through parameter optimization rather than structural complexity.
Data Source
AI summary
Methods, systems, and apparatus, including computer-readable storage media for hardware compression and decompression. A system can include a decompressor device coupled to a memory device and a processor. The decompressor device can be configured to receive, from the memory device, compressed data that has been compressed using an entropy encoding, process the compressed data using the entropy encoding to generate uncompressed data, and send the uncompressed data to the processor. The system can also include a compressor device configured to generate, from uncompressed data, a probability distribution of codewords, generate a code table from the probability distribution, and compress incoming data using the generated code table.