Grouped Weight Compression for Low-Bandwidth Neural Network Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks face significant bandwidth challenges due to the large size of weight arrays stored in memory, leading to high system bandwidth usage during runtime.
Innovation Solution
A data compression method that encodes groups of data items using header and body portions, with interleaved body data, allowing for efficient decompression with reduced hardware requirements and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If weights are stored in uncompressed form in memory, then decompression speed is fast, but system bandwidth usage is high and memory requirements are large
Solution Approach 1:
The weight data is divided into groups of N weights, where each group is encoded with a single h-bit header followed by N body portions of b bits each. This segmentation allows the decompression process to read one header and then sequentially extract N weight values without repeated header parsing, reducing decompression overhead while achieving compression ratios that lower bandwidth usage.
Solution Approach 2:
The body portions are pre-interleaved during compression such that the first b bits of each body portion correspond to the least significant bits of the N weight values in sequence. This preliminary arrangement enables the decompression process to efficiently reconstruct weight values by simply reading consecutive b-bit segments and combining them with previously read more significant bits, minimizing computational complexity during decompression.
2Quantity of substance
If weights are compressed using traditional methods, then memory usage is reduced, but decompression hardware complexity and power consumption increase
Solution Approach 1:
The compression scheme segments weight data into groups with uniform structure (h-bit header + N×b-bit body portions), enabling a simple state-machine decompression approach that reads headers, then sequentially processes body portions. This regular structure avoids complex adaptive decoding algorithms, reducing hardware complexity while achieving memory savings through compression.
Solution Approach 2:
The invention changes the representation parameters of weight data by using variable-length body portions (b bits) within fixed-size groups, where the header indicates the actual weight values and body portions provide additional precision only when needed. This parameter-based approach reduces average memory usage while keeping decompression logic simple through fixed group structures.
3Quantity of substance
If variable-length encoding is used for each weight, then compression ratio is high, but decompression speed decreases due to variable parsing overhead
Solution Approach 1:
Instead of applying variable-length encoding to each individual weight (which would require parsing variable-length fields N times), the invention segments N weights into a single fixed-structure group with one h-bit header followed by N uniform b-bit body portions. This segmentation reduces parsing operations from N variable-length decodes to one header decode plus N simple bit extractions, maintaining high compression while improving decompression speed.
Data Source
AI summary
A data compression method comprises encoding groups of data items by generating, for each group, header data comprising h-bits and a plurality of body portions each comprising b-bits and each body portion corresponding to a data item in the group. The value of h may be fixed for all groups and the value of b is fixed within a group, wherein the header data for a group comprises an indication of b for the body portions of that group. In various examples, b=0 and so there are no body portions. In examples where b is not equal to zero, a body data field is generated for each group by interleaving bits from the body portions corresponding to data items in the group. The resultant encoded data block, comprising the header data and, where present, the body data field can be written to memory.