CNN Weight Compression with Grouped Headers and Interleaved Bits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face significant bandwidth challenges due to large arrays of weights stored in memory, requiring efficient compression and decompression methods to reduce system bandwidth usage.
Innovation Solution
A data compression method that encodes groups of data items using header and body portions, with interleaved body bits, allowing for efficient storage and decompression, reducing hardware requirements and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If weights are stored in memory for CNNs, then the neural network can function properly, but system bandwidth usage increases significantly
Solution Approach 1:
The weight array is divided into groups of data items, where each group is encoded separately with header data and body portions. This segmentation allows efficient compression while maintaining the ability to reconstruct the original weights for neural network operation.
Solution Approach 2:
The patent changes the representation parameters of weight data by encoding groups with variable-length body portions (b-bits) based on the actual data distribution. The header data (h-bits) indicates the body portion size, allowing adaptive compression that reduces bandwidth usage while preserving weight accuracy.
2Loss of energy
If compression encoding is applied to weight arrays, then bandwidth usage is reduced, but decompression complexity increases
Solution Approach 1:
The encoding process segments weight data into groups with header and body portions, making the compression manageable and systematic. The decoder similarly processes data in segmented groups, reducing the complexity burden on any single processing unit.
Solution Approach 2:
The patent performs preliminary analysis of weight data distribution during the encoding phase to determine optimal body portion sizes (b-bits) for each group. This preliminary action allows the decoder to anticipate the decoding requirements from the header data, reducing runtime complexity.
3Productivity
If large arrays of weights are stored, then the neural network can process complex data, but memory requirements and access time increase
Solution Approach 1:
Weight arrays are divided into multiple encoded groups that can be stored efficiently in memory. The segmentation allows selective loading of only the necessary groups for each computation, reducing memory access time while maintaining the ability to process complex neural network data.
Solution Approach 2:
The encoding process is performed in advance during model training or deployment, converting large weight arrays into compressed formats before runtime. This preliminary compression reduces the memory footprint and accelerates runtime access without affecting the neural network's processing capability.
4Quantity of substance
If compression is applied to weight data, then storage efficiency improves, but power consumption during decompression may increase
Solution Approach 1:
The decompression process operates on segmented groups of data items rather than entire weight arrays at once. This segmentation allows power-efficient incremental decompression, where only the necessary groups are decompressed and loaded into processing units, reducing overall power consumption.
Solution Approach 2:
The patent uses variable body portion sizes (b-bits) indicated in header data to optimize the balance between compression ratio and decompression effort. Groups with smaller body portions require less decompression processing, reducing power consumption while maintaining storage efficiency for the overall weight array.
Data Source
AI summary
A data compression method comprises encoding groups of data items by generating, for each group, header data comprising h-bits and a plurality of body portions each comprising b-bits and each body portion corresponding to a data item in the group. The value of h may be fixed for all groups and the value of b is fixed within a group, wherein the header data for a group comprises an indication of b for the body portions of that group. In various examples, b=0 and so there are no body portions. In examples where b is not equal to zero, a body data field is generated for each group by interleaving bits from the body portions corresponding to data items in the group. The resultant encoded data block, comprising the header data and, where present, the body data field can be written to memory.