ML Model Parameter Compression Using Bit-Plane Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models, especially deep learning models, require a large number of parameters, leading to significant memory and computational resource usage, which is problematic in resource-constrained devices.
Innovation Solution
The proposed solution involves a method for lossless compression of machine learning model parameters using a sequence of operations including conversion from initial to target encoding formats, bit plane transformation, and bit masking, which takes advantage of inherent sparsity in the data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of stationary object
If lossy compression is used to reduce model size, then memory footprint is reduced, but model accuracy and reliability are reduced
Solution Approach 1:
The patent segments the model parameters into multiple bit planes based on their significance. Each bit plane represents a specific range of parameter values, allowing selective compression of less significant bit planes while preserving more significant ones. This segmentation enables differential compression where different parts of the parameter space are compressed to different degrees, reducing overall model size while maintaining accuracy for critical parameters.
Solution Approach 2:
The patent transforms the parameter representation by converting from standard floating-point format to a quantized format organized by bit planes. This parameter transformation allows the system to work with discrete bit representations rather than continuous floating-point values, enabling lossless compression through bit-level operations while maintaining the ability to reconstruct original parameter values exactly.
2Volume of stationary object
If model parameters are compressed, then storage requirements are reduced, but computational resources increase due to decompression overhead
Solution Approach 1:
The patent performs preliminary organization of parameters into bit planes during the compression phase, creating a structured representation that facilitates efficient decompression. The bit planes are pre-processed and stored in an optimized format, so that during inference, the decompression operation simply involves reassembling the pre-organized bit planes rather than performing complex computational transformations, thereby reducing runtime computational energy.
3Reliability
If more parameters are used in the model, then prediction accuracy improves, but memory footprint increases substantially
Solution Approach 1:
The patent creates a compressed copy of the model parameters using bit plane representation. Instead of storing the full-precision floating-point parameters, the system stores a compressed bit-plane version that occupies less memory. During inference, the parameters are decompressed from the bit-plane representation back to their original format, allowing the model to operate with full precision while the storage requirement is reduced to the compressed copy.
Data Source
AI summary
Certain aspects of the present disclosure provide techniques and apparatus for improved machine learning model compression. A set of parameters for a machine learning model is accessed, where the set of parameters are formatted according to a first encoding. A converted set of parameters is generated based on applying a conversion operation to format the set of parameters according to a second encoding. A set of bit planes is generated based on applying a bit plane transformation to the converted set of parameters, and a compressed set of parameters for the machine learning model is generated based on applying a bit mask operation to one or more bit planes of the set of bit planes.