Machine Learning Model Compression via Dense Range Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models, particularly large-scale neural networks, face challenges in being deployed on resource-constrained devices due to their large memory footprint, and existing quantization techniques often result in a reduction in model quality.
Innovation Solution
A machine learning model compression system that identifies dense ranges in the distribution of parameter values and converts these values to a compact format, while setting values outside these dense ranges to zero, thereby reducing the model's memory footprint without significantly degrading prediction accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of stationary object
If post-training quantization techniques are applied to reduce model size, then memory footprint is reduced, but prediction accuracy deteriorates due to loss of valuable information
Solution Approach 1:
The patent applies different compression treatments to different portions of the model based on their importance. Important parameters (those with larger magnitudes) are preserved with full precision, while less important parameters (those with smaller magnitudes) are compressed to lower precision. This local differentiation allows the model to maintain high prediction accuracy for critical parameters while achieving significant compression overall.
Solution Approach 2:
The patent dynamically changes the precision parameters for different model parameters based on their magnitude and importance. Instead of applying a uniform quantization level, the system adjusts the number of bits allocated to represent each parameter according to its contribution to the model's output, thereby optimizing the trade-off between model size and accuracy.
2Productivity
If large-scale neural networks are used to solve complex problems, then problem-solving capability is improved, but memory resources required exceed available capacity on mobile devices
Solution Approach 1:
The patent segments the model parameters into different groups based on their importance and magnitude. By dividing the parameter space into segments (e.g., high-magnitude parameters, low-magnitude parameters), the system can apply different storage and computation strategies to each segment, enabling large models to run on devices with limited memory by loading only necessary segments or compressing less critical ones.
Solution Approach 2:
The patent introduces a new dimension of precision control, allowing the model to operate with variable precision across different parameters. This dimensional approach to parameter representation enables the model to maintain functionality with reduced memory requirements by using fewer bits for parameters that contribute less to the overall output.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for compresses a machine learning model having a plurality of parameters. In one aspect, one of the methods includes obtaining trained values of a set of parameters for at least a portion of a machine learning model; identifying one or more dense ranges for the trained values; determining a least number of bits required to represent each trained value within the one or more dense ranges; identifying a second format having a range that is smaller than a range of the first format; and generating a compressed version of the at least a portion of the machine learning model.