Multi-Scale Dynamic Quantization for Deep Learning Model Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional quantized deep learning models face challenges in balancing prediction accuracy and compression rate, as uniform quantization often results in reduced accuracy due to excessive quantization of values in the middle section of the value distribution.
Innovation Solution
The method involves dividing the value distribution into sections using breaking points and applying different quantization parameters to each section, employing multi-scale dynamic quantization to approximate the original distribution more closely, and using techniques like percentile clipping and straight through estimators to improve gradient approximation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If uniform quantization is applied to all parameter values, then compression rate is improved, but prediction accuracy deteriorates due to excessive quantization of values in the middle section
Solution Approach 1:
The value distribution range is divided into multiple sections (e.g., first section for smaller absolute values, second section for larger absolute values) using breaking points. Each section is quantized independently with its own quantization parameter, allowing differential treatment of different value ranges rather than uniform quantization across all values.
Solution Approach 2:
Different quantization parameters are assigned to different sections of the value distribution. Specifically, the first quantization parameter is used for values in the first section, while the second quantization parameter is used for values in the second section. This local differentiation optimizes quantization precision for each region, preserving prediction accuracy while maintaining compression.
2Measurement precision
If different quantization parameters are used for different sections, then prediction accuracy is improved, but device complexity increases
Solution Approach 1:
The quantization parameter is made dynamic rather than static. The system automatically selects appropriate quantization parameters based on the section in which the parameter value falls, determined by breaking points. This dynamic adaptation allows the system to achieve high prediction accuracy without manual configuration of complex quantization schemes.
Data Source
AI summary
An optimizing method and a computing apparatus for a deep learning network and a computer-readable storage medium are provided. In the method, a value distribution is obtained from a pre-trained model. One or more breaking points in a range of the value distribution are determined. Quantization is performed on a part of values of a parameter type in a first section among multiple sections using a first quantization parameter and the other part of values of the parameter type in a second section among the sections using a second quantization parameter. The value distribution is a statistical distribution of values of the parameter type in the deep learning network. The range is divided into the sections by one or more breaking points. The first quantization parameter is different from the second quantization parameter. Accordingly, accuracy drop can be reduced.