Mixed-Precision Quantizer Search for Lower-Cost ANN Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mixed-precision search techniques for artificial neural networks suffer from slow convergence and high training costs, especially with large-scale datasets, limiting the availability of on-device AI applications due to computational and power consumption challenges.
Innovation Solution
The EMS technique employs a folding quantizer module that samples a single bit-width option during training, combined with a full-precision pass, to reduce computational cost and memory usage while optimizing network parameters for efficient mixed-precision design.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If existing mixed-precision search techniques execute all bit-width options to search for optimized bit-width, then network design completeness is improved, but computational cost and training time increase significantly
Solution Approach 1:
The patent applies partial action by executing only a sampled bit-width option rather than all possible bit-width options during training. The folding quantizer module selectively executes one sampled bit-width option at a time, reducing computational cost while still achieving reasonable convergence rates through the combination of sampling and full-precision passes.
Solution Approach 2:
The training process is segmented into different phases: a sampled bit-width option execution phase and a full-precision pass phase. This segmentation allows the system to alternate between low-cost sampling and complete precision updates, balancing exploration of bit-width options with efficient training convergence.
2Productivity
If a sampled bit-width option is executed to reduce computational cost, then training efficiency is improved, but network parameter convergence may be compromised
Solution Approach 1:
The full-precision pass ensures continuous and reliable parameter updates for all bit-width options, maintaining convergence reliability. The sampled bit-width option execution provides efficient progress while the full-precision pass periodically reinforces parameter updates, ensuring both efficiency and reliability are maintained throughout training.
3Ease of manufacture
If uniform quantization is used, then implementation simplicity is improved, but network accuracy decreases compared to mixed-precision approaches
Solution Approach 1:
The patent implements local quality by allowing different bit-width options to be selected for different network layers or regions. The folding quantizer module enables mixed-precision quantization where critical layers can use higher precision while less critical layers use lower precision, optimizing the balance between accuracy and computational efficiency.
Data Source
AI summary
A method for performing efficient mixed-precision search for an artificial neural network (ANN) includes training the ANN by sampling selected candidate quantizers of a bank of candidate quantizer and updating network parameters for a next iteration based on outputs of layers of the ANN. The outputs are computed by processing quantized data with operators (e.g., convolution). The quantizers converge to optimal bit-widths that reduce classification losses bounded by complexity constrains.