Hybrid Quantized Neural Network Layer Precision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for quantizing artificial neural networks struggle to balance accuracy and efficiency, particularly when deployed on edge devices with limited memory and power supply.
Innovation Solution
A method that accesses a floating-point neural network, converts it to a low-bit-width representation, calculates per-layer deviation statistics, and iteratively increases the bit-width of specific layers until a specified accuracy threshold is met, resulting in a hybrid quantized network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the neural network is quantized to low-bit-width representation, then the storage footprint is reduced, but the accuracy deteriorates
Solution Approach 1:
The patent applies local quality by differentiating quantization bit-widths across different layers of the neural network. Instead of uniformly quantizing all layers to low-bit-width (e.g., 8-bit), the method identifies specific layers that require higher precision (e.g., 16-bit or 32-bit) to maintain accuracy, while other layers can use lower precision for compression. This selective approach optimizes the balance between storage footprint reduction and accuracy preservation on a per-layer basis.
Solution Approach 2:
The patent employs parameter changes by dynamically adjusting the quantization bit-width parameter for different layers based on their specific characteristics and contribution to overall accuracy. The method evaluates each layer's importance and sensitivity to quantization, then assigns appropriate bit-width parameters accordingly, transforming the fixed quantization approach into a flexible, adaptive parameter-setting strategy.
2Measurement precision
If the bit-width of all layers is increased to maintain accuracy, then the accuracy is preserved, but the storage footprint increases
Solution Approach 1:
The patent resolves this contradiction by applying local quality through selective precision assignment. Each layer is evaluated individually to determine its specific precision requirements, allowing the system to maintain accuracy only where necessary while compressing other layers. This localized approach ensures that accuracy is preserved for critical layers without unnecessarily increasing the storage footprint of the entire network.
Solution Approach 2:
The patent applies partial action by applying high precision only to the extent necessary for maintaining accuracy. Instead of uniformly increasing bit-width across all layers (excessive action), the method identifies and applies high precision only to specific layers that contribute most to accuracy, leaving other layers at lower precision levels, thus avoiding unnecessary storage overhead.
3Ease of manufacture
If uniform quantization is applied to all layers, then the quantization process is simple, but the accuracy loss is greater
Solution Approach 1:
The patent overcomes the limitation of uniform quantization by implementing local quality through layer-specific quantization parameters. The method evaluates each layer's characteristics and applies tailored quantization strategies, resulting in better accuracy preservation. While this increases complexity compared to uniform quantization, the patent provides automated evaluation and assignment mechanisms that manage this complexity efficiently.
Solution Approach 2:
The patent applies preliminary action by pre-evaluating each layer's sensitivity to quantization and determining optimal bit-width assignments before final quantization. This upfront analysis and planning phase enables the system to make informed decisions about precision requirements for each layer, avoiding excessive accuracy loss while maintaining a systematic approach to the quantization process.
Data Source
AI summary
A method includes, for each floating-point layer in a set of floating-point layers: calculating a set of input activations and a set of output activations of the floating-point layer; converting the floating-point layer to a low-bit-width layer; calculating a set of low-bit-width output activations based on the set of input activations; and calculating a per-layer deviation statistic of the low-bit-width layer. The method also includes ordering the set of low-bit-width layers based on the per-layer deviation statistic of each low-bit-width layer. The method additionally includes, while a loss-of-accuracy threshold exceeds the accuracy of the quantized network: converting a floating-point layer represented by the low-bit-width layer to a high-bit-width layer; replacing the low-bit-width layer with the high-bit-width layer in the quantized network; updating the accuracy of the quantized network; and, in response to the accuracy of the quantized network exceeding the loss-of-accuracy threshold, returning the quantized network.