Automatic Hybrid Quantization for Deep Neural Network Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional hybrid quantization of deep neural networks requires domain expertise and empirical rules, leading to sub-optimal compression and performance, especially in devices with limited computation capacity.
Innovation Solution
A method involving a meta learning network with a quantization function to automatically generate and optimize hybrid quantization vectors, using genetic algorithms and teacher networks for supervised training, to achieve deeper compression while maintaining high performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional hybrid quantization methods are used, then model compression is achieved, but manufacturing precision deteriorates due to sub-optimal quantization parameters
Solution Approach 1:
The system employs self-service through automatic quantization parameter selection where the model itself provides the optimal quantization parameters through meta-learning, eliminating the need for external expert intervention and empirical tuning while achieving optimal compression-accuracy balance
Solution Approach 2:
The system changes parameters dynamically by using meta-learning to automatically adjust quantization parameters (bit-widths, thresholds) based on the specific model architecture and dataset characteristics, rather than using fixed conventional parameters, thereby optimizing both compression and accuracy
2Manufacturing precision
If expert intervention is used for hybrid quantization, then manufacturing precision improves, but device complexity increases due to requirement of domain expertise
Solution Approach 1:
The system eliminates the need for expert intervention by implementing self-service through automatic parameter selection mechanisms where the meta-learning model autonomously determines optimal quantization parameters based on the given model and dataset, simplifying the overall system while maintaining high accuracy
Solution Approach 2:
The meta-learning model acts as an intermediary that bridges the gap between model compression requirements and accuracy maintenance, automatically generating optimal quantization parameters without requiring direct expert involvement, thereby reducing system complexity while preserving precision
3Device complexity
If uniform bit width quantization is applied, then device complexity is reduced, but manufacturing precision deteriorates with sub-optimal performance
Solution Approach 1:
The system applies local quality by allowing different bit-widths for different layers or parameters within the model, rather than using uniform quantization, with the meta-learning model automatically determining which specific parameters benefit from higher or lower precision, thereby optimizing performance without excessive complexity
Data Source
AI summary
Methods, computer program products, and/or systems are provided that perform the following operations: obtaining a target neural network structure and constraints for a target neural network; generating a meta learning network having an associated quantization function based, at least in part, on the target neural network structure; training the meta learning network based, at least in part, on providing a hybrid quantization vector as input to the meta learning network and providing a training dataset to the target neural network; obtaining a plurality of hybrid quantization vectors; determining a new hybrid quantization vector from the plurality of hybrid quantization vectors; and retraining the trained meta learning network based, at least in part, on providing the new hybrid quantization vector as input to the trained meta learning network.