Hybrid Quantization for Deep Learning Network Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning networks face challenges in balancing prediction accuracy and compression ratio due to limitations in current quantization techniques.
Innovation Solution
The proposed method employs a hybrid approach combining power of two quantization and dynamic fixed-point quantization for deep learning networks, allowing for improved prediction accuracy while maintaining a certain compression rate by selecting between one-hot and two-hot encoding formats based on performance indices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If quantization is applied to compress the deep learning network model, then the model size is reduced and compression ratio is improved, but the prediction accuracy deteriorates
Solution Approach 1:
The patent segments the quantization process into two distinct formats: power-of-two quantization format and dynamic fixed-point quantization format. Different data types are assigned to different formats based on their characteristics, allowing the system to maintain prediction accuracy while achieving compression. This segmentation resolves the contradiction by enabling selective quantization strategies for different model components.
Solution Approach 2:
The patent applies different quantization methods to different data types locally. Specifically, weight data undergoes power-of-two quantization while activation data undergoes dynamic fixed-point quantization. This local differentiation allows each data type to be compressed in the most appropriate manner, maintaining overall prediction accuracy while achieving model compression.
2Device complexity
If higher compression ratio is achieved through quantization, then model complexity is reduced, but prediction accuracy deteriorates
Solution Approach 1:
The patent introduces dynamic fixed-point quantization for activation data, where quantization parameters are adjusted dynamically during inference based on input characteristics. This dynamic approach allows the model to maintain higher prediction accuracy even with reduced complexity, as the quantization adapts to different input scenarios rather than using a fixed compression level.
Solution Approach 2:
The patent creates a composite quantization structure that combines power-of-two quantization and dynamic fixed-point quantization within the same model. This composite approach allows different parts of the model to operate at different complexity levels, achieving overall model compression while preserving prediction accuracy in critical pathways.
3Productivity
If power of two quantization is used, then computational efficiency is improved through simpler operations, but prediction accuracy deteriorates compared to full precision
Solution Approach 1:
The patent merges power-of-two quantization and dynamic fixed-point quantization into a unified inference system. By combining the computational efficiency of power-of-two operations with the accuracy-preserving properties of dynamic fixed-point quantization, the system achieves both improved computational efficiency and maintained prediction accuracy simultaneously.
Solution Approach 2:
The patent uses dequantization operations as intermediaries between the quantized representations and the final prediction output. The dequantization process recovers approximate full-precision values from the quantized formats, serving as a mediator that bridges the gap between computational efficiency and prediction accuracy.
Data Source
AI summary
Disclosed are an optimizing method and a computing system used for deep learning networks. The first data is obtained. The first data is quantized through the power of two quantization. The first data after the power of two quantization is the first format or the second format. The numbers of the first values in the first format or the second format is different. The second data is obtained. The second data is quantized through dynamic fixed-point quantization. A computation related to a deep learning network is performed on the quantized first data after the power of two quantization and the quantized second data after dynamic fixed-point quantization. Accordingly, the prediction precision could be increased, and the complexity of the model could be reduced.