Dynamic Quantization Precision for Deep Learning Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deep learning network quantization fixes bit precision for each layer after learning, preventing dynamic adjustment during runtime, leading to inefficient computation and low power consumption efficiency, especially when optimal precision varies with input data.
Innovation Solution
Generating multiple quantization models for different bit numbers and selecting the appropriate model based on input data uncertainty to perform object recognition, allowing dynamic precision adjustment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If quantization precision is fixed for every layer after learning is completed, then device complexity is reduced and ease of operation is improved, but computation efficiency deteriorates and power consumption efficiency decreases
Solution Approach 1:
The patent applies dynamics by making quantization precision adjustable during runtime instead of fixed. The system dynamically selects from multiple pre-trained quantization models with different bit precisions (e.g., 8-bit, 4-bit, 2-bit) based on input data characteristics, allowing the network to adapt its computation precision to match the complexity of each input, thereby improving computation efficiency while maintaining ease of operation through automated selection
2Device complexity
If quantization precision is fixed for every layer after learning is completed, then device complexity is reduced, but adaptability to different input data deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-training multiple quantization models with different bit precisions before runtime. These pre-trained models are stored and ready for selection based on input data characteristics. This approach maintains relatively low device complexity while achieving high adaptability, as the system only needs to store and select from pre-prepared models rather than dynamically training or adjusting precision during inference
3Measurement precision
If higher quantization precision is used for all layers, then measurement precision and reliability are improved, but use of energy increases
Solution Approach 1:
The patent applies local quality by allowing different layers of the neural network to use different quantization precisions dynamically. Instead of uniformly applying high precision to all layers, the system selectively applies higher precision only to layers that require it based on input data characteristics, while using lower precision for layers where it suffices. This localized approach maintains measurement precision where needed while significantly reducing overall power consumption
4Use of energy by moving object
If lower quantization precision is used to reduce computation, then use of energy is reduced, but measurement precision and computation efficiency deteriorate
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the quantization precision parameter based on input data characteristics and network state. The system monitors factors such as input data complexity, intermediate activation distributions, and gradient magnitudes to adaptively select or adjust precision levels. This allows the system to use lower precision when acceptable (reducing power consumption) while maintaining higher precision when needed (preserving measurement accuracy), effectively resolving the trade-off between energy use and precision
Data Source
AI summary
An image recognition method includes the steps of: for a deep learning network that carries out object recognition on a random image, carrying out quantization corresponding to the number of a plurality of different bits to generate a plurality of quantization models respectively corresponding to the number of bits; receiving image data as an input for the deep learning network; determining the uncertainty of the input image data; selecting any one of the plurality of quantization models on the basis of the determined uncertainty; and recognizing an object from the image data by using the selected quantization model, and outputting, as the result of the object recognition, a label corresponding to the image data.