Neural Network Model Quantization via Staged Layerwise Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large-scale neural network models require significant resources for storage and computation, making them challenging to deploy on devices with limited capabilities, necessitating model quantization to reduce their scale while maintaining recognition precision.
Innovation Solution
The staged layerwise quantization-based model quantization training method, which iteratively quantizes network layers, inserting pseudo-quantization operators and adjusting parameters, allows for gradual adaptation to quantization errors, reducing interference and preserving recognition precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large-scale neural network model is deployed, then recognition precision is improved, but storage space and computational resources are excessively consumed
Solution Approach 1:
The patent applies parameter changes by quantizing the neural network model parameters from high precision (e.g., 32-bit floating point) to low precision (e.g., 8-bit integer). This transformation reduces the storage space and computational resource requirements while maintaining recognition precision through iterative training with quantized data sets that compensate for quantization errors.
2Measurement precision
If a large-scale neural network model is deployed, then recognition precision is improved, but computational resources are excessively consumed
Solution Approach 1:
The patent transforms model parameters from high precision to low precision representation, which directly reduces computational resource consumption. The iterative training process with quantized data sets ensures that the quantized model achieves comparable recognition precision to the original large-scale model while requiring significantly fewer computational resources.
3Quantity of substance
If model quantization is applied, then storage space and computational resources are reduced, but recognition precision is lost
Solution Approach 1:
The patent applies preliminary action by performing iterative training with quantized data sets before final model deployment. This preliminary training process pre-compensates for quantization errors, allowing the model to achieve high recognition precision even with low-precision parameters. The training process includes multiple iterations where the model is trained on quantized versions of the training data, gradually adapting to the quantized representation.
Solution Approach 2:
The patent employs feedback mechanisms through iterative training where the model performance is continuously evaluated and used to adjust the training process. The quantized data sets are generated based on the model's output statistics, and the training iterations provide feedback loops that refine the quantization parameters and compensate for precision loss, ensuring the final quantized model maintains high recognition accuracy.
Data Source
AI summary
A data processing method is provided. In the method, a first model that includes N network layers is obtained. The first model is trained with a first data set that includes first data and training label information of the first data, N being a positive integer. The first model is trained with a second data set. The second data set including second data and training label information of the second data, the second data being quantized. A first unquantized target network layer of the N network layers is quantized. Further, an updated first model that includes the quantized first target network layer is trained with the second data set to obtain a second model.