Pure Integer Quantization for Lightweight Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing quantization techniques for lightweight neural networks result in significant accuracy loss and require lengthy retraining times, and most methods only quantize weights and feature maps while leaving offset and quantization coefficients as floating-point numbers, which is not suitable for ASIC and FPGA applications.
Innovation Solution
A pure integer quantization method for lightweight neural networks involves normalizing feature map values by the maximum pixel value and multiplying corresponding weights by this maximum value to transfer channel imbalances, with a tunable imbalance transfer coefficient t to optimize quantization accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional quantization techniques are applied to lightweight neural networks, then quantization is achieved, but accuracy loss occurs significantly
Solution Approach 1:
The patent changes the quantization parameters by introducing imbalance transfer coefficients (alpha and beta) that control how channel imbalances are redistributed. By adjusting these parameters, the method adapts the quantization process to preserve accuracy while achieving pure integer quantization, directly resolving the contradiction between quantization precision and model accuracy.
Solution Approach 2:
The patent segments the quantization process into distinct stages: forward propagation quantization, backward propagation quantization, and imbalance transfer. Each stage handles specific aspects of the quantization problem separately, allowing precise control over accuracy preservation while achieving integer-only quantization.
2Measurement precision
If retraining or training-time quantization techniques are used to recover accuracy loss, then model accuracy is improved, but training time increases significantly
Solution Approach 1:
The patent performs preliminary imbalance transfer during the quantization preparation phase, adjusting weight distributions before the actual quantization inference. This preliminary adjustment prevents accuracy loss from occurring in the first place, eliminating the need for time-consuming retraining while maintaining high model accuracy.
Solution Approach 2:
The patent introduces imbalance transfer coefficients as intermediary parameters that mediate between the original floating-point weights and the quantized integer weights. These coefficients act as a bridge, allowing accurate representation of weight distributions without requiring full retraining, thus reducing time loss while preserving accuracy.
3Manufacturing precision
If only weights and feature maps are quantized while offset and quantization coefficients remain floating-point, then partial quantization is achieved, but ASIC and FPGA applications are not optimized
Solution Approach 1:
The patent merges all quantization parameters (weights, feature maps, offsets, and quantization coefficients) into a unified integer quantization scheme. By combining these previously separate quantization treatments into a single pure integer framework, the method achieves full hardware compatibility for ASIC and FPGA deployments while maintaining high quantization precision.
4Ease of manufacture
If ReLU6 is directly replaced by ReLU to accommodate Nagel et al.'s method, then cross-layer weight balance can be applied, but significant accuracy loss occurs
Solution Approach 1:
The patent applies local quality by treating different activation functions (ReLU6 and ReLU) with specialized handling. Instead of forcing a uniform approach, the method adjusts the imbalance transfer strategy based on the specific activation function being used, preserving accuracy for ReLU6 while enabling the method's applicability to various activation functions including ReLU.
Data Source
AI summary
A pure integer quantization method for a lightweight neural network (LNN) is provided. The method includes the following steps: acquiring a maximum value of each pixel in each of the channels of the feature map of a current layer; dividing a value of each pixel in each of the channels of the feature map by a t-th power of the maximum value, t∈[0,1]; multiplying a weight in each of the channels by the maximum value of each pixel in each of the channels of the corresponding feature map; and convolving the processed feature map with the processed weight to acquire the feature map of a next layer. The algorithm is verified on SkyNet and MobileNet respectively, and lossless INT8 quantization on SkyNet and maximum quantization accuracy so far on MobileNetv2 are achieved.