A Quantization Method for Deep Neural Networks Based on Elastic Significant Bits

Abstract

The present invention provides a deep neural network quantization method based on elastic effective digits, which quantizes fixed-point numbers or floating-point numbers into quantized values ​​with elastic effective digits, discards redundant mantissa parts, and adopts a feasible solution method to quantitatively evaluate quantization The distribution of values ​​differs from the original data. The present invention has quantized values ​​with flexible effective bits. Through different effective bits, the distribution of quantized values ​​can cover a series of bell-shaped distributions from long tail to uniform, adapting to the weight / activation distribution of DNNs, thereby ensuring low precision loss; The multiplication calculation can be realized by multiple shift additions on the hardware, which improves the overall efficiency of the quantization model; the distribution difference function quantitatively estimates the quantization loss caused by different quantization schemes, and can select the optimal quantization scheme under different conditions. Achieve lower quantization loss and improve the accuracy of the quantization model.

Description

technical field [0001] The invention belongs to the technical field of deep neural network compression, in particular to a deep neural network quantization method based on elastic valid bits. Background technique [0002] Deep Neural Networks (DNNs) quantization is an effective way to compress DNNs, which can significantly improve the computational efficiency of DNNs and enable the network to be deployed on edge computing platforms with limited resources. One of the most common ways to achieve quantization in DNNs is to project high-precision floating-point values ​​into low-precision quantized values. For example, DNNs with 32-bit floating point can achieve 32x model size compression by replacing weights with only one bit, and even reduce complex multiplication operations to simple bit operations on hardware. Therefore, in the case of fewer bit values, DNN quantization can significantly reduce the computational scale or memory footprint, thereby improving computational eff...

AI Technical Summary

Problems solved by technology

None of these distributions are suitable for the bell-shaped distribution, but in most cases, the DNN weights / activation values ​​are bell-shaped distributed, which shows that APoT cannot adapt to most DNN quantization, which will bring a large decrease in accuracy

figure 2 The displayed APoT is used to quantize the complex step function of the projection, which is difficult to implement with simple scaling and rounding functions, so its time complexity and space complexity have reached O(n), which will greatly The reduced computational efficiency of the quantized model

Images