Bi-scaled Deep Neural Network Quantization for Range Resolution Tradeoff
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Networks (DNNs) face efficiency and computational challenges due to their large-scale structure and data processing requirements, particularly in representing long-tailed data structures where most elements are of small magnitude with a few being orders of magnitude larger, necessitating a superior tradeoff between range and resolution.
Innovation Solution
The implementation of bi-scaled Deep Neural Networks (DNNs) using two scale factors, scale-fine and scale-wide, for quantization, where scale-fine provides higher resolution for small numbers and scale-wide offers more range for larger numbers, enabling low-precision DNNs with higher accuracy. This is achieved through mechanisms that identify when a data structure's tail starts by building histograms and checking density ratios, and using a multiscale processing element with correction logic and accumulation registers to balance quantization range and resolution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single scale factor is used for quantization, then the quantization process is simple, but it cannot provide both high resolution for small numbers and sufficient range for large numbers
Solution Approach 1:
The patent segments the quantization process into two distinct modes using two different scale factors. The first scale factor (scale-fine) is used for quantizing small-magnitude values to provide high resolution, while the second scale factor (scale-wide) is used for large-magnitude values to provide sufficient range. This segmentation allows the system to optimize for both resolution and range simultaneously by applying the appropriate scale factor based on the input value magnitude.
Solution Approach 2:
The patent implements dynamic scale factor selection where the system automatically chooses between scale-fine and scale-wide based on the magnitude of the input value. This dynamic adaptation allows the quantization process to adjust its characteristics (resolution vs. range) according to the specific requirements of each data point, thereby resolving the contradiction between maintaining high resolution and providing sufficient range.
2Productivity
If low-precision quantization is used, then computational overhead is reduced, but accuracy of the DNN decreases
Solution Approach 1:
The patent applies local quality by using different precision levels for different parts of the data distribution. Small-magnitude values (which constitute the majority of data points in long-tailed distributions) are quantized with high resolution using scale-fine, while large-magnitude values are quantized with lower resolution using scale-wide. This localized approach to precision maintains overall DNN accuracy while enabling efficient low-precision computation for the predominant small values.
Solution Approach 2:
The patent changes the quantization parameters (scale factors) based on the characteristics of the data being processed. By introducing two different scale factors instead of one, the system can adapt the quantization parameters to match the local characteristics of the data distribution, thereby maintaining accuracy even when using low-precision representations for the majority of computations.
3Measurement precision
If high resolution is used for all values, then small numbers are represented accurately, but large values saturate and lose information
Solution Approach 1:
The patent segments the value range into two regions: small-magnitude values and large-magnitude values. For small values, scale-fine provides high resolution to represent them accurately. For large values, scale-wide provides a broader range to prevent saturation. This segmentation ensures that each region is handled with the appropriate quantization characteristics, resolving the contradiction between resolution and range coverage.
4Measurement precision
If two scale factors are used for quantization, then both resolution and range are improved, but the complexity of the quantization process increases
Solution Approach 1:
The patent performs preliminary action by pre-determining the appropriate scale factor for each quantization operation based on the magnitude of the input value. The system establishes clear thresholds and selection criteria before the actual quantization process, allowing for efficient determination of which scale factor to use without requiring complex runtime analysis. This preliminary preparation reduces the complexity burden during execution.
Solution Approach 2:
The patent implements a dynamic but systematic approach to scale factor selection. Rather than requiring complex adaptive algorithms, the system uses straightforward magnitude-based criteria to dynamically choose between scale-fine and scale-wide. This dynamic selection mechanism balances the need for accuracy with the desire to keep the quantization process relatively simple and efficient.
Data Source
AI summary
A method is provided for forming a Deep Neural Network (DNN). The method includes quantizing deep learning data structures of the DNN into at least two modes using at least two scale factors, respectively. Each of the at least two modes corresponds to a respective one of the at least two scale factors. The method further includes identifying which of the at least two scale factors to use for a given one of the data structures based on a data distribution of the given one of the data structures. The quantizing step includes identifying when a tail of the given one of the data structures starts by (i) building a histogram of values in the given one of the data structures using successive bins; (ii) identifying a ratio of density between the successive bins; and (iii) checking whether the ratio of density is greater than a ratio of density threshold.