Value-Dependent Quantization for Machine Learning Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models face challenges in efficiently quantizing linear operations while maintaining privacy, verifiability, and minimizing accuracy loss.
Innovation Solution
A computing-processor-implemented method and system for quantizing linear operations of machine learning computations by determining a maximum field size and scaling factors for weight, input data, and bias matrices, ensuring minimal error and accuracy loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If quantization is applied to reduce computational complexity and improve efficiency, then processing speed and resource utilization are improved, but accuracy loss increases
Solution Approach 1:
The patent changes the parameter representation from fixed-point to floating-point format, allowing the system to maintain high precision while still benefiting from quantization. By representing weights, inputs, and activations as floating-point values with explicit scale factors, the system achieves both computational efficiency and accuracy preservation simultaneously
Solution Approach 2:
The patent introduces dynamic scale factors that can be adjusted per tensor and per operation, rather than using fixed quantization parameters. This dynamic adjustment allows the system to optimize for both speed and accuracy depending on the specific computational context, resolving the trade-off between processing speed and measurement precision
2Measurement precision
If scaling factors are increased to maintain precision, then accuracy is improved, but the range of representable values decreases
Solution Approach 1:
The patent adds an explicit dimension for scale factors separate from the actual data values. By representing numbers as (value, scale) pairs rather than single quantized values, the system gains an additional degree of freedom that allows simultaneous optimization of precision and range without the traditional trade-off
3Device complexity
If fixed-point quantization is used to simplify computations, then computational complexity is reduced, but privacy and verifiability are compromised
Solution Approach 1:
The patent introduces floating-point representation as an intermediary between integer quantization and full-precision floating-point arithmetic. This intermediary representation maintains the simplicity of integer-based computations while incorporating the precision and range characteristics of floating-point numbers, thereby preserving privacy and verifiability properties without sacrificing computational efficiency
Data Source
AI summary
A computing system determines a maximum field size for a field containing values of quantized versions of a weight matrix, an input data matrix, and a bias matrix and determines a weight upper bound of scaling factors for the weight matrix based on values of the weight matrix, an input data upper bound of scaling factors for the input data matrix based on values of the input data matrix, and a bias upper bound of scaling factors for the bias matrix based on values of the bias matrix. The computing system also sets a weight scaling factor of the weight matrix, an input data scaling factor of the input data matrix, and a bias scaling factor for the bias matrix in two different cases: when the sum is less than or equal to the bias upper bound and when the sum is greater than the bias upper bound.