Micro-Scaling Quantization for Low-Latency Numeric Format Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies fail to efficiently convert numerical data formats, resulting in increased computational complexity, energy consumption, and latency due to complex operations like logarithm and exponential calculations during quantization and de-quantization processes.
Innovation Solution
Implement a micro-scaling quantization and de-quantization process that avoids logarithm and exponential calculations, replacing them with shifting and addition/subtraction operations, using formats like MXFP8, MXFP6, or MXFP4 to reduce computational complexity and energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating-point data is used for neural network computations, then computational accuracy is maintained, but processing device size, complexity, and energy consumption increase
Solution Approach 1:
The patent changes the data representation parameters from floating-point (FP32/FP16) to fixed-point (INT8/INT4) formats. This parameter transformation enables lower precision representation that reduces computational complexity and energy consumption while maintaining acceptable accuracy for neural network inference tasks.
Solution Approach 2:
The patent applies quantization that uses fewer bits (INT8 or INT4 instead of FP32) to represent numerical data. This partial action approach sacrifices some precision to achieve significant reductions in computational complexity and energy consumption, which is acceptable for many inference workloads.
2Adaptability or versatility
If traditional quantization methods using logarithm and exponential calculations are used, then data format conversion is achieved, but computational complexity and energy consumption increase
Solution Approach 1:
The patent replaces complex mathematical operations (logarithm and exponential calculations) with simpler arithmetic operations (multiplication and addition). This substitution fundamentally changes the computational mechanism used in quantization, reducing energy consumption while maintaining data format conversion capability.
Solution Approach 2:
The patent introduces a scaling factor parameter that enables efficient quantization through simple multiplication operations instead of complex logarithmic calculations. This parameter transformation allows the system to achieve data format conversion with minimal computational overhead and energy consumption.
3Productivity
If quantization and de-quantization operations are performed, then data compression and efficiency are improved, but processing time and computational complexity increase
Solution Approach 1:
The patent replaces time-consuming logarithmic and exponential calculations with faster multiplication and addition operations. This substitution significantly reduces the processing time required for quantization and de-quantization operations, improving overall computational efficiency without adding significant complexity.
Data Source
AI summary
In one or more aspects, a processing device for numerical data quantization includes processing circuitry configured to determine a maximum exponent from a set of exponents of a set of digital representations of a set of numbers, obtain a set of scaled exponents based on the maximum exponent, and perform one of: (i) obtain a set of quantized significands based on a set of mantissas of the set of digital representations and the set of scaled exponents, or (ii) obtain a set of quantized mantissas based on the set of mantissas. The processing circuitry is configured to output a set of quantized digital representations of the set of numbers, based on the set of quantized significands, or based on the set of quantized mantissas and the set of scaled exponents; and to output a biased exponent scaling factor based on the maximum exponent.