Microscaling Block Conversion With Dynamic Scale Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for converting high-precision floating-point formats to MX-compliant formats result in significant data loss and precision issues due to the clipping of large magnitudes and underrepresentation of smaller magnitudes, leading to inefficiencies in AI computations.
Innovation Solution
A method for determining the block scale (2X) in MX formats by using equations that either round down, round up, or adjust the maximum exponent based on mantissa values to maintain dynamic range and avoid clipping, ensuring accurate conversion from FP32 to MX formats.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If standard conversion methods are used to convert FP32 to MX format, then memory footprint is reduced, but precision is lost due to clipping of large magnitudes and underrepresentation of smaller magnitudes
Solution Approach 1:
The patent implements dynamic block scaling where the scale factor is adjusted based on the actual data distribution within each block. Instead of using a fixed scaling approach, the system dynamically determines the appropriate block scale (2X) by analyzing the exponent values in the block, allowing the representation to adapt to the specific magnitude range of the data being converted. This dynamic adjustment prevents both clipping of large values and underrepresentation of small values.
Solution Approach 2:
The patent changes the scaling parameter (block scale 2X) based on the mantissa values and exponent distribution of the input data. By modifying the scale factor parameter dynamically rather than using a predetermined fixed scale, the conversion process can maintain precision across different magnitude ranges while still achieving memory compression through the MX format.
2Measurement precision
If block scale is increased to maintain precision of large magnitudes, then precision is improved, but computational overhead increases
Solution Approach 1:
The patent divides the data into blocks and processes each block independently with its own optimized scale factor. By segmenting the conversion process at the block level rather than processing all data with a single global scale, the system can use smaller, more efficient scale factors for each block, reducing overall computational overhead while maintaining precision for each segment's magnitude range.
Solution Approach 2:
The patent optimizes the block scale parameter by deriving it from the actual data characteristics (exponent values and mantissa distribution) rather than using conservative large-scale factors. This parameter optimization reduces the computational complexity of scaling operations while maintaining the precision needed for large magnitudes in each block.
3Productivity
If standard MX conversion is used, then conversion speed is improved, but data accuracy deteriorates due to significant data loss
Solution Approach 1:
The patent maintains fast conversion by using a dynamic yet deterministic algorithm that quickly determines the appropriate block scale factor based on simple statistical measures of the input block (such as maximum exponent values). This dynamic approach avoids iterative or complex optimization routines, preserving conversion speed while significantly reducing data loss compared to fixed-scale methods.
Solution Approach 2:
The patent changes the scaling parameter adaptively based on the input data's exponent distribution, allowing the conversion to maintain high accuracy without requiring excessive computation time. By deriving the block scale from key parameters of the input data rather than using trial-and-error or conservative fixed values, the system achieves both speed and accuracy.
Data Source
AI summary
Disclosed herein are various techniques for converting a vector from a high precision floating point format to a microscaling (MX) format. An example of a precision floating point format is the FP32 format described above, however, the initial format may be another type of standard floating point format as well (reference to the FP32 number format hereinafter is merely for exemplary purposes and not intended to be limiting). The techniques for converting to the MX-compliant format are improvements over the standard technique suggested in the MX specification by at least accounting for the amount of data in the mantissa of the original precision floating point format to mitigate the amount of data that is lost during the conversion. Therefore, the benefits of representing multiple data points of a vector in the single MX format representation without sacrificing as much of the data contained in the original high precision format that may occur following the standard technique described in the MX specification (portions of which are described below).