Shared Exponent Floating Point for QSNR-Area Tradeoffs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing floating point data types are inefficient for neural network training and inference at ultra-narrow bit-width, as they fail to maximize the fidelity of each bit while minimizing computing cost, particularly in maximizing Quantization Signal to Noise Ratio (QSNR) while minimizing the area overhead of hardware dot-product units.
Innovation Solution
The implementation of hierarchical and shared exponent floating point data types, which involve determining shared exponent values and quantizing mantissas based on these exponents, allowing for efficient storage and computation by balancing tile size and encoding efficiency, thereby optimizing QSNR in hardware dot-product units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional floating point data types are used for neural network training and inference, then computing operations can be performed, but the fidelity of each bit is not maximized and the area overhead of hardware dot-product units increases
Solution Approach 1:
The patent segments the exponent representation into multiple levels (e.g., 8-bit exponents divided into 4-bit integer part and 4-bit fractional part). This segmentation allows for optimized hardware implementation where only the integer part requires full precision representation, while the fractional part can be handled with reduced precision, thereby reducing the area overhead of dot-product units while maintaining high QSNR for neural network computations
Solution Approach 2:
The patent applies local quality by using different precision levels for different parts of the floating point representation. Specifically, the exponent is divided into regions with different precision requirements: the integer part uses full precision for range representation, while the fractional part uses reduced precision for fine-grained adjustments. This allows the hardware to allocate resources efficiently, focusing full precision on critical regions while reducing precision in less critical regions, thus optimizing the area-QSNR tradeoff
2Area of stationary object
If ultra-narrow bit-width data types are used to minimize computing cost, then area overhead is reduced, but the fidelity of each bit decreases
Solution Approach 1:
The patent introduces a hierarchical dimension to the exponent representation by dividing it into integer and fractional parts with different precision levels. Instead of uniformly reducing bit-width across the entire exponent, the invention creates a multi-dimensional structure where the integer part maintains full precision for range coverage, while the fractional part uses reduced precision for fine adjustments. This dimensional differentiation allows ultra-narrow bit-width implementation while preserving fidelity through the hierarchical structure
3Measurement precision
If full precision exponents are used for all floating point values, then accuracy is maintained, but hardware area overhead increases
Solution Approach 1:
The patent changes the parameter structure of the exponent by introducing a hierarchical representation with different precision levels for different parts of the exponent. Instead of using a uniform full-precision exponent for all values, the invention modifies the exponent parameters into an 8-bit structure with 4-bit integer and 4-bit fractional parts. This parameter transformation allows the hardware to use reduced precision (4-bit) for the fractional part while maintaining full precision (4-bit) for the integer part, thereby reducing overall area overhead while preserving accuracy through the changed parameter structure
Data Source
AI summary
Embodiments of the present disclosure include systems and methods for providing hierarchical and shared exponent floating point data types. First and second shared exponent values are determined based on exponent values of a plurality of floating point values. A third shared exponent value is determined based the first shared exponent value and the second shared exponent value. First and second difference values are determined based on the first shared exponent value, the second shared exponent value, and the third shared exponent value. Sign values and mantissa values are determined for the plurality of floating point values. The sign value and the mantissa value for each floating point value in the plurality of floating point values, the third shared exponent value, the first difference value, and the second difference value are stored in a data structure for a shared exponent floating point data type.