Floating-Point Dynamic Range Expansion via Scaling Circuitry
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Integrated circuits face limitations in representing variables due to restricted number formats available, which can lead to overflow or underflow issues during arithmetic operations, especially in machine learning applications where formats like bfloat16 are beneficial but not always supported by hardware resources.
Innovation Solution
Incorporating pre-scaling and post-scaling circuitry to adjust the representation of variables before and after processing, allowing inputs in one format (e.g., bfloat16) to be scaled to another format (e.g., half-precision floating-point) to perform operations without overflow, and then scaling the output back to the original format, thereby emulating operations in the desired format.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the integrated circuit uses limited hardware resources with fixed number formats, then the device complexity is reduced, but the adaptability to different number formats (such as bfloat16) is limited
Solution Approach 1:
The patent introduces scaling circuitry as an intermediary component between the input data and the arithmetic logic unit. This scaling circuitry converts data from one number format (e.g., bfloat16) to another (e.g., half-precision floating-point) that the hardware natively supports, enabling format adaptability without requiring multiple dedicated hardware units for each format.
Solution Approach 2:
The patent changes the numerical parameters (precision, exponent range, mantissa bits) of the data representation dynamically through scaling operations. By adjusting these parameters via pre-scaling and post-scaling circuitry, the system can emulate support for various number formats using a fixed hardware configuration, thus improving adaptability without proportionally increasing device complexity.
2Reliability
If the integrated circuit performs arithmetic operations directly in the available number format, then the processing speed is improved, but overflow or underflow issues occur when the variable range exceeds the format capacity
Solution Approach 1:
The patent applies pre-scaling to the input data before it enters the arithmetic logic unit. This preliminary action adjusts the exponent and mantissa of the input variables to ensure that the arithmetic operation results will fall within the representable range of the target number format, thereby preventing overflow and underflow before they occur.
Solution Approach 2:
The patent implements post-scaling circuitry that processes the output of the arithmetic operation and adjusts it back to the original number format. This feedback mechanism ensures that the final result maintains the correct numerical properties and range, providing reliability by correcting any potential range issues after the operation.
3Adaptability or versatility
If the integrated circuit emulates operations in a desired number format by scaling inputs and outputs, then the adaptability to unsupported formats is improved, but the processing time increases due to additional scaling operations
Solution Approach 1:
The patent segments the arithmetic operation into distinct phases: pre-scaling of inputs, core arithmetic computation, and post-scaling of outputs. This segmentation allows each component to be optimized independently - the scaling operations can be performed in parallel where possible, and the core arithmetic unit can be dedicated to high-speed computation, minimizing the overall time penalty.
Solution Approach 2:
The patent creates a scaled copy of the input data in a format that the hardware can efficiently process. Rather than modifying the hardware to support every possible format, the system makes a copy of the data in the appropriate format, performs the operation, and then scales the result back. This copying approach maintains adaptability while keeping the core hardware simple and fast.
Data Source
AI summary
The present disclosure relates generally to techniques for adjusting the number representation (e.g., format) of a variable before and/or after performing one or more arithmetic operations on the variable. In particular, the present disclosure relates to scaling the range of a variable to a suitable representation based on available hardware (e.g., hard logic) in an integrated circuit device. For example, an input in a first number format (e.g., bfloat16) may be scaled to a second number format (e.g., half-precision floating-point) so that circuitry implemented to receive inputs in the second number format may perform one or more arithmetic operations on the input. Further, the output produced by the circuitry may be scaled back to the first number format. Accordingly, arithmetic operations, such as a dot-product, performed in a first format may be emulated by scaling the inputs to and/or the outputs from arithmetic operations performed in another format.