Systolic Arithmetic Array Scaling for FP16 Overflow Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systolic arithmetic arrays face performance degradation due to overflow and underflow issues when performing low-accuracy arithmetic operations, particularly in AI computations, as the computation results often exceed or fall below the expressible range of lower-accuracy floating-point formats like FP16, and adjusting the scaling factor using a CPU introduces overhead.
Innovation Solution
A systolic arithmetic array device with integrated arithmetic operators and an adjuster that shifts and updates the exponent part of computation results and adjusts the scaling factor within the array, preventing overflow and underflow by dynamically adjusting the floating-point format range.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by stationary object
If low-accuracy floating-point format (FP16) is used for computation, then power performance and computational efficiency are improved, but overflow and underflow occur more frequently
Solution Approach 1:
The patent implements dynamic scaling factor adjustment within the systolic arithmetic array. Each arithmetic operator dynamically adjusts the scaling factor based on the exponent values of its inputs, allowing the expression range to adapt to the actual computation results. This dynamic adjustment prevents overflow and underflow while maintaining the benefits of low-accuracy formats.
Solution Approach 2:
The patent changes the scaling factor parameter dynamically during computation. By adjusting the scaling factor based on exponent comparisons and arithmetic operations, the system maintains computation results within the valid range of FP16 format, preventing overflow and underflow while preserving energy efficiency.
2Reliability
If scaling factor adjustment is performed using CPU, then overflow and underflow are suppressed, but computational overhead increases
Solution Approach 1:
The systolic arithmetic array performs scaling factor adjustment autonomously without CPU intervention. Each arithmetic operator independently adjusts its own scaling factor based on local exponent information, eliminating the need for CPU-based control and avoiding the associated computational overhead and latency.
Solution Approach 2:
The patent extracts the scaling factor adjustment function from the CPU and relocates it directly into the arithmetic operators within the systolic array. This extraction eliminates the communication overhead and latency associated with CPU-based control, improving computational performance while maintaining reliability.
3Measurement precision
If FP16 format is used, then expression range is reduced, but resolution and precision are improved
Solution Approach 1:
The patent makes the expression range dynamic by adjusting the scaling factor based on the actual values being computed. This allows the system to effectively expand or contract the expression range as needed while maintaining FP16 resolution, adapting to different computation scenarios without sacrificing precision.
Data Source
AI summary
A device includes a systolic arithmetic array including multiple arithmetic operators connected in an array and an adjuster connected to one or more most-downstream arithmetic operators. Each arithmetic operator shifts multiple bits in an exponent part of a result of an arithmetic operation performed on input data in accordance with a value of one or more bits in the exponent part, updates shift information representing a cumulative amount of shifting made by at least one arithmetic operator, and outputs data obtained by shifting the bits in the exponent part, as the input data, to a subsequent arithmetic operator or the adjuster being a downstream entity connected to a downstream side of the arithmetic operator, and outputs the shift information to the downstream entity. The adjuster adjusts, based on the shift information from each most-downstream arithmetic operator, the input data from the each most-downstream arithmetic operator.