FPGA Exponential Datapath Using Lookup Tables for Low-Latency Softmax
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing FPGA implementations of the exponential function for low-precision floating-point numbers are resource-intensive and latency-heavy, particularly in computing the softmax operation for large neural networks, consuming a significant portion of FPGA resources and time.
Innovation Solution
A low-area and low-latency architecture for approximating the exponential function using fixed-point adders, constant shifters, and lookup tables, without floating-point or integer multipliers, specifically designed for low-precision formats like BFloat16, employing a scaled mantissa operation and exponential stage to compute the exponential function efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional FPGA implementations of the exponential function are used, then computation accuracy is maintained, but resource consumption and latency increase significantly
Solution Approach 1:
The patent changes the computational parameters by using fixed-point arithmetic instead of floating-point arithmetic, and by transforming the exponential function computation into a series of simpler operations (scaling, lookup table access, and addition). This parameter change maintains computational accuracy while significantly reducing the hardware resources required, directly resolving the contradiction between accuracy and resource consumption.
Solution Approach 2:
The patent replaces complex floating-point multiplier operations with simpler fixed-point arithmetic operations and lookup table mechanisms. By substituting the mechanical floating-point computation system with a fixed-point system combined with pre-computed lookup tables, the invention reduces hardware complexity while preserving the essential computational function, thereby resolving the resource consumption issue.
2Measurement precision
If traditional FPGA implementations of the exponential function are used, then computation accuracy is maintained, but computation time increases
Solution Approach 1:
The patent applies preliminary action by pre-computing exponential values and storing them in lookup tables during the design phase. During runtime, the system only needs to perform scaling and table lookup operations rather than computing exponentials from scratch, which dramatically reduces computation time and latency while maintaining accuracy through the pre-computed values.
Solution Approach 2:
By changing from floating-point to fixed-point parameter representation and transforming the computational approach to use scaling factors and lookup tables, the patent reduces the number of computational cycles required. This parameter transformation maintains numerical accuracy while enabling faster computation, directly addressing the latency issue.
3Device complexity
If low-precision floating-point formats are used, then resource consumption is reduced, but computation accuracy may deteriorate
Solution Approach 1:
The patent introduces scaling factors as intermediary variables that bridge the gap between low-precision fixed-point representation and the required computational accuracy. These scaling factors allow the system to perform accurate exponential computations using fixed-point arithmetic by appropriately scaling the input, performing the computation, and scaling the result, thereby maintaining accuracy while using resource-efficient fixed-point formats.
Data Source
AI summary
Aspects of embodiments of the present disclosure relate to a field programmable gate array (FPGA) configured to implement an exponential function data path including: an input scaling stage including constant shifters and integer adders to scale a mantissa portion of an input floating-point value by approximately log2 e to compute a scaled mantissa value, where e is Euler's number; and an exponential stage including barrel shifters and an exponential lookup table to: extract an integer portion and a fractional portion from the scaled mantissa value based on the exponent portion of the input floating-point value; apply a bias shift to the integer portion to compute a result exponent portion of a result floating-point value; lookup a result mantissa portion of the result floating-point value in the exponential lookup table based on the fractional portion; and combine the result exponent portion and the result mantissa portion to generate the result floating-point value.