Exponential Approximation Using Bit-Shift Taylor Factorials
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The computation of the exponential function e^x in neural networks is computationally expensive due to the need for significant floating-point operations and large lookup tables, particularly in functions like the softmax function, which hinders efficient processing and increases hardware requirements.
Innovation Solution
Approximate the exponential function e^x using a Taylor series with factorials replaced by nearest powers of 2, allowing bit-shift operations to replace division, and decompose the argument into an integer and a power of 2 for further simplification, reducing hardware complexity and error resilience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the exponential function is computed using standard floating-point operations or large lookup tables, then computation accuracy is maintained, but processing time increases and hardware complexity increases
Solution Approach 1:
The patent changes the computational parameters by representing the exponential function as a polynomial in base 2 (e^x = 2^(x/ln(2))). This transformation allows the use of binary exponentiation and bit-shift operations instead of standard floating-point multiplication and division, significantly reducing processing time while maintaining sufficient accuracy for neural network applications
Solution Approach 2:
The patent uses lookup tables to store pre-computed values of the exponential function for integer and fractional parts separately. By copying and combining these pre-computed values rather than computing the full exponential from scratch, the system reduces processing time while maintaining accuracy
2Measurement precision
If the exponential function is computed using standard floating-point operations or large lookup tables, then computation accuracy is maintained, but hardware size and power consumption increase
Solution Approach 1:
The patent transforms the computational approach to use integer arithmetic and bit-shift operations instead of floating-point operations. This parameter change allows the use of simpler hardware components (integer ALU, shift registers) rather than complex floating-point units, reducing hardware size and power consumption while maintaining sufficient accuracy
Solution Approach 2:
The patent segments the exponential computation into separate handling of integer and fractional parts of the exponent. The integer part is computed using bit-shift operations and the fractional part uses a smaller lookup table, allowing each segment to use optimized, smaller hardware components rather than a single large computational unit
3Device complexity
If the Taylor series approximation is used for e^x, then computation complexity is reduced, but the need for power computation, factorial computation, and division remains
Solution Approach 1:
The patent changes the mathematical parameters by expressing the exponential function in base 2 and using polynomial approximation with binomial coefficients instead of Taylor series. This eliminates the need for factorial computation and replaces division with bit-shift operations, simplifying hardware implementation
Solution Approach 2:
The patent substitutes complex arithmetic operations (division, factorial computation) with simpler mechanical operations (bit-shift, integer multiplication). The binary exponentiation approach replaces the need for sequential Taylor series computation with parallel bit-manipulation operations
Data Source
Figure 1
Figure 2
AI summary
A method (100) for computing an approximate value A of the exponential function ex of an argument x, comprising the steps of: • approximating (110) ex with a Taylor expansion T around x = 0 that comprises a predetermined number n of terms with i-th powers xi of the argument x divided by the respective factorial of i, with i = 1, ..., n; and • in the computation of each term, approximating (140) the factorial of i to the nearest power of 2, p(i!).