Implementation method of general exponential lookup table supporting floating point type data
By using a general exponential lookup table method that supports floating-point data, the problems of high computational latency and high power consumption in traditional floating-point hardware implementations are solved, achieving fast computation and efficient use of hardware resources.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-15
- Publication Date
- 2026-04-10
AI Technical Summary
Traditional floating-point hardware implementations require a large number of floating-point computing units, resulting in high computational latency, high power consumption, and low applicability.
A general exponential lookup table method that supports floating-point data is adopted. By finding the lookup table value, calculating the address, and performing the lookup and linear interpolation calculations, the computation of interpolation coefficients is simplified and the use of hardware resources is reduced.
It achieves faster computing speed and reduced hardware resource requirements, while improving computing accuracy and efficiency.
Smart Images
Figure CN121832879A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of integrated circuit digital technology, in particular to a general exponential lookup table implementation method supporting floating point data. BACKGROUND
[0002] The activation function breaks through the limitation of the linear model by introducing the nonlinear characteristic, so that the neural network can learn and express complex data features and abstract patterns. The introduction of the nonlinear operator not only enhances the expression ability of the model, but also optimizes the network training process through gradient propagation, which promotes the breakthrough progress of deep learning in the fields of computer vision, speech recognition, reinforcement learning, etc. The calculation of the activation function and the nonlinear operator depends on the floating point precision, especially when dealing with continuous values, the floating point operation can more accurately preserve the small differences of the data, thereby improving the model performance.
[0003] At present, the hardware implementation of floating point numbers needs to use a large number of floating point calculation units, and the calculation delay is high, the power consumption is large, and the applicability is also low. The traditional lookup table still needs more floating point calculation units when processing floating point inputs. SUMMARY
[0004] The purpose of the present application is to provide a general exponential lookup table implementation method supporting floating point data, to solve the problem of a large number of floating point calculation units required by traditional floating point hardware implementation.
[0005] To solve the above technical problems, the present application provides a general exponential lookup table implementation method supporting floating point data, comprising:
[0006] First, calculate the exponential lookup table lookup value x lut , and calculate the exponential lookup table data storage address addr according to the lookup value x lut .
[0007] Second, use addr to look up the exponential lookup table, and look up the calculated address addr and the next address addr+1 to obtain lookup table results y0 and y1 respectively.
[0008] Third, obtain the interpolation coefficient k from the linear interpolation formula, and convert the fixed point number (0.M in ) to M FP in the same floating point data format as x lut .
[0009] Fourth, perform linear interpolation calculation, and input the interpolation coefficient k and the two lookup table results y0 and y1 into the interpolation formula to obtain the final output result y.
[0010] In one embodiment, the calculation formula of the exponential lookup table lookup value x lut is:
[0011] wherein x in is a floating-point data input value input to an exponential lookup table, min FP is a floating-point data type consistent with x in , and is a bias of the input value x in ; since x in cannot be less than or equal to 0 in log2 calculation, min FP shifts the value in the log2 calculation to a value greater than 0 as a whole.
[0012] In an embodiment, a formula for calculating an exponential lookup table data storage address addr according to a lookup table value x lut is as follows:
[0013] addr = [log2(x lut )] + offset
[0014] = E lut - 127 + offset
[0015] wherein [] is a floor function, x lut is an FP32 lookup table value finally calculated, offset is an integer lookup table value bias, which shifts the lookup table result addr to a value greater than 0, and E lut represents an exponent of x lut , and is an integer exponent part value of x lut ; the lookup table data storage address addr is finally obtained through the formula, and if x lut is a non-normal number, addr is directly assigned as 0.
[0016] In an embodiment, a calculation formula for obtaining an interpolation coefficient k through a linear interpolation formula is as follows:
[0017]
[0018] wherein x0≤x lut ≤x1, x lut , x0and x1belong to the same floating-point number type and are all greater than 0, x0and x1are floating-point numbers stored in two adjacent addresses addr and addr+1 of an exponential lookup table, x lut has the same exponent value E as x0, and thus the corresponding exponential lookup table address addr is the same. In order to facilitate interpolation calculation, the floating-point data stored in the exponential lookup table has all 0s in the mantissa, and x lut , x0and x1have the same initial bias min FP , and min FP is a left boundary of the lookup table, and thus the FP32 true value of x0and x1is represented as:
[0019]
[0020] Since x0 and x1 are floating point numbers stored in the exponent lookup table at adjacent two addresses, (E1-E0) results in 1, thus (x1-x0)=2^(E0-127), and the floating point true value of x0 and x1 is brought into the formula of the interpolation coefficient k, and the following is obtained:
[0021]
[0022] In the formula, M in is the mantissa part of x in , and x lut is equal to the mantissa part of x in .
[0023] In an embodiment, the interpolation formula into which the interpolation coefficient k and the two lookup table results y0 and y1 are brought is as follows:
[0024]
[0025] to obtain the final output result y.
[0026] The present application provides a general exponential lookup table implementation method supporting floating point data, in which the method of simulating floating point type by integer is used in address calculation, the operation steps are simple, the operation process does not need to convert fixed point number and floating point number, and the operation speed is fast. In interpolation calculation, the calculation amount of the interpolation coefficient is greatly simplified through formula derivation, and the usage amount of hardware resources is reduced. BRIEF DESCRIPTION OF DRAWINGS
[0027] Figure 1 is the overall data processing flowchart of the present application.
[0028] Figure 2 is the address calculation structure block diagram of the present application.
[0029] Figure 3 is the linear interpolation calculation structure block diagram provided by the present application. DETAILED DESCRIPTION
[0030] The present application provides a general exponential lookup table implementation method supporting floating point data, which is further described in detail below in combination with the drawings and specific embodiments. The advantages and features of the present application will be more apparent according to the following description. It should be noted that the drawings are very simplified and use non-precise proportions, which are only used to facilitate and clarify the purpose of assisting the description of the embodiments of the present application.
[0031] In IEEE754 standard, a normalized floating point data type xFP The true value of x is represented as:
[0032] x FP = (-1) s × 2 E-n × (1.M)
[0033] In the IEEE754 standard, the true value of a non-normalized floating-point data type x FP is represented as:
[0034] x FP = (-1) s × 2 -(n-1) × (0.M)
[0035] In the formula, S represents the sign bit of the floating-point number x FP , E is the exponent of the floating-point data, represents the unsigned integer exponent bit part of the floating-point data, n represents the offset of the exponent E, and M represents the mantissa bit part of the floating-point data x FP .
[0036] The present application provides a general exponential lookup table implementation method supporting floating-point data, and the flowchart is as shown in Figure 1 The method of the present application comprises the following steps:
[0037] First, the exponential lookup table lookup value x lut is calculated, as shown in Figure 2 , and the calculation method and formula are as follows:
[0038] x lut = x in -min FP
[0039] In the formula, x in is the floating-point data input value input to the exponential lookup table, min FP is consistent with the floating-point data type of x in , and min FP is the offset of the x in input value. Since x in cannot be less than or equal to 0 when log2 is calculated, min FP can shift the value as a whole to be greater than 0 when log2 is calculated.
[0040] Second, the exponential lookup table data storage address addr is calculated, please continue to refer to Figure 2 , and the calculation method and formula are as follows:
[0041]
[0042] In the formula, [] is the floor function, and x lutFor the final FP32 table lookup value, offset is an integer table value offset number, which can offset the table lookup result addr to a value greater than 0, E lut represents the exponent of x lut , which is also the integer exponent part value of x lut . Through the formula, the final lookup table data storage address addr is obtained. If x lut is a non-standard number, addr is directly assigned as 0.
[0043] Third step, using addr to perform table lookup on the exponential lookup table: performing table lookup on the calculated address addr and the next address addr+1 to obtain lookup table results y0 and y1, respectively;
[0044] Fourth step, calculate the interpolation coefficient: the calculation formula of the interpolation coefficient k is obtained from the linear interpolation formula:
[0045]
[0046] In the formula, x0≤x lut ≤x1, x lut , x0 and x1 belong to the same floating-point number type and are all greater than 0, x0 and x1 are floating-point numbers stored in adjacent two addresses addr and addr+1 of the exponential lookup table, x lut and x0 have the same exponent E (exponent) value, so the corresponding exponential lookup table address addr is equal, in order to facilitate interpolation calculation, the floating-point data stored in the exponential lookup table has all 0s in the mantissa bit, and x lut , x0 and x1 have the same initial offset min FP , min FP is the left boundary of the lookup table, therefore, the FP32 true value of x0 and x1 can be expressed as:
[0047]
[0048] Since x0 and x1 are floating-point numbers stored in adjacent two addresses of the exponential lookup table, (E1-E0) result 1, therefore, (x1-x0)=2^(E0-127), after bringing the FP32 true value of x0 and x1 into the formula of the interpolation coefficient k, we can obtain:
[0049]
[0050] In the formula, M in is the mantissa part of x in , and x lut has the same mantissa part as x in .
[0051] Fifth step, bring the fixed-point number (0.Min ) converted into x lut M FP ;
[0052] The sixth step is to perform linear interpolation calculation to improve the calculation accuracy, as shown in the following formula: Figure 3 The linear interpolation calculation method and calculation formula of y are as follows:
[0053]
[0054] to obtain the final output result y.
[0055] The technical scheme of the present application is further described below in combination with specific embodiments:
[0056] Suppose the input data is 0.123, and the selected activation function in the lookup table is Tanh. The exponential lookup table is made in the range of [0, 4], and the depth is 65. That is, min FP = 0. Then the offset is -63.
[0057] The working mode is executed in the following steps:
[0058] The first step is to calculate the exponential lookup table value x lut , and the calculation method and calculation formula are as follows:
[0059] x lut = x in - min FP = 0.123
[0060] The true value expression of FP32 is 00111101111110101110000100100100.
[0061] The second step is to calculate the exponential lookup table data storage address addr, and the calculation method and calculation formula are as follows:
[0062]
[0063] Then x lut is between the interpolation point 2^-4 and the interpolation point 2^-3.
[0064] The third step is to perform lookup table on the exponential lookup table using addr: the calculated address 59 and the next address 60 are looked up, respectively obtaining the lookup table results y0 = 0.0624200915 and y1 = 0.124354087.
[0065] Fourth step, find the interpolation coefficient: the linear interpolation formula can be obtained from the interpolation coefficient k calculation formula for:
[0066]
[0067] M in =11110101110000100100100, and then k into the decimal system can get the result k = 0.96001775;
[0068] Fifth step, the fixed point number (0.M in ) into the same M lut with x FP floating point data format;
[0069] Sixth step, linear interpolation calculation, improve the calculation accuracy: the interpolation coefficient k and two table results y0 and y1 into the interpolation formula, the final output results y, y linear interpolation calculation method and calculation formula as follows:
[0070]
[0071] Get the final output results 0.121877826508420125 with the input directly into the formula of the exact results error in 10^-4 order of magnitude.
[0072] The above description is only to the preferred embodiments of the present application, not any limit on the scope of the present application, the ordinary skill in the art according to the above disclosure of any change, modification, all belong to the scope of the claims.
Claims
1. A method for implementing a general exponential lookup table that supports floating-point data, characterized in that, include: The first step is to find the value x in the exponent lookup table. lut And based on the lookup table value x lut Find the data storage address addr of the index lookup table; The second step is to use addr to look up the exponential lookup table. The calculated address addr and the next address addr+1 are used to look up the table to obtain the lookup table results y0 and y1 respectively. The third step is to obtain the interpolation coefficients k from the linear interpolation formula, and then convert the fixed-point number (0.M) to k. in ) transformed into x lut M with the same floating-point data format FP ; The fourth step is to perform linear interpolation calculations, substituting the interpolation coefficients k and the two lookup table results y0 and y1 into the interpolation formula to obtain the final output result y.
2. The method for implementing a general exponential lookup table supporting floating-point data as described in claim 1, characterized in that, The value x in the exponent lookup table is obtained by looking up the exponent lookup table. lut The calculation formula is: x lut =x in -my FP In the formula, x in It is the floating-point data input value entered into the exponential lookup table, min FP With x in The floating-point data types are consistent, which is x. in The bias of the input value; due to the calculation of log2, x in It cannot be less than or equal to 0, therefore min FP The values used in the log2 calculation are shifted to be greater than 0.
3. The method for implementing a general exponential lookup table supporting floating-point data as described in claim 2, characterized in that, The value x is based on the table lookup. lut The formula for finding the data storage address addr of the exponential lookup table is: addr=[log2(x lut )]+offset =E lut -127+offset In the formula, the square brackets [] represent floor function, x lut For the final FP32 lookup value, offset is the integer offset of the lookup value, which shifts the lookup result addr to a value greater than 0. lut x represents lut The exponent is also x lut The integer exponent part of the value; the final data storage address addr of the lookup table is obtained through the formula, if x lut Since it is a non-standard number, simply assign the value 0 to addr.
4. The method for implementing a general exponential lookup table supporting floating-point data as described in claim 3, characterized in that, The formula for calculating the interpolation coefficient k obtained from the linear interpolation formula is as follows: In the formula, x0 < x lut < x1, x lut x0 and x1 belong to the same floating-point type and are both greater than 0. x0 and x1 are floating-point numbers stored at two adjacent addresses, addr and addr+1, in the exponent lookup table. lut Since the exponent E of x0 is equal to the value of x0, the corresponding address addr of the exponential lookup table is also equal. For ease of interpolation, the mantissa of the floating-point data stored in the exponential lookup table is all 0. Furthermore, for the same lookup table, x... lut x0 and x1 have the same initial bias min FP min FP To find the left boundary of the lookup table, the FP32 truth values for x0 and x1 are therefore: Since x0 and x1 are floating-point numbers stored at two adjacent addresses in an exponential lookup table, (E1-E0) results in 1. Therefore, (x1-x0) = 2^(E0-127). Substituting the floating-point truth values of x0 and x1 into the formula for the interpolation coefficient k, we get: In the formula, M in It is x in The mantissa part, x lut With x in The last digits are equal.
5. The method for implementing a general exponential lookup table supporting floating-point data as described in claim 4, characterized in that, The interpolation formula that substitutes the interpolation coefficient k and the two lookup table results y0 and y1 is as follows: The final output result y is obtained.