High-precision hyperbolic tangent function calculation device

The high-precision hyperbolic tangent function calculation device for convolutional neural networks addresses the issues of complexity and inaccuracy by using a parameter determination module and calculation execution module to optimize hyperbolic tangent function calculations through direct output, definition calculation, and Taylor expansion, thereby improving computational efficiency and accuracy.

CN119089099APending Publication Date: 2024-12-06NANJING UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411233902.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-04
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Existing activation function calculation schemes for hyperbolic tangent functions in neural networks suffer from high computational complexity and inaccurate results, particularly due to the use of low-accuracy approximations like Hard tanh and Ultra Tanh, and complex methods like Taylor polynomial series approximation, which require extensive calculations and fail to provide precise results when x is a small number.

Method used

A high-precision hyperbolic tangent function calculation device for convolutional neural networks that includes a parameter acquisition module, a parameter determination module, and a calculation execution module, which uses direct output, definition calculation, and Taylor expansion based on a fast interpolation algorithm to determine the appropriate calculation method for single-precision floating-point parameters, optimizing the calculation process through piecewise function handling.

Benefits of technology

The device simplifies calculations and improves accuracy by employing direct output, definition calculation, and Taylor expansion methods tailored to different parameter ranges, reducing computational complexity and enhancing the precision of hyperbolic tangent function calculations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119089099A_ABST
    Figure CN119089099A_ABST
Patent Text Reader

Abstract

The invention provides a high-precision hyperbolic tangent function calculation device, and the device comprises a parameter obtaining module which is configured to obtain a single-precision floating point parameter from a convolutional neural network; the parameter judgment module is configured to judge the relationship between the absolute value of the single-precision floating point parameter and a preset value and generate a corresponding calculation instruction; the calculation execution module is configured to perform hyperbolic tangent function calculation on the single-precision floating point parameters according to the calculation instruction to obtain target output, and the target output is used for representing nonlinear features in the convolutional neural network; the calculation of the hyperbolic tangent function comprises Taylor expansion calculation, direct output calculation and definition calculation, and the definition calculation comprises calculation of a transcendental function by using a fast interpolation algorithm. The problems that an existing activation function calculation scheme is high in calculation complexity and inaccurate in calculation result are solved through the calculation device.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of activation function calculation, and in particular to a high-precision hyperbolic tangent function calculation device. Background Art

[0002] The hyperbolic tangent function (tanh function) is a common nonlinear function in mathematics and statistics, and is currently widely used in neural networks, signal processing, data compression, scientific computing and other fields. For example, in the field of neural networks, tanh can be used as an activation function to introduce nonlinear factors and improve the performance of the model.

[0003] The Tanh function calculation formula is as follows:

[0004]

[0005] In order to quickly calculate tanh, the field of neural networks proposed Hard tanh and Ultra Tanh functions to approximate tanh, but due to their low accuracy, they can still partially reduce the accuracy of the neural network.

[0006] In the field of transcendental function calculation, the Taylor polynomial series approximation algorithm approximates the target function by expanding a finite Taylor polynomial series. However, this method requires expanding enough polynomials to achieve an error of the ulp order of magnitude. Since floating-point functions are used in calculations, this method brings a huge amount of calculations.

[0007] If you choose to directly calculate the output based on the tanh expression, when x is a small number, the numerator is 0 and the accurate calculation result cannot be obtained.<math.h> expm1f is defined in the x -1, but the implementation is overly complex. Summary of the invention

[0008] The present application provides a high-precision hyperbolic tangent function calculation device to solve the problem of high calculation complexity and inaccurate calculation results in existing activation function calculation schemes.

[0009] The computing device is used for a convolutional neural network, and the computing device comprises:

[0010] A parameter acquisition module, wherein the parameter acquisition module is configured to acquire single-precision floating-point parameters from the convolutional neural network;

[0011] A parameter determination module, wherein the parameter determination unit is configured to determine the relationship between the absolute value of the single-precision floating-point parameter and a preset value, and generate a corresponding calculation instruction;

[0012] A calculation execution module, wherein the calculation execution module is configured to calculate a hyperbolic tangent function on the single-precision floating-point parameter according to the calculation instruction to obtain a target output, wherein the target output is used to characterize the nonlinear characteristics in the convolutional neural network; the calculation of the hyperbolic tangent function includes Taylor expansion calculation, direct output calculation and definition calculation, and the definition calculation includes calculating a transcendental function using a fast interpolation algorithm.

[0013] Preferably, the parameter determination module is further configured to:

[0014] Determining whether the absolute value of the single-precision floating-point parameter is greater than a first preset value;

[0015] If yes, generating a first calculation instruction, wherein the first calculation instruction is used to instruct the calculation execution module to perform direct output calculation according to the single-precision floating-point parameter;

[0016] If not, determining whether the absolute value of the single-precision floating-point parameter is greater than a second preset value;

[0017] If yes, generating a second calculation instruction, wherein the second calculation instruction is used to instruct the calculation execution module to perform a definition calculation according to the single-precision floating-point parameter;

[0018] If not, a third calculation instruction is generated, where the third calculation instruction is used to instruct the calculation execution module to perform Taylor expansion calculation according to the single-precision floating-point parameter.

[0019] Preferably, the parameter determination module is further configured to:

[0020] When the absolute value of the single-precision floating-point parameter is greater than the first preset value, determining whether the single-precision floating-point parameter is a positive number;

[0021] If yes, a first sub-calculation instruction is generated, where the first sub-calculation instruction is used to instruct the calculation execution module to output 1 in fp32 format according to the single-precision floating-point parameter;

[0022] If not, a second sub-calculation instruction is output, where the second sub-calculation instruction is used to instruct the calculation execution module to output -1 in the fp32 format according to the single-precision floating-point parameter.

[0023] Preferably, the calculation execution module includes:

[0024] a defined-formula calculation unit, the defined-formula calculation unit being configured to perform defined-formula floating-point calculation on the single-precision floating-point parameter according to the second calculation instruction;

[0025] A Taylor calculation unit is configured to perform Taylor expansion floating-point calculation on the single-precision floating-point parameter according to the third calculation instruction.

[0026] Preferably, the definitional calculation unit is further configured as:

[0027] Preprocessing the single-precision floating-point parameter to obtain a preprocessing parameter;

[0028] Obtaining a calculation function selection signal according to a second calculation instruction;

[0029] Selecting a signal to perform polynomial calculation according to the preprocessing parameter and the calculation function to obtain a polynomial calculation result;

[0030] A signal is selected for post-processing according to the definition result and the calculation function to obtain the target output.

[0031] Preferably, the Taylor calculation unit is further configured as:

[0032] Floating-point FMA processing, floating-point addition processing and floating-point multiplication processing are performed according to the single-precision floating-point parameter, so as to realize Taylor expansion floating-point calculation of the single-precision floating-point parameter.

[0033] Preferably, the definitional calculation unit includes:

[0034] A preprocessing component, wherein the preprocessing component is configured to perform initialization processing corresponding to the transcendental function on the single-precision floating-point parameter to obtain the preprocessing parameter, wherein the preprocessing parameter includes a table lookup index corresponding to different transcendental functions, a polynomial calculation input, and an intermediate result;

[0035] A table lookup index component, wherein the table lookup index component is configured to determine the type of the transcendental function according to the calculation function selection signal; perform table lookup processing according to the table lookup index and the type of the transcendental function to obtain a plurality of polynomial coefficients corresponding to the transcendental function;

[0036] a parameter calculation component, the parameter calculation component being configured to perform polynomial calculation on the polynomial coefficients and the polynomial calculation input according to the type of the transcendental function to obtain a polynomial calculation result;

[0037] A post-processing component is configured to normalize the polynomial calculation result and the intermediate result according to the type of the transcendental function to obtain the target output.

[0038] Preferably, the single-precision floating-point parameter is divided into a plurality of operation parameters before entering the Taylor calculation unit, the operation parameters at least including a first operand, a second operand and a third operand, and the Taylor calculation unit includes:

[0039] an adding component, the adding component being configured to perform an addition calculation on the operating parameters;

[0040] a multiplication component configured to perform a multiplication calculation on the operation parameters;

[0041] An FMA component, wherein the FMA component is configured to perform a multiplication-addition fusion calculation on the operation parameter;

[0042] a negation component, the negation component being configured to perform a negation calculation on the operation parameter;

[0043] The operation parameter is calculated by the addition component, the multiplication component and the FMA component to obtain a Taylor expansion.

[0044] Preferably, the polynomial coefficients include at least a first coefficient, a second coefficient and a third coefficient, and the parameter calculation component includes:

[0045] A squarer, the squarer being configured to square the polynomial calculation input to obtain a square result;

[0046] a first multiplier, wherein the first multiplier is configured to perform a multiplication calculation on the second coefficient and the polynomial calculation input to obtain a first multiplication result;

[0047] a second multiplier, wherein the second multiplier is configured to perform multiplication calculation on the square result and the third coefficient to obtain a second multiplication result;

[0048] A three-input adder is configured to perform an addition calculation on the first coefficient, the first multiplication result and the second multiplication result to obtain the target output.

[0049] Preferably, the computing device further comprises:

[0050] An output selection module is configured to output the target output of the corresponding calculation function according to the calculation function selection signal.

[0051] It can be seen from the above content that the present application provides a high-precision hyperbolic tangent function calculation device, the calculation device is used for convolutional neural networks, the calculation device includes a parameter acquisition module, the parameter acquisition module is configured to obtain single-precision floating-point parameters from the convolutional neural network; a parameter judgment module, the parameter judgment unit is configured to judge the relationship between the absolute value of the single-precision floating-point parameter and a preset value, and generate a corresponding calculation instruction; a calculation execution module, the calculation execution module is configured to calculate the hyperbolic tangent function of the single-precision floating-point parameter according to the calculation instruction to obtain a target output, and the target output is used to characterize the nonlinear characteristics in the convolutional neural network; the calculation of the hyperbolic tangent function includes Taylor expansion calculation, direct output calculation and definition calculation, and the definition calculation includes calculating the transcendental function using a fast interpolation algorithm. The present application solves the problem of high computational complexity and inaccurate calculation results of the existing activation function calculation scheme through the above-mentioned calculation device. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0053] Figure 1 A schematic diagram of a high-precision hyperbolic tangent function calculation device for the present application;

[0054] Figure 2 This is a schematic diagram of a calculation execution module in a high-precision hyperbolic tangent function calculation device of the present application;

[0055] Figure 3 This is a working state diagram of a calculation execution module in a high-precision hyperbolic tangent function calculation device of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] Figure 1 This is a schematic diagram of a high-precision hyperbolic tangent function calculation device of the present application.

[0058] See also Figure 1 It can be seen that this embodiment provides a high-precision hyperbolic tangent function calculation device, the calculation device is used for a convolutional neural network, and the calculation device includes:

[0059] The parameter acquisition module 100 is configured to acquire single-precision floating-point parameters from the convolutional neural network. Specifically, in this embodiment, the parameter acquisition module 100 is used to acquire single-precision floating-point parameters from the convolutional neural network.

[0060] The computing device further comprises:

[0061] The parameter judgment module 200 is configured to judge the relationship between the absolute value of the single-precision floating-point parameter and a preset value, and generate a corresponding calculation instruction. Specifically, in this embodiment, the parameter judgment module 200 judges the absolute value of the single-precision floating-point parameter by the user in this embodiment, and thereby implements different calculation processing for the single-precision floating-point parameters with different values.

[0062] It should be noted that the arrangement of all the single-precision floating-point parameters can be regarded as a piecewise function, and different calculation processes are performed in different segments.

[0063] Furthermore, in some embodiments, the parameter determination module 200 is further configured to:

[0064] Determining whether the absolute value of the single-precision floating-point parameter is greater than a first preset value;

[0065] If yes, a first calculation instruction is generated, wherein the first calculation instruction is used to instruct the calculation execution module 300 to perform direct output calculation according to the single-precision floating-point parameter;

[0066] If not, determining whether the absolute value of the single-precision floating-point parameter is greater than a second preset value;

[0067] If yes, a second calculation instruction is generated, wherein the second calculation instruction is used to instruct the calculation execution module 300 to perform a definition calculation according to the single-precision floating-point parameter;

[0068] If not, a third calculation instruction is generated, where the third calculation instruction is used to instruct the calculation execution module 300 to perform Taylor expansion calculation according to the single-precision floating-point parameter.

[0069] Specifically, in this embodiment, the first preset value example is 9.0109, and the second preset value example is 0.625; based on this, the absolute value of the single-precision floating-point parameter is judged by the parameter judgment module 200, and the specific judgment process is as follows:

[0070] When the absolute value of the single-precision floating-point parameter is greater than 9.0109, the parameter determination module 200 generates a first calculation instruction for instructing the calculation execution module 300 to perform direct output calculation according to the single-precision floating-point parameter;

[0071] Among them, since the absolute value of the single-precision floating-point parameter is judged, and the single-precision floating-point parameter itself has positive and negative distinctions, the following process is also used for the case where the absolute value of the single-precision floating-point parameter is greater than 9.0109:

[0072] When the absolute value of the single-precision floating-point parameter is greater than 9.0109 and the single-precision floating-point parameter is a positive number, the parameter determination module 200 generates the first sub-calculation instruction for instructing the calculation execution module 300 to output 1 in the fp32 format according to the single-precision floating-point parameter;

[0073] When the absolute value of the single-precision floating-point parameter is greater than 9.0109 and the single-precision floating-point parameter is a negative number, the parameter judgment module 200 generates the second sub-calculation instruction for instructing the calculation execution module 300 to output -1 in the fp32 format according to the single-precision floating-point parameter, thereby completing all embodiments for the case where the absolute value of the single-precision floating-point parameter is greater than 9.0109.

[0074] When the absolute value of the single-precision floating-point parameter is less than 9.0109 and the absolute value of the single-precision floating-point parameter is greater than 0.625, the parameter determination module 200 generates the second calculation instruction for instructing the calculation execution module 300 to perform a definition-based calculation according to the single-precision floating-point parameter;

[0075] When the absolute value of the single-precision floating-point parameter is less than 0.625, the parameter judgment module 200 generates the third calculation instruction for instructing the calculation execution module 300 to perform Taylor expansion calculation according to the single-precision floating-point parameter; thereby, the segmented calculation of the single-precision floating-point parameter under different numerical conditions is completed, which not only simplifies the calculation of the calculation function, but also improves the accuracy of the calculation to a certain extent.

[0076] The computing device further comprises:

[0077] A calculation execution module 300 is configured to calculate a hyperbolic tangent function on the single-precision floating-point parameter according to the calculation instruction to obtain a target output, wherein the target output is used to characterize the nonlinear characteristics in the convolutional neural network; the calculation of the hyperbolic tangent function includes Taylor expansion calculation, direct output calculation and definition calculation, and the definition calculation includes calculating a transcendental function using a fast interpolation algorithm. Specifically, in this embodiment, the first sub-calculation instruction, the second sub-calculation instruction, the third calculation instruction and the second calculation instruction are executed by the calculation execution module 300; when executing the first sub-calculation instruction and the second sub-calculation instruction, no redundant calculation is required, and 1 or -1 in the corresponding format of fp32 can be directly output; when executing the second calculation instruction and the third calculation instruction, the Taylor expansion calculation or the definition calculation needs to be performed according to the corresponding instructions.

[0078] It should be noted that in the definitional calculation, the transcendental function is calculated using a fast interpolation algorithm.

[0079] Further, in some embodiments, the calculation execution module 300 includes:

[0080] A definitional calculation unit 310, wherein the definitional calculation unit 310 is configured to perform a definitional floating-point calculation on the single-precision floating-point parameter according to the second calculation instruction. Specifically, in the present embodiment, the definitional calculation unit 310 is used to preprocess the single-precision floating-point parameter to obtain a preprocessing parameter; perform the polynomial calculation according to the preprocessing parameter and the calculation function selection signal to complete the definitional floating-point calculation to obtain a polynomial calculation result; and perform post-processing according to the polynomial calculation result and the calculation function selection signal to obtain the target output.

[0081] Taylor calculation unit 320, the Taylor calculation unit 320 is configured to perform Taylor expansion floating-point calculation on the single-precision floating-point parameter according to the third calculation instruction. Specifically, in the present embodiment, the Taylor calculation unit 320 is used to perform floating-point FMA processing, floating-point addition processing and floating-point multiplication processing according to the single-precision floating-point parameter, so as to implement Taylor expansion floating-point calculation of the single-precision floating-point parameter.

[0082] The specific structures of the definitional calculation unit 310 and the Taylor calculation unit 320 are shown in Figure 2 .

[0083] Figure 2 This is a schematic diagram of a calculation execution module in a high-precision hyperbolic tangent function calculation device of the present application.

[0084] See also Figure 2It can be seen that, further, in some embodiments, the definition calculation unit 310 includes:

[0085] A preprocessing component 311, wherein the preprocessing component 311 is configured to perform initialization processing corresponding to the transcendental function on the single-precision floating-point parameter to obtain the preprocessing parameter, wherein the preprocessing parameter includes a table lookup index corresponding to different transcendental functions, a polynomial calculation input, and an intermediate result;

[0086] A table lookup index component 312, wherein the table lookup index component 312 is configured to determine the type of the transcendental function according to the calculation function selection signal; perform table lookup processing according to the table lookup index and the type of the transcendental function to obtain a plurality of polynomial coefficients corresponding to the transcendental function;

[0087] A parameter calculation component 313, wherein the parameter calculation component 313 is configured to perform polynomial calculation on the polynomial coefficients and the polynomial calculation input according to the type of the transcendental function to obtain a polynomial calculation result;

[0088] A post-processing component 314 is configured to normalize the exponent bits of the polynomial calculation result and the intermediate result pair according to the type of the transcendental function to obtain the target output.

[0089] Specifically, in this embodiment, the components of the definitional calculation unit 310 can be found in Figure 2 In the middle left half, in the definition calculation unit 310, the single-precision floating-point parameter is initialized and preprocessed with the corresponding transcendental function by the preprocessing component 311, wherein the corresponding transcendental function is determined by the calculation function selection signal, and the table lookup index, polynomial calculation input and intermediate result are outputted thereby, wherein the intermediate result is used for the calculation of the post-processing component 314; the table lookup index component 312 performs table lookup processing according to the table lookup index and the type of the transcendental function, and obtains several corresponding polynomial coefficients by table lookup; the parameter calculation component 313 performs polynomial calculation corresponding to the definition calculation unit 310 on the obtained polynomial coefficients and the polynomial calculation input, and obtains the definition result. It should be noted that the data calculated by the parameter calculation component 313 is not the final data, and it needs to be output after post-processing; the polynomial result and the intermediate result are normalized by the post-processing component 314 according to the type of the transcendental function, so as to obtain the target output for output.

[0090] The polynomial coefficients include at least a first coefficient, a second coefficient and a third coefficient, and the parameter calculation component 313 includes:

[0091] A squarer 3131, wherein the squarer 3131 is configured to square the polynomial calculation input to obtain a square result;

[0092] a first multiplier 3132, wherein the first multiplier 3132 is configured to perform a multiplication calculation on the second coefficient and the polynomial calculation input to obtain a first multiplication result;

[0093] a second multiplier 3133, wherein the second multiplier 3133 is configured to perform multiplication calculation on the square result and the third coefficient to obtain a second multiplication result;

[0094] A three-input adder 3134 is configured to add the first coefficient, the first multiplication result and the second multiplication result to obtain the target output.

[0095] Specifically, in this embodiment, the polynomial calculation is performed by the squarer 3131 , the first multiplier 3132 , the second multiplier 3133 and the three-input adder 3134 .

[0096] See also Figure 2 It can be seen that, further, in some embodiments, the single-precision floating-point parameter is divided into a plurality of operation parameters before entering the Taylor calculation unit 320, and the operation parameters at least include a first operand, a second operand, and a third operand. The Taylor calculation unit 320 includes:

[0097] An adding component 321, wherein the adding component 321 is configured to perform an addition calculation on the operating parameters;

[0098] a multiplication component 322, the multiplication component 322 being configured to perform a multiplication calculation on the operation parameters;

[0099] A negation component 323, wherein the negation component 323 is configured to perform a negation calculation on the operation parameter;

[0100] An FMA component 324, wherein the FMA component 324 is configured to perform a multiplication-addition fusion calculation on the operation parameter;

[0101] The operation parameter is calculated by the addition component 321, the multiplication component 322 and the FMA component 324 to obtain a Taylor expansion;

[0102] Specifically, in this embodiment, the Taylor calculation unit 320 constructs a calculation formula through the addition component 321, the multiplication component 322 and the FMA component 324, so as to complete the Taylor expansion calculation.

[0103] Furthermore, in some embodiments, the computing device further includes:

[0104] The output selection module 400 is configured to select the target output of the calculation function according to the calculation function selection signal output. Specifically, in the present embodiment, the final output of the computing device is completed through the output selection module 400, and the target output of the calculation function is selected according to the calculation function selection signal output.

[0105] The principles of this embodiment are as follows:

[0106] This solution uses a piecewise calculation of high-precision hyperbolic tangent function. The input is a single-precision floating point number. When the absolute value of the input is greater than 9.0109: if the input is a negative number, the fp32 format representation of -1 is directly output; if the input is a positive number, the fp32 format representation of 1 is directly output. When the absolute value of the input is in the interval (0.0625, 9.0109], it is directly calculated by the definition formula. The e exponential calculation and division in the definition formula use the fast interpolation calculation of the e exponential with base 2 and the reciprocal for equivalent calculation. The conversion is as follows:

[0107]

[0108] log2e uses the fp32 data format inside the device, and the multiplication and addition units in the formula use single-precision multiplication, addition, and FMA (multiplication and addition fusion x·y+z). When the absolute value of the input is less than 0.0625, the first two terms of Taylor expansion are used for approximate calculation:

[0109]

[0110] The calculation consists of two floating-point multiplications and one FMA.

[0111] For example, a 3-bit wide calculation function selection signal is used to select the calculation operation to be performed, and the device can complete six basic calculations: exponential, reciprocal, floating-point multiplication, floating-point addition, FMA, and negation with a base of 2. The first operand, the second operand, and the third operand are all floating-point inputs.

[0112] In order to speed up the calculation of the hyperbolic tangent function without losing accuracy, fast interpolation is used to calculate the base 2 exponential and reciprocal.

[0113] First, after the preprocessing component 311, for the inverse:

[0114] (1) Initialize the mantissa segment first: If the 8 bits of the exponent E are not all 0, it indicates a normalized number. Just add 1 before the mantissa M, and the bit width will be expanded from 23 bits to 24 bits. If the exponent E bits are all 0, but the highest bit of M (index is 22) is 1, the mantissa segment is shifted left by 1 bit to get a 24-bit initialization result. If the exponent E bits are all 0 and the highest bit of M is 0, and the second highest bit (index is 21) is 1, the mantissa segment is shifted left by 2 bits and the lower 24 bits are taken as the mantissa initialization result. In other cases, since the maximum representation value of the FP32 data format is (2-2 -23 )×2 127 , the result after the inverse will exceed the maximum representation value of the FP32 data format, so it is only calculated in the post-processing component 314.

[0115] (2) Calculate the intermediate result norm_exp of the exponent segment: If the exponent bits E are all 0 and the highest bit of M is 0 and the second highest bit is 1, norm_exp = 255-E; in other cases, norm_exp = 254-E.

[0116] (3) The table lookup index component 312 determines the table lookup index and the polynomial calculation input x: the table lookup index is the 17th to 23rd bits of the mantissa segment initialization result, and the intermediate calculation input x is the lower 16 bits of the mantissa segment initialization result with 3 bits of 0 added at the end;

[0117] For exponents with base 2, according to the fp32 data format, the exponent bits of the output result are the integer part of the input result.

[0118] (1) Similarly, the mantissa is initialized first. For normalized numbers, the integer 1 is added to the 24th bit. The denormalized number is not processed and is recorded as frac_1. Based on the exponent value, the fixed-point representation of the true value with the decimal point between the 23rd and 24th bits is obtained. real =E-127, if E real If it is a positive number, shift frac_1 left by E. real bit; if E real If it is a negative number, shift frac_1 right by -E real bit, and the result is recorded as frac_2. Then the integer part int is the 24th to 31st bits of frac_2 (frac_2[30:23]), and the fractional part frac is the lower 23 bits of frac_2 (frac_2[22:0]).

[0119] (2) Exponential intermediate result norm_exp: If the input is a negative number, then:

[0120] 2 -(int+frac) =2 -(int+1-1+frac) =2 -(int+1) 2(1-frac) =2 -(int+1) 2 frac补 ;

[0121] Since the exponent bit of the fp32 data format is biased, norm_exp=127-(int+1). In particular, if frac is 0, norm_exp=127-int. The two's complement of frac is recorded as frac_3. If the input is a positive number norm_exp=127+int, frac is recorded as frac_3.

[0122] (3) Lookup index and determination of x: The lookup index is the 17th to 23rd bits of frac_3 (frac_1[22:16]), and x is the lower 16 bits of frac_3 with 3 bits of 0 added to the end. Interpolation calculation C0+C1x+C2x 2 The coefficients C0, C1, and C2 of the polynomial are stored in a lookup table, which is 52 bits in total. The lookup table index is composed of the calculation function selection signal and the 7-bit lookup table index obtained by preprocessing. After the polynomial coefficients are taken out, they are passed through the multiplier and the three-input adder 3134 to obtain the polynomial calculation result, which is recorded as sum with a bit width of 64 bits and sent to the post-processing module.

[0123] For the reciprocal, if the input is infinite or nan type, the output result is 0 or nan directly; if the input is less than or equal to 2 in absolute value -128 Denormalized numbers directly output positive / negative infinity; perform the following processing on other inputs: confirm the number of 0s before the 1 in the highest bit of the 64-bit result sum, record it as zero_num_sum. shift_num = zero_num_sum-1. Shift sum to the left by shift_num and record it as sum_norm_mid. If exp_mid is less than zero_num_sum, it proves that the output result is a denormalized number, shift sum_norm_mid to the right by zero_num_sum-exp_mid, record it as sum_norm, and the exponent bit is 0; otherwise, no shift is performed, and the exponent bit result is exp_mid-shift_num. Finally, the 40th to 62nd bits of sum_norm (sum_norm[61:39]) are intercepted as the mantissa segment output, and the lower 39 bits are rounded to obtain the final exponent and mantissa output. The sign is determined by the sign of the input value.

[0124] For exponent: output 0 for negative infinity; output positive infinity for positive infinity if input is positive infinity; output nan for nan if input is positive, output positive infinity directly if the exponent plus bias is greater than 133, and the output exceeds the maximum value that the fp32 data format can represent; output positive infinity directly if input is negative, output 0 directly if the exponent plus bias is greater than 134, and the output exceeds the absolute minimum value that the fp32 denormalized number can represent; output 0 directly if the exponent plus bias is less than 103, which represents a very small number, output 1 directly; perform the following steps for other cases: the sign bit is 0, which represents a positive number. Confirm the number of 0s before the 1 in the highest bit of the 64-bit result sum, recorded as zero_num_sum. shift_num = zero_num_sum-1. At this time, the decimal point is between the 62nd and 63rd bits. If the result_exp obtained by preprocessing is less than 1, it proves to be a non-normalized output. The sum is shifted right by 1-result_exp, recorded as sum_norm, and the output exponent is 0. In other cases, the sum is shifted left by shift_num, recorded as sum_norm, and the output exponent is result_exp-shift_num. Finally, the 40th to 62nd bits of sum_norm (sum_norm[61:39]) are intercepted as the mantissa segment output, and the lower 39 bits are rounded to obtain the final exponent and mantissa output.

[0125] The floating point multiplication component 322 and the floating point addition component 321 are both implemented with a one-stage pipeline, completing the calculation in two time cycles. The FMA component 324 is implemented with a two-stage pipeline, completing the calculation in three time cycles.

[0126] Figure 3 This is a working state diagram of a calculation execution module in a high-precision hyperbolic tangent function calculation device of the present application.

[0127] Figure 3 When the calculation execution module 300 shown in the figure performs calculation, it is necessary to determine which step of the hyperbolic tangent function calculation is currently being performed. A finite state machine is used for control, and a state jump is performed when a done signal indicating that the calculation is completed is received. Specifically, it includes:

[0128] IDLE: In idle state, no calculation is performed. If any signal of one_sign, minus_one_sign, or zero_sign is pulled high, it will jump directly to the DONE state, without performing any calculation, and directly output ±1 or 0.

[0129] TAYLORCOPY: When the smaller signal is pulled high, the proof uses the first two terms of Taylor expansion for calculation, jumping from IDLE to TAYLORCOPY, which stores the input in register 1. When the operation is completed, the done signal is pulled high and jumps to the next state.

[0130] TAYLORMUL_1: The calculation function selects multiplication and multiplies the input by the value of register 1 to obtain x 2 , the calculation result is stored in register 0. When the operation is completed, the done signal is pulled high and jumps to the next state.

[0131] TAYLORMUL_2: The calculation function selects multiplication and multiplies the value of register 0 by the value of register 1 to obtain x 3 , the calculation result is stored in register 0. When the operation is completed, the done signal is pulled high and jumps to the next state.

[0132] TAYLORFMA: Select FMA as the calculation function and execute The calculation is performed and the result is stored in the target register. When the operation is completed, the done signal is pulled high and the state jumps to the DONE state.

[0133] DIRECTCOPY: When the large signal is pulled high, it proves that the fast interpolation algorithm is used for direct calculation, jumping from IDLE to DIRECTCOPY, which stores the input in register 1. When the operation is completed, the done signal is pulled high and jumps to the next state.

[0134] DIRECTMUL_1: The calculation function selects multiplication, multiplies the constant log2e by the value of register 1 to obtain xlog2e, and stores the calculation result in register 0. When the operation is completed, the done signal is pulled high and jumps to the next state.

[0135] DIRECTINV_1: The calculation function selects inversion, inverts the value of register 0 to get -xlog2e, and stores the calculation result in register 2. When the operation is completed, the done signal is pulled high and jumps to the next state.

[0136] DIRECTEXP_1: The calculation function selects the exponential function and calculates the exponential of the value of register 2 to obtain e -x , the calculation result is stored in register 2. When the operation is completed, the done signal is pulled high and jumps to the next state.

[0137] DIRECTEXP_2: The calculation function selects the exponential function and calculates the exponential of the value of register 0 to obtain e x , the calculation result is stored in register 0. When the operation is completed, the done signal is pulled high and jumps to the next state.

[0138] DIRECTADD_1: The calculation function selects addition and multiplies the value of register 0 by the value of register 2 to obtain e x +e -x , the calculation result is stored in register 1. When the operation is completed, the done signal is pulled high and jumps to the next state.

[0139] DIRECTREC: Calculate the function to select the inverse, and take the inverse of the value of register 1 to get 1 / (e x +e -x) , the calculation result is stored in register 1. When the operation is completed, the done signal is pulled high and jumps to the next state.

[0140] DIRECTINV_2: The calculation function selects inversion, and the value of register 2 is inverted to get -e -x , the calculation result is stored in register 2. When the operation is completed, the done signal is pulled high and jumps to the next state.

[0141] DIRECTADD_2: The calculation function selects addition and multiplies the value of register 0 by the value of register 2 to obtain e x -e -x , the calculation result is stored in register 0. When the operation is completed, the done signal is pulled high and jumps to the next state.

[0142] DIRECTMUL_1: The calculation function selects multiplication and multiplies the value of register 0 by the value of register 1 to obtain (e x -e -x) / (e x +e -x) , the calculation result is stored in the target register. When the operation is completed, the done signal is pulled high and jumps to the DONE state.

[0143] DONE: Calculation completed state, the next clock cycle will jump back to the IDLE state.

[0144] The advantages of this embodiment are as follows:

[0145] The device uses segmented calculations, combined with interpolation calculations and Taylor polynomial approximation. When the input absolute value is less than 0.0625, the first two terms of Taylor expansion are used for approximate calculations; when the input absolute value is in the interval (0.0625, 9.0109], it is directly calculated through the definition formula, and the e exponential calculation and division in the definition formula are equivalently calculated using the e exponential and reciprocal calculated by fast interpolation calculations with a base of 2; when the input absolute value is greater than 9.0109, ±1 is directly output. Fast interpolation calculations store the coefficients of different sections of the exponential and reciprocal with a base of 2 in the lookup table. The input data is preprocessed to obtain intermediate results and lookup table indexes, and the polynomial coefficients are obtained through the lookup table, and C0+C1x+C2x is calculated. 2The final output result is obtained through post-processing, and the internal fixed-point calculation greatly reduces the transcendental function calculation time.

Claims

1. A high-precision hyperbolic tangent function calculation device, characterized in that: The computing device is used for a convolutional neural network, and the computing device comprises: A parameter acquisition module (100), the parameter acquisition module (100) being configured to acquire single-precision floating-point parameters from the convolutional neural network; A parameter determination module (200), wherein the parameter determination unit (200) is configured to determine the relationship between the absolute value of the single-precision floating-point parameter and a preset value, and generate a corresponding calculation instruction; A calculation execution module (300), wherein the calculation execution module (300) is configured to calculate a hyperbolic tangent function on the single-precision floating-point parameter according to the calculation instruction to obtain a target output, wherein the target output is used to characterize the nonlinear characteristics in the convolutional neural network; the calculation of the hyperbolic tangent function includes Taylor expansion calculation, direct output calculation and definition calculation, wherein the definition calculation includes calculating a transcendental function using a fast interpolation algorithm.

2. A high-precision hyperbolic tangent function calculation device according to claim 1, characterized in that: The parameter determination module (200) is further configured to: Determining whether the absolute value of the single-precision floating-point parameter is greater than a first preset value; If yes, generating a first calculation instruction, the first calculation instruction is used to instruct the calculation execution module (300) to perform direct output calculation according to the single-precision floating-point parameter; If not, determining whether the absolute value of the single-precision floating-point parameter is greater than a second preset value; If yes, generating a second calculation instruction, the second calculation instruction is used to instruct the calculation execution module (300) to perform a definition calculation according to the single-precision floating-point parameter; If not, a third calculation instruction is generated, wherein the third calculation instruction is used to instruct the calculation execution module (300) to perform Taylor expansion calculation according to the single-precision floating-point parameter.

3. A high-precision hyperbolic tangent function calculation device according to claim 2, characterized in that: The parameter determination module (200) is further configured to: When the absolute value of the single-precision floating-point parameter is greater than the first preset value, determining whether the single-precision floating-point parameter is a positive number; If yes, a first sub-calculation instruction is generated, wherein the first sub-calculation instruction is used to instruct the calculation execution module (300) to output 1 in the fp32 format according to the single-precision floating-point parameter; If not, a second sub-calculation instruction is output, wherein the second sub-calculation instruction is used to instruct the calculation execution module (300) to output -1 in the fp32 format according to the single-precision floating-point parameter.

4. A high-precision hyperbolic tangent function calculation device according to claim 3, characterized in that: The calculation execution module (300) comprises: A definitional calculation unit (310), the definitional calculation unit (310) being configured to perform a definitional floating-point calculation on the single-precision floating-point parameter according to the second calculation instruction; A Taylor calculation unit (320), wherein the Taylor calculation unit (320) is configured to perform Taylor expansion floating-point calculation on the single-precision floating-point parameter according to the third calculation instruction.

5. A high-precision hyperbolic tangent function calculation device according to claim 4, characterized in that: The definitional calculation unit (310) is further configured to: Preprocessing the single-precision floating-point parameter to obtain a preprocessing parameter; Obtaining a calculation function selection signal according to a second calculation instruction; Selecting a signal to perform polynomial calculation according to the preprocessing parameter and the calculation function to obtain a polynomial calculation result; A signal is selected for post-processing according to the polynomial calculation result and the calculation function to obtain the target output.

6. A high-precision hyperbolic tangent function calculation device according to claim 5, characterized in that: The Taylor calculation unit (320) is further configured to: Floating-point FMA processing, floating-point addition processing and floating-point multiplication processing are performed according to the single-precision floating-point parameter, so as to realize Taylor expansion floating-point calculation of the single-precision floating-point parameter.

7. A high-precision hyperbolic tangent function calculation device according to claim 6, characterized in that: The definitional calculation unit (310) comprises: A preprocessing component (311), wherein the preprocessing component (311) is configured to perform initialization processing corresponding to the transcendental function on the single-precision floating-point parameter to obtain the preprocessing parameter, wherein the preprocessing parameter includes a table lookup index corresponding to different transcendental functions, a polynomial calculation input, and an intermediate result; A table lookup index component (312), the table lookup index component (312) being configured to determine the type of the transcendental function according to the calculation function selection signal; perform table lookup processing according to the table lookup index and the type of the transcendental function to obtain a plurality of polynomial coefficients corresponding to the transcendental function; A parameter calculation component (313), wherein the parameter calculation component (313) is configured to perform polynomial calculation on the polynomial coefficients and the polynomial calculation input according to the type of the transcendental function to obtain a polynomial calculation result; A post-processing component (314) is configured to perform normalization processing on the polynomial calculation result and the intermediate result according to the type of the transcendental function to obtain the target output.

8. A high-precision hyperbolic tangent function calculation device according to claim 7, characterized in that: The single-precision floating-point parameter is divided into a plurality of operation parameters before entering the Taylor calculation unit (320), wherein the operation parameters at least include a first operand, a second operand and a third operand. The Taylor calculation unit (320) includes: An adding component (321), the adding component (321) being configured to perform an addition calculation on the operating parameters; a multiplication component (322), the multiplication component (322) being configured to perform a multiplication calculation on the operating parameter; A negation component (323), wherein the negation component (323) is configured to perform a negation calculation on the operation parameter; An FMA component (324), the FMA component (324) being configured to perform a fused multiply-add calculation on the operation parameter; The operation parameters are calculated by the addition component (321), the multiplication component (322) and the FMA component (324) to obtain a Taylor expansion.

9. A high-precision hyperbolic tangent function calculation device according to claim 8, characterized in that: The polynomial coefficients include at least a first coefficient, a second coefficient and a third coefficient, and the parameter calculation component (313) includes: A squarer (3131), the squarer (3131) being configured to square the polynomial calculation input to obtain a square result; a first multiplier (3132), wherein the first multiplier (3132) is configured to perform a multiplication calculation on the second coefficient and the polynomial calculation input to obtain a first multiplication result; a second multiplier (3133), wherein the second multiplier (3133) is configured to perform multiplication calculation on the square result and the third coefficient to obtain a second multiplication result; A three-input adder (3134), wherein the three-input adder (3134) is configured to perform an addition calculation on the first coefficient, the first multiplication result and the second multiplication result to obtain the target output.

10. A high-precision hyperbolic tangent function calculation device according to claim 9, characterized in that: The computing device further comprises: An output selection module (400), the output selection module (400) is configured to output the target output of the corresponding calculation function according to the calculation function selection signal.