Hardware-friendly high-precision floating point transcendental function calculation system

Abstract

The invention provides a hardware-friendly high-precision floating point transcendental function calculation system, and the system comprises a parameter obtaining module which is configured to obtain a to-be-calculated single-precision floating point parameter and a calculation function selection signal; the transcendental function preprocessing module is configured to perform preprocessing on the single-precision floating point parameters respectively and output a target preprocessing result; the calculation module is configured to select a target preprocessing result and perform calculation according to the target preprocessing result to obtain a target polynomial calculation result; the leading zero detection module is configured to perform leading zero detection on the target polynomial calculation result to obtain a detection result; and the transcendental function post-processing module is configured to perform standardization processing on the detection result and output a target result corresponding to the transcendental function. According to the system, the problems that an existing transcendental function calculation scheme is low in efficiency and long in calculation period are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of transcendental function calculation, and in particular to a hardware-friendly high-precision floating-point transcendental function calculation system. Background Art

[0002] In order to meet the needs of transcendental function calculation brought about by the development of deep neural networks, more and more methods are applied to transcendental function implementation. The main methods are Taylor polynomial expansion and CORDIC (Coordinate Rotation Digital Computer) algorithm.

[0003] The Taylor polynomial expansion algorithm approximates the target function value by expanding the Taylor polynomial series. Due to the actual calculation time cycle and hardware resource limitations, this method can often only expand Taylor polynomials with a limited number of terms. When the number of expanded terms of the Taylor series is low, it is difficult to achieve the corresponding accuracy. A large number of expanded terms will lead to great hardware resource consumption. In addition, in order to obtain better approximation accuracy, Taylor polynomial expansion usually involves floating-point multiplication and addition, and the hardware implementation is complex. In addition, different hardware architectures must be used for different transcendental functions, which consumes a lot of hardware resources.

[0004] The CORDIC algorithm approximates the final rotation angle by rotating a series of small angles multiple times, and can effectively calculate trigonometric functions such as sine and cosine functions. The CORDIC algorithm uses fixed-point calculations and can be calculated using only shift registers and adders, but it requires multiple iterations to achieve high accuracy and has a long calculation cycle. Summary of the invention

[0005] The present application provides a hardware-friendly high-precision floating-point transcendental function calculation system to solve the problems of low efficiency and long calculation cycle of existing transcendental function calculation solutions.

[0006] The system is applied to a neural network, and the system comprises:

[0007] A parameter acquisition module, wherein the parameter acquisition module is configured to acquire a single-precision floating-point parameter to be calculated and a calculation function selection signal from the neural network;

[0008] A transcendental function preprocessing module, wherein the transcendental function preprocessing module is configured to perform preprocessing of different transcendental functions on the single-precision floating-point parameters respectively, and output corresponding target preprocessing results, wherein the target preprocessing results include table lookup indexes corresponding to different transcendental functions, polynomial calculation inputs, and intermediate results;

[0009] A calculation module, wherein the calculation module has a parameter lookup table corresponding to the table lookup index, and the calculation module is configured to select a target preprocessing result corresponding to the transcendental function according to the calculation function selection signal, and perform calculation according to the corresponding target preprocessing result to obtain a target polynomial calculation result;

[0010] A leading zero detection module, wherein the leading zero detection module is configured to perform a leading zero detection on the target polynomial calculation result to obtain a detection result; the leading zero detection includes detecting the first parameter position of the target polynomial calculation result having a value of 1;

[0011] A transcendental function post-processing module is configured to perform normalization processing on the detection result and select a target result of the transcendental function according to the calculation function selection signal output, wherein the target result is used to characterize the nonlinear characteristics in the neural network.

[0012] Preferably, the transcendental function preprocessing module includes:

[0013] A mantissa initialization unit, wherein the mantissa initialization unit is configured to perform initialization processing corresponding to the transcendental function on the single-precision floating-point parameter to obtain an initialization parameter;

[0014] An intermediate result unit, the intermediate result unit being configured to perform intermediate result calculation on the initialization parameter to obtain the intermediate result;

[0015] A table lookup index unit is configured to obtain the table lookup index and the polynomial calculation input according to the mantissa segment of the initialization parameter.

[0016] Preferably, the transcendental function preprocessing module further includes:

[0017] A quadrant determination unit, wherein the quadrant determination unit is configured to calculate a quadrant parameter of a sine function or a cosine function according to the initialization parameter when the transcendental function is a sine / cosine function.

[0018] Preferably, the calculation module includes:

[0019] a table lookup unit, the table lookup unit being configured to determine the type of the transcendental function according to the calculation function selection signal, and determine a plurality of polynomial coefficients according to the type of the transcendental function, the table lookup index, and the parameter lookup table corresponding to the transcendental function, the polynomial coefficients comprising at least a first coefficient, a second coefficient, and a third coefficient;

[0020] a first polynomial calculation unit, the first polynomial calculation unit being configured to perform a first operation on the polynomial calculation input and the polynomial coefficients to obtain a first multiplication result and a second multiplication result;

[0021] A second polynomial calculation unit is configured to perform a second operation on the first multiplication result, the second multiplication result and the first coefficient to obtain the target polynomial calculation result.

[0022] Preferably, the first polynomial calculation unit comprises:

[0023] A squarer, the squarer being configured to perform a square calculation on the polynomial calculation input to obtain a square result;

[0024] a first Wallace tree multiplier, wherein the first Wallace tree multiplier is configured to perform a multiplication calculation on the third coefficient and the square result to obtain a first multiplication result;

[0025] The second Wallace tree multiplier is configured to perform a multiplication calculation on the second coefficient and the polynomial calculation input to obtain a second multiplication result.

[0026] Preferably, the second polynomial calculation unit comprises:

[0027] a shifter, the shifter being configured to perform decimal point alignment on the second multiplication result and the first coefficient respectively according to the calculation function selection signal, so as to obtain a first alignment result and a second alignment result respectively;

[0028] A complement calculator, wherein the complement calculator is configured to calculate a function complement on the first multiplication result, the first alignment result and / or the second alignment result when there is a need to calculate a function complement on the first multiplication result, the first alignment result and / or the second alignment result;

[0029] A carry-save adder, wherein the carry-save adder is configured to perform operand reduction processing on the first multiplication result, the first alignment result and the second alignment result of completing the calculation function complement, respectively, to obtain a first retained result and a second retained result;

[0030] A carry-lookahead adder is configured to perform an addition calculation on the first retained result and the second retained result to obtain the target polynomial calculation result.

[0031] Preferably, the transcendental function post-processing module is further configured to output the target result corresponding to the transcendental function according to the numerical value of the target polynomial calculation result and the calculation function selection signal.

[0032] Preferably, the transcendental function includes a reciprocal function, a reciprocal square root function, a logarithmic function, a sine function, a cosine function and an exponential function.

[0033] Preferably, the parameter lookup tables of the sine function and the cosine function are the same.

[0034] Preferably, the system further comprises:

[0035] An input selector, the input selector being configured to obtain various data output by the transcendental function preprocessing module, and input various data corresponding to the transcendental function into the calculation module according to the calculation function selection signal;

[0036] An output selector, wherein the output selector is configured to obtain the target results corresponding to all the transcendental functions output by the transcendental function post-processing module, and output the target results corresponding to the transcendental functions according to the calculation function selection signal.

[0037] It can be seen from the above content that the present application provides a hardware-friendly high-precision floating-point transcendental function calculation system, the system comprising a parameter acquisition module, the parameter acquisition module being configured to obtain single-precision floating-point parameters to be calculated and a calculation function selection signal from the neural network; a transcendental function preprocessing module, the transcendental function preprocessing module being configured to perform preprocessing of different transcendental functions on the single-precision floating-point parameters respectively, and output corresponding target preprocessing results, the target preprocessing results comprising table lookup indexes corresponding to the different transcendental functions, polynomial calculation inputs and intermediate results; a calculation module, the calculation module having a parameter lookup table corresponding to the table lookup index, the calculation module being configured to calculate the target preprocessing results according to the target preprocessing results; and a calculation module having a parameter lookup table corresponding to the table lookup index. The calculation function selection signal selects the target preprocessing result corresponding to the transcendental function, performs calculation according to the corresponding target preprocessing result, and obtains the target polynomial calculation result; the leading zero detection module, the leading zero detection module is configured to perform leading zero detection on the target polynomial calculation result to obtain the detection result; the leading zero detection includes detecting the first parameter position with a value of 1 in the target polynomial calculation result; the transcendental function post-processing module, the transcendental function post-processing module is configured to normalize the detection result, and output the target result corresponding to the transcendental function according to the calculation function selection signal, and the target result is used to characterize the nonlinear characteristics in the neural network. The present application solves the problem of low efficiency and long calculation cycle of the existing transcendental function calculation scheme through the above system. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] Figure 1 A schematic diagram of a hardware-friendly high-precision floating-point transcendental function calculation system for this application;

[0040] Figure 2 A schematic diagram of a transcendental function preprocessing module in a hardware-friendly high-precision floating-point transcendental function calculation system of the present application;

[0041] Figure 3 A schematic diagram of a computing module in a hardware-friendly high-precision floating-point transcendental function computing system for the present application;

[0042] Figure 4 This is a schematic diagram illustrating the FP32 data format. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Figure 1 A schematic diagram of a hardware-friendly high-precision floating-point transcendental function calculation system for this application.

[0045] See also Figure 1 It can be seen that this embodiment provides a hardware-friendly high-precision floating-point transcendental function calculation system, the system is applied to a neural network, the neural network includes a convolutional neural network and a Transformer neural network, and the system includes:

[0046] The parameter acquisition module 100 is configured to obtain the single-precision floating-point parameters to be calculated and the calculation function selection signal from the neural network. Specifically, in the present embodiment, the parameter acquisition module 100 is used to obtain the single-precision floating-point parameters to be calculated and the calculation function selection signal corresponding to the transcendental function to be calculated.

[0047] It should be noted that the subsequent modules not only perform calculations according to the calculation function selection signal, but the calculation function selection signal is used to indicate the target result of the output corresponding to the transcendental function.

[0048] The system further comprises:

[0049] The transcendental function preprocessing module 200 is configured to perform preprocessing of different transcendental functions on the single-precision floating-point parameters, and output corresponding target preprocessing results, wherein the target preprocessing results include table lookup indexes, polynomial calculation inputs and intermediate results corresponding to the different transcendental functions. Specifically, in the present embodiment, the transcendental function preprocessing module 200 is used to perform preprocessing on the single-precision floating-point parameters, wherein due to the difference in transcendental functions, the preprocessing of the single-precision floating-point parameters corresponding to different transcendental functions is also different; the table lookup indexes, the polynomial calculation inputs and the intermediate results are extracted from the single-precision floating-point parameters through the preprocessing function of the transcendental function preprocessing module 200, and the table lookup indexes, the polynomial calculation inputs and the intermediate results are used for subsequent calculations.

[0050] Figure 2 A schematic diagram of a transcendental function preprocessing module in a hardware-friendly high-precision floating-point transcendental function calculation system of the present application.

[0051] See also Figure 2 It can be seen that, further, in some embodiments, the transcendental function preprocessing module 200 includes:

[0052] A mantissa initialization unit 210, wherein the mantissa initialization unit 210 is configured to perform initialization processing corresponding to the transcendental function on the single-precision floating-point parameter to obtain an initialization parameter;

[0053] An intermediate result unit 220, wherein the intermediate result unit 220 is configured to perform intermediate result calculation on the initialization parameter to obtain the intermediate result;

[0054] A table lookup index unit 230, wherein the table lookup index unit 230 is configured to obtain the table lookup index and the polynomial calculation input according to the mantissa segment of the initialization parameter;

[0055] The quadrant determining unit 240 is configured to calculate the quadrant parameters of the sine function or the cosine function according to the initialization parameters when the transcendental function is a sine / cosine function.

[0056] Specifically, in the present embodiment, the single-precision floating-point parameter is initialized corresponding to the transcendental function by the mantissa segment initialization unit 210, so as to convert the single-precision floating-point parameter into the corresponding specification; the table lookup index, the polynomial calculation input and the intermediate result are extracted by the intermediate result unit 220 and the table lookup index unit 230; it should be noted that, due to the difference of the transcendental function, specific processing is required in a specific transcendental function, based on this, when the transcendental function is a sine / cosine function, the quadrant parameter of the sine function or the cosine function is calculated by the quadrant determination unit 240 according to the initialization parameter.

[0057] Exemplary:

[0058] Figure 4 This is a schematic diagram illustrating the FP32 data format.

[0059] See also Figure 4 It can be seen that the lowest bit index of the FP32 data type is 0, the highest bit index is 31, and the total is 32 bits.

[0060] According to the IEEE754 standard, the formula for normalized binary floating point numbers is as follows:

[0061] V=(-1) S ×M×2 E ;

[0062] Where S is the sign bit, S is 0 for positive numbers, S is 1 for negative numbers. M is the mantissa, and for normalized numbers, the range is [1, 2). The left end of the binary decimal is always 1, and since it is not stored, it is also called the implicit leading bit. For denormalized numbers, the implicit leading 1 is not included, and the range is [0, 1). E is the exponent. At the same time, since the exponent needs to be positive or negative, a biased exponent is generated. For normalized numbers, the actual exponent value is Eb, where b is the offset value, which is 2 e-1 -1, where e is the length of the bits used to store the exponent. For denormalized numbers, the actual exponent is 1-b. For denormalized numbers, the calculation formula for FP32 normalized numbers is as follows:

[0063] V=(-1) S ×1.M×2 (E-127) ;

[0064] For FP32 denormalized numbers, we have:

[0065] V=(-1) S ×0.M×2 -126 ;

[0066] Therefore, the storage range of FP32 data format is 2-149 ~(2-2 -23 )×2 127 .

[0067] Preprocessing for different transcendental functions includes:

[0068] (1) Reciprocal preprocessing

[0069] Initialize the mantissa segment: If the 8 bits of the exponent E are not all 0, it indicates a normalized number. Just add 1 before the mantissa M, and the bit width will be expanded from 23 bits to 24 bits. If the exponent E bits are all 0, but the highest bit of M (index is 22) is 1, the mantissa segment is shifted left by 1 bit to get a 24-bit initialization result. If the exponent E bits are all 0 and the highest bit of M is 0, and the second highest bit (index is 21) is 1, the mantissa segment is shifted left by 2 bits and the lower 24 bits are taken as the mantissa initialization result. In other cases, since the maximum representation value of the FP32 data format is (2-2 -23 )×2 127 , the result after the inverse will exceed the maximum representation value of the FP32 data format, so it is only calculated in the post-processing module.

[0070] Calculate the intermediate result exp_mid of the exponent segment: if the exponent bits E are all 0 and the highest bit of M is 0 and the second highest bit is 1, exp_mid = 255-E; in other cases, exp_mid = 254-E.

[0071] Determination of table lookup index and x: The table lookup index is the 17th to 23rd bits of the initialization result of the mantissa segment, and the polynomial calculation input x is the lower 16 bits of the initialization result of the mantissa segment with 3 bits of 0 added at the end;

[0072] (2) Sine and cosine function preprocessing

[0073] Since the cosine function can be obtained by translating the sine function, the same polynomial coefficient table is used. To further improve the calculation accuracy, the input of the trigonometric function is the original value and the original value multiplied by The preprocessed object is the original value multiplied by fp32 input.

[0074] Mantissa initialization processing: If the exponent bits E are all 0, it means a denormalized number and no processing is done; otherwise, add 1 before the mantissa bit M to restore it to 1.M, and the decimal point is between the 23rd and 24th bits. The result after processing is recorded as frac_1. Calculate the shift value based on the exponent bit: shift_num = |E-127|. If E>127, frac_1 is shifted to the left by shift_num; otherwise, it is shifted to the right by shift_num to obtain the mantissa initialization processing result frac_2.

[0075] Quadrant parameter determination: If the sine function is being calculated, the 27th to 28th bits of frac_2 (index starts from 0, frac[27:26]) are intercepted as the quadrant parameter quad. If the cosine function is being calculated, it is necessary to add 1 to the quadrant parameter of the sine function and map it to the sine function for calculation, quad = frac[27:26] + 1. The last 26 bits of frac_2 are recorded as frac_3.

[0076] Lookup table index and determine x: Knowing frac_3 multiply by Get the original value to be divisible The remainder after .

[0077] When quad=0: Direct calculation

[0078] When quad=1:

[0079]

[0080] When quad=2:

[0081] When quad=3:

[0082]

[0083] Therefore, when the lowest bit of quad is 1, it means it is in the second and fourth quadrants of the sine function. We only need to find the complement of frac_3 to get frac_4 = frac_3 for calculation. 补 When the lowest bit of quad is 0, it means the sine function is in the first and third quadrants, frac_4 = frac_3. The table index is frac_4 bits 20 to 26 (frac_4[25:19]), and the polynomial calculation value x is the lower 19 bits of frac_4 (frac_4[18:0]).

[0084] Integer division signal: If frac_3 is equal to 0, it means the original value is An integer multiple of , divisible = 1;

[0085] Negative output signal: When the highest bit of quad is equal to 1, it means that it is located in the third or fourth quadrant of the sine function, and the output formula contains a negative sign, at this time neg = 1; conversely, when the highest bit of quad is equal to 0, it means that it is located in the first or second quadrant of the sine function, the output is a positive value, neg = 0.

[0086] (3) Logarithmic function preprocessing

[0087] When calculating the logarithmic function with base 2, the following formula can be used for transformation. Since 1.M is less than 2, the result is composed of the true exponential value as the integer part and log2(1.M) as the decimal part.

[0088]

[0089] Mantissa initialization: Since the denormalized number has no hidden integer 1, we need to first confirm the number of zeros before the highest bit 1 of the 23-bit mantissa M, recorded as zero_num. Since the decimal point is between the 23rd and 24th bits, M is shifted left by zero_num+1, so that the 24th bit integer is 1, and the format becomes 1.M', recorded as frac_1. Correspondingly, the actual exponent value E real =-126-(zero_num+1). For normalized numbers, we only need to add 1 to the 24th bit to get frac_1, E real =E-127.

[0090] Negative output signal and integer part result_int: Since the fp32 data format contains a sign bit to indicate positive and negative, for the 9-bit E real If the highest bit is 1, it means the output is the complement code, E real is a negative number, then neg=1. real Find the complement code to get E real The absolute value of -E real Since log2(1.M)∈[0,1), when E real When it is a negative number, there are:

[0091] E real +log2(1.M)=-(-E real -log2(1.M))=-(E real |-1+(1-log2(1.M)))

[0092] =-((E real -1)+[log2(1.M)] 补 )=-(result_int+result_frac);

[0093] Special case M=0, result_int=|E real |; In other cases, result_int=|E real |-1, result_int bit width is 8 bits. real If the highest bit is 0, it means E real Is a positive number, result_int=|E real |.

[0094] Determination of table lookup index and x: The table lookup index is the 17th to 23rd bits of frac_1 (frac_1[22:16]), and x is the lower 16 bits of frac_1 with 3 bits of 0 at the end.

[0095] (4) Exponential function preprocessing

[0096] Calculation 2 x When using the function, according to the fp32 data format, the output result E is the integer part of the input result.

[0097] Mantissa initialization: For normalized numbers, add the integer 1 at the 24th bit. For denormalized numbers, do not process them and record them as frac_1. Based on the exponent value, get the fixed-point representation of the true value between the 23rd and 24th bits of the decimal point. real =E-127, if E real If it is a positive number, shift frac_1 left by E. real bit; if E real If it is a negative number, shift frac_1 right by -E real bit, and the result is recorded as frac_2. Then the integer part int is the 24th to 31st bits of frac_2 (frac_2[30:23]), and the fractional part frac is the lower 23 bits of frac_2 (frac_2[22:0]).

[0098] Exponential intermediate result result_exp: If the input is a negative number, there are:

[0099] 2 -(int+frac) =2 -(int+1-1+frac) =2 -(int+1) 2 (1-frac) =2 -(int+1) 2 frac补 ;

[0100] Since the exponent of the fp32 data format is biased, result_exp = 127-(int+1). In particular, if frac is 0, result_exp = 127-int. The two's complement of frac is recorded as frac_3.

[0101] If the input is a positive number result_exp=127+int, frac is recorded as frac_3.

[0102] Determination of table lookup index and x: The table lookup index is the 17th to 23rd bits of frac_3 (frac_1[22:16]), and x is the lower 16 bits of frac_3 with 3 bits of 0 added to the end.

[0103] (5) Square root inverse preprocessing

[0104] Mantissa initialization: For the normalized number, add the integer 1 at the 24th bit, denoted as frac_1, and the actual exponent value E real =E-127; For denormalized numbers, first determine the number of zeros before the 1 in the highest bit of the 23-bit mantissa M, recorded as zero_num. Since the decimal point is between the 23rd and 24th bits, M is shifted left by zero_num+1, so that the 24th bit is 1, and becomes 1.M' format, recorded as frac_1, and the actual exponent value is E real =-126-(zero_num+1).

[0105] Exponential intermediate result result_exp: Since the inverse square root is calculated, the true exponential value is converted to an even number. If the exponent is an odd number, E even =(E real -1) / 2, the mantissa is doubled and is in the interval [2, 4), and the table is looked up using Table 1, table_index = 1; if the index is an even number, E even =E real / 2, the mantissa is in the interval [1, 2), use table 0 to look up the table, table_index = 0. result_exp = 127-E even .

[0106] Determination of table lookup index and x: The table lookup index is the table value concatenated with the 18th to 23rd bits of frac_3 ({table_index, frac_1[22:17]}), and x is the lower 17 bits of frac_3 with 2 bits of 0 added to the end.

[0107] The system further comprises:

[0108] The calculation module 300 has a parameter lookup table corresponding to the lookup table index. The calculation module 300 is configured to select a target preprocessing result corresponding to the transcendental function according to the calculation function selection signal, and perform calculation according to the corresponding target preprocessing result to obtain a target polynomial calculation result. Specifically, in this embodiment, the calculation of the transcendental function is implemented by the calculation module 300, wherein the specific calculation process and components are shown in Figure 3 .

[0109] Figure 3 A schematic diagram of a computing module in a hardware-friendly high-precision floating-point transcendental function computing system for the present application.

[0110] See also Figure 3 It can be seen that, further, in some embodiments, the calculation module 300 includes:

[0111] A table lookup unit 310, wherein the table lookup unit 310 is configured to determine the type of the transcendental function according to the calculation function selection signal, and determine a plurality of polynomial coefficients according to the type of the transcendental function, the table lookup index, and the parameter lookup table corresponding to the transcendental function, wherein the polynomial coefficients include at least a first coefficient, a second coefficient, and a third coefficient;

[0112] A first polynomial calculation unit 320, wherein the first polynomial calculation unit 320 is configured to perform a first operation on the polynomial calculation input and the polynomial coefficient to obtain a first multiplication result and a second multiplication result;

[0113] The second polynomial calculation unit 330 is configured to perform a second operation on the first multiplication result, the second multiplication result and the first coefficient to obtain the target polynomial calculation result.

[0114] Specifically, in this embodiment, the type of the transcendental function is determined according to the calculation function selection signal by the lookup table unit 310, and a number of polynomial coefficients are determined according to the lookup table index and the type of the transcendental function in combination with the parameter lookup table of the transcendental function; the corresponding calculation of all the polynomial coefficients is implemented by the first polynomial calculation unit 320 and the second polynomial calculation unit 330.

[0115] Among them, the first polynomial calculation unit 320 includes a squarer 321, which is configured to perform a square calculation on the polynomial calculation input to obtain a square result; a first Wallace tree multiplier 322, which is configured to multiply the third coefficient and the square result to obtain a first multiplication result; and a second Wallace tree multiplier 323, which is configured to multiply the second coefficient and the polynomial calculation input to obtain a second multiplication result.

[0116] The second polynomial calculation unit 330 includes a shifter 331, which is configured to align the decimal points of the second multiplication result and the first coefficient according to the calculation function selection signal, respectively, to obtain a first alignment result and a second alignment result; a complement calculator 332, which is configured to calculate the function complement of the first multiplication result, the first alignment result and / or the second alignment result when there is a need to calculate the function complement of the first multiplication result, the first alignment result and / or the second alignment result; a carry-save adder 333, which is configured to reduce the number of operands of the first multiplication result, the first alignment result and the second alignment result after completing the calculation of the function complement, respectively, to obtain a first retained result and a second retained result; a carry-lookahead adder 334, which is configured to add the first retained result and the second retained result to obtain the target polynomial calculation result.

[0117] Exemplarily, the lookup table takes out the polynomial coefficients C0, C1, and C2 according to the lookup table index and the number of types of calculated transcendental functions, and the bit widths are 25 bits, 16 bits, and 11 bits, respectively. The squarer 321 calculates the square of x, and the result is multiplied by C2 through the first Wallace tree multiplier 322. x is also multiplied by C1 through the first Wallace tree multiplier 322. The result is aligned with the decimal point through the shifter 331 according to the type of function calculated, and it is determined whether to calculate the function complement. The output result of the complement calculator 332 is sent to the carry-save adder 333 to reduce the number of operands, and then the carry-lookahead adder 334 completes C0+C1x+C2x. 2 Calculate and record the result as sum, with the decimal point between the 63rd and 62nd bits.

[0118] Transcendental function coefficient lookup table: Sin and cosine functions share the same lookup table with a depth of 128 and a data bit width of 52. The square root function is composed of two tables with data located at [1, 2) and [2, 4). The depth of each table is 64 and the data bit width is 52. The lookup tables of other functions are all 128 deep and 52 in width.

[0119] The system further comprises:

[0120] A leading zero detection module 400 is configured to perform a leading zero detection on the target polynomial calculation result to obtain a detection result; the leading zero detection includes detecting the position of the first parameter whose value is 1 in the target polynomial calculation result. Specifically, in this embodiment, the leading zero detection module 400 performs a leading zero detection on the target polynomial calculation result to determine the position of the parameter in the target polynomial calculation result.

[0121] The system further comprises:

[0122] The transcendental function post-processing module 500 is configured to perform normalization processing on the detection result, and select a target result corresponding to the transcendental function according to the calculation function selection signal output, and the target result is used to characterize the nonlinear characteristics in the neural network. Specifically, in this embodiment, the detection result is normalized by the transcendental function post-processing module 500, and a target result that meets the IEEE754 standard is obtained.

[0123] Exemplary:

[0124] For different transcendental functions, the post-processing methods are slightly different. The following are the specific post-processing solutions for different transcendental functions:

[0125] (1) Countdown post-processing

[0126] If the input is infinite or nan type, the output result is 0 or nan directly; if the input is an absolute value less than or equal to 2 -128 Denormalized numbers directly output positive / negative infinity; for other inputs, perform the following processing:

[0127] Confirm the number of 0s before the 1 in the highest bit of the 64-bit result sum, recorded as zero_num_sum. shift_num = zero_num_sum-1. Shift sum to the left by shift_num and record it as sum_norm_mid. If exp_mid is less than zero_num_sum, it proves that the output result is a denormalized number. Shift sum_norm_mid to the right by zero_num_sum-exp_mid, recorded as sum_norm, and the exponent bit is 0; otherwise, no shift is performed, and the exponent bit result is exp_mid-shift_num. Finally, the 40th to 62nd bits of sum_norm (sum_norm[61:39]) are intercepted as the mantissa segment output, and the lower 39 bits are rounded to obtain the final exponent and mantissa output. The sign is determined by the sign of the input value.

[0128] (2) Post-processing of sine and cosine functions

[0129] If the original value is infinite or nan, nan is output directly. If divisible=1, it means it is on the boundary. When the lowest bit of quad is 1, the absolute value of the output result is 1. When the lowest bit of quad is 0, 0 is output. Since sin(-x)=-sin(x), cos(-x)=cos(x), the sign of the final output result can be determined according to the input sign, the type of function calculated, and the neg obtained by preprocessing. If the sine function is calculated, the output sign is determined by the input sign s and neg, neg^s (XOR). If the cosine function is calculated, the output sign is determined by neg.

[0130] Confirm the number of zeros before the 1 in the highest bit of the 64-bit result sum, record it as zero_num_sum. shift_num = zero_num_sum-1. Shift sum left by shift_num, record it as sum_norm, and the exponent bit is 127-shift_num. Finally, intercept the 40th to 62nd bits of sum_norm (sum_norm[61:39]) as the mantissa segment output, and round the lower 39 bits to obtain the final exponent and mantissa output.

[0131] (3) Logarithmic function post-processing

[0132] If the input data is a negative number, directly output -nan; if the input is +0, output negative infinity; if the input is positive infinity, output positive infinity; if the input is +nan, output +nan. The rest are processed as follows:

[0133] The output sign has been calculated in the preprocessing module. First, keep the high 54 bits of sum, round according to the low 8 bits, and record the result as frac_round. If neg=1, the output is negative, and the decimal part should be converted into a complement code, so the complement code is calculated for frac_round. If the output is positive, no additional processing is required. After concatenating the integer part and the decimal part, the decimal point is between the 54th and 55th bits, recorded as result. Confirm the number of 0s before the 1 of the highest bit of the 62-bit result result, recorded as zero_num_sum. Shift result left by zero_num_sum to ensure that the highest bit is 1, recorded as o_norm. The exponent part of the output is: 134-zero_num_sum, and the mantissa segment intercepts the 39th to 61st bits of o_norm (o_norm[60:38]).

[0134] (4) Exponential function post-processing

[0135] If the input is positive infinity, the output is positive infinity; if the input is nan, the output is nan; if the input is a positive number, the exponent plus the bias is greater than 133, the output result exceeds the maximum value that the fp32 data format can represent, and the positive infinity is directly output; if the input is a negative number, the exponent plus the bias is greater than 134, the output result exceeds the absolute minimum value that the fp32 denormalized number can represent, and 0 is directly output; if the exponent plus the bias is less than 103, it represents a very small number, and 1 is directly output; in other cases, perform the following steps:

[0136] The sign bit is 0, which represents a positive number. Confirm the number of 0s before the 1 in the highest bit of the 64-bit result sum, recorded as zero_num_sum. shift_num = zero_num_sum-1. At this time, the decimal point is between the 62nd and 63rd bits. If the result_exp obtained by preprocessing is less than 1, it proves to be a non-normalized output. Shift sum right by 1-result_exp, recorded as sum_norm, and the output exponent bit is 0; in other cases, shift sum left by shift_num bits, recorded as sum_norm, and the output exponent bit is result_exp-shift_num. Finally, intercept the 40th to 62nd bits of sum_norm (sum_norm[61:39]) as the mantissa segment output, and round the lower 39 bits to obtain the final exponent and mantissa output.

[0137] (5) Post-processing of the inverse square root

[0138] When the input is ±0, the output is ±∞; when the input is nan, the output is nan, and in other cases, perform the following steps.

[0139] Confirm the number of zeros before the 1 in the highest bit of the 64-bit result sum, and record it as zero_num_sum. shift_num = zero_num_sum-1. Shift sum left by shift_num bits, record it as sum_norm, and output the exponent bits as result_exp-shift_num. Finally, intercept the 40th to 62nd bits of sum_norm (sum_norm[61:39]) as the mantissa segment output, and round the lower 39 bits to obtain the final exponent and mantissa output.

[0140] It is also possible to insert a pipeline at the point where the table lookup result and the squarer result are obtained to store all the intermediate results and input signals used subsequently. This cycle mainly performs preprocessing, table lookup and square result calculation. A second pipeline is inserted at the result of the shifter 331. This cycle is mainly used to calculate the multiplication C1x, C2x. 2After the leading zero detection module 400 is completed, the third stage of pipeline is inserted. This cycle is mainly used for calculating the complement code, summing and leading zero detection. The last cycle is used for post-processing and outputting the final result.

[0141] Furthermore, in some embodiments, the system further comprises:

[0142] An input selector 600, wherein the input selector 600 is configured to obtain various data output by the transcendental function preprocessing module 200, and input various data corresponding to the transcendental function into the calculation module 300 according to the calculation function selection signal;

[0143] The output selector 700 is configured to obtain the target results corresponding to all the transcendental functions output by the transcendental function post-processing module 500, and output the target results corresponding to the transcendental functions according to the calculation function selection signal.

[0144] Specifically, in this embodiment, the parameters to be input are selected through the input selector 600, and the parameters to be output are selected through the output selector 700.

[0145] The principles of this embodiment are as follows:

[0146] The workflow is as follows: 32-bit single-precision floating-point numbers are input into the module, and first sent to the transcendental function preprocessing module 200, which outputs a 7-bit table lookup index, a 19-bit polynomial calculation input x, and intermediate results required for post-processing of each function. The preprocessing result of the corresponding transcendental function is selected by the input selector 600 according to the calculation function selection signal. The preprocessing result is sent to the calculation module 300 for fixed-point calculation C0+C1x+C2x 2 Polynomial. Subsequently, the 64-bit polynomial calculation result will be sent to the leading zero detection module 400 for leading zero detection. The detection result and each intermediate calculation result will be sent to the transcendental function post-processing module 500 for normalization processing to obtain a result that meets the IEEE754 standard. Finally, the correct output result is obtained by the output selector 700 according to the calculation function selection signal.

[0147] This embodiment has the following advantages:

[0148] 1. High-precision calculation: The absolute error of the sine function in the range [-Π, Π] is no more than 2 -21.41 ; The absolute error of the cosine function in the range [-Π,Π] is no more than 2 -22; The maximum precision error of the exponential function with base 2 is 2ulp; the maximum precision error of the reciprocal function is 2ulp; the maximum error of the logarithmic function with base 2 in the interval [0.5,2] is 82ulp, which is equivalent to the maximum absolute error of no more than 2 -23 , the maximum precision error in the rest range is 1ulp; the maximum precision error of the inverse square root function is 2ulp.

[0149] 2. The system adopts the polynomial fitting method to expand different transcendental functions including sine function, cosine function, reciprocal, reciprocal square root, logarithm with base 2, and exponential with base 2 into quadratic polynomials, and stores the polynomial coefficients of different sections of the transcendental function obtained using the maximum and minimum approximation method in the lookup table.

[0150] 3. Fixed-point calculation is used inside the system, which greatly reduces the hardware complexity compared to floating-point calculation.

[0151] 4. The calculation does not require multiple iterations, and a three-stage pipeline design is adopted to further improve the efficiency of transcendental function calculation.

[0152] 5. High resource utilization: Multiple transcendental function calculations use the same polynomial calculation module and leading zero detection module.

Claims

1. A hardware-friendly high-precision floating-point transcendental function calculation system, characterized in that: The system is applied to a neural network, and the system comprises: A parameter acquisition module (100), the parameter acquisition module (100) being configured to acquire a single-precision floating-point parameter to be calculated and a calculation function selection signal from the neural network; A transcendental function preprocessing module (200), the transcendental function preprocessing module (200) being configured to perform preprocessing of different transcendental functions on the single-precision floating-point parameters respectively, and output corresponding target preprocessing results, the target preprocessing results comprising table lookup indexes corresponding to the different transcendental functions, polynomial calculation inputs, and intermediate results; A calculation module (300), the calculation module (300) having a parameter lookup table corresponding to the table lookup index, the calculation module (300) being configured to select a target preprocessing result corresponding to the transcendental function according to the calculation function selection signal, and to perform calculation according to the corresponding target preprocessing result to obtain a target polynomial calculation result; A leading zero detection module (400), the leading zero detection module (400) being configured to perform leading zero detection on the target polynomial calculation result to obtain a detection result; the leading zero detection comprises detecting the first parameter position of the target polynomial calculation result having a value of 1; A transcendental function post-processing module (500) is configured to perform normalization processing on the detection result and select a target result of the transcendental function according to the calculation function selection signal output, wherein the target result is used to characterize the nonlinear characteristics in the neural network.

2. A hardware-friendly high-precision floating-point transcendental function calculation system according to claim 1, characterized in that: The transcendental function preprocessing module (200) comprises: A mantissa initialization unit (210), wherein the mantissa initialization unit (210) is configured to perform initialization processing corresponding to the transcendental function on the single-precision floating-point parameter to obtain an initialization parameter; An intermediate result unit (220), the intermediate result unit (220) being configured to perform intermediate result calculation on the initialization parameter to obtain the intermediate result; A table lookup index unit (230), the table lookup index unit (230) being configured to obtain the table lookup index and the polynomial calculation input according to the mantissa segment of the initialization parameter.

3. A hardware-friendly high-precision floating-point transcendental function calculation system according to claim 2, characterized in that: The transcendental function preprocessing module (200) further comprises: A quadrant determination unit (240), wherein the quadrant determination unit (240) is configured to calculate a quadrant parameter of a sine function or a cosine function according to the initialization parameter when the transcendental function is a sine / cosine function.

4. A hardware-friendly high-precision floating-point transcendental function calculation system according to claim 3, characterized in that: The calculation module (300) comprises: A table lookup unit (310), the table lookup unit (310) being configured to determine the type of the transcendental function according to the calculation function selection signal, and to determine a plurality of polynomial coefficients according to the type of the transcendental function, the table lookup index and the parameter lookup table corresponding to the transcendental function, the polynomial coefficients comprising at least a first coefficient, a second coefficient and a third coefficient; A first polynomial calculation unit (320), the first polynomial calculation unit (320) being configured to perform a first operation on the polynomial calculation input and the polynomial coefficients to obtain a first multiplication result and a second multiplication result; A second polynomial calculation unit (330), wherein the second polynomial calculation unit (330) is configured to perform a second operation on the first multiplication result, the second multiplication result and the first coefficient to obtain the target polynomial calculation result.

5. A hardware-friendly high-precision floating-point transcendental function calculation system according to claim 4, characterized in that: The first polynomial calculation unit (320) comprises: A squarer (321), the squarer (321) being configured to perform a square calculation on the polynomial calculation input to obtain a square result; A first Wallace tree multiplier (322), wherein the first Wallace tree multiplier (322) is configured to perform a multiplication calculation on the third coefficient and the square result to obtain a first multiplication result; The second Wallace tree multiplier (323) is configured to perform a multiplication calculation on the second coefficient and the polynomial calculation input to obtain a second multiplication result.

6. A hardware-friendly high-precision floating-point transcendental function calculation system according to claim 5, characterized in that: The second polynomial calculation unit (330) comprises: A shifter (331), the shifter (331) being configured to perform decimal point alignment on the second multiplication result and the first coefficient respectively according to the calculation function selection signal, to obtain a first alignment result and a second alignment result respectively; A complement calculator (332), wherein the complement calculator (332) is configured to calculate a function complement on the first multiplication result, the first alignment result and / or the second alignment result when there is a need to calculate a function complement on the first multiplication result, the first alignment result and / or the second alignment result; A carry-save adder (333), wherein the carry-save adder (333) is configured to respectively perform operand reduction processing on the first multiplication result, the first alignment result and the second alignment result of completing the calculation function complement, so as to obtain a first retained result and a second retained result; A carry look-ahead adder (334) is configured to perform an addition calculation on the first retained result and the second retained result to obtain the target polynomial calculation result.

7. A hardware-friendly high-precision floating-point transcendental function calculation system according to claim 6, characterized in that: The transcendental function post-processing module (500) is further configured to output the target result corresponding to the transcendental function according to the numerical value of the target polynomial calculation result and the calculation function selection signal.

8. A hardware-friendly high-precision floating-point transcendental function calculation system according to claim 7, characterized in that: The transcendental functions include reciprocal functions, reciprocal square root functions, logarithmic functions, sine functions, cosine functions and exponential functions.

9. A hardware-friendly high-precision floating-point transcendental function calculation system according to claim 8, characterized in that: The parameter lookup tables of the sine function and the cosine function are the same.

10. A hardware-friendly high-precision floating-point transcendental function calculation system according to claim 9, characterized in that: The system further comprises: An input selector (600), the input selector (600) being configured to obtain various data output by the transcendental function preprocessing module (200), and input various data corresponding to the transcendental function into the calculation module (300) according to the calculation function selection signal; An output selector (700), wherein the output selector (700) is configured to obtain the target results corresponding to all the transcendental functions output by the transcendental function post-processing module (500), and output the target results corresponding to the transcendental functions according to the calculation function selection signal.

Images