Hardware system and method for implementing softmax function calculation

Abstract

The present invention provides a hardware system and method for implementing softmax function calculation, relating to the field of hardware acceleration of neural networks. The present invention optimizes softmax function calculation by sparsifying only the output valid bit values ​​for exponential calculation and storage to reduce computational redundancy and storage requirements. Furthermore, the present invention employs a dynamic shifting method to update the maximum value, hiding the maximum value in the pipeline, thereby improving computational efficiency. Furthermore, the present invention addresses complex exponential and division operations, identifies a scheme for selecting the precision of exponential and division units for different softmax precision requirements, and improves the piecewise linear fitting algorithm. The system has strong applicability and can implement operations on general nonlinear functions such as exponential, division, and s-type functions. The system is simple to operate, requires a small footprint, and consumes low power, thus having great practical value.

Description

Technical Field

[0001] The present invention relates to the field of hardware acceleration of neural networks, and in particular to a hardware system and method for implementing softmax function calculation. Background Art

[0002] Softmax is a normalized exponential function that converts linear predictions into class probabilities. It is widely used in deep learning. Existing deep learning hardware architectures are mostly designed based on the characteristics of CNNs, LSTMs, and MLPSs, largely ignoring the softmax operation. This is because softmax is typically used as the final layer in these networks, representing only a small portion of computational time and resource usage. However, this is no longer applicable for networks such as transformers, which use softmax as a key component of the attention mechanism. For these networks, softmax can become a significant bottleneck.

[0003] Softmax involves complex exponential and division operations, which consume a large amount of computing resources and cycles, making hardware implementation difficult. Currently, many designs focus on optimizing methods such as lookup tables, segmented fitting, rotating coordinate systems, and neural network fitting to provide efficient implementation of exponential and division units. However, these designs have problems. First, different tasks require different computational accuracy, but there is a lack of solutions for selecting the accuracy of key units such as exponential and division units. Second, since the number of classification outputs can be thousands or millions, and sequence signals such as speech have strong local information correlation, in tasks such as speech recognition, a large number of weak score items will be input into the softmax, resulting in computational redundancy. A large amount of storage space is required to store the exponential results, resulting in reduced computational efficiency. Therefore, it is necessary to explore efficient softmax function calculation methods and hardware implementation solutions. Summary of the Invention

[0004] In view of the above problems, the present invention provides a hardware system and method for implementing softmax function calculation, thereby improving the computational efficiency of the softmax function and reducing hardware resource overhead and power consumption while ensuring the accuracy required by the neural network.

[0005] In one aspect, the present invention provides a method for implementing softmax function calculation, comprising:

[0006] Step S1, determining the structure of a hardware system according to the implementation accuracy of the exponential unit and the division unit, wherein the hardware system includes an input data processing unit, an exponential unit, a maximum value update unit, a shift accumulation unit, an intermediate result storage unit, a highest non-zero bit detection unit, and a division unit;

[0007] Step S2: quantize the input data x i The input data processing unit is multiplied by the constant log2e, and the calculated fractional part v is i With the integer part u i inputting the index unit and the maximum value updating unit respectively;

[0008] Step S3, the exponential unit converts the decimal part v i As an exponent with base 2, the exponent result is 2 vi , the maximum value updating unit updates the integer part u i Filter to obtain valid integer data and its index;

[0009] Step S4: the shift accumulation unit performs a shift on the exponential result 2 according to the valid integer data. vi Shift and get the natural exponential calculation result e xi ;

[0010] Step S5, the index result 2 is converted into vi Storing in the intermediate result storage unit;

[0011] Step S6: the shift accumulation unit converts the natural exponent calculation result e into xi Perform accumulation to obtain a first cumulative sum;

[0012] Step S7: The highest non-zero bit detection unit converts the first accumulated sum into an f×2 Q Form, where 1≤f<2;

[0013] Step S8, inputting the parameter f into the division unit, and calculating the reciprocal 1 / f of the parameter f;

[0014] Step S9: subtract the valid integer data from the parameter Q, and use the subtraction result to shift the reciprocal 1 / f to obtain a softmax function value.

[0015] Another aspect of the present invention provides a hardware system for implementing softmax function calculation, the hardware system comprising an input data processing unit, an exponent unit, a maximum value update unit, a shift accumulation unit, an intermediate result storage unit, a highest non-zero bit detection unit, and a division unit, wherein:

[0016] The input data processing unit is used to convert the quantized input data x i Multiply it with the constant log2e to get the fractional part v i With the integer part u i ;

[0017] The exponent unit is used to receive the fractional part v i , and the decimal part v i As an exponent with base 2, the exponent result is 2 vi The maximum value updating unit is used to receive the integer part u i , and for the integer part u i Filter to obtain valid integer data and its index;

[0018] The shift-accumulate unit is used to receive the valid integer data and the exponent result 2 vi , and the exponent result 2 according to the valid integer data vi Shift and get the natural exponential calculation result e xi The shift accumulation unit is further used to calculate the natural exponent result e according to the index of the valid integer data. xi Perform accumulation to obtain a first cumulative sum;

[0019] The intermediate result storage unit is used to store the exponential result 2 according to the index of the valid integer data. vi ;

[0020] The highest non-zero bit detection unit is used to receive the first accumulated sum and convert the first accumulated sum into f×2 containing parameters f and parameter Q. Q Form, where 1≤f<2;

[0021] The division unit is used to receive the parameter f and calculate the reciprocal 1 / f of the parameter f; the division unit is also used to subtract the valid integer data from the parameter Q, and use the subtraction result to shift the reciprocal 1 / f to obtain the softmax function value.

[0022] Compared with the prior art, the hardware system and method for implementing softmax function calculation provided by the present invention have at least the following beneficial effects:

[0023] (1) The present invention optimizes the calculation of the softmax function. By sparsifying the output, only valid bit values ​​are selected for exponential calculation and storage to reduce computational redundancy and storage requirements. The maximum value is updated dynamically by shifting to hide the maximum value in the pipeline, thereby improving computational efficiency.

[0024] (2) Aiming at the complex exponential and division operations, the present invention determines the scheme for selecting the exponential and division unit precision for different softmax precision requirements, improves the piecewise linear fitting algorithm, has strong applicability, and can realize the operation of general nonlinear functions such as exponential, division and s-type functions. It is simple to operate, has a small area, low power consumption, and has great practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0026] Figure 1 Schematically shows an architecture diagram of a hardware system for implementing softmax function calculation according to an embodiment of the present invention;

[0027] Figure 2 Schematically shows a flow chart of a method for implementing softmax function calculation according to an embodiment of the present invention;

[0028] Figure 3 A flowchart schematically illustrates a process for calculating a fractional part and an integer part according to an embodiment of the present invention;

[0029] Figure 4 Schematically shows a flow chart of an integer part screening process according to an embodiment of the present invention;

[0030] Figure 5 A diagram schematically illustrates a maximum optimization design for an integer part screening process according to an embodiment of the present invention;

[0031] Figure 6 Schematically shows a flow chart of a piecewise linear fitting algorithm according to an embodiment of the present invention;

[0032] Figure 7 A schematic diagram illustrating a geometric principle of a piecewise linear fitting algorithm according to an embodiment of the present invention is shown;

[0033] Figure 8 Schematically shows a circuit diagram of a maximum value updating unit according to an embodiment of the present invention;

[0034] Figure 9 Schematically shows a circuit diagram of a shift-accumulate unit according to an embodiment of the present invention;

[0035] Figure 10 Schematically shows a circuit diagram of an index unit according to an embodiment of the present invention;

[0036] Figure 11 The figure schematically shows a circuit diagram of a division unit according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] To make the objectives, technical solutions, and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to specific embodiments and the accompanying drawings. It is apparent that the embodiments described are only a portion of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0038] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the presence of the features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0039] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0040] When implementing the softmax function in hardware, in order to avoid the overflow of exponential operation, the input data x is normalized. i Subtract the global maximum value MAX of all input data to limit the input of the exponent to the domain of definition (-∞, 0). The calculation formula of the softmax function is as follows:

[0041]

[0042] Where N is the number of input data.

[0043] When implementing softmax in hardware, the exponential and division units are key. While hardware circuits are relatively user-friendly for addition, subtraction, shifting, and logical operations, exponential and division operations require additional circuit design, consuming significant resources.

[0044] In view of this, the present invention considers simplifying the exponential unit and division unit in softmax through operations such as addition and shift, so as to improve the computational efficiency of the softmax function and reduce hardware resource overhead and power consumption while ensuring the accuracy required by the neural network.

[0045] Figure 1 The following schematically shows an architecture diagram of a hardware system for implementing softmax function calculation according to an embodiment of the present invention. Figure 2 The flowchart of the method for implementing softmax function calculation according to an embodiment of the present invention is schematically shown.

[0046] Combine Figure 1 and Figure 2As shown, the method for implementing softmax function calculation according to this embodiment includes steps S1 to S9.

[0047] Step S1, according to the implementation accuracy of the exponential unit and the division unit, determine the structure of the hardware system, wherein the hardware system includes an input data processing unit, an exponential unit, a maximum value update unit, a shift accumulation unit, an intermediate result storage unit, a highest non-zero bit detection unit (LOD) and a division unit.

[0048] In an embodiment of the present invention, step S1 includes: simulating the precision selection scheme of the exponential unit and the division unit in the softmax function calculation process under different precision requirements through Python to determine the structure of the hardware system.

[0049] Specifically, by determining the input sequence length num, the sequence starting point start, the sequence ending point end, the exponential error b_mae, and the division error c_mae, the softmax calculation formula after the error is introduced can be obtained, and the result is expressed as Ans:

[0050]

[0051] Where x i Take the random decimal value of num between the sequence interval (start, end); b i Take a random decimal value between the exponential error (-b_mae, +b_mae); c i Take a random decimal value between the exponential error (-c_mae, +c_mae); part1 and part2 represent two intermediate parameters.

[0052] Introducing the influence of truncation error during hardware implementation, the simulation truncation error formula is as follows:

[0053]

[0054] Where qw_cut represents the truncation error function, IN is the input, qw is the truncation bit width, and floor represents rounding down. After introducing the truncation error, the softmax calculation formula is as follows, and the result is expressed as Ans_cut:

[0055] Ans_cut

[0056] =qw_cut[(qw_cut(Part1, len1)×qw_cut(Part2, len2)), len3]

[0057] Among them, len1 represents the truncated bit width of Part1, len2 represents the truncated bit width of Part2, and len3 represents the truncated bit width of the product of Part1 and Part2.

[0058] The absolute error absERROR, maximum absolute error MAE, mean absolute error AVE, and mean square error MSE between the original softmax result and the softmax result after truncation of the simulated error are expressed as follows:

[0059] absERROR=|Ans_cut-softmax|

[0060] MAE = max(absERROR)

[0061]

[0062]

[0063] Therefore, by determining different exponential errors and division errors, testing the softmax results of N random sequence pairs and taking the average of the errors, the exponential and division unit accuracy under the target softmax accuracy can be obtained.

[0064] Step S2: quantize the input data x i The input data processing unit is multiplied by the constant log2e (approximately 1.4375), and the calculated fractional part v is i With the integer part u i Enter the exponential unit and the maximum value update unit respectively.

[0065] Figure 3 The flowchart of the process of calculating the fractional part and the integer part according to an embodiment of the present invention is schematically shown.

[0066] like Figure 3 As shown, in the embodiment of the present invention, the decimal part v i With the integer part u i Calculate it according to the following steps S21 to S22.

[0067] Step S21: Input data x i Multiply by the constant log2e to get the product.

[0068] Step S22: input data x i The sign bit is used to judge, if the input data x i If it is a positive number, the product result is rounded according to the bit width to get the integer part u i , based on the product result and the integer part u i The difference between the two gets the fractional part v i Otherwise, round the product and add 1 to get the integer part u i , based on the product result and the integer part u i The difference is negated and then added 1 to get the fractional part vi .

[0069] For ease of understanding, the input data x i When it is a positive number, the decimal part v i With the integer part u i It can be expressed as:

[0070]

[0071] Where, Indicates rounding up.

[0072] The calculated fractional part v i Input the exponential unit and the integer part u i Enter the maximum value to update the cell.

[0073] Step S3, the exponent unit converts the decimal part v i As an exponent with base 2, the exponent result is 2 vi , the maximum value update unit updates the integer part u i Filter to obtain valid integer data and its index.

[0074] Figure 4 The flowchart of the integer part screening process according to an embodiment of the present invention is schematically shown. Figure 5 The maximum value optimization design diagram of the integer part screening process according to an embodiment of the present invention is schematically shown.

[0075] like Figure 4 and Figure 5 As shown, in the embodiment of the present invention, the maximum value updating unit updates the integer part u i Screening is performed to obtain valid integer data and its index, including the following steps S31 to S32.

[0076] Step S31, based on the integer part u i , calculate the global maximum value MAX and the local maximum value m according to the following formula i , the difference between the local maximum and the integer part ΔMAXu i , and the difference between the local maximum before and after Δmax i ;

[0077] MAX=max(u i ), i = 0, 1, ..., N-1

[0078] m i =max(u1, u2, ..., u i )

[0079] ΔMAXu i =m i -ui

[0080] Δmax i =m i -m i-1

[0081] Where N is the integer part u i The number of

[0082] Specifically, since the integer part u of the input to the maximum update unit i is a sequence number, i = 0, 1, ..., N-1, which has a constant global maximum value MAX. Figure 5 , local maximum m i The initial value m0 is the first input u0. Assuming that the second input u1 is less than u0, the local maximum value m i The second value remains at m0. When the third input u2 is greater than u1 and u0, the local maximum value m i The third value is updated to m1 (i.e. u2), and so on, the local maximum value m represented by the sequence number can be obtained. i At the same time, the difference ΔMAXu expressed as a sequence number can be calculated i and the difference Δmax i .

[0083] Step S32, at the local maximum m i With the integer part u i The difference ΔMAXu i If it is greater than the preset threshold, the integer part u i Filter out valid integer data and their indexes.

[0084] For example, taking 8-bit output as an example, that is, the preset threshold is 8, then the difference ΔMAXu i When it is greater than 8, output the filtered valid integer data u i and index.

[0085] Furthermore, in step S3, the maximum value updating unit updates the integer part u i Screening is performed to obtain valid integer data and its index, which may also include: i The difference between the before and after Δmax i Accumulate and get the second cumulative sum; at the local maximum m i With the integer part u i The difference ΔMAXu i When the second cumulative sum is greater than the first preset threshold and the second cumulative sum is greater than the second preset threshold, the integer part u i Filter out valid integer data and their indexes.

[0086] For example, in a specific task such as speech recognition, data distribution is characterized by locality. To reduce missed detection, the cumulative comparison unit can be added according to the specific situation to determine the local maximum value m. i The difference between the before and after Δmax i The accumulated sum is the second accumulated sum (i.e. m-m0). When m-m0 is greater than 8 (taking 8-bit output as an example) and the difference ΔMAXu i When it is greater than 8, the output is the filtered valid integer data u i and index.

[0087] In this step, the index unit is converted mathematically

[0088]

[0089] The natural exponential is converted to a hardware-friendly base-2 exponential, and an optimized piecewise linear fitting algorithm is employed to avoid the use of multipliers. Generally speaking, a piecewise linear fitting algorithm uses several linear segment functions kx+b to approximate the target function y, requiring only the slope k, intercept b, and segment endpoints of each linear segment to be stored. A greedy algorithm is employed, using the maximum absolute error (MAE) as a metric, to reduce the bit width of the slope k, convert multiplication operations into shifts and additions, and introduce dynamic error compensation to further correct coefficients, continuously optimizing the segmented solution.

[0090] Figure 6 The flowchart of the piecewise linear fitting algorithm according to an embodiment of the present invention is schematically shown. Figure 7 The geometric principle diagram of the piecewise linear fitting algorithm according to an embodiment of the present invention is schematically shown.

[0091] like Figure 6 and Figure 7 As shown, specifically, in step S3, the index unit uses a piecewise linear fitting algorithm to calculate the decimal part v i The corresponding index result is 2 vi , wherein the piecewise linear fitting algorithm includes the following steps S3A to S3D.

[0092] Step S3A, the decimal part v i Discretize into multiple discrete points x iw , to simulate the quantization error implemented by the hardware system.

[0093] This step is used to simulate the quantization error of the hardware system. iw Expressed as;

[0094]

[0095] Where iw is the decimal part v iThe number of digits; N, M are the starting point and the ending point of multiple discrete points respectively, the number of discrete points NUM=(NM) / (2 iw )+1.

[0096] Step S3B, using multiple initial linear segment functions f0(x)=kx+b to fit multiple discrete points, to obtain the slope k, intercept b and segment endpoints of the interval to which each initial linear segment belongs.

[0097] Step S3C, introduce the slope control factor kw and intercept control factor bw related to the bit width, and modify the slope k and intercept b of each initial linear segment according to the following formula to obtain multiple modified linear segment functions f1(x)=k kw ·x iw +b bw ;

[0098]

[0099] Where k kw is the corrected fitting slope; b bw is the corrected fitting intercept; round(·) is a function that rounds to the specified number of digits.

[0100] This step is used to further simulate the quantization and truncation errors in the hardware system implementation process. The slope control factor kw and intercept control factor bw related to the bit width can be introduced in the Python implementation process, and rounding simulation can be performed to finally obtain multiple corrected linear segment functions.

[0101] Step S3D, using multiple modified linear segment functions f1(x)=k kw ·x iw +b bw To approximate the objective function y, specify the search interval, calculate the error between each modified linear segment function and the objective function within the search interval, and when the error between a modified linear segment function and the objective function is less than the preset maximum absolute error MAE, retain the fitting slope k of the linear segment function kw , update the linear segment to exceed the upper limit slope and the lower limit slope k The intersection point and the fitting slope is k kw and continue to expand the search range, where the upper limit slope and the lower limit slope k Defined as;

[0102]

[0103] k =k kw -MAE

[0104] Otherwise, when the error between a corrected linear segment function and the objective function is greater than the maximum absolute error MAE, the fitting slope k of the linear segment function is kw Updated to Until the error between each calculated corrected linear segment function and the objective function is less than the maximum absolute error MAE.

[0105] This step takes into account the hardware system's implementation accuracy and uses the maximum absolute error (MAE) as the criterion for dividing each linear segment. A greedy algorithm is used to search the entire domain interval to ensure that each segment error does not exceed the predetermined maximum absolute error (MAE). The maximum absolute error (MAE) is expressed as:

[0106] MAE=max|f2(x)-y|

[0107] The idea of ​​piecewise linear fitting algorithm is as follows Figure 7 As shown, the double dashed arc line represents the actual function curve, that is, the corrected linear segment function f1(x), and the straight line k represents the fitting slope of the function f1(x) (that is, the fitting slope k kw ), first search the interval (s1, s2), when the error between the fitting function f1(x) and the actual function is small Error If the error is less than the maximum absolute error (MAE), the fitting slope does not need to be updated, and the fitting function exceeds the upper limit slope. and the lower limit slope k The intersection point and the fitting slope is k kw The search range can be expanded; on the contrary, when the error Error If it is greater than the maximum absolute error MAE, the slope needs to be updated to Until the maximum absolute error MAE requirement is met.

[0108] It can be seen that the goal of the piecewise linear fitting algorithm is to achieve a high-precision function with as few resources as possible, and to reduce the number of segments and bit width as much as possible while ensuring accuracy.

[0109] The introduction of the bit-width-dependent slope control factor kw and intercept control factor bw allows for comprehensive consideration in selecting the number of segments and data bit width. During hardware system implementation, the largest resource cost to implement the function f(x) = kx + b lies in the multiplication calculation. Therefore, by reducing the bit width of the fitted slope k and further converting complex multiplications into shifts and additions, resource overhead can be significantly reduced. However, this algorithm is slope-sensitive, and rounding errors and reducing the slope bit width can affect algorithm accuracy. Therefore, dynamic error compensation is introduced to further correct the coefficients.

[0110] Furthermore, step S3C further includes: introducing dynamic error compensation, and correcting the multiple corrected linear segment functions f1(x)=k kw ·x iw +b bw , and get the updated multiple linear segment functions:

[0111]

[0112] Where f2(x) is the updated linear segment function; MAE is the maximum absolute error.

[0113] Step S4: the shift accumulation unit adjusts the exponent result 2 according to the valid integer data. vi Shift and get the natural exponential calculation result e xi .

[0114] In the embodiment of the present invention, step S4 specifically includes: the shift accumulation unit shifts the difference ΔMAXu greater than the preset threshold i As the shift value, the exponent results in 2 vi Shift and get the natural exponential calculation result e xi The complete operation of natural exponential is shown below:

[0115]

[0116] Step S5, index the result 2 according to the index of the valid integer data vi Stored in the intermediate result storage unit FIFO.

[0117] Step S6: The shift accumulation unit converts the natural exponent calculation result e into xi Perform accumulation to obtain a first cumulative sum.

[0118] In the embodiment of the present invention, step S6 further includes: the shift accumulation unit calculates the local maximum value according to the difference Δmax before and after the local maximum value. i The first accumulated sum is shifted to update the first accumulated sum.

[0119] Since the local maximum m i Real-time changes, the difference is expressed as Δmax i , for example Δmax1=m1-m0, and, due to the difference ΔMAXu i And the natural index calculation result e xi It also changes in real time. To ensure the correctness of the final result, the first cumulative sum needs to be updated in real time. The shift difference Δmax during the accumulation process i , the effect of the maximum value change on the result can be corrected, thereby hiding the maximum value in the pipeline. The cumulative sum update in step S6 is shown as follows:

[0120]

[0121] Where SUM is the first cumulative sum.

[0122] Therefore, the shift-accumulation unit shifts the difference Δmax i By updating the first accumulated sum, the calculation of the maximum value can be hidden in the pipeline, thereby improving the calculation efficiency.

[0123] Step S7: The highest non-zero bit detection unit converts the first accumulated sum into an f×2 Q The form is, where 1≤f<2.

[0124] After the calculation of the exponential unit, due to the exponential result 2 vi The domain of is (0, 1), the range of is (1, 2), and the highest non-zero bit detection unit LOD is used to convert the denominator's cumulative sum SUM into f×2 Q The domain of parameter f is [1, 2).

[0125] Step S8: input the parameter f into the division unit and calculate the reciprocal 1 / f of the parameter f.

[0126] In the embodiment of the present invention, step S8 further includes: the division unit is preset with a low-precision division mode and a high-precision division mode, wherein:

[0127] In low-precision division mode, the reciprocal 1 / f is calculated using a piecewise linear fitting algorithm, and the reciprocal 1 / f is compared with the natural exponential calculation result e xi Multiply to get the low-precision division result;

[0128] In the high-precision division mode, the low-precision division calculation result is iteratively calculated using the Newton iteration method to obtain the high-precision division calculation result.

[0129] Specifically, after LOD, the division operation can be converted into a shift and multiplication operation. Two different precision division units can be selected according to the mode selector. The low-precision division mode selector provides two different precision division units. The mode selection signal select is 1 to select the low-precision division mode. The optimized piecewise linear fitting algorithm is used to calculate the inverse 1 / f, and then compared with f i (ie index result 2 vi ) are multiplied to get the decimal part, and the integer part can be realized by shifting. The calculation formula is as follows:

[0130]

[0131] Where Q i and Q are intermediate equivalent parameters, Qi =u i -MAX, Q is the integer that SUM needs to shift after LOD transformation.

[0132] The mode selection signal select is 0 to select the high-precision division mode. The Newton iteration method is introduced using a piecewise linear fitting algorithm with a low number of segments. Two additional multipliers are introduced to perform an iterative calculation to obtain a high-precision division result. The Newton iteration principle and the division calculation are shown in the following formula:

[0133]

[0134]

[0135] Where y i is the function of Newton iterative approximation, i represents the number of iterations; f i Calculates the value 2 for the exponential unit vi ; f is a number in the interval [1, 2) after LOD transformation.

[0136] Step S9: Subtract the valid integer data from the parameter Q, and use the subtraction result to shift the reciprocal 1 / f to obtain the softmax function value.

[0137] Through the method for implementing softmax function calculation provided by the above-mentioned disclosed embodiment, the present invention optimizes the calculation of the softmax function, selects only the output valid bit value for exponential calculation and storage through sparseness to reduce computational redundancy and storage requirements, and adopts a dynamic shift to update the maximum value to hide the maximum value in the pipeline, thereby improving computational efficiency. In addition, for the complex exponential and division operations, the present invention determines a scheme for selecting the precision of the exponential and division units with different softmax precision requirements, improves the piecewise linear fitting algorithm, has strong applicability, can realize the operation of general nonlinear functions such as exponential, division and s-type functions, is simple to operate, has a small area, low power consumption, and has great practical value.

[0138] The exponential unit and division unit each employ an optimized piecewise linear fitting algorithm, using several linear segment functions kx+b to approximate the target function y. Only the slope k, intercept b, and segment endpoints of each linear segment need to be stored. A greedy algorithm, using the maximum absolute error (MAE) as a metric, reduces the bit width of the slope k, converts multiplication operations into shifts and additions, and introduces dynamic error compensation to further correct coefficients, continuously optimizing the segmented solution.

[0139] Based on the method disclosed in the above embodiment, the present invention also provides a hardware system for realizing softmax function calculation. Figure 1 、 Figures 8-11The hardware system is described in detail.

[0140] Please continue reading Figure 1 The hardware system for implementing the softmax function calculation according to this embodiment includes an input data processing unit, an exponential unit, a maximum value updating unit, a shift accumulation unit, an intermediate result storage unit, a highest non-zero bit detection unit, and a division unit, wherein:

[0141] The input data processing unit is used to convert the quantized input data x i Multiply it with the constant log2e to get the fractional part v i With the integer part u i ;

[0142] The exponent unit is used to receive the decimal part v i , and the decimal part v i As an exponent with base 2, the exponent result is 2 vi , the maximum value update unit is used to receive the integer part u i , and for the integer part u i Filter to obtain valid integer data and its index;

[0143] The shift-accumulate unit is used to receive valid integer data and exponent result 2 vi , and the exponent result 2 is calculated based on the valid integer data vi Shift and get the natural exponential calculation result e xi The shift-accumulate unit is also used to convert the natural exponent calculation result e into the value of the index of the valid integer data. xi Perform accumulation to obtain a first cumulative sum;

[0144] The intermediate result storage unit is used to store the exponential result 2 according to the index of the valid integer data vi ;

[0145] The highest non-zero bit detection unit is used to receive the first accumulated sum and convert the first accumulated sum into f×2 containing the parameter f and the parameter Q Q Form, where 1≤f<2;

[0146] The division unit is used to receive the parameter f and calculate the reciprocal 1 / f of the parameter f; the division unit is also used to subtract the valid integer data from the parameter Q, and use the subtraction result to shift the reciprocal 1 / f to obtain the softmax function value.

[0147] In an embodiment of the present invention, the input data processing unit includes a multiplier and a selector, wherein:

[0148] The multiplier is used to convert the quantized input data x i Multiply by the constant log2e to get the product result;

[0149] The selector is used to select the input data x i The sign bit is used to judge, if the input data x i If it is a positive number, the product result is rounded according to the bit width to get the integer part u i , based on the product result and the integer part u i The difference between the two gets the fractional part v i Otherwise, round the product and add 1 to get the integer part u i , based on the product result and the integer part u i The difference is negated and then added 1 to get the fractional part v i .

[0150] Figure 8 The figure schematically shows a circuit diagram of a maximum value updating unit according to an embodiment of the present invention.

[0151] like Figure 8 As shown, in this embodiment of the present invention, the maximum value updating unit includes a comparator and a selector, wherein:

[0152] Integer part u i The maximum value update unit is input in the form of data stream, and the comparator and selector are used to determine whether the local maximum value m needs to be updated. i ; When the select value in the selector is 1, the comparator is used to find the local maximum value m i With the integer part u i The difference ΔMAXu i When it is greater than the preset threshold, the integer part u i The effective value of the signal is pulled up, from the integer part u i Filter out valid integer data and its index;

[0153] Otherwise, when the select value in the selector is 0, the comparator is used to find the local maximum value m. i With the integer part u i The difference AMAXu i is greater than the first preset threshold, and the local maximum value m i The difference between the before and after Δmax i When the second accumulated sum is greater than the second preset threshold, the integer part u i The effective value of the signal is pulled high, and the invalid redundant calculation is skipped, and the integer part u i Filter out valid integer data and their indexes.

[0154] Figure 9 The circuit diagram of the shift-accumulate unit according to an embodiment of the present invention is schematically shown.

[0155] like Figure 9 As shown, in the embodiment of the present invention, the shift accumulation unit includes an accumulator and a shifter, wherein:

[0156] The accumulator is used to calculate the natural exponential result e according to the index of the valid integer data. xi Perform accumulation to obtain a first cumulative sum;

[0157] The shifter is used to calculate the difference Δmax before and after the local maximum value. i The first accumulated sum is shifted to update the first accumulated sum.

[0158] The shift-accumulate unit calculates the natural exponential result e xi Accumulate and shift by Δmax through the shifter i By updating the result of the accumulator, the impact of the change in the maximum value on the result can be corrected, thereby hiding the maximum value calculation in the pipeline.

[0159] The highest non-zero detection unit LOD converts the accumulated sum SUM of the denominator into f×2 Q The domain of parameter f is [1, 2).

[0160] Figure 10 The figure schematically shows a circuit diagram of an index unit according to an embodiment of the present invention.

[0161] like Figure 10 As shown, in the embodiment of the present invention, the exponential unit includes a decoder, a selector, a shifter and an adder.

[0162] The exponential unit is simulated in Python to determine its implementation accuracy according to different softmax accuracy requirements. The optimized piecewise linear fitting algorithm is used to implement exponential operations with different accuracies, and the use of multipliers is avoided during the implementation process.

[0163] Furthermore, the exponential unit achieves an exponent with a maximum error of 1.95e-3, requiring only 16 segments, and the slope bit width is controlled to two significant bits. The piecewise linear fitting algorithm only needs to store the starting point, slope k, and intercept b of each linear segment. The input data x is judged by the decoder and selector to obtain the slope k and intercept b values ​​of each linear segment, and then added together. The calculation formula is as follows:

[0164]

[0165] Where B0~B15 represent the intercepts of different segments respectively; PWL_EXP represents the exponential function of piecewise linear fitting.

[0166] After the calculation of the exponential unit, due to the exponential result 2 viThe domain of is (0, 1), the range of is (1, 2), and the highest non-zero bit detection unit LOD is used to convert the denominator's cumulative sum SUM into f×2 Q The domain of parameter f is [1, 2).

[0167] in, Therefore SUM≥1. The LOD unit operation assumes that the fixed-point input is (0011.00110010) b , the decoder composed of multiplexers determines the highest non-zero bit and gives the corresponding shift number as (001.100110010)2. Similarly, if the input is (0000.00110010)2, the corresponding right shift of 3 bits becomes (0000001.10010)2.

[0168] Figure 11 The figure schematically shows a circuit diagram of a division unit according to an embodiment of the present invention.

[0169] like Figure 11 As shown, in this embodiment of the present invention, the division unit includes a mode selector and a multiplier, wherein:

[0170] When the signal select of the mode selector is 1, the low-precision division mode is selected. The division unit uses a piecewise linear fitting algorithm to calculate the reciprocal 1 / f, and uses a multiplier to multiply the reciprocal 1 / f with the natural exponential calculation result e xi Multiply to get the low-precision division result;

[0171] When the signal select of the mode selector takes a value of 0, the high-precision division mode is selected, and the multiplier is used to iteratively calculate the low-precision division calculation result using the Newton iteration method to obtain a high-precision division calculation result.

[0172] Specifically, the division unit determines the implementation accuracy of the division unit through Python simulation according to different softmax accuracy requirements, and can select two different precision division modes according to the mode selector. In the low-precision division mode, an optimized piecewise linear fitting algorithm is used to obtain the preliminary result of the piecewise linear fitting (i.e., the inverse 1 / f) and e through a multiplier. xi Multiply to obtain the low-precision division calculation result; in the high-precision division mode, based on the piecewise linear fitting algorithm in the low-precision division mode, two additional multipliers are introduced to perform an iterative calculation using the Newton iteration method to obtain the high-precision division calculation result.

[0173] Furthermore, using the piecewise linear fitting algorithm to implement the division 1 / f only requires 10 segments, and the slope bit width is controlled to two significant bits.

[0174]

[0175] Where B0~B9 represent the intercepts of different segments respectively; PWL_DIV represents the reciprocal function of the piecewise linear fitting.

[0176] By providing a hardware system for implementing softmax function calculations in the above-disclosed embodiments, the present invention improves the computational efficiency of the softmax function and reduces hardware resource overhead and power consumption while ensuring the accuracy required by the neural network.

[0177] It should be noted that the implementation method of the system part is similar to the implementation method of the method part, and the technical effects achieved are also similar. For the specific details of the system part, please refer to the above-mentioned method implementation method part, which will not be repeated here.

[0178] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some blocks or combinations thereof in the block diagrams and / or flow charts can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, or other processor programmable to implement the method for calculating the softmax function, so that when these instructions are executed by the processor, a device for implementing the functions / operations described in these block diagrams and / or flow charts can be created.

[0179] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly specifying the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined. Furthermore, the word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements.

[0180] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for realizing softmax function calculation, characterized in that: include: Step S1, determining the structure of a hardware system according to the implementation accuracy of the exponential unit and the division unit, wherein the hardware system includes an input data processing unit, an exponential unit, a maximum value update unit, a shift accumulation unit, an intermediate result storage unit, a highest non-zero bit detection unit, and a division unit; Step S2: quantize the input data x i The input data processing unit is multiplied by the constant log2e, and the calculated fractional part v is i With the integer part u i inputting the index unit and the maximum value updating unit respectively; Step S3, the exponential unit converts the decimal part v i As an exponent with base 2, the exponent result is 2 vi , the maximum value updating unit updates the integer part u i Filter to obtain valid integer data and its index; Step S4: the shift accumulation unit performs a shift on the exponential result 2 according to the valid integer data. vi Shift and get the natural exponential calculation result e xi ; Step S5, the index result 2 is converted into vi Storing in the intermediate result storage unit; Step S6: the shift accumulation unit converts the natural exponent calculation result e into xi Perform accumulation to obtain a first cumulative sum; Step S7: The highest non-zero bit detection unit converts the first accumulated sum into an f×2 Q Form, where 1≤f<2; Step S8, inputting the parameter f into the division unit, and calculating the reciprocal 1 / f of the parameter f; Step S9: subtract the valid integer data from the parameter Q, and use the subtraction result to shift the reciprocal 1 / f to obtain a softmax function value.

2. The method according to claim 1, characterized in that The step S1 comprises: The structure of the hardware system is determined by simulating the precision selection scheme of the exponential unit and the division unit in the softmax function calculation process under different precision requirements through Python.

3. The method according to claim 1, characterized in that In step S2, the fractional part v i With the integer part u i Calculated as follows: The input data x i Multiply by the constant log2e to get the product result; For input data x i The sign bit is used to judge, if the input data x i If it is a positive number, the product result is rounded according to the bit width to obtain the integer part u i , based on the product result and the integer part u i The difference between the two values ​​is used to obtain the fractional part v i ;otherwise , round the product result and add 1 to get the integer part u i , based on the product result and the integer part u i The difference is negated and then added with 1 to obtain the fractional part v i .

4. The method according to claim 1, wherein In step S3, the maximum value updating unit updates the integer part u i Filter to obtain valid integer data and their indexes, including: Based on the integer part u i , calculate the global maximum value MAX and the local maximum value m according to the following formula i , the difference ΔMAXu between the local maximum value and the integer part i , and the difference Δmax before and after the local maximum i ; MAX=max(u i ),i=0,1,…,N-1 m i =max(u1,u2,…,u i ) ΔMAXu i =m i -in i Δmax i =m i -m i-1 Wherein, N is the integer part u i the number of At the local maximum m i With the integer part u i The difference ΔMAXu i If the integer part u is greater than the preset threshold, i Filter out valid integer data and their indexes.

5. The method according to claim 4, characterized in that In step S3, the maximum value updating unit updates the integer part u i Filter to get valid integer data and its index, including: The local maximum m i The difference between the before and after Δmax i Perform accumulation to obtain a second cumulative sum; At the local maximum m i With the integer part u i The difference ΔMAXu i When the second accumulated sum is greater than the first preset threshold and the second accumulated sum is greater than the second preset threshold, the integer part u i Filter out valid integer data and their indexes.

6. The method according to claim 1, characterized in that In step S3, the index unit calculates the fractional part v using a piecewise linear fitting algorithm. i The corresponding index result is 2 vi , wherein the piecewise linear fitting algorithm includes: Step S3A, the decimal part v i Discretize into multiple discrete points x iw , to simulate the quantization error achieved by the hardware system, wherein the multiple discrete points x iw Expressed as; Where iw is the decimal part v i The number of digits; N, M are the starting point and the ending point of multiple discrete points respectively, the number of discrete points NUM=(NM) / (2 iw )+1; Step S3B, fitting the multiple discrete points using multiple initial linear segment functions f0(x)=kx+b to obtain the slope k, intercept b, and segment endpoints of the interval to which each initial linear segment belongs; Step S3C, introduce the slope control factor kw and intercept control factor bw related to the bit width, and modify the slope k and intercept b of each interval to which the initial linear segment belongs according to the following formula to obtain multiple modified linear segment functions f1(x)=k kw ·x iw +b bw ; Where k kw is the corrected fitting slope; b bw is the corrected fitting intercept; round(.) is a function that rounds to the specified number of digits; Step S3D, using multiple modified linear segment functions f1(x)=k kw ·x iw +b bw To approximate the objective function y, specify a search interval, calculate the error between each modified linear segment function and the objective function within the search interval, and retain the fitting slope k of the linear segment function when the error between a modified linear segment function and the objective function is less than the preset maximum absolute error MAE kw , update the linear segment to exceed the upper limit slope The intersection point with the lower limit slope k and the fitted slope is k kw straight line, and continue to expand the search interval, where the upper limit slope and the lower limit slope k Defined as; k=k kw -MAE Otherwise, when the error between a modified linear segment function and the objective function is greater than the maximum absolute error MAE, the fitting slope k of the linear segment function is kw Updated to Until the errors between each calculated corrected linear segment function and the objective function are less than the maximum absolute error MAE.

7. The method according to claim 6, characterized in that The step S3C further includes: Introduce dynamic error compensation and correct the multiple corrected linear segment functions f1(x)=k again kw ·x iw +b bw , and get the updated multiple linear segment functions: Where f2(x) is the updated linear segment function; MAE is the maximum absolute error.

8. The method according to claim 4, characterized in that The step S4 specifically includes: The shift accumulation unit will be greater than the difference ΔMAXu of the preset threshold i As the shift value, the index results in 2 vi Shift and get the natural exponential calculation result e xi .

9. The method according to claim 4, characterized in that The step S6 further includes: The shift accumulation unit is based on the difference Δmax before and after the local maximum value. i The first accumulated sum is shifted to update the first accumulated sum.

10. The method according to claim 6, characterized in that The step S8 further includes: The division unit is preset with a low-precision division mode and a high-precision division mode, wherein: In the low-precision division mode, the piecewise linear fitting algorithm is used to calculate the reciprocal 1 / f, and the reciprocal 1 / f is compared with the natural exponential calculation result e xi Multiply to get the low-precision division result; In the high-precision division mode, the low-precision division calculation result is iteratively calculated using the Newton iteration method to obtain a high-precision division calculation result.

11. A hardware system for implementing softmax function calculation, characterized in that: The hardware system includes an input data processing unit, an exponent unit, a maximum value updating unit, a shift accumulation unit, an intermediate result storage unit, a highest non-zero bit detection unit and a division unit, wherein: The input data processing unit is used to convert the quantized input data x i Multiply it with the constant log2e to get the fractional part v i With the integer part u i ; The exponent unit is used to receive the fractional part v i , and the decimal part v i As an exponent with base 2, the exponent result is 2 vi The maximum value updating unit is used to receive the integer part u i , and for the integer part u i Filter to obtain valid integer data and its index; The shift-accumulate unit is used to receive the valid integer data and the exponent result 2 vi , and the exponent result 2 according to the valid integer data vi Shift and get the natural exponential calculation result e xi The shift accumulation unit is further used to calculate the natural exponent result e according to the index of the valid integer data. xi Perform accumulation to obtain a first cumulative sum; The intermediate result storage unit is used to store the exponential result 2 according to the index of the valid integer data. vi ; The highest non-zero bit detection unit is used to receive the first accumulated sum and convert the first accumulated sum into f×2 containing parameters f and parameter Q. Q Form, where 1≤f<2; The division unit is used to receive the parameter f and calculate the reciprocal 1 / f of the parameter f; the division unit is also used to subtract the valid integer data from the parameter Q, and use the subtraction result to shift the reciprocal 1 / f to obtain the softmax function value.

12. The hardware system according to claim 11, characterized in that: The input data processing unit includes a multiplier and a selector, wherein: The multiplier is used to convert the quantized input data x i Multiply by the constant log2e to get the product result; The selector is used to select the input data x i The sign bit is used to judge, if the input data x i If it is a positive number, the product result is rounded according to the bit width to obtain the integer part u i , based on the product result and the integer part u i The difference between the two values ​​is used to obtain the fractional part v i Otherwise, round the product result and add 1 to obtain the integer part u i , based on the product result and the integer part u i The difference is negated and then added with 1 to obtain the fractional part v i .

13. The hardware system according to claim 11, characterized in that: The maximum value updating unit includes a comparator and a selector, wherein: The integer part u i The maximum value updating unit is input in the form of data stream, and the comparator and selector are used to determine whether the local maximum value m needs to be updated. i When the selector se1ect value is 1, the comparator is used to calculate the local maximum value m i With the integer part u i The difference ΔMAXu i When the integer part u is greater than the preset threshold, i The effective value of the signal is pulled high, from the integer part u i Filter out valid integer data and its index; Otherwise, when the selector value is 0, the comparator is used to determine the local maximum value m. i With the integer part u i The difference ΔMAXu i is greater than the first preset threshold, and the local maximum value m i The difference between the before and after Δmax i When the second accumulated sum is greater than the second preset threshold, the integer part u i The effective value of the signal is pulled high, from the integer part u i Filter out valid integer data and their indexes.

14. The hardware system according to claim 13, wherein: The shift-accumulate unit includes an accumulator and a shifter, wherein: The accumulator is used to add the natural exponent calculation result e according to the index of the valid integer data xi Perform accumulation to obtain a first cumulative sum; The shifter is used to determine the difference Δmax before and after the local maximum value. i The first accumulated sum is shifted to update the first accumulated sum.

15. The hardware system according to claim 11, characterized in that: The exponential unit includes a decoder, a selector, a shifter and an adder.

16. The hardware system according to claim 11, characterized in that: The division unit comprises a mode selector and a multiplier, wherein: When the signal select of the mode selector is 1, the low-precision division mode is selected, the division unit calculates the reciprocal 1 / f using a piecewise linear fitting algorithm, and uses the multiplier to multiply the reciprocal 1 / f with the natural exponential calculation result e xi Multiply to get the low-precision division result; When the signal select of the mode selector takes a value of 0, the high-precision division mode is selected, and the multiplier is used to iteratively calculate the low-precision division calculation result using Newton's iteration method to obtain a high-precision division calculation result.

Images