An apparatus for approximating a softmax function

Abstract

The application discloses a device for approximating calculation of a softmax function. The device comprises a maximum value and last input maximum value unit, a subtraction operation unit, an approximate exp solution unit, a tree-shaped solution unit, a local and accumulation unit and an approximate ln solution unit. The maximum value and last input maximum value unit is used to obtain the maximum value of input data and store the final maximum value after comparison in an input data storage unit. The subtraction operation unit is used to perform subtraction operation on the data output by the maximum value and last input maximum value unit and the input data storage unit. The approximate exp solution unit is used to obtain the result of the exponential function of any input. The tree-shaped solution unit is used to perform tree-shaped accumulation summation on the input data. The local and accumulation unit is used to accumulate the local and of multiple inputs, and finally obtain the accumulated value of multiple inputs. The approximate ln solution unit is used to obtain the result of the logarithmic function of any input, and store the result in the input data storage unit. The device can reduce the power consumption, area and delay cost of the hardware architecture while maintaining a certain precision.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of very large scale integrated circuit design, and relates to a hardware implementation method for solving a softmax function. BACKGROUND

[0002] In recent years, deep learning has made breakthroughs in image recognition, natural language processing and other fields, and has become a tool widely used in various fields. Therefore, the design of inference acceleration for deep learning to speed up the calculation and reduce the calculation time has become one of the current trends. When designing a hardware-based deep neural network, the operation of the activation function becomes a major challenge. The activation function is a commonly used operator in deep neural networks, which mainly plays a role in nonlinear transmission. Due to this feature, it is generally necessary to implement complex operations in hardware, which makes it difficult to maintain a relatively ideal accuracy while performing high-speed calculations. The softmax function is a common activation function in deep neural networks, which can map the outputs of multiple neurons to the interval (0, 1), and can generally be used in multi-classification tasks. Therefore, it is urgent to develop a new hardware design scheme that can implement approximate softmax calculation under high accuracy requirements, and has low power consumption, small area and low delay.

[0003] In the use environment of deep neural networks, the softmax function calculation usually refers to a layer of preprocessing, which subtracts the maximum value of the input vectors of the function from these vectors, and then normalizes the exponential function of these vectors to obtain the result. This method helps the stability of training, but at the cost of introducing additional input vector traversal. If the calculation method of "online normalization" is used, that is, the maximum value is continuously calculated while the normalization value (i.e. the sum of the denominators) is calculated, then this can reduce the restriction on the input data, thereby reducing the requirements on the hardware design and improving the operation speed. SUMMARY

[0004] Therefore, the present application aims to provide an apparatus for approximately calculating the softmax function to achieve higher calculation accuracy and lower calculation delay.

[0005] In order to achieve the above-mentioned purpose, the technical scheme adopted by the method of the present application is as follows:

[0006] An apparatus for approximating a softmax function, comprising a maximum value and last input maximum value unit, a subtraction operation unit, an approximate exp solution unit, a tree-shaped summation unit, a local sum accumulation unit, an approximate ln solution unit, and an input data temporary storage unit; the maximum value and last input maximum value unit is configured to obtain the maximum value of input data and temporarily store the final maximum value after comparison in the input data temporary storage unit; the subtraction operation unit is configured to perform subtraction operation on the data output by the maximum value and last input maximum value unit and the input data temporary storage unit respectively, and output the operation result to the approximate exp solution unit; the approximate exp solution unit is configured to approximately solve the result of the exponential function of any input and output to the tree-shaped solution unit; the tree-shaped solution unit is configured to perform tree-shaped accumulation summation on the input data and output the summation result to the local sum accumulation unit; the local sum accumulation unit is configured to accumulate the local sum of multiple inputs, finally obtain the accumulated value of multiple inputs, and then output to the approximate ln solution unit; the approximate ln solution unit is configured to approximately solve the result of the logarithmic function of any input and temporarily store it in the input data temporary storage unit; and the input data temporary storage unit is configured to temporarily store data.

[0007] Further, the maximum value unit and the last input maximum value unit are configured in a tree structure, the maximum value of input data is solved in parallel every two times to obtain the local maximum value, and the maximum value of all input data is obtained by solving every two data in parallel; the current obtained maximum value is compared with the maximum value of the last input data, if the maximum value of the last input data is larger, the current obtained maximum value is replaced by the maximum value of the last input data for output.

[0008] Further, the approximate exp solution unit is configured to convert the exponential function with base e into the function with base 2, i.e. Then the exponential is divided into an integer part y int and a decimal part y frac , i.e. The integer part is divided into a shifted form, and the decimal part is approximately solved by a lookup table.

[0009] Further, the approximate ln solution unit is configured to first decompose the input data x into the form of 2 i multiplied by a number a between (1, 2] according to the formula x=2 i ·a, and then disassemble it to obtain ln 2·(i+log2 a); the value of ln 2·log2 a is approximately solved by a lookup table, and finally the approximate solution result y=ln2·i+g(y frac ) is obtained after accumulation.

[0010] The application provides a device for innovatively realizing approximate calculation of a softmax function, utilizes a calculation formula of exp-log-exp, simultaneously uses a local accumulation mode of multiple summations, and splits complex function data to realize approximate calculation of values of exponential functions and logarithmic functions, so that the calculation speed is faster under the condition of ensuring the accuracy of the approximate calculation result, and the universality of hardware circuit implementation is enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is a structural diagram of the device of the application;

[0012] Figure 2 is a structural diagram of an approximate exp unit;

[0013] Figure 3 is a structural diagram of an approximate ln unit. DETAILED DESCRIPTION

[0014] The technical solutions of the application will be further described in detail below in combination with the drawings and specific embodiments.

[0015] The following formula is used in the embodiment to approximate the softmax function:

[0016]

[0017] wherein x is an input vector, f(x i ) is an output result, and x max is the maximum value in a series of input vectors. Here, i can be an arbitrary value, but the device of the embodiment only supports input of 8 data, so multiple inputs are required for i greater than 8 to traverse all independent variables. The specific mode can be introduced through the following steps:

[0018] Step 1, decompose the input vector sequence x into a t x 8 sequence, and the specific meaning is that 8 elements are simultaneously input as modules at each of the 1st, 2nd, 3rd,..., tth time, and the maximum value m t of the current input sequence is calculated:

[0019] m t = max(x 8t , x 8t+1 ,..., x 8t+7 ), t = 0, 1, 2,...

[0020] Step 2, use c0, c1, c2,..., c tdenotes the maximum value of the 8 numbers found in the tth iteration of the input vector, and the maximum value of the t-1th iteration c t-1 the last iteration of the 0th iteration c -1 is compared with the maximum value c

[0021] c t = max(m t , c t-1 ), t = 0, 1, 2,...

[0022] output.

[0023] Step 3, the local and accumulation unit finds the result of the next iteration according to the result of the last iteration, and combines the input of the last iteration to find the result of the next iteration, which is called the local sum. The local sum is denoted by s0, s1, s2,..., s t denotes the local sum of the tth iteration of the local and accumulation unit (the last iteration of the 0th iteration s -1 = 0), at this time, the input sequence x 8t , x 8t+1 ,..., x 8t+7 at the current time is subtracted from the maximum value of the tth iteration, then the value of the exponential function with the natural constant e as the base of the result is found, and the exponential function with the natural constant e as the base of the difference between the tth iteration and the t-1th iteration is found, and finally, the local sum s t-1 at the previous time s t is updated to obtain the local sum s t at the current time:

[0024]

[0025] Step 4, after the tth iteration, all the inputs of the entire calculation are processed, and the final local sum s t is found, the value of the corresponding natural logarithm is found, and then the input sequence is finally calculated to obtain the result of the softmax function calculation: f(x i ) = exp(x i - x max - log(s t )).

[0026] When the input i is not a multiple of 8, the end of the input sequence can be padded with 0 to make the input variable a multiple of 8.

[0027] The device for approximating the softmax function of the embodiment has an input vector x of 16-bit fixed-point numbers, including 1-bit sign bit, 8-bit integer part, and 7-bit decimal part. The overall hardware architecture diagram is as Figure 1As shown, mainly includes the following order of work six modules: the maximum value and the last input maximum value unit, approximate solution exp unit, tree sum unit, local and cumulative unit, approximate solution ln unit, input data staging unit. Each module is described in detail as follows.

[0028] The maximum value and the last input maximum value unit, for finding the maximum value and its staging for the next data operations. Here the maximum value can be used in tree structure, that is, in each of the two values to find the maximum value, so that in a single register can be obtained in half the greater number of data, and then by analogy can be obtained the final maximum value, while there are comparison logic unit, the need to compare it with the last input maximum value, if the last input maximum value is greater, then the current maximum value output by the last input maximum value is replaced, so that the next subtraction calculator results will be 0 to avoid negative numbers in the subtraction operation.

[0029] Approximate solution exp unit for approximate solution of any input exponential function results. By converting the exponential function with base e into a function with base 2, that is, Then the index is divided into integer part y int And decimal part y frac That is The integer part is divided into the form of shift, and the decimal part is approximately solved by looking up the table.

[0030] Tree solution unit for tree-shaped accumulation of 8 input data, so as to quickly get the desired value.

[0031] Local and cumulative unit for cumulative input of local and local cumulative value. The unit according to the number of dependent variables of the softmax function, the cumulative sum of the input value, specifically, it first stores the previous output, if there is no previous output, it is set to 0, then it is multiplied by the difference between the maximum value in the path, so as to achieve the effect of scaling, followed by the result and the local sum output by the tree sum unit to accumulate, so as to obtain the output. The unit will be iterated in this way, and finally all the dependent variables of the softmax function will be included, and the output will be sent to the approximate solution ln unit for the next calculation.

[0032] Approximate solution ln unit is used to approximate the solution of any input logarithmic function results. First, the input is decomposed into 2 power 2 i Multiplying a number between (1, 2] a, that is, x = 2 i ·a, and then it is disassembled to get ln 2·(i+log2 a), which can be approximated by looking up the table g(yfrac ) = ln 2 · log2 a, and the final accumulation will give the approximate solution, i.e., y = ln 2 · i + g(y frac ).

[0033] The input data temporary storage unit is configured to temporarily store the input data.

[0034] The approximate exp solution unit in the embodiment is configured to receive the result of the subtraction as the input, as shown in the figure. Figure 2 The approximate exp solution unit in the embodiment is configured to receive the result of the subtraction as the input, as shown in the figure. After the input, the internal calculation is implemented by the logic shown in the figure. Specifically, first, a constant multiplication is performed, and then the first "1" is detected from the result to obtain the exponential part in the form of 2 power and the remaining mantissa part. Since the input of exp is negative, the influence of the sign bit is not considered. Then, the mantissa part is subjected to the lookup table operation to complete the approximate calculation process and obtain the approximate value of the calculation result. Then, the result is input into the controllable bit shift register, and the extracted exponential part of 2 is used to control the direction and size of the bit shift to finally obtain the calculation result of the approximate exp solution unit. In order to be consistent with the subsequent calculation, the bit width of the result is still 16 bits, including 1 bit of sign bit, 8 bits of integer part and 7 bits of decimal part.

[0035] The tree solution unit used in the embodiment is mainly configured to use the addition of the addition number to perform the tree-shaped accumulation summation of 8 inputs. In the first calculation, 8 numbers are added two by two to obtain 4 local sums. In the second calculation, 4 local sums are added two by two to obtain 2 local sums. Finally, the two numbers are added to obtain the calculation result of the summation of 8 numbers. In this way, through the tree structure, the operation delay can be greatly reduced.

[0036] The local sum accumulation unit used in the embodiment is mainly used for the case where the input argument of the softmax function is greater than 8. In this case, the weighted sum obtained by the tree summation unit is not the expected value, so in the case of multiple inputs, the device needs to re-weight and sum the input values. The maximum value finding and last input maximum value unit shown in the figure is used to adjust the weight, and the module is responsible for accumulating the values after the weight adjustment.

[0037] The approximate ln solution unit in the embodiment is configured to receive the result of the subtraction as the input, as shown in the figure. Figure 3As shown, the input is the result of the cumulative evaluation. After input into the approximate solution ln unit, the internal calculation is realized by the logic shown in the figure. Specifically, first, a first "1" detection is performed to obtain the exponential part in the form of 2 power and the remaining mantissa part. Then, the mantissa part is subjected to a lookup table operation to complete the approximate calculation process to obtain the approximate value of the calculation result. The integer part is subjected to a constant multiplication operation. Then, the results are accumulated to obtain the calculation result of the approximate solution ln unit. The bit width used here is still 16 bits, including 1 bit of sign bit, 8 bits of integer part and 7 bits of decimal part.

[0038] The input data temporary storage unit in the embodiment mainly uses FIFO to temporarily store data. After the data on which the subsequent calculation depends are all calculated, the input temporary storage data are output one by one and sent into the two subtracter units and the approximate solution exp unit in sequence, and the calculation results are finally output in groups of 8 to obtain the desired softmax calculation result.

[0039] The subtraction operation unit in the embodiment includes a subtracter arranged between the maximum value calculation unit and the last input maximum value unit and the approximate solution exp unit, and a subtracter arranged between the input data temporary storage unit and the approximate solution exp unit. Both of them use 16-bit fixed-point numbers, including 1 bit of sign bit, 8 bits of integer part and 7 bits of decimal part. The specific implementation of the subtraction operation module is as follows: the complement of the minuend is directly taken, and then addition calculation is performed. Specifically, if the sign bit is 0, that is, the minuend is positive, the complement result is obtained by inverting the minuend and adding 1, and then addition is performed. If the sign bit is 1, that is, the minuend is negative, the original result is obtained by restoring the positive number from the complement form of the negative number. The number after the subtraction operation still uses 16-bit fixed-point numbers, including 1 bit of sign bit, 8 bits of integer part and 7 bits of decimal part.

[0040] The above only describes the preferred embodiments of the present application, but the protection scope of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed by the present application can be easily thought of by those skilled in the art, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An apparatus for approximating the softmax function, characterized in that, It includes a unit for finding the maximum value and the previous input maximum value, a subtraction operation unit, an approximate exp solution unit, a tree-structured summation unit, a local and cumulative unit, an approximate ln solution unit, and an input data temporary storage unit; The unit for finding the maximum value and the unit for finding the previous maximum value is used to find the maximum value of the input data and temporarily store the final maximum value after comparison in the input data temporary storage unit. The subtraction unit is used to perform subtraction operations on the data output by the maximum value and previous input maximum value unit and the input data temporary storage unit respectively, and output the operation results to the approximate solution exp unit; The approximate solution exp unit is used to approximate the result of an exponential function with any input and output it to the tree-shaped solution unit; The tree-based solution unit is used to perform tree-based summation on the input data and output the summation result to the local and cumulative units. The local sum accumulation unit is used to accumulate the local sums of multiple inputs, and finally obtain the accumulated value of multiple inputs, and then output it to the approximate solution ln unit; The approximate solution unit is used to approximate the result of the logarithmic function of any input and temporarily store it in the input data temporary storage unit; the input data temporary storage unit is used to temporarily store data. The approximate solution for the exp unit is achieved by transforming the exponential function with base e into a function with base 2, i.e. Then the exponent is divided into integer parts y int and the decimal part y frac ,Right now The integer part is divided into a shifted form, while the fractional part is approximated by a lookup table; The approximate solution for the ln element is first based on the formula x = 2. i ·a, decompose the input data x into powers of 2. i Multiply by a number 'a' in the range (1,2], then decompose it to obtain ln2·(i+log2a); approximate the value of ln2·log2a using a lookup table, and finally sum them up to obtain the approximate result, i.e., y=ln2·i+g(y frac ).

2. The apparatus for approximating the softmax function according to claim 1, characterized in that, The maximum value unit and the previous input maximum value unit are constructed using a tree structure. The maximum value of each pair of input data is calculated in parallel to obtain a local maximum value. This process is repeated for each pair of data in parallel to obtain the maximum value among all input data. The current maximum value is compared with the maximum value of the previous input data. If the maximum value of the previous input data is larger, then the current maximum value is replaced by the maximum value of the previous input data and output.

Images