Softmax function hardware accelerator based on approximate calculation

Abstract

The Softmax function hardware accelerator based on approximate calculation comprises a parallel comparison tree module, an approximate e index module, an accumulation summation module, an approximate division module, a bit width conversion ping-pong cache and a 16bit base-4 Booth approximate multiplier, a complete handshake protocol is arranged among the submodules, and the Softmax function hardware accelerator has a back pressure function. According to the method, the calculation formula of the Softmax function is approximately simplified, the multiplier used by the sub-module is approximately optimized, and the assembly line between the modules is optimized by using the ping-pong cache, so that the hardware resource overhead is reduced and the calculation delay is reduced under the condition of keeping relatively high accuracy, and the method can be applied to a neural network hardware accelerator.

Description

Technical Field

[0001] The invention relates to the technical field of very large scale integrated circuits, and in particular to a Softmax function hardware accelerator based on approximate calculation. Background Art

[0002] Nowadays, with the continuous optimization of neural network algorithms and the support of high-performance hardware, deep learning has been widely used in computer vision, speech recognition, natural language processing and other fields, becoming a hot research field in artificial intelligence.

[0003] As the parameter size of deep learning grows exponentially, traditional software processing has computing bottlenecks and latency issues, and how to use hardware to accelerate the computing process of these models has become a major challenge. The normalized exponential function (Softmax function) is a commonly used activation function in deep learning, usually used in multi-classification problems to output the probability distribution of each category.

[0004] The Softmax function converts the output of the neural network into a probability distribution, so that the output value is between 0 and 1, and the sum of the output probabilities of all categories is 1. However, the traditional Softmax function involves complex exponential operations, and precise calculations will consume a lot of hardware resources. Therefore, it is necessary to develop a new Softmax hardware accelerator to reduce the computing cost and delay while maintaining a high accuracy. Summary of the invention

[0005] The purpose of the present invention is to provide a Softmax function hardware accelerator based on approximate calculation to solve the problems in the background technology.

[0006] In order to solve the above technical problems, the present invention provides a Softmax function hardware accelerator based on approximate calculation, including a parallel comparison tree module, an approximate e Exponential module, cumulative summation module, approximate division module, bit width conversion ping-pong cache module and 16-bit base-4 Booth approximate multiplier, with complete handshake protocol between modules and back pressure function; The parallel comparison tree module inputs multiple input vector data for calculating Softmax in parallel, and is used to solve the maximum value of the input data and output the result to the approximate e Index module; The approximation e The exponential module solves the input data e The exponential approximation result is output to the bit width conversion ping-pong buffer module and the accumulation and summation module; The cumulative summation module cumulatively sums a plurality of input data and outputs the sum to the approximate division module; The approximate division module converts the bit width into the ping-pong buffer module. e The exponential approximation result is approximated by the output of the cumulative summation module to obtain the approximate calculation result of the Softmax function; The bit width conversion ping-pong buffer module is used to input data and approximate e Index modules, approximation e Pipeline optimization between the exponential module and the division module to support inconsistent input and output parallelism; The 16-bit radix-4 Booth approximation multiplier is used to approximate e Multiplication approximation in the exponential module and the approximate division module.

[0007] In one embodiment, the parallel comparison tree module supports multiple parallel data inputs, including an eight-input comparator, a four-input comparator, a two-input comparator and a maximum value register, and supports a maximum of 32 parallel inputs. The four local maxima obtained by the first-level comparison are compared with the final result obtained by the second-level comparison and stored in the maximum value register; the result obtained in each round of comparison is compared with the value in the maximum value register. If it is greater than the value in the maximum value register, the original value is overwritten; if it is less than, it remains unchanged, and finally the maximum values ​​of a batch of input vectors are obtained.

[0008] In one embodiment, the approximation e The index module will e Exponentials are converted to base 2 functions using the base conversion formula: , and divide the exponential part into integer and decimal. The integer is realized by the shifter, and the decimal is approximated as polynomial multiplication and addition through the Taylor expansion formula: The approximation e All multiplications within the exponential module use 16-bit radix-4 Booth approximation multipliers.

[0009] In one implementation, the approximate division module uses the Goldschmidt approximation algorithm: Approximate division as multiplication: All multiplications within the approximate division module employ 16-bit radix-4 Booth approximate multipliers.

[0010] In one embodiment, the 16-bit radix-4 Booth approximate multiplier uses an approximate 4-2 compressor, an approximate full adder and a segmented approximate compression tree. The approximate 4-2 compressor and the approximate full adder use a Karnaugh map numerical replacement method to simplify the calculation process; the segmented approximate compression tree will perform approximate compression on the low bits and retain accurate compression on the high bits.

[0011] In one implementation, the bit width conversion ping-pong cache module supports input and output data with different degrees of parallelism, and is composed of two RAMs and a read-write controller. The read-write controller changes the degree of parallelism of input and output according to configuration parameters.

[0012] In one embodiment, the complete handshake protocol is a Ready / Valid handshake protocol; the Ready signal indicates whether the receiving module is ready to receive data. When the receiving module is ready to receive data, the Ready signal is set to a high level; the Valid signal indicates whether the sending module has valid data to send. When the sending module has valid data, the Valid signal is set to a high level.

[0013] In one implementation, the back pressure function is that the module receiving data transmits feedback information to the module sending data by controlling the Ready signal, pausing or continuing to send data, so as to avoid data loss.

[0014] The present invention provides a Softmax function hardware accelerator based on approximate calculation, which calculates the approximate result of the Softmax function through a parallel comparison tree module, an approximate e-index module, an accumulative summation module and an approximate division module. Each submodule has a complete handshake protocol and a back pressure function to avoid data loss. The present invention uses a 16-bit base-4 Booth approximate multiplier to optimize the multiplication calculation within the module, and uses a bit width conversion ping-pong cache to optimize the pipeline between modules, ultimately maintaining a high accuracy rate while reducing hardware resource consumption and reducing calculation delays. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a schematic diagram of the Softmax hardware accelerator architecture provided by the present invention.

[0016] Figure 2 This is a schematic diagram of the parallel comparison tree structure provided by the present invention.

[0017] Figure 3 Schematic diagram of the approximate e-exponential pipeline algorithm provided by the present invention.

[0018] Figure 4 A schematic diagram of the approximate division pipeline algorithm provided by the present invention.

[0019] Figure 5A schematic diagram of the coding structure of an unsigned 16-bit radix-4 Booth multiplier provided by the present invention.

[0020] Figure 6 This is a schematic diagram of the approximate compressed tree structure provided by the present invention. DETAILED DESCRIPTION

[0021] The following is a further detailed description of a Softmax function hardware accelerator based on approximate calculation proposed by the present invention in conjunction with the accompanying drawings and specific embodiments. According to the following description, the advantages and features of the present invention will become clearer. It should be noted that the accompanying drawings are all in a very simplified form and are not in precise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0022] The Softmax function is also called the normalized exponential function, and its calculation formula is: in xi is the first i values, a total of n indivual xi The present invention provides a Softmax function hardware accelerator based on approximate calculation. Figure 1 As shown, by comparing tree modules in parallel, approximation e The exponential module, the cumulative summation module and the approximate division module calculate the approximate result of the Softmax function. xi Including positive and negative numbers, by finding the maximum value and subtracting the two, you can ensure that e The input signs before the exponent are unified.

[0023] The structure of the parallel comparison tree module is as follows Figure 2 As shown, it supports multiple parallel data inputs. The parallel comparison tree module includes an eight-input comparator, a four-input comparator, a two-input comparator and a maximum value register. It supports up to 32 parallel inputs. The four local maximum values ​​obtained by the first-level comparison are compared in the second level and the final result is stored in the maximum value register. The result obtained in each round of comparison is compared with the value in the maximum value register. If it is greater than the value in the maximum value register, the original value is overwritten. If it is less than, it remains unchanged. Finally, the maximum value of a batch of input vectors can be obtained.

[0024] approximate e The exponential module is used to calculate the Softmax formula. e The exponential function operation is approximated and simplified. e Exponential conversion is a shift and addition operation. First, the original formula needs to be changed to the base: in is a constant, which can be further simplified to 1 / ln 2. Therefore Can be simplified to . Treat the exponent as an integer m Add decimals n , the original formula can be transformed into .in, This can be achieved in hardware by shifting. n Between -1 and 0, so It can be approximated as a polynomial sum by a second-order Taylor expansion at -1 / 2: Therefore, designing corresponding multipliers and adders can achieve To facilitate calculation and further simplify, each input vector xi Subtract the maximum value xmax .because xi - xmax must be a negative number, which not only saves the subsequent additional sign bit operation, but also e The output of the exponential function is controlled between 0 and 1 to avoid overflow caused by too large a calculation result. e The exponential part is designed with hardware. In order to increase parallelism and reduce latency, while finding the maximum value, the ping-pong cache method is used to continuously input the input data into the bit width conversion ping-pong cache module. After the maximum value is obtained, the previously input data can be read immediately to perform the next subtraction operation. In order to further optimize the hardware design, all operations involving subtraction are converted into addition. When implementing the Taylor expansion approximation polynomial, a pipeline design is used to calculate addition and multiplication in parallel to reduce latency. The internal algorithm flow of this module is as follows: Figure 3 shown.

[0025] Approximate division operation uses the Goldschmidt approximation algorithm to approximate division to reduce the resource overhead and computational delay caused by the divider. The algorithm is based on the following formula: in, is the dividend, is a divisor less than 1, 1- ,for s The maximum exponent of , the denominator can be simplified to ,and s Less than 1, so the denominator tends to 1, and the quotient can be found by solving the numerator. n The larger the value, the more accurate the result. Considering the resource consumption and calculation accuracy of hardware design, n Set to 4. Then we get the following formula: The numerator and denominator are divided by the leading 1 detection module and the shift It can save resource consumption and improve computing efficiency. The leading 1 detection module is used to detect the position of the first 1 in the input data, and then calculate how many bits the numerator and denominator need to be shifted right. Based on the above derivation, the division module is designed in a pipelined manner. The algorithm flow is as follows: Figure 4 shown.

[0026] The Booth algorithm can reduce the number of partial products by removing zeros from the encoded multiplier, thereby reducing latency. The original Booth algorithm is also called the radix-2 Booth algorithm, and its degree of optimization for latency depends on the distribution of 1s in the multiplier. The radix-4 Booth multiplier is an improvement on the radix-2 Booth multiplier, and its principle is as follows: set up N Bit signed number A and B Multiply to get 2 N The product of bits P , the result can be simplified to is the sum of the multiplication and accumulation of the base coefficients. The base coefficients have five possibilities: -2, -1, 0, 1, and 2, so the partial product is -2 A ,- A ,0, A , 2 A Five results. The partial product can be obtained by encoding, and the specific corresponding relationship is as follows.

[0027]

[0028] Through base-4 Booth encoding, the number of partial product rows is changed from N The line is reduced to N / 2+1 line, effectively reducing latency and resource overhead. Partial products need to be shifted and added to get the final result, usually using 3-2 compressors, 4-2 compressors, compression trees and other structures for fast accumulation. e The exponential module and the approximate division module contain a large number of multiplication operations, so an approximate base-4 Booth multiplier is used to further reduce resource consumption. The approximate base-4 Booth multiplier uses precise encoding, and its expression is shown in the following formula: in, Indicates x OK y The partial product of the columns, { }From the multiplicand A , { }From the multiplier B Based on the logical expression of partial products, an unsigned 16-bit exact radix-4 Booth multiplier generates the partial product structure as Figure 5 As shown, each circle represents a digit. s Represents the sign extension bit, that is, the sign bit of each row of partial products. Right now s Negate. Multiplier B It is necessary to add 2 bits in the high position to ensure a positive partial sum, so the partial product of the last row is always positive and the bit width is 16 bits. For partial accumulation and addition, a compression tree consisting of a 4-2 compressor, a full adder and a half adder can be used to quickly complete the partial accumulation and addition. After 4 rounds of compression, the exact result is obtained by adding. By analyzing the internal data of Softmax, the partial product array generated by the multiplier is segmented and approximated: approximate calculations are used for bits that have a relatively small impact on the result, and exact calculations are used for bits that have a greater impact on the result. For example, if the 15th to 30th bits of the result are needed, bits 0 to 11 are approximately compressed during accumulation, bits 12 to 30 are accurately compressed, and bit 31 is discarded. When approximating the compression of partial products, approximate 4-2 compressors and approximate full adders are used.

[0029] The approximate 4-2 compressor uses the Karnaugh map numerical replacement method to achieve approximate design and discards the original input carry and . Let the input be A 1, A 2, A 3. A 4. Its output Sum The corresponding Karnaugh maps are shown below: “0→1” means changing “0” to “1”, and “1→0” means changing “1” to “0”.

[0030] 3412 00 01 11 10 00 0 1→0 0 1→0 01 1→0 0 1→0 0 11 0→1 1 1 1 10 1 0 1 0 10 0 1 1 1 Get an approximate 4-2 compressor Sum The logical expression is: In a similar 4-2 compressor Carry The corresponding Karnaugh map is shown below: 3412 00 01 11 10 00 0 0→1 1 0→1 01 0 1 1 1 11 1→0 1 1 1 10 0 1 1 1 The logical expression of Carry is: Assuming the input of the approximate full adder is B1, B2, B3, the expressions of the output FSum and FCarry are as follows: Based on the approximate full adder and the approximate 4-2 compressor, the compression tree is approximated. Taking the unsigned 16-bit radix-4booth multiplier as an example, the approximate compression process is as follows: Figure 6 As shown, there are three levels of approximate compression trees.

[0031] Ping-pong cache is a cache mechanism for improving data processing efficiency. Its basic principle is to use a double buffer to alternately perform data read and write operations to achieve continuous data processing and reduce waiting time. The bit width conversion ping-pong cache in this embodiment includes a double buffer and a read-write controller. During the data processing process, the system will alternately use the double buffer. When Buffer A is output, the read-write controller will switch to Buffer B for output, and Buffer A can be used to receive new input data. Considering that the parallelism of input and output may be different, the read-write controller has added a parallelism configurable register. When the configured parallelism is less than the maximum parallelism that can be supported, the controller splices the addresses through an internal counter. This embodiment supports a minimum parallelism of 8 and a maximum parallelism of 32.

[0032] The above description is only a description of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention. Any changes or modifications made by a person skilled in the art in the field of the present invention based on the above disclosure shall fall within the scope of protection of the claims.

Claims

1. A Softmax function hardware accelerator based on approximate calculation, characterized in that: Includes parallel comparison tree module, approximation e Exponential module, cumulative summation module, approximate division module, bit width conversion ping-pong cache module and 16-bit base-4 Booth approximate multiplier, with complete handshake protocol between modules and back pressure function; The parallel comparison tree module inputs multiple input vector data for calculating Softmax in parallel, and is used to solve the maximum value of the input data and output the result to the approximate e Index module; The approximation e The exponential module solves the input data e The exponential approximation result is output to the bit width conversion ping-pong buffer module and the accumulation and summation module; The cumulative summation module cumulatively sums a plurality of input data and outputs the sum to the approximate division module; The approximate division module converts the bit width into the ping-pong buffer module. e The exponential approximation result is approximated by the output of the cumulative summation module to obtain the approximate calculation result of the Softmax function; The bit width conversion ping-pong buffer module is used to input data and approximate e Index modules, approximation e Pipeline optimization between the exponential module and the division module to support inconsistent input and output parallelism; The 16-bit radix-4 Booth approximation multiplier is used to approximate e Multiplication approximation in the exponential module and the approximate division module.

2. The Softmax function hardware accelerator based on approximate calculation as claimed in claim 1, characterized in that: The parallel comparison tree module supports multiple parallel data inputs, including an eight-input comparator, a four-input comparator, a two-input comparator and a maximum value register, and supports a maximum of 32 parallel inputs. The four local maximum values ​​obtained by the first-level comparison are compared with the final result obtained by the second-level comparison and stored in the maximum value register; the result obtained in each round of comparison is compared with the value in the maximum value register. If it is greater than the value in the maximum value register, the original value is overwritten; if it is less than, it remains unchanged, and finally the maximum values ​​of a batch of input vectors are obtained.

3. The Softmax function hardware accelerator based on approximate calculation as claimed in claim 1, characterized in that: The approximation e The index module will e Exponentials are converted to base 2 functions using the base conversion formula: , and divide the exponential part into integer and decimal. The integer is realized by the shifter, and the decimal is approximated as polynomial multiplication and addition through the Taylor expansion formula: The approximation e All multiplications within the exponential module use 16-bit radix-4 Booth approximation multipliers.

4. The Softmax function hardware accelerator based on approximate calculation as claimed in claim 1, characterized in that: The approximate division module uses the Goldschmidt approximation algorithm: Approximate division as multiplication: All multiplications within the approximate division module employ 16-bit radix-4 Booth approximate multipliers.

5. The Softmax function hardware accelerator based on approximate calculation as claimed in claim 3 or 4, characterized in that: The 16-bit radix-4 Booth approximate multiplier adopts an approximate 4-2 compressor, an approximate full adder and a segmented approximate compression tree. The approximate 4-2 compressor and the approximate full adder adopt a Karnaugh map numerical replacement method to simplify the calculation process; the segmented approximate compression tree will perform approximate compression on the low bits and retain accurate compression on the high bits.

6. The Softmax function hardware accelerator based on approximate calculation as claimed in claim 1, characterized in that: The bit width conversion ping-pong cache module supports input and output data with different parallelisms, and is composed of two RAMs and a read-write controller. The read-write controller changes the parallelism of input and output according to configuration parameters.

7. The Softmax function hardware accelerator based on approximate calculation as claimed in claim 1, characterized in that: The complete handshake protocol is a Ready / Valid handshake protocol; the Ready signal indicates whether the receiving module is ready to receive data. When the receiving module is ready to receive data, the Ready signal is set to a high level; the Valid signal indicates whether the sending module has valid data to send. When the sending module has valid data, the Valid signal is set to a high level.

8. The Softmax function hardware accelerator based on approximate calculation as claimed in claim 7, characterized in that: The back pressure function is that the module receiving data transmits feedback information to the module sending data by controlling the Ready signal, pausing or continuing to send data, so as to avoid data loss.

Images