Softmax function hardware accelerator and acceleration method thereof

Abstract

The invention provides a Softmax function hardware accelerator and an acceleration method thereof, and is applied to the field of hardware acceleration. The hardware accelerator comprises a preprocessing module, a maximum value updating module, an index module, an accumulation module, a digital standardization conversion module and a reciprocal module. The method comprises the following steps: inputting a plurality of initial vectors into the preprocessing module to determine a plurality of effective elements in each group and a maximum element in each group; comparing the maximum elements of each group through a maximum value updating module to obtain a global maximum element; inputting each group of effective elements into an index module for piecewise linear fitting to obtain an index approximate value of each effective element; an accumulation module shifts and accumulates the index approximate value according to the relative offset to obtain an accumulated sum; converting the accumulated sum into a standardized floating point form by using a digital standardized conversion module; and finally, obtaining a Softmax probability value corresponding to the effective element through piecewise linear fitting and shift operation by a reciprocal module.

Description

Technical Field

[0001] This application relates to the field of hardware acceleration, specifically to a Softmax function hardware accelerator and its acceleration method. Background Technology

[0002] In the field of deep learning, the Softmax function, as one of the core activation functions, is widely used in classification tasks of neural networks. Its core function is to transform the multiple raw scores output by the model into normalized results that conform to a probability distribution, so that each output value is in the interval [0,1] and the sum of all output values ​​is 1, thereby achieving probability prediction for different categories. As the scale of deep learning models continues to expand, the dimension of the vectors input to the Softmax function continues to increase (for example, in tasks such as natural language processing and computer vision, the vector dimension often reaches thousands or even tens of thousands of dimensions), causing the computational cost of the Softmax function to grow exponentially, placing stringent demands on computational efficiency. The Softmax function optimizes numerical stability by introducing a global maximum value to avoid numerical overflow problems during exponential operations. However, this computational process involves a large number of natural exponent calculations, accumulation operations, and reciprocal calculations, resulting in high computational complexity and strong data dependencies. Software implementation is insufficient to meet the real-time inference requirements of deep learning models. Traditional hardware implementations often suffer from the following drawbacks: First, they frequently employ full-precision exponentiation units, leading to high hardware resource consumption, high power consumption, and long latency, making them unsuitable for scenarios with strict limitations on hardware resources and power consumption, such as edge computing devices. Second, for high-dimensional sparse input vectors, traditional solutions fail to effectively filter valid values, performing full calculations on all elements, wasting computational resources and further reducing efficiency. Furthermore, during accumulation and normalization, traditional solutions lack a balance between data precision and computational efficiency, easily leading to numerical precision loss or excessive latency. Therefore, designing a hardware acceleration scheme for the Softmax function that balances computational efficiency, numerical precision, and hardware resource consumption has become a pressing technical problem in the field of deep learning hardware acceleration. Summary of the Invention

[0003] In view of this, this application provides a hardware accelerator for the Softmax function and its acceleration method. By optimizing the hardware implementation logic of key links such as data preprocessing, exponential operation, and accumulation normalization, it significantly improves computational efficiency, reduces hardware resource consumption and power consumption while ensuring that the numerical calculation accuracy meets the requirements of deep learning tasks. It is also suitable for diverse application scenarios such as high-dimensional vector operations and edge computing devices.

[0004] This application provides a hardware acceleration method for the Softmax function, applied to a Softmax function hardware accelerator. The Softmax function hardware accelerator includes a preprocessing module, a maximum value update module, an exponentiation module, an accumulation module, a number normalization conversion module, and a reciprocal module. The method is characterized by: inputting multiple initial vectors into the preprocessing module; dividing the multiple initial vectors into multiple vector groups according to a predetermined number of elements; processing the initial vectors to determine the maximum element within each vector group and multiple valid elements within each vector group; receiving the maximum element within each vector group through the maximum value update module and comparing it to determine the global maximum element of the multiple initial vectors; and calculating the difference between the global maximum element and the maximum element within each vector group. The relative offset of each vector group is obtained; each valid element in each vector group is input into the exponent module for piecewise linear fitting to obtain the exponential approximation value of each valid element; the accumulation module shifts the exponential approximation value of each valid element according to the relative offset in the vector group to which each exponential approximation value belongs, and accumulates all the exponential approximation values ​​corresponding to all valid elements after shifting to obtain the sum; the number standardization conversion module converts the sum to standardized floating-point form to obtain the mantissa corresponding to the sum; the reciprocal module merges the mantissa and the fractional part of the valid elements, performs piecewise linear fitting on the merged result, performs shift operation, obtains the Softmax probability value corresponding to the valid element, and stores it in a register.

[0005] According to an embodiment of this application, the preprocessing module includes an extremum determination unit and a valid value filtering unit. Multiple initial vectors are divided into multiple vector groups according to a predetermined number of elements and input into the preprocessing module for processing to determine the maximum element and multiple valid elements within each vector group. This includes: using the extremum determination unit to compare multiple elements within each vector group based on a binary tree algorithm to obtain the maximum element among the multiple elements in each vector group; and using the valid value filtering unit to perform sparsification processing on the elements within each vector group to obtain multiple valid elements within each vector group. According to an embodiment of this application, the maximum value update module receives and compares the maximum element within each vector group to determine the global maximum element of multiple initial vectors. This includes: the maximum value update module sequentially receives the maximum element within each vector group in a preset order and performs loop processing: if the received maximum element is the maximum element of the first received vector group, the maximum element is temporarily stored as the global maximum element; if it is not the maximum element of the first received vector group, the maximum element is compared with the temporarily stored global maximum element, and the larger value is retained as the new global maximum element.

[0006] According to an embodiment of this application, the exponent module includes a linear fitting unit and a shifting unit; each valid element in each vector group is input into the exponent module for piecewise linear fitting to obtain an exponential approximation value for each valid element, including: performing piecewise approximation fitting on a preset natural exponential function corresponding to each valid element to obtain a fitting result corresponding to each valid element; inputting the fitting result into the shifting unit to perform a shifting operation on the fitting result of each valid element to obtain an exponential approximation value for each valid element; wherein the shifting magnitude and shifting direction of the shifting operation are determined based on the integer part of each valid element.

[0007] According to an embodiment of this application, based on the relative offset in the vector group to which each exponential approximation belongs, the exponential approximation of each valid element is shifted by the accumulation module, including: if the relative offset in the vector group to which the exponential approximation belongs is positive, the accumulation module performs a left shift operation on the corresponding exponential approximation, and the left shift magnitude is equal to the value of the relative offset.

[0008] According to an embodiment of this application, the accumulation module further includes an accumulation register; accumulating all exponent approximations corresponding to all valid elements after shifting to obtain an accumulation sum includes: the accumulation module first initializes the accumulation sum in the accumulation register to 0, then reads each shifted exponent approximation value in a preset order, performs an arithmetic accumulation operation between the currently read exponent approximation value and the current accumulation sum stored in the accumulation register to obtain a new accumulation sum; after each accumulation operation is completed, the original accumulation sum stored in the accumulation register is updated to the new accumulation sum; after all shifted exponent approximations have been read and accumulated, the value stored in the accumulation register is the final required accumulation sum.

[0009] According to an embodiment of this application, the cumulative sum input is converted into a standardized floating-point form using a digital normalization conversion module to obtain the mantissa corresponding to the cumulative sum. This includes: converting the cumulative sum into binary data corresponding to the cumulative sum; converting the binary data corresponding to the cumulative sum into a standardized floating-point form by locating the position information of the most significant bit in the binary data; and splitting the data to obtain the mantissa corresponding to the cumulative sum.

[0010] According to an embodiment of this application, the reciprocal module includes an addition unit, a linear fitting unit, and a shift unit. The reciprocal module merges the mantissa and the fractional part of the effective elements, performs piecewise linear fitting on the merged result, and then performs a shift operation to obtain the Softmax probability value corresponding to the effective element, which is stored in a register. This includes: merging the mantissa and the fractional part of the effective elements using the addition unit to obtain a merged result; inputting the merged result into the linear fitting unit to obtain a fitted function approximation; and shifting the function approximation using the integer part of the effective element, the global maximum element, and the mantissa corresponding to the cumulative sum as shift control parameters to obtain the Softmax probability value corresponding to the effective element. The shift control parameters include shift amplitude and shift direction.

[0011] According to an embodiment of this application, a fitting result for each valid element is obtained by performing piecewise approximate fitting on a preset natural exponential function corresponding to each valid element. This includes: dividing the domain of the preset natural exponential function corresponding to each valid element into multiple segments, wherein the division of each segment is based on the derivative distribution characteristics of the natural exponential function and a preset scaling factor, and is dynamically divided according to preset rules; using a binary search algorithm on each segment to determine the farthest endpoint of each segment while meeting preset error requirements; based on the farthest endpoint, using a greedy algorithm to find the minimum slope bit width of each segment to obtain fitting parameters for each segment, including slope and intercept; and fitting the natural exponential function based on the fitting parameters for each segment.

[0012] A second aspect of this application provides a hardware accelerator for the Softmax function, comprising: a preprocessing module for dividing multiple initial vectors into multiple vector groups according to a predetermined number of elements, and processing them to determine the maximum element and multiple valid elements within each vector group; a maximum value update module for receiving the maximum element within each vector group and comparing it to determine the global maximum element of the multiple initial vectors, calculating the difference between the global maximum element and the maximum element within each vector group, and obtaining the relative offset of each vector group; and an exponentiation module for performing piecewise operations on each valid element within each vector group based on the natural exponential function. The algorithm employs a linear fitting process to obtain exponential approximations of the effective input vectors. An accumulation module shifts the exponential approximation of each effective element based on its relative offset within the vector group, then sums all the shifted exponential approximations of all effective elements to obtain a cumulative sum. A number standardization conversion module converts the cumulative sum to standardized floating-point form, obtaining the mantissa of the cumulative sum. A reciprocal module merges the mantissa and the fractional parts of the effective elements, performs piecewise linear fitting on the merged result, and then performs a shift operation to obtain the Softmax probability value corresponding to the effective element, storing it in a register.

[0013] The hardware accelerator and acceleration method for the Softmax function proposed in this application have the following beneficial effects:

[0014] (1) High computational efficiency: By sparsification to remove redundancy, binary tree to quickly find the maximum and minimum values, and piecewise fitting to replace full-precision exponential operation, combined with parallel processing logic, latency is greatly reduced and throughput is increased.

[0015] (2) Low hardware overhead: Segmented fitting and shift operations replace complex operation units, optimize the bit width of fitting parameters, and combine sparsification to reduce the burden, significantly reducing the number of logic gates and power consumption.

[0016] (3) Stable and controllable accuracy: Based on dynamic segmentation of function derivative, the fitting error is controlled, and with the global maximum value and offset calibration, overflow is avoided, thus meeting the accuracy requirements of deep learning;

[0017] (4) Strong adaptability: It supports flexible grouping and sparse threshold adjustment. The modular architecture can be integrated into various hardware platforms and adapt to different dimensions and scenarios.

[0018] (5) Easy to implement and expandable: The modules are clearly divided, the data interaction is simple, the parameters can be configured by software, and the computing scale can be expanded horizontally. Attached Figure Description

[0019] The embodiments of this application are described below with reference to the accompanying drawings, in which:

[0020] Figure 1aA flowchart illustrating a hardware acceleration method for the Softmax function according to an embodiment of this application is shown schematically.

[0021] Figure 1b This illustration schematically shows a hardware accelerator architecture diagram for the Softmax function according to an embodiment of this application;

[0022] Figure 2 A schematic diagram of a piecewise linear fitting unit according to an embodiment of this application is shown.

[0023] Figure 3 A block diagram of a Softmax function hardware accelerator according to an embodiment of this application is shown schematically. Detailed Implementation

[0024] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0025] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0027] In the field of deep learning, the Softmax function, as one of the core activation functions, is widely used in classification tasks of neural networks. The mathematical expression for the Softmax function is:

[0028]

[0029] Where X is an input vector with N elements, i.e., X = [ ], where N is a positive integer. Let N be the largest element. For the Nth element The value of the Softmax function.

[0030] The core function of the Softmax function is to convert the multiple raw scores of the model output into normalized scores that conform to the probability distribution, so that the normalized score of each output is in the interval [0,1] and the sum of the normalized scores of all outputs is 1, thereby realizing the calculation of the probability value of each input.

[0031] To improve the computation speed of the Softmax function, this application provides a hardware accelerator for the Softmax function and its acceleration method.

[0032] Figure 1a A flowchart illustrating a hardware acceleration method for the Softmax function according to an embodiment of this application is shown schematically. Figure 1b An architectural diagram of a Softmax function hardware accelerator according to an embodiment of this application is illustrated.

[0033] like Figure 1a As shown, the hardware acceleration method for the Softmax function in this application includes steps S1 to S6; as Figure 1b As shown, the architecture of the Softmax function hardware accelerator according to an embodiment of this application includes a preprocessing module, a maximum value update module, an exponentiation module, an accumulation module, a digital normalization conversion module, and a reciprocal module. The modules are interconnected through a high-speed bus inside the FPGA. This accelerator architecture provides physical support for the aforementioned Softmax function hardware acceleration method.

[0034] The specific steps of the Softmax function hardware acceleration method in this embodiment are as follows:

[0035] In step S1, multiple initial vectors are input into the preprocessing module. The initial vectors are divided into multiple vector groups according to a predetermined number of elements and processed to determine the maximum element and multiple valid elements within each vector group; for example... Figure 1b As shown, the preprocessing module includes an extremum determination unit and an effective value filtering unit.

[0036] For example, if the total number of initial vectors X is N, then the N initial vectors can be divided into M = ... A set of vectors, namely V1, V2...V M ,in, This indicates rounding up. Each vector group contains at most p elements; if the last group has fewer than p elements, it is padded with zeros. That is, for any vector group... The elements inside can be represented as , where k=1,2,3,4…M.

[0037] Furthermore, the maximum / minimum value finding unit uses a binary tree algorithm to compare multiple elements in each vector group to obtain the maximum element among multiple elements in each vector group.

[0038] Using a binary tree algorithm, multiple valid elements within each vector group are compared pairwise and then merged level by level (e.g., when p=8, it requires...). (Level comparator), ultimately obtaining each vector group The largest element among multiple valid elements Each comparator unit consists of four full adders, capable of comparing two 4-bit binary numbers. For an 8-bit input, a layered comparison method is used: first, the higher 4 bits are compared; if the higher 4 bits are the same, then the lower 4 bits are compared to determine the maximum value.

[0039] Furthermore, the effective value filtering unit performs sparsification processing on the elements in each vector group based on the largest element among the multiple elements in each vector group, thereby obtaining multiple effective elements in each vector group.

[0040] Specifically, for each vector group The effective value filtering unit calculates multiple elements within each vector group. The maximum element within each corresponding vector group The difference ,Right now (i=1,2,3,4…p).

[0041] By As a dynamic benchmark for each vector group to filter valid elements, it can simultaneously adapt to each vector group with different amplitude scales, avoiding the defect of falsely filtering valid elements in each vector group due to amplitude differences, and ensuring that each vector group can accurately filter valid elements.

[0042] A threshold T can be set if ,but Invalid element; if ,but If it is a valid element, it will be retained and participate in subsequent calculations.

[0043] Where, threshold T= , The effective bit width of the result of the exponentiation operation, such as 8 bits or 12 bits.

[0044] In this embodiment, as Figure 1b As shown, the preprocessing module may also include a constant multiplier unit for multiplying the valid elements within each vector group. and the maximum element within each vector group This is converted into hardware-friendly shift and addition operations. This allows the valid elements within each vector group to be... Or the maximum element within each vector group It is split into two parts: an integer and a decimal.

[0045] For example, the effective elements within each vector group are multiplied by a constant multiplier unit. and the maximum element within each vector group respectively with Multiplication, using ≈1+2 −2 +2 −4 +2 −6 The approximate expression converts multiplication operations into addition and shift operations, which can significantly reduce hardware overhead.

[0046] Furthermore, for each valid element ,make ,in For the integer part, The decimal part.

[0047] For the largest element in the group ,make ,in The integer part The decimal part.

[0048] Because comparing integers in hardware requires only simple comparator logic operations—for example, comparing 4-bit integers only requires 4 full adders, with a latency of less than one clock cycle—comparing floating-point numbers while retaining the fractional part requires an additional floating-point comparator (increasing the number of logic gates by more than three times), resulting in a latency of more than three clock cycles, it is generally preferable to use the integer part. Replace the maximum element in each vector group , that is to say It is an approximation of the largest element within each vector group.

[0049] In step S2, the maximum element in each vector group is received and compared by the maximum value update module to determine the global maximum element of multiple initial vectors. The difference between the global maximum element and the maximum element in each vector group is calculated to obtain the relative offset of each vector group.

[0050] In this embodiment, the maximum value update module receives the maximum element in each vector group in a preset order and performs loop processing: if the received maximum element is the maximum element in the first received vector group, the maximum element is temporarily stored as the global maximum element; if it is not the maximum element in the first received vector group, the maximum element is compared with the temporarily stored global maximum element, and the larger value is retained as the new global maximum element.

[0051] For example, using a cyclic update strategy, the maximum value update module receives the approximate value of the maximum element in each vector group in a predetermined order. If the received value is an approximation of the largest element of the first vector group. Then the approximate value of the largest element will be... Temporarily stored in the intermediate result storage unit, i.e. .in This is the first maximum value element stored in the intermediate result storage unit.

[0052] If the received value is an approximation of the largest element in the k-th vector group. ( Then the approximate value of the largest element in the k-th group is... and the largest element in the intermediate result storage unit Compare the two and select the maximum value. .

[0053] After all vector groups have been processed, the global maximum element is finally obtained. The above steps can be specifically expressed as follows: .

[0054] Furthermore, for multiple valid elements within each vector group, the relative offset is obtained by calculating the difference between the global maximum element and the maximum element within the group. That is, relative offset .

[0055] In step S3, each effective element in each vector group is input into the exponential module for piecewise linear fitting to obtain the exponential approximation value of each effective element.

[0056] In this embodiment, as Figure 1b As shown, the exponential module includes a linear fitting unit and a shifting unit.

[0057] In this embodiment, each effective element within each vector group is input into the exponential module for piecewise linear fitting to obtain an exponential approximation value for each effective element. This includes: performing piecewise approximation fitting on a preset natural exponential function corresponding to each effective element to obtain a fitting result for each effective element; inputting the fitting result into a shift unit to perform a shift operation on the fitting result for each effective element to obtain an exponential approximation value for each effective element; wherein the shift magnitude and shift direction of the shift operation are determined based on the integer part of each effective element after exponential transformation.

[0058] Due to the high computational complexity of exponential operations, this application uses a piecewise linear fitting algorithm built into the linear fitting unit to approximate the exponent.

[0059] For example, in the Softmax function To avoid directly calculating the exponent (such as...) When it is large, (This can lead to exponential explosion and numerical overflow); linear fitting units and shift units can be used to replace full-precision exponential operations.

[0060] First, use the change-of-base formula. Converting the natural base to a base-2 exponent, i.e.:

[0061]

[0062] Furthermore, due to , .in, for The integer part, The decimal part; To extract the integer part, we obtain the result by extracting the integer part. integer part .

[0063] Therefore, it can be Split into:

[0064]

[0065] in, for The integer part of the form; It is in decimal form, and Suitable for calculation by linear fitting unit.

[0066] For example, assuming valid elements Maximum element =6.8, the largest element in the world 8.5. Then The integer part can be split into -3.4625 = -4 + 0.5375. The decimal part is .

[0067] Furthermore, it can Converted to a base-2 exponent, i.e. For the decimal part Then, the input is used for fitting within the linear fitting unit.

[0068] Through linear fitting unit, The domain (-1, 1) is dynamically segmented. For each segment, a binary search is used to find the farthest endpoint, and then a greedy algorithm is used to find the slope that meets the accuracy requirements. and intercept The minimum required bit width, thus saving chip area and power consumption.

[0069] For example, for The slope is obtained after linear fitting unit. =0.693, intercept .

[0070] Ultimately, it can be obtained through linear fitting units. Fitted values ,in for decimal part Approximate value after linear fitting.

[0071] Furthermore, regarding the linear fitting results Perform a shift operation to perform an equivalent computation. Finally obtained exponential approximation .

[0072] In this embodiment, the shift magnitude and shift direction of the shift operation are based on the integer part of each valid element after exponential transformation. Sure.

[0073] For example, for valid elements integer part Then the linear fitting results Perform a shift operation to obtain the valid element. exponential approximation ,Right now .

[0074] And the true value Absolute error ;

[0075] The relative error is RE=AE / 0.09071795≈5.44%.

[0076] Based on the absolute error and relative error, it can be seen that the fitting accuracy of this embodiment is relatively high.

[0077] In step S4, the accumulation module calculates each exponential approximation value. The relative offset in the vector group to which it belongs The exponential approximation for each effective element Perform a shift operation to approximate the exponent values ​​corresponding to all valid elements after the shift. Perform the summation to obtain the cumulative sum.

[0078] In this embodiment, the accumulation module receives the approximate exponent values ​​corresponding to each group of valid elements output by the exponent module in groups. Based on the relative offset in the vector group to which each exponential approximation belongs. The exponential approximation for each effective element Perform a shift to unify all exponential approximations. The global benchmark.

[0079] For example, the approximate exponent value corresponding to the effective elements within each group is received in groups. , where k is the group number.

[0080] In this embodiment, if the relative offset in the vector group to which the exponential approximation belongs... For positive values, the accumulation module approximates the corresponding exponent. Perform a left shift operation, the left shift magnitude and the relative offset. If the values ​​are equal, and the exponents are approximate values... Relative offset within the vector group If the value is 0, then no shift operation is performed.

[0081] In the shift operation, the shift magnitude is determined by the relative offset. Decide.

[0082] like Then shift to the left bit, equivalent to ;like ,but No shift, that is ,in It is the shifted exponential approximation of each valid element in the current vector group.

[0083] Furthermore, the shifted exponential approximation value of each valid element in the current vector group is read in a preset order. For example, first read the approximate values ​​of the exponent in the order within the group, then read the approximate values ​​within the next group, and so on, and then accumulate them.

[0084] In this embodiment, the accumulation module further includes an accumulation register; the accumulation of all exponent approximations corresponding to all valid elements after shifting to obtain a cumulative sum includes: the accumulation module first initializes the cumulative sum in the accumulation register to 0, then reads each shifted exponent approximation in a preset order, performs an arithmetic accumulation operation between the currently read exponent approximation and the current cumulative sum stored in the accumulation register, and obtains a new cumulative sum; after each accumulation operation is completed, the original cumulative sum stored in the accumulation register is updated to the new cumulative sum; after all shifted exponent approximations have been read and accumulated, the value stored in the accumulation register is the final required cumulative sum.

[0085] For example, when the accumulator receives the exponential approximation value of each vector group, it first clears the accumulator register to zero, that is, it ensures that the accumulated sum in the accumulator register is zero. 0, to avoid residual data affecting the final result.

[0086] Furthermore, the exponential approximations are read sequentially, assuming the first exponential approximation read is... =0.3432, the second exponent approximation is 0.3432. =0.0642.

[0087] First, The current accumulated sum stored in the current accumulator register Performing arithmetic addition: 0 + 0.3432 = 0.3432, which gives the new cumulative sum. ;

[0088] Then read the second exponential approximation in sequence. =0.0642, and Accumulate, and you will get .

[0089] Repeat the above process continuously until all exponential approximations are accumulated, which is the required sum SUM:

[0090]

[0091] When accumulating element by element, the temporary register retains the complete result of each accumulation (including carry and high-order bits), avoiding precision truncation caused by accumulating multiple data at the same time; at the same time, only one accumulator temporary register is required, reducing hardware resource consumption and making it suitable for edge devices.

[0092] In step S5, the sum is converted into a normalized floating-point form using the digital normalization conversion module to obtain the mantissa corresponding to the sum.

[0093] In this embodiment, the accumulated sum is converted into corresponding binary data. By locating the position information of the most significant bit in the binary data, the binary data corresponding to the accumulated sum is converted into a standardized floating-point form, and the mantissa corresponding to the accumulated sum is obtained by splitting it. In this embodiment, the number standardization conversion module is implemented through a most significant bit detector.

[0094] First, convert SUM to binary data form. bin Then SUM bin =b n b n−1 ...b1b0 (where b) n It is the most significant bit in n-bit binary data, i.e., the first 1.

[0095] Furthermore, the binary data SUM bin Convert to normalized floating-point form SUM bin =F×2 Q F is the mantissa, and 1≤F<2; Q is the exponential factor (Q=n, where n is the position index of the most significant bit).

[0096] For example, if SUM = 11.25, then the binary SUM... bin =1011.01, then the last digit is F=1.01101, Q=3, that is, SUM=11.25=F×2 Q =1.01101×2 3 .

[0097] In step S6, the decimal part of the mantissa and the effective element is merged using the reciprocal module. The merged result is then subjected to piecewise linear fitting and shift operation to obtain the Softmax probability value corresponding to the effective element, which is then stored in a register.

[0098] In this embodiment, the reciprocal module includes an addition unit, a linear fitting unit, and a shifting unit. The addition unit merges the decimal parts of the mantissa and the significant elements to obtain the merged result. Both the linear fitting unit and the shifting unit can be reused.

[0099] The merged results are input into the linear fitting unit to obtain the approximate value of the fitted function;

[0100] The shift unit uses the integer part of the effective element, the global maximum element, and the mantissa of the sum as shift control parameters for the approximation of the function corresponding to the effective element. The function approximation is shifted to obtain the Softmax probability value corresponding to the effective element. The shift control parameters include the shift amplitude and the shift direction.

[0101] For example, due to

[0102] for can Convert to exponentiation with base 2:

[0103]

[0104] At the same time, due to

[0105] therefore:

[0106]

[0107] in, for The Softmax value.

[0108] Furthermore, we can obtain the result after combining the mantissa and the decimal part of the significant elements, that is... ,and Suitable for linear fitting calculations.

[0109] like Figure 1b As shown, the linear fitting unit in the exponential module is reused for... By fitting the data, we obtain its slope as... Intercept position .

[0110] therefore, ;

[0111] Because the linear fitting unit only processes the fractional part, i.e. Therefore, it is necessary to approximate the fractional part. Shift the position to ensure improved accuracy.

[0112] pass Figure 1b The shift unit in the middle, for the fractional part To perform a shift, that is:

[0113]

[0114] Thus obtain The definite value, that is, the Softmax of all input elements is a probability distribution with { }

[0115] pass Figure 1a The methods and processes in Figure 1b The deep collaboration of hardware modules in the system solves the core pain points of traditional Softmax function hardware acceleration. Specifically, it focuses on sparsity reduction to eliminate redundancy, piecewise fitting to reduce complexity, and hardware reuse to reduce overhead. While ensuring accuracy, it achieves high energy efficiency, high throughput, and high adaptability in Softmax acceleration, providing a feasible hardware solution for edge deployment of deep learning models.

[0116] Figure 2 A schematic diagram of a piecewise linear fitting unit structure according to an embodiment of this application is shown.

[0117] like Figure 2 The diagram shown is a piecewise linear fitting unit structure, including a decoder and a multiplexer.

[0118] In this embodiment, the domain of the preset natural exponential function corresponding to each valid element is divided into multiple segments. The division of each segment is based on the derivative distribution characteristics of the natural exponential function and a preset scaling factor, and is dynamically divided according to preset rules. A binary search algorithm is used for each segment to determine the farthest segment endpoint under the premise of meeting preset error requirements. Based on the farthest segment endpoint, a greedy algorithm is used to find the minimum slope bit width of each segment to obtain the fitting parameters of each segment, including the slope and intercept. Based on the fitting parameters of each segment, the natural exponential function is fitted.

[0119] For example, the natural exponential function (which becomes 2 after a base change transformation) h The domain of the function is (-1, 1). Based on the distribution characteristics of the function derivative and the preset scaling factor, the segments are dynamically divided. When the absolute value of the derivative is larger (the function changes more drastically, such as when h is close to 0), the segments are denser; when the absolute value of the derivative is smaller (the function changes more gently, such as when h is close to ±1), the segments are sparser, ensuring that the fitting error is evenly distributed.

[0120] The results of dynamic segmentation (e.g., 8 segments) are pre-stacked into the decoder's hardware comparator. The core function of the decoder is to replicate the segmentation rules of the algorithm in real time. When the input h (e.g., v from the exponent module) is... i The countdown module's v i After entering the decoder (−F+1), the hardware comparator compares h with the pre-stored segment boundaries one by one; through combinational logic, the interval determination is completed within one clock cycle, and a 3-bit binary segment index (corresponding to 8 segments) is output to provide a basis for subsequent parameter selection.

[0121] Furthermore, for each initial segment, a binary search is performed to find the segment that meets a preset error threshold (e.g., ...). Under the premise of finding the farthest segment endpoint, first estimate the upper limit of the segment length based on the derivative of the starting point, and then iterate and test within the search interval to ensure that the segment coverage is maximized. For example, if the farthest endpoint of an initial segment is determined to be -0.75 by the binary search algorithm, then the boundary threshold of that segment in the decoder is fixed to -0.75.

[0122] The decoder's comparator strictly follows this threshold to divide the interval, ensuring that the segmentation at the hardware level is completely consistent with the segmentation after algorithm optimization, thus avoiding accuracy loss due to hardware implementation deviations.

[0123] For example, in this embodiment, the number of segments is ultimately optimized to 8, so the decoder only needs 3 bits of encoding to complete the index output, which greatly simplifies the hardware logic.

[0124] Furthermore, based on the determined segment endpoint, a greedy algorithm is used to find the minimum slope bit width that meets the error requirements. Starting from the maximum slope bit width (e.g., 16 bits), the algorithm traverses downwards to select the minimum feasible bit width (e.g., 8 bits).

[0125] The optimal fitting parameters for each segment, such as the slope K, are obtained. opt ={K1,K2,K3,……,K8} and intercept B opt ={B1,B2,B3,……,B8}.

[0126] The optimized eight sets of fitting parameters ({K1, B1}, {K2, B2}...{K8, B8}) are pre-stored in the input register of the multiplexer, achieving a deep integration of hardware and algorithm optimization.

[0127] Specifically, the register bit width strictly matches the minimum slope bit width (e.g., 8 bits) to avoid resource waste caused by redundant bit width; the fitting parameters are stored using fixed-point quantization, and the quantization accuracy is determined by the algorithm's error threshold (e.g., the smaller the allowable error, the higher the accuracy requirement), ensuring that the hardware operation error does not exceed the algorithm's preset range; at the same time, the input ports of the multiplexer correspond one-to-one with the parameter registers, and the corresponding parameters can be directly selected through segmented indexing without real-time calculation, achieving parameter reuse and low latency.

[0128] Furthermore, through linear operations Approximate substitution 2 h In the corresponding segmented output, the nonlinear operation is transformed into a simple linear operation.

[0129] At the same time, due to the slope and intercept For quantized fixed-point numbers (e.g., 8-bit), and multiplication operations It can be broken down into multiple shifts and additions.

[0130] For example, for The input is quantized into an 8-bit fixed-point number, resulting in the number 10110001. Since the quantized slope is fixed-point, its multiplication with the input h can be decomposed into multiple left shifts and addition operations (without requiring a complex multiplier). For the addition operation… The operation is performed by a single hardware adder, and the entire linear operation is completed within one clock cycle. The result is directly output to the shift unit of the exponent module or the reciprocal module.

[0131] In this embodiment, the index module is used as an example. Taking the fitting process with a value of 0.5375 as an example:

[0132] First, =0.5375 converted to exponential form 2 0.5375 Based on function 2h (h represents the derivative distribution characteristics of the fractional part. The domain (0,1) of h is dynamically divided into 8 segments, such as the first segment [0,0.125); the second segment [0.125,0.5); the third segment [0.5,0.75); the fourth segment [0.75,0.875); the fifth segment [0.875,0.9375); the sixth segment [0.9375,0.96875); the seventh segment [0.96875,0.984375); and the eighth segment [0.984375,1].)

[0133] because =0.5375 belongs to the 3rd segment (interval [0.5, 0.75)), and the built-in optimal fitting parameter corresponding to this segment is: =0.693、 =1.0, the result of linear operation is 0.693×0.5375+1.0≈1.3725.

[0134] At the same time, the decoder receives =0.5375, the comparator determines that it belongs to the 3rd segment, and outputs index 101; the multiplexer selects the pre-stored value based on index 101. =0.693 and =1.0; Linear operations are performed using hardware shifting and addition, outputting 1.3725.

[0135] because (True value); Absolute error It meets the accuracy requirements of deep learning tasks, while the hardware operation only takes one clock cycle.

[0136] Figure 3 A block diagram of a Softmax function hardware accelerator according to an embodiment of this application is shown schematically.

[0137] like Figure 3 As shown, the Softmax function hardware accelerator 300 in this embodiment includes a preprocessing module 310, a maximum value update module 320, an exponentiation module 330, an accumulation module 340, a number normalization conversion module 350, and a reciprocal module 360.

[0138] Preprocessing module 310: Divides multiple initial vectors into multiple vector groups according to a predetermined number of elements, and inputs these groups into the preprocessing module for processing to determine multiple valid elements within each vector group and the largest element among these valid elements. In one embodiment, preprocessing module 310 can be used to perform step S1 described above, which will not be repeated here.

[0139] In this embodiment, the preprocessing module 310 includes an extremum determination unit, an effective value filtering unit, and a constant multiplier unit.

[0140] Maximum / minimum value finding unit: used to perform pairwise comparisons and level-by-level merging of elements within a group based on a binary tree algorithm to obtain the maximum value within the group.

[0141] Valid value filtering unit: Used to filter valid elements by calculating the difference between an element and the maximum value in the group and comparing it with a threshold, thereby achieving sparsity processing.

[0142] Constant multiplier unit: used to multiply the effective elements and the maximum value within the group by... Multiplication is converted into shift and addition operations, and the result is split into integer and fractional parts.

[0143] Maximum value update module 320: This module receives the maximum element within each vector group and compares it to determine the global maximum element of multiple initial vectors. It then calculates the difference between the global maximum element and the maximum element within each vector group to obtain the relative offset of each vector group. In one embodiment, the maximum value update module 320 can be used to execute step S2 as described above, which will not be repeated here.

[0144] In this embodiment, the maximum value update module 320 also includes an intermediate result storage unit, a comparator, and an offset calculator to realize dynamic updating of the global maximum value and real-time calculation of the relative offset.

[0145] Exponent module 330: Used to perform piecewise linear fitting based on the natural exponential function on the effective input vector to obtain an exponential approximation of the effective input vector. In one embodiment, exponent module 330 can be used to perform step S3 described above, which will not be repeated here.

[0146] In this embodiment, the exponent module 330 further includes a linear fitting unit and a shift unit. The linear fitting unit uses an error-aware fitting algorithm to achieve a high-precision, low-complexity exponent approximation, and the shift unit performs shift calibration of the exponent approximation based on the integer part difference.

[0147] Accumulation module 340: This module shifts the exponential approximation of each valid element according to the relative offset within the vector group to which each exponential approximation belongs, and then accumulates all the exponential approximations corresponding to all the shifted valid elements to obtain a sum. In one embodiment, accumulation module 340 can be used to execute step S4 described above, which will not be repeated here.

[0148] In this embodiment, the accumulation module 340 further includes an accumulation register and a shift unit to realize the shift calibration of the exponential approximation and element-by-element accumulation.

[0149] Number normalization conversion module 350: used to convert the accumulated sum input into normalized floating-point form to obtain the mantissa of the accumulated sum. In one embodiment, the number normalization conversion module 350 can be used to perform step S5 described above, which will not be repeated here.

[0150] The reciprocal module 360 ​​is used to perform addition and piecewise linear fitting operations on the mantissa and the valid input vector, followed by a shift operation to obtain the Softmax probability value corresponding to the valid input vector, and then output it to the result register for storage. In one embodiment, the reciprocal module 360 ​​can be used to execute step S6 described above, which will not be repeated here.

[0151] In this embodiment, the reciprocal module 360 ​​further includes an addition unit, a linear fitting unit, and a shifting unit to realize the approximate calculation of the reciprocal and the calibration of the final probability value.

[0152] In this embodiment, the Softmax function hardware accelerator also includes a result register, which includes an intermediate result storage unit and an output register. The intermediate result storage unit is used to store intermediate data such as the integer part, fractional part, maximum value within a group, and global maximum value during the preprocessing process. The output register is used to store and output the final Softmax probability distribution result.

[0153] It should be noted that the embodiments of the accelerator section are similar to those of the method section, and the technical effects achieved are also similar. For specific details, please refer to the method embodiment section above, which will not be repeated here.

[0154] According to embodiments of this application, any plurality of modules of the preprocessing module 310, the maximum value update module 320, the exponentiation module 330, the accumulation module 340, the digital normalization conversion module 350, and the reciprocal module 360 ​​can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this application, at least one of the preprocessing module 310, the maximum value update module 320, the exponentiation module 330, the accumulation module 340, the digital normalization conversion module 350, and the reciprocal module 360 ​​can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the preprocessing module 310, the maximum value update module 320, the exponentiation module 330, the accumulation module 340, the number standardization conversion module 350, and the reciprocal module 360 ​​may be implemented at least partially as a computer program module, which can perform the corresponding function when the computer program module is run.

[0155] Those skilled in the art will understand that the features described in the various embodiments of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments of this disclosure can be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0156] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A hardware acceleration method for the Softmax function, applied to a Softmax function hardware accelerator, wherein the Softmax function hardware accelerator includes a preprocessing module, a maximum value update module, an exponentiation module, an accumulation module, a number normalization conversion module, and a reciprocal module, characterized in that, The method includes: Multiple initial vectors are input into the preprocessing module, which divides the multiple initial vectors into multiple vector groups according to a predetermined number of elements and processes them to determine the maximum element in each vector group and multiple valid elements in each vector group. The maximum value update module receives and compares the maximum element in each vector group to determine the global maximum element of the plurality of initial vectors. It then calculates the difference between the global maximum element and the maximum element in each vector group to obtain the relative offset of each vector group. Each effective element in each vector group is input into the exponential module for piecewise linear fitting to obtain an exponential approximation value for each effective element. The accumulation module shifts the exponential approximation of each valid element according to the relative offset in the vector group to which each exponential approximation belongs, and accumulates all the exponential approximation of all valid elements after shifting to obtain the sum. The sum is converted into a normalized floating-point form using the digital normalization conversion module to obtain the mantissa of the sum. The decimal part of the mantissa and the effective element is merged using the reciprocal module. The merged result is then subjected to piecewise linear fitting and shift operation to obtain the Softmax probability value corresponding to the effective element, which is then stored in a register.

2. The method according to claim 1, characterized in that, The preprocessing module includes an extremum determination unit and an effective value filtering unit; Determining the maximum element within each vector group and multiple valid elements within each vector group includes: The maximum / minimum value finding unit compares multiple elements in each vector group based on a binary tree algorithm to obtain the maximum element among the multiple elements in each vector group. The effective value filtering unit performs sparsification on the elements in each vector group based on the largest element among the multiple elements in each vector group, thereby obtaining multiple effective elements in each vector group.

3. The method according to claim 1, characterized in that, The maximum value update module receives the maximum element within each vector group and compares it to determine the global maximum element of the plurality of initial vectors, including: The maximum value update module receives the maximum element in each vector group in a preset order and performs loop processing: If the received maximum element is the first received maximum element in the vector group, then the maximum element is temporarily stored as the global maximum element. If it is not the largest element in the first received vector group, then the largest element is compared with the temporarily stored global largest element, and the larger value is retained as the new global largest element.

4. The method according to claim 1, characterized in that, The exponential module includes a linear fitting unit and a shifting unit; Each effective element within each vector group is input into the exponential module for piecewise linear fitting to obtain an exponential approximation value for each effective element, including: By performing piecewise approximate fitting on the preset natural exponential function corresponding to each effective element, the fitting result corresponding to each effective element is obtained; The fitting result is input into the shifting unit, and a shift operation is performed on the fitting result of each effective element to obtain an exponential approximation value of each effective element; wherein the shifting magnitude and shifting direction of the shifting operation are determined based on the integer part of each effective element.

5. The method according to claim 1, characterized in that, The accumulation module shifts the exponential approximation of each valid element according to the relative offset in the vector group to which each exponential approximation belongs, including: If the relative offset in the vector group to which the exponential approximation belongs is positive, the accumulation module performs a left shift operation on the corresponding exponential approximation, and the left shift magnitude is equal to the value of the relative offset. If the relative offset in the vector group to which the exponential approximation belongs is 0, then no shift operation is performed.

6. The method according to claim 5, characterized in that, The accumulation module also includes an accumulation register; The step of summing up all the exponent approximations corresponding to all the shifted valid elements to obtain the sum includes: The accumulation module first initializes the sum in the accumulation register to 0, and then reads each shifted exponent approximation value in a preset order. It then performs an arithmetic accumulation operation between the currently read exponent approximation value and the current sum stored in the accumulation register to obtain a new sum. After each accumulation operation is completed, the original accumulated sum stored in the accumulator is updated to the new accumulated sum; after all the shifted exponent approximations have been read and accumulated, the value stored in the accumulator is the final required accumulated sum.

7. The method according to claim 1, characterized in that, The accumulated sum is converted into normalized floating-point form using the digital normalization conversion module to obtain the mantissa corresponding to the accumulated sum, including: The accumulated sum is converted into binary data corresponding to the accumulated sum. By locating the position information of the most significant bit in the binary data, the binary data corresponding to the accumulated sum is converted into a standardized floating-point form, and the mantissa corresponding to the accumulated sum is obtained by splitting it.

8. The method according to claim 1, characterized in that, The reciprocal module includes an addition unit, a linear fitting unit, and a shifting unit; The decimal parts of the mantissa and the effective elements are merged using a reciprocal module. The merged result is then subjected to piecewise linear fitting followed by a shift operation to obtain the Softmax probability value corresponding to the effective element, which is stored in a register. This includes: The addition unit combines the mantissa and the fractional part of the effective element to obtain the combined result. The merged result is input into the linear fitting unit to obtain the approximate value of the fitted function; The shift unit uses the integer part of the effective element, the global maximum element, and the mantissa of the sum as shift control parameters for the approximate value of the function corresponding to the effective element, and shifts the approximate value of the function to obtain the Softmax probability value corresponding to the effective element; wherein, the shift control parameters include shift amplitude and shift direction.

9. The method according to claim 1, characterized in that, The step of obtaining a fitting result for each effective element by performing piecewise approximation fitting on a preset natural exponential function corresponding to each effective element includes: The domain of the preset natural exponential function corresponding to each of the effective elements is divided into multiple segments, wherein the division of each segment is based on the derivative distribution characteristics of the natural exponential function and a preset scaling factor, and is dynamically divided according to preset rules. For each segment, a binary search algorithm is used to determine the farthest segment endpoint of each segment under the premise of meeting the preset error requirements; Based on the farthest segment endpoint, a greedy algorithm is used to find the minimum slope bit width of each segment to obtain the fitting parameters of each segment, which include slope and intercept. The natural exponential function is fitted based on the fitting parameters for each segment.

10. A hardware accelerator for the Softmax function, characterized in that, include: The preprocessing module is used to divide multiple initial vectors into multiple vector groups according to a predetermined number of elements, and process them to determine the maximum element in each vector group and multiple valid elements in each vector group; The maximum value update module is used to receive the maximum element in each of the vector groups and compare it to determine the global maximum element of the plurality of initial vectors, calculate the difference between the global maximum element and the maximum element in each of the vector groups, and obtain the relative offset of each vector group. The exponential module is used to perform piecewise linear fitting based on the natural exponential function on each effective element in each vector group to obtain an exponential approximation of the effective input vector. The accumulation module is used to shift the exponential approximation of each valid element according to the relative offset in the vector group to which each exponential approximation belongs, and to accumulate all the exponential approximation values ​​corresponding to all the shifted valid elements to obtain the accumulated sum. A digital normalization conversion module is used to convert the accumulated sum into a normalized floating-point form to obtain the mantissa of the accumulated sum; The reciprocal module is used to merge the mantissa and the fractional part of the effective element, perform piecewise linear fitting on the merged result and then perform a shift operation to obtain the Softmax probability value corresponding to the effective element, and store it in a register.

Images