Normalization calculation method, device and equipment based on FPGA (Field Programmable Gate Array)
By shifting and splitting the input vector into integer and fractional parts on the FPGA through subtraction, and then using lookup tables and Taylor expansion to perform exponent calculation, the problem of high-precision normalized exponent functions on FPGA is solved, and efficient normalization calculation is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- JIANGSU HIRAIN AUTOMOTIVE ELECTRONICS CO LTD
- Filing Date
- 2026-02-03
- Publication Date
- 2026-05-26
AI Technical Summary
In the existing technology, it is difficult to perform normalized exponential function (softmax function) calculation with high accuracy on FPGA, especially due to the large resource consumption and long calculation cycle caused by complex exponent and division operations.
The input vector is shifted into the negative domain by subtraction, compressing the exponent domain. The complex exponent calculation is split into integer and fractional parts, which are calculated separately. The exponent is calculated using lookup tables and Taylor expansion. Finally, the normalized result is obtained through the normalized reference value.
It simplifies the computational logic, reduces resource consumption, achieves high-precision normalized exponential function operations, and improves the computational efficiency of FPGA.
Smart Images
Figure CN122087256A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a normalization calculation method, apparatus and device based on FPGA. Background Technology
[0002] With technological advancements, deep learning models have been widely applied in multi-classification tasks such as image recognition and natural language processing. As artificial intelligence is increasingly deployed to edge computing and embedded devices, higher demands are being placed on the computational efficiency and resource consumption of the model inference process. Field-Programmable Gate Arrays (FPGAs), due to their customizable hardware logic, parallel processing capabilities, and low power consumption, have become an important platform for achieving efficient deep learning.
[0003] In existing technologies, the calculation of the normalized exponential function (softmax function) in model operation is an important process, which includes complex exponentiation and division operations. Currently, the main solutions for exponent calculation are to perform piecewise linear fitting or to perform an n-term Taylor expansion. These methods consume a lot of resources and have a long calculation cycle. On resource-constrained FPGAs, it is not possible to perform softmax function calculation with high precision. Summary of the Invention
[0004] This application provides a normalization calculation method, apparatus, and device based on FPGA, which can solve the technical problem of difficulty in performing normalized exponential function calculations with high precision.
[0005] In a first aspect, this application provides a normalization calculation method based on FPGA, the method comprising: Find the maximum value among multiple elements of the input vector, subtract the maximum value from each element of the input vector to obtain the translation value corresponding to each element; the translation value is less than or equal to 0. Extract the integer and fractional parts corresponding to each of the translation values, and determine the first result corresponding to each integer part and the second result corresponding to each fractional part; the first result is the result of exponentiation based on a preset base and the integer part; the second result is the result of exponentiation based on a preset base and the fractional part. Based on the first result and the second result corresponding to each translation value, the exponential result corresponding to the translation value is obtained; Based on the exponential results of all translation values, a normalized reference value is determined; Based on the exponential results of each translation value and the normalization reference value, the normalization results corresponding to each element in the input vector are obtained.
[0006] In some possible implementations, the step of extracting the integer and fractional parts corresponding to each of the translation values, and determining a first result corresponding to each of the integer parts and a second result corresponding to each of the fractional parts, includes: Extract the integer and fractional parts corresponding to each of the translation values, and determine the absolute value of each integer part and the absolute value of each fractional part; The absolute value of each integer part is used as the first absolute value, and the result of the operation corresponding to the exponentiation of each first absolute value based on a preset base is determined as the first result; The absolute value of each decimal part is used as the second absolute value, and the result of the exponentiation operation of each second absolute value based on a preset base is determined as the second result.
[0007] In some possible implementations, the step of using the absolute value of each of the integer parts as a first absolute value, and determining the result of the exponentiation operation corresponding to each of the first absolute values based on a preset base, as a first result, includes: The absolute value of each of the integer parts is taken as the first absolute value; Based on each of the first absolute values, a first result corresponding to each of the first absolute values is determined from a lookup table; the lookup table is a data table storing multiple mapping relationships, and the mapping relationship is the correspondence between integers and the first result.
[0008] In some possible implementations, the maximum integer value in the lookup table corresponds to a preset calculation precision, and for each first absolute value, determining the first result corresponding to each first absolute value from the lookup table includes: Based on each of the first absolute values, determine the mapping relationship in the lookup table; If the first absolute value is less than the maximum integer value, the mapping value corresponding to the first absolute value in the lookup table is taken as the first result corresponding to the first absolute value. If the first absolute value is greater than or equal to the maximum integer value, the mapping value corresponding to the maximum integer value in the lookup table is used as the first result corresponding to the first absolute value.
[0009] In some possible implementations, the step of determining the result of exponentiation of each of the second absolute values based on a preset base, using the absolute values of the integer parts and the absolute values of the fractional parts as second absolute values, as the second result, includes: The absolute value of each of the aforementioned decimal parts is taken as the second absolute value; A Taylor expansion with a preset number of expansion terms is performed on each of the second absolute values to calculate the second result corresponding to each of the fractional parts.
[0010] In some possible implementations, the Taylor expansion is pre-set to have three expansion terms.
[0011] In some possible implementations, obtaining the exponential result corresponding to the translation value based on the first result and the second result corresponding to each translation value includes: The exponential result corresponding to each translation value is obtained by multiplying the first result and the second result corresponding to each translation value.
[0012] In some possible implementations, obtaining the normalization result corresponding to each element in the input vector based on the exponential result of each translation value and the normalization reference value includes: Convert the exponential results of each translation value to floating-point numbers; Based on the floating-point divider, each of the exponent results is divided by the normalized reference value to obtain the corresponding calculation result; Each of the calculation results is converted into a fixed-point number to obtain the normalized result corresponding to each element in the input vector.
[0013] Secondly, this application provides a normalization computing device based on FPGA, the device comprising: The search module is used to find the maximum value among multiple elements of the input vector, subtract the maximum value from each element of the input vector to obtain the translation value corresponding to each element; the translation value is less than or equal to 0. The extraction module is used to extract the integer part and the fractional part corresponding to each of the translation values, and to determine the first result corresponding to each of the integer parts and the second result corresponding to each of the fractional parts; the first result is the result of the exponentiation operation based on a preset base and the integer part; the second result is the result of the exponentiation operation based on a preset base and the fractional part. The calculation module is used to obtain the exponential result corresponding to the translation value based on the first result and the second result corresponding to each translation value; The determination module is used to determine the normalized reference value based on the exponential results of all translation values; The normalization module is used to obtain the normalization result corresponding to each element in the input vector based on the exponential result of each translation value and the normalization reference value, respectively.
[0014] Thirdly, this application provides a normalization computing device based on FPGA, the device comprising: a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the FPGA-based normalization computing method described above.
[0015] Fourthly, this application provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the FPGA-based normalization calculation method described above.
[0016] Fifthly, this application provides a computer program product in which the instructions are executed by the processor of an electronic device, causing the electronic device to perform the FPGA-based normalization calculation method described above.
[0017] The FPGA-based normalization calculation method, apparatus, and device provided in this application utilize subtraction and translation to bring the input vector into the negative value domain, compressing the exponent value domain and eliminating the risk of overflow. Furthermore, the complex exponent calculation is broken down into integer and fractional parts, which are calculated separately to complete the final exponent calculation and obtain the exponent result. Then, a normalization reference value is determined based on all the exponent results, and each exponent result is divided by the normalization reference value to obtain the normalized result. This simplifies the calculation logic, reduces resource consumption, and enables high-precision normalized exponent function calculation. Attached Figure Description
[0018] This application can be better understood from the following description of specific embodiments in conjunction with the accompanying drawings, wherein: Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings, wherein the same or similar reference numerals denote the same or similar features.
[0019] Figure 1 This is a flowchart illustrating a normalization calculation method based on FPGA provided in one embodiment of this application; Figure 2 This is a flowchart illustrating a normalization calculation method based on FPGA provided in another embodiment of this application; Figure 3 This is a flowchart illustrating a normalization calculation method based on FPGA provided in another embodiment of this application; Figure 4 This is a schematic diagram of a fixed-point exponent calculation process provided in one embodiment of this application; Figure 5 This is a schematic diagram of the structure of a FPGA-based normalized computing device provided in one embodiment of this application; Figure 6 This is a schematic diagram of the hardware structure of the FPGA-based normalized computing device provided in the embodiments of this application. Detailed Implementation
[0020] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.
[0021] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.
[0022] To address the problems of existing technologies, embodiments of this application provide a normalization calculation method, apparatus, and device based on FPGA. The FPGA-based normalization calculation method provided in this application will be described first.
[0023] Figure 1 This illustration shows a flowchart of a normalization calculation method based on an FPGA according to an embodiment of this application. This method is applied to a corresponding FPGA, such as... Figure 1 As shown, the method may include the following steps: S101 to S105.
[0024] S101: Find the maximum value among multiple elements of the input vector, subtract the maximum value from each element in the input vector, and obtain the translation value corresponding to each element. The translation value is less than or equal to 0.
[0025] An input vector can be converted into a set containing multiple elements, each of which can correspond to a specific numerical value.
[0026] In the specific implementation, the element with the largest value among multiple elements in the input vector is found and its value is taken as the maximum value. This maximum value is then subtracted from all elements in the input vector to obtain the translation value of each element. This maximum value ensures that the translation value of each element will not be greater than 0, because after subtracting the maximum value, the position of other elements relative to the maximum value will be less than or equal to 0.
[0027] S102: Extract the integer and fractional parts corresponding to each translation value, and determine the first result corresponding to each integer part and the second result corresponding to each fractional part. The first result is the result of the exponentiation operation based on the preset base and the integer part; the second result is the result of the exponentiation operation based on the preset base and the fractional part.
[0028] In the specific implementation, for the translation value of each element, its integer part and decimal part need to be extracted separately. Then, a first result and a second result are calculated based on the two parts respectively. The integer part is processed according to a first preset rule to obtain the first result; the decimal part is processed according to a second preset rule to obtain the second result.
[0029] S103: Based on the first and second results corresponding to each translation value, obtain the exponential result corresponding to the translation value.
[0030] In practice, the exponential result is calculated from the first and second results of the translation value using a preset mathematical formula, which can be obtained by weighted summation or other combinations.
[0031] S104: Determine the normalized reference value based on the exponential results of all translation values.
[0032] The normalized reference value is a benchmark value used for subsequent standardization calculations, which can be determined by the maximum or minimum value of the index result or statistical value.
[0033] In practice, a reference value is calculated using the exponential results of all translation values to standardize the benchmark for subsequent calculations. Specifically, the minimum and maximum values of the data can be mapped to a specific range, and then the standard deviation and mean of the translation values can be calculated to obtain the normalized reference value.
[0034] S105: Based on the exponential results of each translation value and the normalization reference value, obtain the normalization results corresponding to each element in the input vector.
[0035] In the specific implementation, the final normalized result is obtained by calculating the exponential results of each translation value and the normalization reference value, for example, by dividing each exponential result by the normalization reference value, or by adjusting the magnitude of each value through addition, subtraction, multiplication, and division operations, so that the final output completes the normalization result.
[0036] The FPGA-based normalization calculation method provided in this application utilizes subtraction and shifting to bring the input vector into the negative value domain, compressing the exponent range and eliminating overflow risk. It then breaks down the complex exponent calculation into integer and fractional parts, performing separate calculations for each. This completes the final exponent calculation, yielding the exponent result. Subsequently, a normalization reference value is determined based on all the exponent results, and each exponent result is divided by the normalization reference value to obtain the normalized result. This simplifies the calculation logic, reduces resource consumption, and enables high-precision normalized exponent function calculation.
[0037] In order to improve the computing efficiency of FPGA, in some implementations, S102 may include the following steps: S201 to S202.
[0038] S201: Extract the integer and decimal parts corresponding to each translation value, and determine the absolute value of each integer part and the absolute value of each decimal part.
[0039] In the specific implementation, each shift value is decomposed into an integer part and a fractional part. After extracting the integer part and the fractional part, their absolute values are calculated separately. For example, for a binary number, if the sign bit is 0, both the fractional part and the integer part are 0; if the sign bit is 1, the integer part discards the sign bit and is inverted, and the fractional part is inverted and incremented by 1 to obtain the absolute value.
[0040] S202: Using the absolute value of each of the integer parts as the first absolute value, determine the operation result corresponding to the exponentiation operation of each of the first absolute values based on a preset base, and use it as the first result.
[0041] In the specific implementation, based on the absolute value of the extracted and calculated integer part, the result of the exponentiation operation with a preset base and the first absolute value as the exponent is determined, thereby determining the first result corresponding to each integer part.
[0042] S203: Using the absolute value of each of the decimal parts as the second absolute value, determine the operation result corresponding to the exponentiation operation of each of the second absolute values based on a preset base, and use it as the second result.
[0043] In the specific implementation, for the absolute value of the decimal part, the result of the exponentiation operation with a preset base and a second absolute value as the exponent is determined to obtain the second result corresponding to each decimal part.
[0044] The FPGA-based normalization calculation method provided in this application extracts the integer and fractional parts corresponding to each translation value, determines the absolute value of each integer part and the absolute value of each fractional part, performs classification processing for different numerical characteristics, avoids negative number operations through absolute values, and improves the computing efficiency of FPGA.
[0045] In order to perform rapid exponent calculation, in some implementations, S202 may include the following steps: S301: The absolute value of each of the integer parts is taken as the first absolute value.
[0046] In the specific implementation, the first absolute value refers to the result obtained after the integer part has undergone the absolute value operation.
[0047] S302: Based on each of the first absolute values, determine the first result corresponding to each of the first absolute values from the lookup table; the lookup table is a data table storing multiple mapping relationships, and the mapping relationship is the correspondence between integers and the first result.
[0048] In its implementation, a Look-Up Table (LUT) is a data structure that can quickly find the corresponding output result based on a given input. In this step, the lookup table maps the first absolute value of the integer part to the corresponding first result. By inputting the first absolute value of the integer part, the lookup table returns the corresponding value, which is the first result.
[0049] The FPGA-based normalization calculation method provided in this application improves the computational efficiency of the algorithm by using a lookup table, which replaces complex calculations with a direct lookup process. The lookup table is pre-built, suitable for fast retrieval, and can perform rapid exponential calculations when processing large amounts of data.
[0050] In order to reduce the storage space required by the lookup table, in some implementations, the maximum integer value in the lookup table corresponds to a preset calculation precision. S302 may include the following steps: S401 to S403.
[0051] S401: Determine the mapping relationship in the lookup table based on each of the first absolute values.
[0052] In the actual implementation, the mapping relationship in the corresponding lookup table is found based on the absolute value of each integer part, thus avoiding repeated calculations each time.
[0053] S402: If the first absolute value is less than the maximum integer value, the mapping value corresponding to the first absolute value in the lookup table is taken as the first result corresponding to the first absolute value.
[0054] In the specific implementation, the absolute value of each integer part is compared with the preset maximum integer value. If the absolute value of the integer part is less than the maximum integer value, the corresponding mapping value can be directly obtained from the lookup table. Then, the corresponding mapping value in the lookup table is directly used as the first result corresponding to the integer part.
[0055] S403: If the first absolute value is greater than or equal to the maximum integer value, the mapping value corresponding to the maximum integer value in the lookup table is used as the first result corresponding to the first absolute value.
[0056] In the actual implementation, if the absolute value of the integer part is greater than or equal to the maximum integer value, it may not be possible to find the corresponding mapping value directly in the lookup table. In this case, the mapping value corresponding to the maximum integer value in the lookup table will be directly used as the first result corresponding to the integer part.
[0057] The FPGA-based normalization calculation method provided in this application can avoid storing data exceeding the calculation precision in the lookup table by setting a limit on the maximum integer value in the lookup table, thereby reducing the storage space required by the lookup table.
[0058] To improve computational efficiency, in some implementations, S203 may include the following steps: S501 to S502.
[0059] S501: The absolute value of each of the decimal parts is used as the second absolute value.
[0060] S502: Perform a Taylor expansion with a preset number of expansion terms on each of the second absolute values to calculate the second result corresponding to each of the fractional parts.
[0061] In the specific implementation, based on the second absolute value of the fractional part of the translation value, a Taylor expansion is performed to approximate the second result corresponding to the fractional part. The Taylor expansion approximates the calculation by representing the function as a polynomial expansion around a certain point. Here, the second result is calculated using the Taylor expansion formula. For example, a Taylor expansion can be used for approximate expansion with a finite number of terms. The number of terms determines the calculation accuracy. In practical calculations, a third-order Taylor expansion can be used to balance calculation speed and accuracy. The calculation result obtained based on the Taylor expansion is taken as the second result for the fractional part.
[0062] The FPGA-based normalization calculation method provided in this application can quickly calculate the fractional part while maintaining high precision through Taylor expansion. It can avoid excessive computational complexity and improve computational efficiency while maintaining reasonable precision.
[0063] In order to accurately obtain the third result, in some implementations, S103 may include the following step: S601.
[0064] S601: Based on the product of the first and second results corresponding to each translation value, obtain the exponential result corresponding to each translation value.
[0065] In the specific implementation, a first result and a second result associated with each translation value are obtained. These two results are multiplied, and the product is used as the exponent result corresponding to the translation value. Since the first result and the second result are the exponent operation results of the fractional part and integer part of the translation value, respectively, their product is the exponent operation result corresponding to the translation value.
[0066] The FPGA-based normalization calculation method provided in this application combines the product of the first result and the second result to merge the results of the two calculation processes and accurately derive the third result.
[0067] To obtain more accurate calculation results, in some implementations, reference is made to... Figure 2 The exponent result is a fixed point number, and S105 may include the following steps: S701 to S703.
[0068] S701: Converts the exponent results of each translation value into floating-point numbers.
[0069] Fixed-point numbers are a way of representing numerical values in computers where the position of the decimal point is fixed. Floating-point numbers, on the other hand, are a more flexible representation method that allows the decimal point to move freely depending on the size of the value, and can be used to represent a wider range of decimal values.
[0070] In the specific implementation, the exponential result in fixed-point form is converted into a floating-point number through a preset conversion method.
[0071] S702: Based on the floating-point divider, divide each of the exponent results by the normalized reference value to obtain the corresponding calculation result.
[0072] In the specific implementation, after obtaining the normalization reference value, a floating-point divider is used to perform division operations on the exponent results of each floating-point number. Each exponent result is divided by the normalization reference value to obtain the corresponding calculation result, thereby normalizing the range of all exponent results.
[0073] S703: Convert each of the calculation results into fixed-point numbers to obtain the normalized result corresponding to each element in the input vector.
[0074] In the specific implementation, after the calculation is completed, the calculation results are converted from floating-point format back to fixed-point number, and all elements in the input vector are uniformly standardized, with the size of each element being normalized to a reasonable range, thus obtaining the normalized result corresponding to each element in the input vector.
[0075] The FPGA-based normalization calculation method provided in this application first converts all exponential results into floating-point numbers, performs summation and division operations using floating-point numbers, and converts fixed-point numbers into floating-point numbers for summation and normalization. This can effectively avoid the precision loss problem in fixed-point number operations. Using floating-point number division speeds up the processing and yields more accurate calculation results.
[0076] In another implementation, to address the difficulty of implementing the Softmax function for fixed-point decimal calculation on FPGA, a feasible design method is proposed. By separating the integer part and the fractional part and using lookup tables and Taylor expansion respectively to complete the exponent calculation, the calculation logic can be simplified, resource consumption can be reduced, the depth required for lookup tables can be reduced, and high-precision calculations can be performed on this basis.
[0077] Specifically, the formula for calculating the Softmax function is: The FPGA used specifically includes: an input buffer module, a maximum value lookup module, a subtraction shift module, an exponentiation calculation module, an accumulation and summation module, a division normalization module, and an output buffer module. The calculation flow in the FPGA is as follows: Figure 3 As shown, the steps include: S1 to S7.
[0078] S1: Receives the input vector from the preceding fully connected layer via the Input Buffer Module. This module is configured with a register array to buffer the input data. The input vector elements can be fixed-point numbers of arbitrary bit width; in specific implementations, its format can be defined as 1 sign bit, 7 integer bits, and 8 fractional bits (i.e., 1.7Q8 format).
[0079] S2: Max Finder Module, consisting of a parallel comparator and tree-structured selection logic. This module receives vector data output from the input buffer module and quickly calculates the maximum value of all elements in the input vector using a hardware parallel comparison circuit.
[0080] S3: Subtraction & Offset Module, containing a set of parallel fixed-point subtractors. This module receives the output vector from the input buffer module and the maximum value output from the maximum value lookup module, and subtracts this maximum value from each element of the input vector in parallel. This operation implements the shifting of the input data in hardware, restricting the input for subsequent exponent calculations to a range ≤0, thereby constraining the value range of the exponent calculation result to the interval (0, 1], effectively avoiding the fixed-point number overflow problem caused by the excessively large range of the exponent result in subsequent calculations.
[0081] S4: Exponentiation Module. This module uses a lookup table (LUT) to perform the calculation. It receives the translated element values output by the subtraction translation module and calculates the exponential function result of each element. Since the value range of the exponential result of non-positive numbers is limited to the interval (0,1], the calculation result can be quantized into a 16-bit fixed-point decimal format to meet the calculation requirements.
[0082] S5: Accumulation Module, containing a pipelined addition tree. This module receives the exponents of all elements output by the exponent calculation module, pipelines them together to obtain the exponent sum. This sum is quantized into a fixed-point format with 2 integer bits and 16 decimal bits (i.e., 0.2Q16 format) to meet the dynamic range requirements of the accumulation sum.
[0083] S6: The Division & Normalization Module contains a pipelined fixed-point divider. This module receives the individual exponents from the exponent calculation module and the sum of exponents from the accumulation and summing module, and divides each exponent by the sum of exponents in parallel. This step performs the normalization operation in hardware, ensuring that the sum of all elements in the output vector is 1, thus obtaining the distribution representing the class probability. The calculation result is quantized into a 16-bit fixed-point decimal format.
[0084] S7: Output Buffer Module, which receives the normalized result vector (i.e., probability vector) output by the division normalization module, buffers it, and outputs it to subsequent processing units or external interfaces.
[0085] Specifically, for S4, you can refer to... Figure 4 First, separate the integer digits of the fixed-point number. and decimal places Since the maximum value was subtracted in S3, this fixed-point number must be negative or 0.
[0086] Next, the absolute values of the fractional and integer parts are extracted to facilitate the subsequent lookup table and Taylor expansion calculations. Specifically, the sign bit is checked first. If the sign bit is 0, both the fractional and integer parts are 0; if the sign bit is 1, the integer part is discarded and inverted, and the fractional part is inverted and incremented by 1 (if the fractional part is 0, it indicates a carry-over to the integer part, in which case the integer part should also be incremented by 1). Through this process, the absolute values of the integer and fractional parts of the input data can be obtained, respectively. and .
[0087] Calculating the exponent for the integer part requires a lookup table, which stores the function. Given that the input is an 8-bit integer and the output is an 8-bit binary decimal, the lookup table is shown in Table 1: Table 1 When the input data is 00000001, the output data is 01011110, which is 0.3671875 in decimal. This completes the process. The calculation. Furthermore, by Figure 3 As we can see, when the address is 6, the result exceeds the precision limit of 8 bits, so the result is set to 0. Therefore, a larger address index is meaningless. Thus, a lookup table depth of 6 is sufficient. When the input integer is greater than 6, it is regarded as the address of the lookup table being 6, which greatly reduces the resources occupied by the lookup table. The specific depth of the lookup table is determined by the precision limit that the result decimal places can express.
[0088] The calculation of the exponent in the decimal part is accomplished by Taylor expansion, and the formula is as follows: In a preferred embodiment, the approximate calculation of the exponential function in this step uses a Taylor series expansion up to the third term. When the input range is in the interval [-1,0], the fitting accuracy of this third-order Taylor expansion meets the requirements. This step uses an 8-bit*8-bit fixed-point multiplier, and the output result is 16-bit decimal places, retaining the first 8 bits as the decimal part of the output.
[0089] Next, multiply the exponent of the integer part by the exponent of the fractional part, specifically: in, .
[0090] Since it involves multiplying 8 bits by 8 bits, this step can reuse the multiplier mentioned above, further reducing resource consumption. The final output is a 16-bit decimal fixed-point number, which is the output of the exponent calculation. When the third term of the Taylor expansion is used, the error between the result calculated by this method and the original function is that the range of x is always negative. When x < -10, the result of both the Taylor expansion and the original function approaches 0, so it is not considered.
[0091] For the fixed-point division calculation in S6 above, because the resource consumption of a fixed-point divider is higher than that of a floating-point divider in an FPGA, one embodiment can use a fixed-point to floating-point conversion, followed by a floating-point divider, and finally a conversion back to fixed-point. Specifically, the 16-bit fixed-point decimal output from the exponent calculation module and the sum of the exponents (2 integer bits and 16 decimal bits) are both expanded to 16 integer bits. This 16-bit decimal fixed-point number is then input to the fixed-point to floating-point conversion module, which outputs a 24-bit floating-point number. The exponent result converted to a 24-bit floating-point number is used as the dividend, and the sum of the exponents is used as the divisor. These are then input to the floating-point divider to calculate the 24-bit floating-point division result. Then, the division result is input into the floating-to-fixed module, which outputs the division result in 16 bits of integer and 16 bits of decimal. The normalized result must be less than or equal to 1. When it is equal to 1, the output result is set to all 1s in 16 bits. When the result is less than 1, the last 16 bits of data are directly truncated to obtain the final softmax result of the 16-bit fixed-point decimal.
[0092] Based on the FPGA-based normalization calculation method provided in the above embodiments, this application also provides specific implementation methods of an FPGA-based normalization calculation device. Please refer to the following embodiments.
[0093] First see Figure 5 The FPGA-based normalized computing device 500 provided in this application embodiment includes the following modules: The search module 501 is used to find the maximum value among multiple elements of the input vector, and subtract the maximum value from each element in the input vector to obtain the translation value corresponding to each element. The translation value is less than or equal to 0.
[0094] The extraction module 502 is used to extract the integer part and the fractional part corresponding to each of the translation values, and to determine the first result corresponding to each of the integer parts and the second result corresponding to each of the fractional parts respectively; the first result is the result of the exponentiation operation based on the preset base and the integer part; the second result is the result of the exponentiation operation based on the preset base and the fractional part.
[0095] The calculation module 503 is used to obtain the exponential result corresponding to the translation value based on the first result and the second result corresponding to each translation value.
[0096] Module 504 is used to determine a normalized reference value based on the exponential results of all translation values.
[0097] The normalization module 505 is used to obtain the normalization result corresponding to each element in the input vector based on the exponential result of each translation value and the normalization reference value.
[0098] As one implementation of this application, the extraction module 502 includes: The extraction unit is used to extract the integer and fractional parts corresponding to each translation value, and to determine the absolute value of each integer part and the absolute value of each fractional part.
[0099] The determining unit is used to determine the operation result corresponding to the exponentiation operation of each of the integer parts based on a preset base, using the absolute value of each of the integer parts as the first absolute value, and using the result as the first result.
[0100] The determining unit is further configured to use the absolute value of each of the decimal parts as the second absolute value, and determine the operation result corresponding to the exponentiation operation of each of the second absolute values based on a preset base, as the second result.
[0101] As one implementation of this application, the defined unit includes: A subunit is determined for using the absolute value of each of the integer parts as the first absolute value.
[0102] The subunit is further configured to determine, based on each of the first absolute values, the first result corresponding to each of the first absolute values from a lookup table; the lookup table is a data table storing multiple mapping relationships, and the mapping relationship is the correspondence between an integer and the first result.
[0103] As one implementation of this application, the maximum integer value in the lookup table corresponds to a preset calculation precision to determine the sub-unit, including: A sub-unit is determined for determining the mapping relationship in the lookup table based on each of the first absolute values.
[0104] The determined subunit is further configured to, when the first absolute value is less than the maximum integer value, take the mapping value corresponding to the first absolute value in the lookup table as the first result corresponding to the first absolute value.
[0105] The determined subunit is further configured to, when the first absolute value is greater than or equal to the maximum integer value, use the mapping value corresponding to the maximum integer value in the lookup table as the first result corresponding to the first absolute value.
[0106] As one implementation of this application, the FPGA-based normalization computing device 500 includes: A determining module is used to take the absolute value of each of the decimal parts as the second absolute value.
[0107] The calculation module is used to perform a Taylor expansion of each of the second absolute values with a preset number of expansion terms, and calculate the second result corresponding to each of the fractional parts.
[0108] As one implementation of this application, the computing module includes: The determining unit is used to obtain the exponential result corresponding to each translation value based on the product of the first result and the second result corresponding to each translation value.
[0109] As one implementation of this application, the exponent result is a fixed-point number, and the normalization module 505 includes: The conversion unit is used to convert the exponential result of each translation value into a floating-point number.
[0110] The calculation unit is used to divide each of the exponent results by the normalized reference value based on a floating-point divider to obtain the corresponding calculation result.
[0111] The conversion unit is also used to convert each of the calculation results into fixed-point numbers to obtain the normalized result corresponding to each element in the input vector.
[0112] Each module in the FPGA-based normalization calculation device provided in this application embodiment can implement each step in the above-mentioned FPGA-based normalization calculation method and achieve the corresponding effect. For the sake of brevity, it will not be described in detail here.
[0113] Figure 6 A schematic diagram of the structure of the FPGA-based normalized computing hardware provided in an embodiment of this application is shown.
[0114] An FPGA-based normalized computing device may include a processor 601 and a memory 602 storing computer program instructions.
[0115] Specifically, the processor 601 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.
[0116] Memory 602 may include mass storage for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 602 may include removable or non-removable (or fixed) media. Where appropriate, memory 602 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 602 is non-volatile solid-state memory.
[0117] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the FPGA-based normalized computation method according to any embodiment of this disclosure.
[0118] The processor 601 reads and executes computer program instructions stored in the memory 602 to implement any of the FPGA-based normalization calculation methods in the above embodiments.
[0119] In one example, the FPGA-based normalized computing device may further include a communication interface 603 and a bus 610. Wherein, as Figure 6 As shown, the processor 601, memory 602, and communication interface 603 are connected through bus 610 and complete communication with each other.
[0120] The communication interface 603 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.
[0121] Bus 610 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 610 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.
[0122] Furthermore, in conjunction with the FPGA-based normalization calculation method described in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the FPGA-based normalization calculation methods described in the above embodiments.
[0123] This application also provides a computer program product, including a computer program that, when executed, implements any of the FPGA-based normalization calculation methods described in the above embodiments.
[0124] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.
[0125] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.
[0126] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.
[0127] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that the various blocks in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to create a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that the various blocks in the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.
[0128] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.
Claims
1. A normalization calculation method based on FPGA, characterized in that, The method includes: Find the maximum value among multiple elements of the input vector, subtract the maximum value from each element of the input vector to obtain the translation value corresponding to each element; the translation value is less than or equal to 0. Extract the integer and fractional parts corresponding to each of the translation values, and determine the first result corresponding to each integer part and the second result corresponding to each fractional part; the first result is the result of exponentiation based on a preset base and the integer part; the second result is the result of exponentiation based on a preset base and the fractional part. Based on the first result and the second result corresponding to each translation value, the exponential result corresponding to the translation value is obtained; Based on the exponential results of all translation values, a normalized reference value is determined; Based on the exponential results of each translation value and the normalization reference value, the normalization results corresponding to each element in the input vector are obtained.
2. The FPGA-based normalization calculation method according to claim 1, characterized in that, The step of extracting the integer and fractional parts corresponding to each of the translation values, and determining the first result corresponding to each integer part and the second result corresponding to each fractional part, includes: Extract the integer and fractional parts corresponding to each of the translation values, and determine the absolute value of each integer part and the absolute value of each fractional part; The absolute value of each integer part is used as the first absolute value, and the result of the operation corresponding to the exponentiation of each first absolute value based on a preset base is determined as the first result; The absolute value of each decimal part is used as the second absolute value, and the result of the exponentiation operation of each second absolute value based on a preset base is determined as the second result.
3. The FPGA-based normalization calculation method according to claim 2, characterized in that, The step of using the absolute value of each of the integer parts as the first absolute value, and determining the operation result corresponding to the exponentiation operation of each of the first absolute values based on a preset base, as the first result, includes: The absolute value of each of the integer parts is taken as the first absolute value; Based on each of the first absolute values, a first result corresponding to each of the first absolute values is determined from a lookup table; the lookup table is a data table storing multiple mapping relationships, and the mapping relationship is the correspondence between integers and the first result.
4. The FPGA-based normalization calculation method according to claim 3, characterized in that, The maximum integer value in the lookup table corresponds to a preset calculation precision. For each of the first absolute values, the first result corresponding to each first absolute value is determined from the lookup table, including: Based on each of the first absolute values, determine the mapping relationship in the lookup table; If the first absolute value is less than the maximum integer value, the mapping value corresponding to the first absolute value in the lookup table is taken as the first result corresponding to the first absolute value. If the first absolute value is greater than or equal to the maximum integer value, the mapping value corresponding to the maximum integer value in the lookup table is used as the first result corresponding to the first absolute value.
5. The FPGA-based normalization calculation method according to claim 2, characterized in that, The step of using the absolute value of each of the decimal parts as the second absolute value, and determining the result of the exponentiation operation corresponding to each of the second absolute values based on a preset base, as the second result, includes: The absolute value of each of the aforementioned decimal parts is taken as the second absolute value; A Taylor expansion with a preset number of expansion terms is performed on each of the second absolute values to calculate the second result corresponding to each of the fractional parts.
6. The FPGA-based normalization calculation method according to claim 5, characterized in that, The Taylor expansion is pre-set to have three expansion terms.
7. The FPGA-based normalization calculation method according to claim 1, characterized in that, The step of obtaining the exponential result corresponding to the translation value based on the first result and the second result corresponding to each translation value includes: The exponential result corresponding to each translation value is obtained by multiplying the first result and the second result corresponding to each translation value.
8. The FPGA-based normalization calculation method according to any one of claims 1 to 6, characterized in that, The exponent result is a fixed-point number. The process of obtaining the normalization result corresponding to each element in the input vector based on the exponent results of each translation value and the normalization reference value includes: Convert the exponential results of each translation value to floating-point numbers; Based on the floating-point divider, each of the exponent results is divided by the normalized reference value to obtain the corresponding calculation result; Each of the calculation results is converted into a fixed-point number to obtain the normalized result corresponding to each element in the input vector.
9. A normalized computing device based on FPGA, characterized in that, The device includes: The search module is used to find the maximum value among multiple elements of the input vector, subtract the maximum value from each element of the input vector to obtain the translation value corresponding to each element; the translation value is less than or equal to 0. The extraction module is used to extract the integer part and the fractional part corresponding to each of the translation values, and to determine the first result corresponding to each of the integer parts and the second result corresponding to each of the fractional parts; the first result is the result of the exponentiation operation based on a preset base and the integer part; the second result is the result of the exponentiation operation based on a preset base and the fractional part. The calculation module is used to obtain the exponential result corresponding to the translation value based on the first result and the second result corresponding to each translation value; The determination module is used to determine the normalized reference value based on the exponential results of all translation values; The normalization module is used to obtain the normalization result corresponding to each element in the input vector based on the exponential result of each translation value and the normalization reference value, respectively.
10. A normalized computing device based on FPGA, characterized in that, The device includes: a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the FPGA-based normalized calculation method as described in any one of claims 1-8.