Accelerated operation method and device suitable for neural network normalization calculation
By accelerating normalization calculation through pipelined concurrent computation and interpolation fitting, the problem of cumbersome and time-consuming steps in the existing technology is solved, and efficient normalization calculation is achieved.
Patent Information
- Application Number
- CN202411501566.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-25
- Publication Date
- 2025-11-14
- Estimated Expiration
- 2044-10-25
AI Technical Summary
In existing technologies, the normalization calculation steps are numerous, time-consuming, and inefficient, making it difficult to meet practical needs.
A pipelined concurrent computation approach is adopted, which loads and computes the normalized intermediate array in parallel through multiple data channels, including the calculation of intermediate numbers of the numerator and denominator. Interpolation fitting method is used to accelerate the square root reciprocal calculation and reduce hardware overhead.
It improves the efficiency of normalized calculations, shortens the calculation time, and reduces hardware costs.
Smart Images
Figure CN119558365B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method and apparatus for accelerating operation, and more particularly to a method and apparatus for accelerating operation suitable for normalized computation of neural networks. Background Technology
[0002] Normalization is a common data preprocessing method that maps data to decimals between [0, 1] or [-1, 1], making it easier for the model to converge to the optimal solution. Therefore, before constructing a machine learning or deep learning model, it is necessary to normalize the data, which not only speeds up the solution process but may also improve the model's accuracy.
[0003] Currently, normalization typically occurs at the software level, where the hardware directly calculates each intermediate result. This involves progressively calculating multiplication, addition, square roots, and division, then multiplication again, and so on, until the entire normalization operation is completed. Those skilled in the art will recognize that this current normalization method involves numerous steps, is time-consuming, and inefficient, making it difficult to meet practical normalization needs. Summary of the Invention
[0004] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method and apparatus for accelerating the normalization calculation of neural networks, which can effectively accelerate the normalization calculation, improve the efficiency of the normalization calculation, and reduce the hardware cost of the normalization calculation.
[0005] According to the technical solution provided by the present invention, an accelerated operation method suitable for neural network normalization calculation is provided, the accelerated operation method comprising:
[0006] When performing normalization calculations within a neural network, N data channels are configured to load channel data groups respectively, and a normalization calculation source dataset is generated based on the N channel data groups.
[0007] When performing normalization calculations on any source data within the normalization calculation source dataset, a pipelined concurrent calculation method is used to generate a normalization calculation intermediate array, and the number of normalization calculation principal items is calculated based on the normalization calculation intermediate array.
[0008] The normalized calculation intermediate array includes the intermediate values of the numerator and the intermediate values of the denominator.
[0009] The numerator intermediates include either the first numerator intermediate or the second numerator intermediate corresponding to the current source data. The first numerator intermediate is calculated based on the mean difference variation, and the second numerator intermediate and the first numerator intermediate are consistent with the product of the learnable parameters.
[0010] The denominator item number is generated based on the middle number of the denominator items. When generating the denominator item number, at least the reciprocal of the square root of the middle number of the denominator items is calculated.
[0011] Multiply the intermediate number of the numerator with the standard number of the denominator to generate a normalized base for calculating the principal term after the multiplication operation;
[0012] Based on the normalized calculation cardinality of the principal terms, the normalized calculation principal term number is generated;
[0013] The normalized primary term and the normalized secondary term are added together, and the result of the addition is used as the normalized result of the current source data.
[0014] When using a pipelined concurrent computation method to generate the normalized intermediate computation array, at least two channel data group loading processes must be configured, among which,
[0015] Each time a channel data group is loaded, a corresponding normalized calculation source dataset is generated;
[0016] The intermediate values of the numerator terms are calculated using one normalized source dataset, and the intermediate values of the denominator terms are calculated using another normalized source dataset.
[0017] When calculating the middle part of the denominator, the following is included:
[0018] For each of the N data channels loaded with channel data groups, calculate the cumulative sum of the channel data and the cumulative sum of the squared channel data for each corresponding channel data group.
[0019] Add the sums of the corresponding channel data for N data channels to calculate the sum of the source data.
[0020] Add the sum of squares of the corresponding channel data for N data channels to calculate the sum of squares of the source data;
[0021] Based on the cumulative sum of the source data and the cumulative sum of the squares of the source data, the median of the denominator is calculated, wherein the median of the denominator is... This is the sum of the source data. x is the sum of squared values of the source data, n is the number of source data points in the normalized dataset, and x is the sum of squared values of the source data. i ε is a constant and is used to normalize the i-th source data in the source dataset.
[0022] When performing the square root reciprocal operation on the middle number of the denominator, an interpolation fitting method is used to perform iterative calculations on the middle number of the denominator to generate the denominator index after the iterative calculations.
[0023] Each time a channel data group is loaded, N data channels are configured to load the corresponding channel data group in parallel.
[0024] When the median of the numerator is the first median of the numerator, for the first median of the numerator calculated based on the mean difference variation, we have:
[0025]
[0026] Where n is the number of source data points in the source dataset for normalization calculation, and x a For the current source data, This is the sum of the source data.
[0027] When the median of the numerator is the first median of the numerator, when calculating the normalized principal term, we have:
[0028] When multiplying the middle number of the numerator and the item number of the denominator, the first middle number of the numerator is multiplied by the item number of the denominator to generate the normalized calculation base number of the principal term. Then, the normalized calculation base number of the principal term is multiplied by the learnable parameter to generate the normalized calculation principal term number.
[0029] or,
[0030] When multiplying the intermediate number of the numerator and the item number of the denominator, the first intermediate number of the numerator is first multiplied by the learnable parameter, and the result is configured as the second intermediate number of the numerator. Then, the second intermediate number of the numerator is multiplied by the item number of the denominator to generate the normalized calculation principal term base, and the normalized calculation principal term base is used as the normalized calculation principal term number.
[0031] When the median of the numerator is the second median of the numerator, the calculation of the second median of the numerator includes:
[0032] First, the first intermediate number of the numerator is calculated based on the mean difference variation. Then, the first intermediate number of the numerator is multiplied by the learnable parameter, and the result of the multiplication is configured as the second intermediate number of the numerator.
[0033] or,
[0034] First, the learnable parameter is multiplied by each source data. Then, the second intermediate number of the numerator is calculated based on the mean difference transformation.
[0035] When calculating the normalized principal term number, we have:
[0036] The second intermediate number of the numerator is multiplied by the item number of the denominator to generate the normalized calculation base number of the principal term, and the normalized calculation base number of the principal term is used as the normalized calculation principal term number.
[0037] It also includes data format conversion processing, among which,
[0038] Data format conversion processing includes:
[0039] The normalization calculation source dataset is converted to generate target format data corresponding to each source data. The normalization calculation target format dataset is generated based on all target format data, wherein the data bit width of the target format data is consistent with the data bit width of the source data.
[0040] Based on the normalized calculation target format dataset, when calculating and generating the corresponding normalized calculation intermediate array, normalized calculation principal term number and normalized calculation result value, the data bit width of the numerator intermediate number, denominator intermediate number, normalized calculation principal term number and normalized calculation result value are consistent with the data bit width of the target format data.
[0041] The normalized calculation result obtained above is processed for data format output to generate the normalized calculation result value in the target output format.
[0042] An accelerated operation device suitable for neural network normalization calculation includes an accelerated operation processor, wherein at least one neural network is deployed within the accelerated operation processor. When performing normalization calculation within the neural network, for any source data in the normalization calculation source dataset, the accelerated operation processor performs normalization calculation using the accelerated operation method described above and generates a normalization operation result value corresponding to the current source data.
[0043] The advantages of this invention are as follows: For the source data in the normalization calculation, multiple data channels are used for parallel loading, and a pipelined concurrent calculation method is employed to generate the intermediate normalization calculation array, thereby improving data throughput and computational efficiency. During the normalization calculation, the source data is transmitted and loaded twice, which significantly reduces the hardware area consumption for retaining input and intermediate variables compared to existing normalization calculation methods. The pipelined concurrent calculation method reduces the computation time, thus improving the efficiency of the normalization calculation and lowering the hardware overhead cost. Attached Figure Description
[0044] Figure 1 This is a schematic diagram of one embodiment of the accelerated operation of the present invention.
[0045] Figure 2 This is a schematic diagram of an embodiment of the present invention based on accelerated operation during pipeline concurrency. Detailed Implementation
[0046] The present invention will be further described below with reference to specific accompanying drawings and embodiments.
[0047] To effectively accelerate normalization calculations and improve their efficiency, this invention provides an accelerated operation method suitable for neural network normalization calculations. Specifically, the accelerated operation method includes:
[0048] When performing normalization calculations within a neural network, N data channels are configured to load channel data groups respectively, and a normalization calculation source dataset is generated based on the N channel data groups.
[0049] When performing normalization calculations on any source data within the normalization calculation source dataset, a pipelined concurrent calculation method is used to generate a normalization calculation intermediate array, and the number of normalization calculation principal items is calculated based on the normalization calculation intermediate array.
[0050] The normalized calculation intermediate array includes the intermediate values of the numerator and the intermediate values of the denominator.
[0051] The numerator intermediates include either the first numerator intermediate or the second numerator intermediate corresponding to the current source data. The first numerator intermediate is calculated based on the mean difference variation, and the second numerator intermediate and the first numerator intermediate are consistent with the product of the learnable parameters.
[0052] The denominator item number is generated based on the middle number of the denominator items. When generating the denominator item number, at least the reciprocal of the square root of the middle number of the denominator items is calculated.
[0053] Multiply the intermediate number of the numerator with the standard number of the denominator to generate a normalized base for calculating the principal term after the multiplication operation;
[0054] Based on the normalized calculation cardinality of the principal terms, the normalized calculation principal term number is generated;
[0055] The normalized primary term and the normalized secondary term are added together, and the result of the addition is used as the normalized result of the current source data.
[0056] Figure 1 The diagram illustrates an embodiment of normalization computation in a neural network. As shown, normalization computation requires acquiring a source dataset; therefore, the object of normalization computation is this source dataset. To acquire this source dataset, N data channels are typically configured, and channel data groups are loaded through each of these channels. That is, during a single normalization computation, one data channel loads one channel data group. Thus, the source dataset is formed based on these N channel data groups. Generally, each channel data group includes one or more source data sets, and the number of source data sets in each channel data group can vary.
[0057] It should be understood that when the normalization calculation source dataset includes n source data, then for any source data, the expression for calculating the normalization calculation result value of the current source data can be:
[0058]
[0059] Where, x a For the current source data, y a E(x) is the normalized result of the current source data; E(x) is the mean of all source data in the normalized source dataset; Var[x] is the variance of all source data in the normalized source dataset; γ is the mean of the normalized source data. a These are learnable parameters.
[0060] In practice, Related to the current source data and the normalized calculation of the source dataset, this can be called the normalized calculation of the principal term number. In this case, β can be... a This refers to the number of auxiliary terms in the normalization calculation. Therefore, for a given source dataset for normalization calculation, when any source data within that dataset is used as the current source data, the normalization calculation expression should include both the number of main normalization terms and the number of auxiliary normalization terms. After calculating the number of main normalization terms, the main normalization terms are added to the number of auxiliary normalization terms to obtain the normalized result data. It should be noted that the number of auxiliary normalization terms β... a This is generally a learnable parameter, and the number of auxiliary terms β is calculated using normalization. a Learnable parameter γ a The values used in the normalization calculation can be consistent with the existing calculations. Specific values will not be illustrated here.
[0061] As explained above regarding normalization calculation, when performing normalization calculation, it is necessary to calculate the corresponding number of normalized main terms based on the source dataset. To accelerate normalization calculation, in one embodiment of this invention, a pipelined concurrent calculation method is first used to generate an intermediate normalization array. Subsequently, the corresponding number of normalized main terms can be calculated based on this intermediate normalization array.
[0062] As can be seen from the above expression for the normalized principal term count, when calculating the normalized principal term count, the numerator target data at the numerator position should be divided by the denominator target data at the denominator position. That is, the normalized principal term count can be calculated based on the quotient of the numerator and denominator target data. To obtain the numerator and denominator target data, the normalized intermediate array calculated by the above pipelined concurrent calculation method should include the intermediate numerator and denominator values. Subsequently, the corresponding numerator target data should be calculated based on the intermediate numerator values, and the corresponding denominator target data should be calculated based on the intermediate denominator values.
[0063] As can be seen from the above explanation of the expression for the normalized calculation result, the molecular target data should be [x a -E(x)]*γ a The target data for the denominator should be The following section will provide a detailed explanation of the method and process for generating a normalized intermediate array using a pipelined concurrent computation approach, and then calculating the normalized principal term count.
[0064] In one embodiment of the present invention, when generating the normalized intermediate array using a pipelined concurrent computation method, at least two channel data group loading processes are configured, wherein,
[0065] Each time a channel data group is loaded, a corresponding normalized calculation source dataset is generated;
[0066] The intermediate values of the numerator terms are calculated using one normalized source dataset, and the intermediate values of the denominator terms are calculated using another normalized source dataset.
[0067] It should be noted that the median of the numerator and denominator should be generated through two different calculation processes. Therefore, at least two channel data set loading processes should be configured. As described above, N data channels should be used to load the channel data sets twice, and the channel data sets loaded in the two processes should be identical, that is, the normalized calculation source datasets generated in the two loading processes should be consistent. Subsequently, the median of the numerator or denominator can be calculated using the normalized calculation source dataset loaded once. At this time, the corresponding median of the denominator or numerator can be calculated using the normalized calculation source dataset loaded in the other process.
[0068] In practice, each time a channel data group is loaded, N data channels are configured to load the corresponding channel data group in parallel. That is, each data channel loads the corresponding channel data group simultaneously. It can be understood that in a neural network, the upper interface of the normalization layer can be any layer structure in the neural network layer, or it can be a structure that can store data, such as SRAM or DRAM in the physical storage model, or it can be a control module that controls the memory to perform read and write operations. The neural network here can be a classic neural network architecture, and the situation of the neural network is not limited.
[0069] As can be seen from the expression of the normalized calculation result above, the median of the numerator can be either the first median or the second median of the numerator. Both the first and second medians of the numerator are related to or correspond to the current source data. For example, the expression for calculating the first median of the numerator can be x. a -E(x), the expression for calculating the second intermediate of the numerator can be [x a -E(x)]*γ a Therefore, it can be seen that the product of the second and first intermediate numbers of the numerator is consistent with the learnable parameters; the situation of the intermediate numbers of the numerator is related to the use of pipelined concurrent calculation method, and can be referred to the corresponding explanation below.
[0070] As explained above, the median of the denominator should be Var[x]. From the normalized calculation result expression above, it is clear that the denominator's item number should be calculated based on the median of the denominator. Specifically, when calculating the denominator's item number, at least the reciprocal of the square root of the median of the denominator must be performed. This includes sequentially performing the square root operation and the reciprocal operation on the median of the denominator; that is, first calculating the square root value of the median of the denominator, and then performing the reciprocal operation on the calculated square root value. The method and process of performing the reciprocal operation on the median of the denominator will be explained in detail below.
[0071] After obtaining the denominator item number, the median of the numerator can be multiplied by the denominator item number to obtain the normalized principal term cardinality. As explained above, the median of the numerator can be either the first or second median of the numerator. When the median of the numerator is the second median, the normalized principal term cardinality is consistent with the normalized principal term number. When the median of the numerator is the first median, the normalized principal term cardinality should be multiplied by the learnable parameter γ. a After multiplication, the normalized principal term number can be generated. Therefore, when calculating the normalized principal term number based on the normalized principal term cardinality, it should be related to the type of the intermediate number of the numerator.
[0072] As explained above, after obtaining the normalized principal term number, the normalized principal term number and the normalized auxiliary term number should be added together. Then, the result of the addition operation can be used as the normalized result of the current source data, thus completing the normalization calculation of the current source data.
[0073] As can be seen from the above normalization calculation process, the present invention uses a pipelined concurrent calculation method to generate a normalization calculation intermediate array. Subsequently, the normalization calculation principal number can be generated based on the normalization calculation intermediate array, and finally the normalization calculation result data can be obtained. The pipelined concurrent calculation method can effectively shorten the normalization calculation time, that is, effectively accelerate the normalization calculation and improve the efficiency of the normalization calculation.
[0074] In one embodiment of the present invention, calculating the median of the denominator includes:
[0075] For each of the N data channels loaded with channel data groups, calculate the cumulative sum of the channel data and the cumulative sum of the squared channel data for each corresponding channel data group.
[0076] Add the sums of the corresponding channel data for N data channels to calculate the sum of the source data.
[0077] Add the sum of squares of the corresponding channel data for N data channels to calculate the sum of squares of the source data;
[0078] Based on the cumulative sum of the source data and the cumulative sum of the squares of the source data, the median of the denominator is calculated, wherein the median of the denominator is... This is the sum of the source data. x is the sum of squared values of the source data, n is the number of source data points in the normalized dataset, and x is the sum of squared values of the source data. i ε is a constant and is used to normalize the i-th source data in the source dataset.
[0079] As explained above, calculating the intermediate value of the denominator requires an independent and complete normalized calculation source dataset, which is formed by N channel data groups loaded in parallel by N data channels. To accelerate the normalization calculation, the cumulative sum of channel data and the cumulative sum of the squares of channel data are calculated in parallel for each of the N data channel data groups. For example, if the channel data groups are loaded in parallel by three data channels, with the source data x1, x2, and x3 in the first data channel, x4, x5, and x6 in the second data channel, and x7, x8, and x9 in the third data channel, then when calculating the cumulative sum of channel data for each corresponding channel data group, for the first data channel: x1 + x2 + x3, for the second data channel: x4 + x5 + x6, and for the third data channel: x7 + x8 + x9; and when calculating the cumulative sum of the squares of channel data, for the first data channel: For the second data channel, we have: For the third data channel, we have: When the number of channel data N is any other value, and there are other situations regarding the source data within each channel data, please refer to the explanation here. Examples will not be given here.
[0080] After calculating the cumulative sum of data for each channel, the cumulative sums of the N channel data are added together to obtain the cumulative sum of the source data. Similarly, the cumulative sum of the squares of the N channel data is added together to obtain the cumulative sum of the squares of the source data. In practice, the intermediate value of the denominator can be calculated based on the cumulative sum of the source data and the cumulative sum of the squares of the source data. Specifically, the intermediate value of the denominator can be: The constant ε can generally be a small constant, mainly to avoid the situation where the denominator is too small and close to zero when performing normalization calculations.
[0081] The following section derives and explains the expression used for the intermediate number of the denominator in normalization calculations.
[0082] In the above normalized calculation result expression, for the mean E(x) and variance Var[x], we have:
[0083]
[0084] In practice, based on the corresponding expressions for the mean E(x) and variance Var[x], and substituting them into the normalized calculation result expression, the calculation expression for the denominator item scale can be derived as follows: This gives us the middle number of the denominator.
[0085] As is known to those skilled in the art, directly performing square root and division operations is generally time-consuming. In other words, existing technologies are time-consuming when performing square root and division operations on the middle number of the denominator, resulting in low efficiency in normalization calculations. To accelerate the efficiency of normalization calculations, in one embodiment of the present invention, when performing the square root reciprocal operation on the middle number of the denominator, an interpolation fitting method is used for iterative calculation to generate the denominator index after the iterative calculation.
[0086] Specifically, when the square root operation uses an interpolation fitting method, the interpolation fitting method includes Newton's interpolation method. Of course, the interpolation fitting method can also be other interpolation fitting methods, such as approximate function fitting methods or other variations of Taylor's formula expansion. The specific method can be selected according to actual needs.
[0087] The following example illustrates the mixed operation of using Newton's interpolation to replace square root operations and division operations. Specifically, for the target number B in the square root operation, when the domain of the target number B is greater than 0, then:
[0088]
[0089] When using an iterative method for calculation, we have: m is the iteration number, y m y is the result of the m-th iteration. m+1 For the (m+1)th iteration, f′(ym) is f(y m The reciprocal of (). It should be noted that when using the above iterative method, an initial value for the iteration should be configured. The initial value can be configured according to the data format of the target result. The method of configuring the initial value should be to speed up the iteration while meeting the required accuracy. In specific implementation, at least one iteration should be performed during each iteration. The number of iterations can be selected as needed, based on the requirements of both normalization accuracy and computational efficiency.
[0090] As can be seen from the above explanation, when an interpolation method is used for fitting, such as Newton's interpolation method, instead of the square root and division operations in the existing technology, the calculation time for generating the denominator item scale can be greatly reduced.
[0091] In one embodiment of the present invention, when the median of the numerator is the first median of the numerator, the first median of the numerator calculated based on the mean difference variation is as follows:
[0092]
[0093] Where n is the number of source data points in the source dataset for normalization calculation, and x a For the current source data, This is the sum of the source data.
[0094] As explained above, when calculating the median of the numerator, it is necessary to use the normalized source dataset from another loading, and the sum of the source data can be calculated using the method described above. When the current source data x is determined a Then, the first intermediate value of the numerator can be calculated based on the mean difference variation. It should be noted that the mean difference variation is the calculation form derived from the expression of the normalized calculation result above.
[0095] The above illustrates the calculation methods for the numerator intermediate and the denominator item number. Subsequently, the normalized principal term number can be calculated using multiplication based on the numerator intermediate and the denominator item number. In one embodiment of the present invention, when the numerator intermediate is the first intermediate of the numerator, the normalized principal term number is calculated as follows:
[0096] When multiplying the middle number of the numerator and the item number of the denominator, the first middle number of the numerator is multiplied by the item number of the denominator to generate the normalized calculation base number of the principal term. Then, the normalized calculation base number of the principal term is multiplied by the learnable parameter to generate the normalized calculation principal term number.
[0097] or,
[0098] When multiplying the intermediate number of the numerator and the item number of the denominator, the first intermediate number of the numerator is first multiplied by the learnable parameter, and the result is configured as the second intermediate number of the numerator. Then, the second intermediate number of the numerator is multiplied by the item number of the denominator to generate the normalized calculation principal term base, and the normalized calculation principal term base is used as the normalized calculation principal term number.
[0099] When the median of the numerator is the first median of the numerator, as explained above regarding the normalization calculation of the principal term, different pipeline calculation methods can be used. For example, the first median of the numerator can be multiplied by the standard number of the denominator, and then multiplied by the learnable parameter γ. a Multiply; or, first multiply the first intermediate number of the numerator with the learnable parameter, and then multiply it with the denominator's item number to finally obtain the normalized calculation of the principal term number.
[0100] In one embodiment of the present invention, when the median of the numerator is the second median of the numerator, the calculation of generating the second median of the numerator includes:
[0101] First, the first intermediate number of the numerator is calculated based on the mean difference variation. Then, the first intermediate number of the numerator is multiplied by the learnable parameter, and the result of the multiplication is configured as the second intermediate number of the numerator.
[0102] or,
[0103] First, the learnable parameter is multiplied by each source data. Then, the second intermediate number of the numerator is calculated based on the mean difference transformation.
[0104] When calculating the normalized principal term number, we have:
[0105] The second intermediate number of the numerator is multiplied by the item number of the denominator to generate the normalized calculation base number of the principal term, and the normalized calculation base number of the principal term is used as the normalized calculation principal term number.
[0106] As explained above, the product of the second intermediate number and the first intermediate number of the numerator with the learnable parameter is consistent. Therefore, when generating the second intermediate number of the numerator, the product of the first intermediate number of the numerator with the corresponding learnable parameter should be calculated. In practice, different methods can be used to calculate and generate the second intermediate number of the numerator. The specific calculation method can be selected as needed, that is, different pipeline calculation methods can be used to improve the flexibility of pipeline parallel computing and accelerated operation.
[0107] In one embodiment of the present invention, a data format conversion process is further included, wherein,
[0108] Data format conversion processing includes:
[0109] The normalization calculation source dataset is converted to a different format to generate target format data corresponding to each source data, and a normalization calculation target format dataset is generated based on all the target format data.
[0110] Based on the normalized calculation target format dataset, when calculating and generating the corresponding normalized calculation intermediate array, normalized calculation principal term number and normalized calculation result value, the data bit width of the numerator intermediate number, denominator intermediate number, normalized calculation principal term number and normalized calculation result value are consistent with the data bit width of the target format data.
[0111] The normalized calculation result obtained above is processed for data format output to generate the normalized calculation result value in the target output format.
[0112] It is understandable that, when normalizing the source data within the source dataset, such as when the source data is an input variable in the format of a 16-bit wide signed integer, the calculation of intermediate variables such as... If integer operations are performed without loss of precision, then many additional bits need to be added to retain these intermediate variables, which would make the intermediate resources required for the calculation enormous.
[0113] In one embodiment of the present invention, the source data can be converted to a target format, such as a custom format like IEEE 754 FP16. In this case, the source data can be expressed using a 1-bit sign bit width, a 7-bit exponent bit width, and an 8-bit mantissa bit width, that is, using only 16 bits to represent the source data. The range of the source data can reach 2^32. 64 The intermediate results of the calculation can still be represented using a 16-bit width, which greatly reduces the hardware resource consumption caused by expanding the bit width.
[0114] In practice, once the target format is determined, such as the custom format of IEEE 754 FP16 mentioned above, the source data can be converted to the target format to obtain the target format data. The data conversion method can be found in the relevant IEEE 754 documentation. Afterwards, following the normalization operation, the normalized calculation result value of the target format can be obtained. However, the normalized calculation result value of the target format should undergo data format output processing. This data format output can include inverse data format conversion or data format transformation. Inverse data format conversion means converting the above data format back to the original data format. Data format transformation specifically refers to converting the normalized calculation result value to other data formats, such as converting the normalized calculation result value to a data format that can be processed subsequently, to obtain the normalized calculation result value of the target output format. The data format output processing can be selected as needed to meet actual requirements. It is understandable that by using the custom format of IEEE 754 FP16 during the above data format conversion, more hardware resources can be saved while sacrificing slightly higher precision, while still ensuring a fairly high level of accuracy.
[0115] It should be understood that if the source data format is consistent with the target format, the above data format conversion process can be omitted.
[0116] Figure 2 The image shows an embodiment of accelerating normalized computation using a pipelined concurrent computing approach. The following section will combine... Figure 2 This section provides a detailed explanation of an accelerated method for normalized computation, specifically including:
[0117] Step S0: Input the channel data sets required for a total of N data channels one by one;
[0118] Step S1: Convert the input source data to obtain the corresponding target format data;
[0119] Step S2: Calculate the cumulative sum of the corresponding channel data for each data channel in parallel;
[0120] Step S3: Calculate the cumulative sum of squared data for each data channel in parallel;
[0121] Step S4: Determine whether the source data required for the normalization calculation has been sent. If the source data has been sent, proceed to step S5; otherwise, return to the state of step S2 and step S3.
[0122] Step S5: Based on the calculation results of steps S2 and S3, calculate the cumulative sum of the source data and the cumulative sum of the squares of the source data;
[0123] Step S6: Calculate the middle number of the denominator terms;
[0124] Step S7: Calculate and generate the standard values for the denominator items using interpolation methods;
[0125] Step S8: Input the corresponding channel data group for data channel N again;
[0126] Step S9: Perform a uniform format conversion on the input source data to obtain the corresponding target format data;
[0127] Step S10: Calculate the first intermediate number of the numerator;
[0128] Step S11: Calculate the product between the first intermediate number of the numerator and the denominator item number;
[0129] Step S12: Perform multiplication and addition calculations quickly. During multiplication and addition calculations, the intermediate base and the learnable parameter are multiplied by normalized calculations and then added to the number of auxiliary terms calculated by normalization.
[0130] Step S13: Determine whether the source data for normalization calculation has been sent. If it has been sent, proceed to step S14; otherwise, proceed to step S8.
[0131] Step S14: Perform a reverse data format conversion on the normalization result value to obtain the normalized calculation result value in the original data format. Of course, as can be seen from the above description, when the requirements such as the accuracy of the normalization operation are met, the normalization result value can be directly output, that is, the reverse data format conversion is not required here. Alternatively, the normalization result value can be converted to other types required in subsequent steps, such as converting the normalization result value to integer format data. The specific situation can be selected according to the actual needs, and will not be listed here.
[0132] S15: End this calculation, that is, complete the normalization calculation.
[0133] Depend on Figure 2As explained above, this illustrates an embodiment of pipelined concurrent computation where intermediate values for the denominator are first calculated, followed by the calculation of the denominator's index, and then the first intermediate value for the numerator is calculated. Other pipelined concurrent computation methods can be selected according to actual needs, and can be referred to the corresponding explanations above; further examples will not be provided here.
[0134] As can be seen from the above description, an accelerated running device suitable for neural network normalization calculation can be obtained. In one embodiment of the present invention, an accelerated running processor is included, wherein at least a neural network is deployed in the accelerated running processor. When normalization calculation is performed in the neural network, for any source data in the normalization calculation source dataset, the accelerated running processor performs normalization calculation using the accelerated running method described above and generates the normalization calculation result value corresponding to the current source data.
[0135] Specifically, the accelerated processor can be any existing commonly used processing equipment, such as a computing device capable of deploying and running neural networks. The type of accelerated processor can be selected according to actual needs. When performing normalization calculations on the source data within the normalization calculation source dataset, please refer to the above explanation, which will not be repeated here.
Claims
1. A method for accelerating the operation of normalized computation in neural networks, characterized in that, The accelerated operation method includes: When performing normalization calculations within a neural network, N data channels are configured to load channel data groups respectively, and a normalization calculation source dataset is generated based on the N channel data groups. When performing normalization calculations on any source data within the normalization calculation source dataset, a pipelined concurrent calculation method is used to generate a normalization calculation intermediate array, and the number of normalization calculation principal items is calculated based on the normalization calculation intermediate array. The normalized calculation intermediate array includes the intermediate values of the numerator and the intermediate values of the denominator. The numerator intermediates include either the first numerator intermediate or the second numerator intermediate corresponding to the current source data. The first numerator intermediate is calculated based on the mean difference variation, and the second numerator intermediate and the first numerator intermediate are consistent with the product of the learnable parameters. The denominator item number is generated based on the middle number of the denominator items. When generating the denominator item number, at least the reciprocal of the square root of the middle number of the denominator items is calculated. Multiply the intermediate number of the numerator with the standard number of the denominator to generate a normalized base for calculating the principal term after the multiplication operation; Based on the normalized calculation cardinality of the principal terms, the normalized calculation principal term number is generated; The normalized primary term and the normalized secondary term are added together, and the result of the addition is used as the normalized result of the current source data.
2. The accelerated operation method suitable for normalized computation of neural networks according to claim 1, characterized in that: When using a pipelined concurrent computation method to generate the normalized intermediate computation array, at least two channel data group loading processes must be configured, among which, Each time a channel data group is loaded, a corresponding normalized calculation source dataset is generated; The intermediate values of the numerator terms are calculated using one normalized source dataset, and the intermediate values of the denominator terms are calculated using another normalized source dataset.
3. The accelerated operation method suitable for normalized computation of neural networks according to claim 2, characterized in that, When calculating the middle part of the denominator, the following is included: For each of the N data channels loaded with channel data groups, calculate the cumulative sum of the channel data and the cumulative sum of the squared channel data for each corresponding channel data group. Add the sums of the corresponding channel data for N data channels to calculate the sum of the source data. Add the sum of squares of the corresponding channel data for N data channels to calculate the sum of squares of the source data; Based on the cumulative sum of the source data and the cumulative sum of the squares of the source data, the median of the denominator is calculated, wherein the median of the denominator is... This is the sum of the source data. x is the sum of squared values of the source data, n is the number of source data points in the normalized dataset, and x is the sum of squared values of the source data. i ε is a constant and is used to normalize the i-th source data in the source dataset.
4. The accelerated operation method suitable for normalized computation of neural networks according to claim 3, characterized in that, When performing the square root reciprocal operation on the middle number of the denominator, an interpolation fitting method is used to perform iterative calculations on the middle number of the denominator to generate the denominator index after the iterative calculations.
5. The accelerated operation method suitable for normalized computation of neural networks according to claim 2, characterized in that, Each time a channel data group is loaded, N data channels are configured to load the corresponding channel data group in parallel.
6. The accelerated operation method suitable for normalized computation of neural networks according to any one of claims 1 to 5, characterized in that, When the median of the numerator is the first median of the numerator, for the first median of the numerator calculated based on the mean difference variation, we have: Where n is the number of source data points in the source dataset for normalization calculation, and x a For the current source data, This is the sum of the source data.
7. The accelerated operation method suitable for normalized computation of neural networks according to claim 6, characterized in that, When the median of the numerator is the first median of the numerator, when calculating the normalized principal term, we have: When multiplying the middle number of the numerator and the item number of the denominator, the first middle number of the numerator is multiplied by the item number of the denominator to generate the normalized calculation base number of the principal term. Then, the normalized calculation base number of the principal term is multiplied by the learnable parameter to generate the normalized calculation principal term number. or, When multiplying the intermediate number of the numerator and the item number of the denominator, the first intermediate number of the numerator is first multiplied by the learnable parameter, and the result is configured as the second intermediate number of the numerator. Then, the second intermediate number of the numerator is multiplied by the item number of the denominator to generate the normalized calculation principal term base, and the normalized calculation principal term base is used as the normalized calculation principal term number.
8. The accelerated operation method suitable for normalized computation of neural networks according to claim 6, characterized in that, When the median of the numerator is the second median of the numerator, the calculation of the second median of the numerator includes: First, the first intermediate number of the numerator is calculated based on the mean difference variation. Then, the first intermediate number of the numerator is multiplied by the learnable parameter, and the result of the multiplication is configured as the second intermediate number of the numerator. or, First, the learnable parameter is multiplied by each source data. Then, the second intermediate number of the numerator is calculated based on the mean difference transformation. When calculating the normalized principal term number, we have: The second intermediate number of the numerator is multiplied by the item number of the denominator to generate the normalized calculation base number of the principal term, and the normalized calculation base number of the principal term is used as the normalized calculation principal term number.
9. The accelerated operation method suitable for normalized computation of neural networks according to any one of claims 1 to 5, characterized in that, It also includes data format conversion processing, among which, Data format conversion processing includes: The normalization calculation source dataset is converted to generate target format data corresponding to each source data. The normalization calculation target format dataset is generated based on all target format data, wherein the data bit width of the target format data is consistent with the data bit width of the source data. Based on the normalized calculation target format dataset, when calculating and generating the corresponding normalized calculation intermediate array, normalized calculation principal term number and normalized calculation result value, the data bit width of the numerator intermediate number, denominator intermediate number, normalized calculation principal term number and normalized calculation result value are consistent with the data bit width of the target format data. The normalized calculation result obtained above is processed for data format output to generate the normalized calculation result value in the target output format.
10. An accelerated operating device suitable for normalized computation of neural networks, characterized in that, The system includes an accelerated execution processor, wherein at least a neural network is deployed within the accelerated execution processor. When performing normalization calculations within the neural network, for any source data within the normalization calculation source dataset, the accelerated execution processor performs normalization calculations using the accelerated execution method described in any one of claims 1 to 9, and generates a normalization result value corresponding to the current source data.
Citation Information
Patent Citations
Convolutional neural network accelerator based on FPGA
CN115018062A
Neural network acceleration method and device, accelerator and storage medium
CN117973443A