Accelerated operation method and device suitable for neural network normalization calculation

Abstract

The invention relates to an accelerated operation method and device suitable for neural network normalization calculation. The method comprises the steps that when normalization calculation is conducted on any source data in a normalization calculation source data set, a normalization calculation intermediate array is generated through calculation in a pipeline concurrent calculation mode, and a normalization calculation main item number is generated through calculation based on the normalization calculation intermediate array, the normalization calculation intermediate array comprises a sub-term intermediate number and a denominator term intermediate number, and the sub-term intermediate number and a denominator term target number are multiplied to generate a normalization calculation main term cardinal number after multiplication; and performing additive operation on the normalized calculation main item number and the normalized calculation auxiliary item number which are generated by calculation, and taking a result value of the additive operation as a normalized calculation result value of the current source data. According to the method, the normalization calculation can be effectively accelerated, the efficiency of the normalization calculation is improved, and the hardware overhead cost of the normalization calculation is reduced.

Description

Technical Field

[0001] The present invention relates to an accelerated operation method and device, and in particular to an accelerated operation method and device suitable for normalized calculation of a neural network. Background Art

[0002] Normalization is a common data preprocessing method that maps data into decimals between [0, 1] or [-1, 1], making it easier for the model to converge to the optimal solution. Therefore, before constructing a machine learning or deep learning model, it is necessary to normalize the data, which will not only speed up the solution, but also improve the accuracy of the model.

[0003] At present, when normalizing, usually at the software level, the hardware is directly asked to calculate each intermediate result, that is, to gradually calculate multiplication, addition, square root and division, then multiplication... addition, etc. After each step is calculated, the next step is calculated until the entire normalization operation is completed. It is known to those skilled in the art that the normalization calculation method currently used has many steps, is time-consuming, and is inefficient, and is difficult to meet the actual normalization calculation needs. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and to provide an accelerated operation method and device suitable for normalized calculation of neural networks, which can effectively accelerate normalized calculation, improve the efficiency of normalized calculation, and reduce the hardware overhead cost of normalized calculation.

[0005] According to the technical solution provided by the present invention, an accelerated operation method suitable for normalized calculation of a neural network comprises:

[0006] When performing normalization calculation in the neural network, N data channels are configured to load channel data groups respectively, and a normalized calculation source data set is generated based on the N channel data groups;

[0007] When normalizing any source data in the normalized calculation source data set, a pipeline concurrent calculation method is used to calculate and generate a normalized calculation intermediate array, and a normalized calculation main item number is calculated based on the normalized calculation intermediate array, where:

[0008] The normalized calculation intermediate array includes the intermediate numbers of the numerator and the intermediate numbers of the denominator.

[0009] The numerator item intermediate number includes a numerator item first intermediate number or a numerator item second intermediate number corresponding to the current source data, wherein the numerator item first intermediate number is calculated and generated based on the mean difference variant, and the numerator item second intermediate number is consistent with the product result of the numerator item first intermediate number and the learnable parameter;

[0010] The target number of the denominator item is calculated based on the intermediate number of the denominator item, wherein when calculating and generating the target number of the denominator item, at least the inverse square root operation is performed on the intermediate number of the denominator item;

[0011] Multiply the intermediate number of the numerator item and the target number of the denominator item to generate a normalized calculation main item base after the multiplication operation;

[0012] Based on the normalized calculation main item cardinality, the normalized calculation main item number is calculated and generated;

[0013] The normalized calculation main item number and the normalized calculation auxiliary item number generated by the calculation are added, and the result value of the addition operation is used as the normalized calculation result value of the current source data.

[0014] When using pipeline concurrent computing to generate normalized computing intermediate arrays, at least two channel data group loading processes are configured, where:

[0015] Each time a channel data set is loaded, a corresponding normalized calculation source data set is generated;

[0016] A normalized calculation source data set is used to calculate and generate the corresponding intermediate number of the numerator term, and another normalized calculation source data set is used to calculate and generate the corresponding intermediate number of the denominator term.

[0017] When calculating the middle number of the denominator, include:

[0018] For the channel data groups loaded respectively into the N data channels, the channel data cumulative sum and the channel data square cumulative sum of the channel data group corresponding to each data channel are calculated;

[0019] Performing an addition operation on the channel data accumulation sums corresponding to the N data channels to calculate and generate the source data accumulation sum;

[0020] Performing an addition operation on the square cumulative sums of the channel data corresponding to the N data channels to calculate and generate the square cumulative sum of the source data;

[0021] Based on the source data cumulative sum and the source data square cumulative sum, the denominator intermediate number is calculated and generated, where the denominator intermediate number is is the cumulative sum of the source data, is the cumulative sum of the squares of the source data, n is the number of source data in the normalized calculation source data set, x i To normalize the i-th source data in the source data set, ε is a constant.

[0022] When the inverse square root operation is performed on the intermediate number of the denominator, the interpolation fitting method is used to iteratively operate the intermediate number of the denominator to generate the target number of the denominator after the iterative operation.

[0023] Each time a channel data group is loaded, N data channels are configured to load the corresponding channel data groups in parallel.

[0024] When the middle of the numerator is the first middle of the numerator, for the first middle of the numerator calculated based on the variation of the mean difference, we have:

[0025]

[0026] Among them, n is the number of source data in the normalized calculation source data set, x a is the current source data, Cumulative sum of source data.

[0027] When the middle number of the numerator is the first middle number of the numerator, and the normalized number of principal terms is calculated, then:

[0028] When the intermediate number of the numerator item is multiplied by the target number of the denominator item, the first intermediate number of the numerator item is multiplied by the target number of the denominator item to generate a normalized calculation main item base number, and then the normalized calculation main item base number is multiplied by the learnable parameter to generate a normalized calculation main item number;

[0029] or,

[0030] When the intermediate number of the numerator item is multiplied by the target number of the denominator item, the first intermediate number of the numerator item is first multiplied by the learnable parameter, and the multiplication result is configured as the second intermediate number of the numerator item; thereafter, the second intermediate number of the numerator item is multiplied by the target number of the denominator item to generate a normalized calculation main item base, and the normalized calculation main item base is used as the normalized calculation main item number.

[0031] When the middle number of the numerator is the second middle number of the numerator, the calculation to generate the second middle number of the numerator includes:

[0032] First, the first intermediate number of the numerator is calculated based on the mean difference variation, and the calculated first intermediate number of the numerator is multiplied by the learnable parameter, and the result of the multiplication is configured as the second intermediate number of the numerator;

[0033] or,

[0034] First, the learnable parameter is multiplied by each source data, and then the second intermediate number of the numerator is calculated based on the mean difference variant;

[0035] When calculating the normalized number of main items, we have:

[0036] The second intermediate number of the numerator item is multiplied by the target number of the denominator item to generate a normalized calculation main item base number, and the normalized calculation main item base number is used as the normalized calculation main item number.

[0037] It also includes data format conversion processing, in which:

[0038] Data format conversion processing includes:

[0039] Convert all source data in the normalized computing source data set to a target format corresponding to each source data, and generate a normalized computing target format data set based on all the target format data, wherein the data bit width of the target format data is consistent with the data bit width of the source data;

[0040] When generating the corresponding normalized calculation intermediate array, normalized calculation main item number and normalized calculation result value based on the normalized calculation target format data set, the data bit widths of the numerator intermediate number, the denominator intermediate number, the normalized calculation main item number and the normalized calculation result value are consistent with the data bit widths of the target format data;

[0041] The normalized calculation result values ​​obtained above are subjected to data format output processing to generate normalized calculation result values ​​in a target output format after the data format output processing.

[0042] An accelerated operation device suitable for normalized calculation of a neural network includes an accelerated operation processor, wherein at least a neural network is deployed in the accelerated operation processor, and when normalized calculation is performed in the neural network, the accelerated operation processor uses the above-mentioned accelerated operation method to perform normalized calculation on any source data in the normalized calculation source data set, and generates a normalized operation result value corresponding to the current source data.

[0043] The advantages of the present invention are as follows: for normalized calculation source data in normalized calculation, multiple data channels are used for parallel loading, and a pipeline concurrent calculation method is used to calculate and generate a normalized calculation intermediate array, so as to improve data throughput and calculation efficiency; in normalized calculation, two data transmissions are used to load the normalized calculation source data, which can greatly save the hardware area consumption of retaining input and intermediate variables compared with the existing normalized calculation; the scheme adopts a pipeline concurrent calculation method, which can reduce the time required for calculation, and can also improve the efficiency of normalized calculation and reduce the hardware overhead cost of normalized calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of an embodiment of the accelerated operation of the present invention.

[0045] Figure 2 It is a schematic diagram of an embodiment of the present invention based on accelerated operation during pipeline concurrency. DETAILED DESCRIPTION

[0046] The present invention will be further described below in conjunction with specific drawings and embodiments.

[0047] In order to effectively accelerate normalization calculation and improve the efficiency of normalization calculation, the present invention provides an accelerated operation method suitable for neural network normalization calculation. Specifically, the accelerated operation method includes:

[0048] When performing normalization calculation in the neural network, N data channels are configured to load channel data groups respectively, and a normalized calculation source data set is generated based on the N channel data groups;

[0049] When normalizing any source data in the normalized calculation source data set, a pipeline concurrent calculation method is used to calculate and generate a normalized calculation intermediate array, and a normalized calculation main item number is calculated based on the normalized calculation intermediate array, where:

[0050] The normalized calculation intermediate array includes the intermediate numbers of the numerator and the intermediate numbers of the denominator.

[0051] The numerator item intermediate number includes a numerator item first intermediate number or a numerator item second intermediate number corresponding to the current source data, wherein the numerator item first intermediate number is calculated and generated based on the mean difference variant, and the numerator item second intermediate number is consistent with the product result of the numerator item first intermediate number and the learnable parameter;

[0052] The target number of the denominator item is calculated based on the intermediate number of the denominator item, wherein when calculating and generating the target number of the denominator item, at least the inverse square root operation is performed on the intermediate number of the denominator item;

[0053] Multiply the intermediate number of the numerator item and the target number of the denominator item to generate a normalized calculation main item base after the multiplication operation;

[0054] Based on the normalized calculation main item cardinality, the normalized calculation main item number is calculated and generated;

[0055] The normalized calculation main item number and the normalized calculation auxiliary item number generated by the calculation are added, and the result value of the addition operation is used as the normalized calculation result value of the current source data.

[0056] Figure 1 An embodiment of normalization calculation in a neural network is shown in FIG. As can be seen from the figure, when performing normalization calculation, it is necessary to obtain a normalization calculation source data set. It can be seen that the object of normalization calculation is the normalization calculation source data set. In order to obtain the normalization calculation source data set, N data channels can generally be configured, and channel data groups can be loaded through N data channels respectively, that is, during a normalization calculation, one data channel will load a channel data group. At this time, a normalization calculation source data set can be formed based on N channel data groups. Generally, each channel data group includes one or more source data, and the number of source data contained in each channel data group may be different.

[0057] It should be understood that when the normalized calculation source data set includes n source data, for any source data, the expression for calculating the normalized calculation result value of the current source data may be:

[0058]

[0059] Among them, x a is the current source data, y a is the normalized calculation result value of the current source data; E(x) is the mean of all source data in the normalized calculation source data set, Var[x] is the variance of all source data in the normalized calculation source data set, γ a are learnable parameters.

[0060] When implementing it specifically, Related to the current source data and the normalized calculation source data set, it can be called the normalized calculation main item number. At this time, β a It can be seen that for a certain normalized calculation source data set, when any source data in the normalized calculation source data set is used as the current source data, the expression of the normalized calculation should include the normalized calculation main item number and the normalized calculation auxiliary item number. After the normalized calculation main item number is calculated, the normalized calculation main item number and the normalized calculation auxiliary item number are accumulated to obtain the normalized calculation result data. It should be noted that the normalized calculation auxiliary item number β a It is also a learnable parameter, and the auxiliary number β is calculated by normalization. a , learnable parameter γ a The values ​​taken during normalization calculation may be consistent with existing calculations, and specific values ​​will not be given examples here.

[0061] From the above description of normalized calculation, it can be seen that when performing normalized calculation, it is necessary to calculate the corresponding number of normalized calculation main items based on the normalized calculation source data set. In order to speed up the normalized calculation, in one embodiment of the present invention, a pipeline concurrent calculation method is first used to calculate and generate a normalized calculation intermediate array, and then the corresponding number of normalized calculation main items can be calculated based on the normalized calculation intermediate array.

[0062] It can be seen from the above expression of the normalized calculation main item number that when calculating the normalized calculation main item number, at least the numerator item target data at the numerator position should be divided by the denominator position target data, that is, based on the quotient of the numerator item target data and the denominator item target data, the normalized calculation main item number can be calculated. In order to obtain the numerator item target data and the denominator item target data, the normalized calculation intermediate array calculated by the above pipeline concurrent calculation method should include the numerator item intermediate number and the denominator item intermediate number. Thereafter, the corresponding numerator item target data should be calculated based on the numerator item intermediate number, and the corresponding denominator item target data should be calculated based on the denominator item intermediate number.

[0063] From the above expression for normalized calculation result value, we can know that the molecular project target data should be [x a -E(x)]*γ a , the denominator item data should be The following will specifically describe the method and process of using pipeline concurrent computing to generate a normalized computing intermediate array and then calculating the number of normalized computing main items.

[0064] In one embodiment of the present invention, when a pipeline concurrent computing method is used to calculate and generate a normalized computing intermediate array, at least two channel data group loading processes are configured, wherein:

[0065] Each time a channel data set is loaded, a corresponding normalized calculation source data set is generated;

[0066] A normalized calculation source data set is used to calculate and generate the corresponding intermediate number of the numerator term, and another normalized calculation source data set is used to calculate and generate the corresponding intermediate number of the denominator term.

[0067] It should be noted that the intermediate number of the numerator and the intermediate number of the denominator should be calculated and generated by two different calculation processes. Therefore, at least two channel data group loading processes should be configured. As described above, N data channels are used to load the channel data group twice, and the channel data groups loaded twice should be the same, that is, the normalized calculation source data sets generated by the two loading processes are consistent. After that, the intermediate number of the numerator or the intermediate number of the denominator can be calculated and generated using the normalized calculation source data set loaded once. At this time, the corresponding intermediate number of the denominator or the intermediate number of the numerator can be calculated and generated using the normalized calculation source data set loaded once.

[0068] In specific implementation, each time a channel data group is loaded, N data channels are configured to load the corresponding channel data group in parallel, that is, each data channel loads the corresponding channel data group at the same time. It can be understood that in the neural network, the upper interface of the normalization layer can be any layer of structure in the neural network layer, or it can be a structure that can store data such as SRAM, DRAM, etc. in the physical storage model, or it can be a control module that controls the memory to perform read and write operations; the neural network here can be a classic neural network architecture, and the situation of the neural network is not limited.

[0069] From the above expression of the normalized calculation result value, it can be known that the middle number of the numerator item can be the first middle number of the numerator item or the second middle number of the numerator item, wherein the first middle number of the numerator item and the second middle number of the numerator item are both related to or corresponding to the current source data. For example, the calculation expression of the first middle number of the numerator item can be x a -E(x), the calculation expression of the second intermediate number of the numerator can be [x a -E(x)]*γ a , it can be seen that the product of the second intermediate number of the numerator and the first intermediate number of the numerator is consistent with the product of the learnable parameter; the situation of the intermediate number of the numerator is related to the use of pipeline concurrent calculation method, please refer to the corresponding description below.

[0070] It can be seen from the above description that the middle number of the denominator term should be Var[x], and it can be seen from the above normalized calculation result expression that the target number of the denominator term should be calculated based on the middle number of the denominator term, wherein when calculating and generating the target number of the denominator term, at least the middle number of the denominator term is subjected to a square root reciprocal operation. Specifically, when the square root reciprocal operation is performed on the middle number of the denominator term, it includes sequentially performing a square root operation and a reciprocal operation on the middle number of the denominator term, that is, first calculating the square root value of the middle number of the denominator term, and then performing a reciprocal operation on the calculated square root value. The manner and process of performing the square root reciprocal operation on the middle number of the denominator term will be specifically described below.

[0071] After obtaining the target number of the denominator item, the middle number of the numerator item can be multiplied by the target number of the denominator item to obtain the normalized calculation main item cardinality. From the above description, it can be seen that the middle number of the numerator item can be the first middle number of the numerator item or the second middle number of the numerator item. When the middle number of the numerator item is the second middle number of the numerator item, it can be seen from the above description that the normalized calculation main item cardinality is consistent with the normalized calculation main item number; when the middle number of the numerator item is the first middle number of the numerator item, the normalized calculation main item cardinality should be multiplied by the learnable parameter γ a After multiplication, the normalized number of calculated main terms can be generated. Therefore, when calculating and generating the normalized number of calculated main terms based on the normalized number of calculated main terms, it should be related to the type of the intermediate number of the numerator term.

[0072] From the above description, it can be seen that after obtaining the normalized calculation main item number, the normalized calculation main item number and the normalized calculation auxiliary item number should be added. After that, the result value of the addition operation can be used as the normalized calculation result value of the current source data. At this time, the normalized calculation of the current source data is completed.

[0073] It can be seen from the above normalized calculation process that the present invention adopts a pipeline concurrent calculation method to calculate and generate a normalized calculation intermediate array. Thereafter, the normalized calculation main item number can be calculated based on the normalized calculation intermediate array, and finally the normalized calculation result data can be obtained; the use of a pipeline concurrent calculation method can effectively shorten the normalized calculation time, that is, effectively accelerate the normalized calculation and improve the efficiency of the normalized calculation.

[0074] In one embodiment of the present invention, the calculation of the intermediate number of the denominator includes:

[0075] For the channel data groups loaded respectively into the N data channels, the channel data cumulative sum and the channel data square cumulative sum of the channel data group corresponding to each data channel are calculated;

[0076] Performing an addition operation on the channel data accumulation sums corresponding to the N data channels to calculate and generate the source data accumulation sum;

[0077] Performing an addition operation on the square cumulative sums of the channel data corresponding to the N data channels to calculate and generate the square cumulative sum of the source data;

[0078] Based on the source data cumulative sum and the source data square cumulative sum, the denominator intermediate number is calculated and generated, where the denominator intermediate number is is the cumulative sum of the source data, is the cumulative sum of the squares of the source data, n is the number of source data in the normalized calculation source data set, x i To normalize the i-th source data in the source data set, ε is a constant.

[0079] It can be seen from the above description that when calculating the intermediate number of the denominator, an independent and complete normalized calculation source data set is required, and the normalized calculation source data set is formed by N channel data groups loaded in parallel by N data channels. In order to speed up the normalized calculation, for the channel data groups of N data channels, the channel data cumulative sum and the channel data square cumulative sum of each data channel are calculated in parallel. For example, the channel data group is loaded in parallel by 3 data channels, and the source data in the channel data group loaded by the first data channel are x1, x2, and x3 respectively, the source data in the channel data group loaded by the second data channel are x4, x5, and x6 respectively, and the source data in the channel data group loaded by the third data channel are x7, x8, and x9 respectively. When calculating the channel data cumulative sum of the corresponding channel data group of each data channel, for the first data channel, it is: x1+x2+x3, for the second data channel, it is: x4+x5+x6, and for the third data channel, it is: x7+x8+x9; and when calculating the channel data square cumulative sum, for the first data channel, it is: For the second data channel: For the third data channel: When the number N of channel data is other values, and there are other situations of the source data in each channel data, please refer to the description here, and no examples will be given here one by one.

[0080] After calculating the cumulative sum of each channel data, the N channel data cumulative sums are added, that is, the N channel data cumulative sums are accumulated, so that the source data cumulative sum can be calculated after the accumulation. Similarly, the square cumulative sum of the N channel data is added to obtain the source data square cumulative sum. In specific implementation, the intermediate number of the denominator term can be calculated based on the source data cumulative sum and the source data square cumulative sum. Specifically, the intermediate number of the denominator term can be: The constant ε can generally be a small constant, mainly to avoid the situation where the denominator item number is too small and close to zero when performing normalization calculations.

[0081] The following is a derivation and explanation of the expression used for the intermediate number of the denominator in the normalization calculation.

[0082] In the above normalized calculation result expression, for the mean E(x) and variance Var[x], we have:

[0083]

[0084] In specific implementation, according to the corresponding expressions of the above mean E(x) and variance Var[x], substitute them into the above normalized calculation result expression, and after deduction, the calculation expression of the denominator item standard number should be obtained as follows: So we can get the middle number of the denominator as

[0085] It is well known to those skilled in the art that it is generally time-consuming to directly perform square root operations and division operations, that is, it is time-consuming to perform development operations and division operations on the intermediate numbers of the denominator in the prior art, resulting in low efficiency of normalized calculation. In order to speed up the efficiency of normalized calculation, in one embodiment of the present invention, when performing the inverse square root operation on the intermediate numbers of the denominator, an interpolation fitting method is used to perform iterative operations on the intermediate numbers of the denominator to generate the target number of the denominator after the iterative operation.

[0086] Specifically, when the square root operation adopts the interpolation fitting method, the interpolation fitting method includes Newton interpolation method. Of course, the interpolation fitting method can also be other interpolation fitting methods, such as approximate function fitting method, other variants of Taylor formula expansion, which can be selected according to actual needs.

[0087] The following example illustrates the mixed operation method of using Newton interpolation method to replace square root operation and division operation. Specifically, for the target number B of the square root operation, when the domain of the target number B is greater than 0, then:

[0088]

[0089] Among them, when the iterative calculation is adopted, we have: m is the number of iterations, y m is the result of the mth iteration, y m+1 is the result of the m+1th iteration, f′(ym) is f(y m ). It should be noted that when the above method is used for iteration, the initial value of the iteration should be configured. The initial value of the iteration can be configured according to the data format of the target result. The method of configuring the initial value of the iteration is to speed up the iteration and meet the accuracy of the iteration. In specific implementation, at least one iteration should be performed during iteration. The number of iterations can be selected as needed to meet the accuracy of the normalized calculation and the requirements of calculation efficiency.

[0090] It can be seen from the above description that when an interpolation method is used for fitting, such as Newton interpolation method, to replace the square root and division operations in the prior art, the calculation time for calculating the number of denominator items can be greatly reduced.

[0091] In one embodiment of the present invention, when the middle number of the numerator item is the first middle number of the numerator item, for the first middle number of the numerator item calculated based on the variation of the mean difference, the following is true:

[0092]

[0093] Among them, n is the number of source data in the normalized calculation source data set, x a is the current source data, Cumulative sum of source data.

[0094] From the above description, it can be seen that when calculating the intermediate number of the numerator, it is necessary to use the normalized calculation source data set loaded again, and the cumulative sum of the source data can be calculated using the above method. When determining the current source data x a After that, the first intermediate number of the numerator based on the mean difference variant calculation can be calculated. It should be noted that the mean difference variant calculation is a calculation form derived from the expression of the above normalized calculation result value.

[0095] The above shows the calculation method of the intermediate number of the numerator item and the target number of the denominator item. After that, the normalized calculation main item number can be calculated by multiplication based on the intermediate number of the numerator item and the target number of the denominator item. In one embodiment of the present invention, when the intermediate number of the numerator item is the first intermediate number of the numerator item, when the normalized calculation main item number is calculated, then:

[0096] When the intermediate number of the numerator item is multiplied by the target number of the denominator item, the first intermediate number of the numerator item is multiplied by the target number of the denominator item to generate a normalized calculation main item base number, and then the normalized calculation main item base number is multiplied by the learnable parameter to generate a normalized calculation main item number;

[0097] or,

[0098] When the intermediate number of the numerator item is multiplied by the target number of the denominator item, the first intermediate number of the numerator item is first multiplied by the learnable parameter, and the multiplication result is configured as the second intermediate number of the numerator item; thereafter, the second intermediate number of the numerator item is multiplied by the target number of the denominator item to generate a normalized calculation main item base, and the normalized calculation main item base is used as the normalized calculation main item number.

[0099] When the numerator intermediate number is the numerator first intermediate number, we can use different pipeline calculation methods according to the above description of normalized calculation of the principal number. For example, we can multiply the numerator first intermediate number by the denominator target number, and then multiply it by the learnable parameter γ. a Alternatively, the first intermediate number of the numerator is multiplied by the learnable parameter, and then multiplied by the number of denominator items, and finally the normalized number of calculated main items is obtained.

[0100] In one embodiment of the present invention, when the numerator item intermediate number is the numerator item second intermediate number, the calculation to generate the numerator item second intermediate number includes:

[0101] First, the first intermediate number of the numerator is calculated based on the mean difference variation, and the calculated first intermediate number of the numerator is multiplied by the learnable parameter, and the result of the multiplication is configured as the second intermediate number of the numerator;

[0102] or,

[0103] First, the learnable parameter is multiplied by each source data, and then the second intermediate number of the numerator is calculated based on the mean difference variant;

[0104] When calculating the normalized number of main items, we have:

[0105] The second intermediate number of the numerator item is multiplied by the target number of the denominator item to generate a normalized calculation main item base number, and the normalized calculation main item base number is used as the normalized calculation main item number.

[0106] From the above description, it can be seen that the second intermediate number of the numerator item is consistent with the product of the first intermediate number of the numerator item and the learnable parameter. Therefore, when generating the second intermediate number of the numerator item, the corresponding product of the first intermediate number of the numerator item and the learnable parameter should be calculated. In specific implementation, different methods can be used to calculate and generate the second intermediate number of the numerator item. The specific calculation method can be selected according to needs, that is, different pipeline calculation methods can be used to improve the flexibility of pipeline parallel calculation acceleration operation.

[0107] In one embodiment of the present invention, data format conversion processing is also included, wherein:

[0108] Data format conversion processing includes:

[0109] Convert all source data in the normalized computing source data set to a target format corresponding to each source data, and generate a normalized computing target format data set based on all the target format data;

[0110] When generating the corresponding normalized calculation intermediate array, normalized calculation main item number and normalized calculation result value based on the normalized calculation target format data set, the data bit widths of the numerator intermediate number, the denominator intermediate number, the normalized calculation main item number and the normalized calculation result value are consistent with the data bit widths of the target format data;

[0111] The normalized calculation result values ​​obtained above are subjected to data format output processing to generate normalized calculation result values ​​in a target output format after the data format output processing.

[0112] It can be understood that, for the source data in the normalized calculation source data set, if the source data is an input variable in a 16-bit wide integer signed number format, when calculating the intermediate variable such as At the same time, if all integer operations are performed without losing precision, the bit width needs to be expanded by many extra bits to retain these intermediate variables, which will make the intermediate resources required for the calculation become huge.

[0113] In one embodiment of the present invention, the source data can be converted into target format data, such as the source data can be converted into a custom format such as FP16 of IEEE754. At this time, the source data can be expressed with a 1-bit sign width, a 7-bit exponent width, and an 8-bit mantissa width, that is, the source data is represented with only a 16-bit width, and the source data range can reach 2 64 , the intermediate results of the calculation can still be represented by 16-bit bit width, which greatly reduces the hardware resource consumption caused by the expansion of bit width.

[0114] In specific implementation, after determining the target format, such as the above-mentioned custom format of FP16 of IEEE754, the source data can be converted into the target format to obtain the target format data. The data conversion method can refer to the corresponding description of IEEE754. Thereafter, after the above-mentioned normalization operation, the normalized calculation result value of the target format can be obtained, but the normalized calculation result value of the target format should be processed for data format output, wherein the data format output may include the inverse conversion of the data format or the transformation of the data format. The inverse conversion of the data format is to convert the above-mentioned data format into the original data format. The transformation of the data format specifically refers to converting the normalized calculation result value into other data formats, such as converting the normalized calculation result value into a data format that can be processed subsequently, so as to obtain the normalized calculation result value of the target output format. The data format output processing can be selected as needed to meet the actual needs. It can be understood that when the above-mentioned data format conversion is performed, if the custom format of FP16 of IEEE754 is adopted, more hardware resource space can be saved at the expense of less precision, but still a fairly high accuracy can be guaranteed.

[0115] It should be understood that if the source data format of the source data is consistent with the target format, the above-mentioned data format conversion process can be omitted.

[0116] Figure 2 An embodiment of using pipeline concurrent computing to realize normalized computing acceleration is shown in FIG. Figure 2 A method of accelerating normalized calculation is specifically described, specifically including:

[0117] Step S0: inputting the channel data groups required by a total of N data channels one by one;

[0118] Step S1: convert the input source data into a format to obtain corresponding target format data;

[0119] Step S2: Calculate the cumulative sum of the channel data corresponding to each data channel in parallel;

[0120] Step S3: Calculate the square cumulative sum of the channel data corresponding to each data channel in parallel;

[0121] Step S4: Determine whether the source data required for the normalization calculation has been sent. If the source data has been sent, proceed to step S5, otherwise return to step S2 and step S3;

[0122] Step S5: Based on the calculation results of step S2 and step S3, the cumulative sum of source data and the cumulative sum of squares of source data are calculated and generated;

[0123] Step S6: Calculate the middle number of the denominator;

[0124] Step S7: using an interpolation calculation method to calculate and generate the number of denominator item markers;

[0125] Step S8: inputting the channel data group corresponding to the N data channels again;

[0126] Step S9: performing a unified format conversion on the input source data to obtain corresponding target format data;

[0127] Step S10: Calculate and generate the first intermediate number of the numerator;

[0128] Step S11: Calculate the product of the first intermediate number of the numerator and the target number of the denominator;

[0129] Step S12: quickly perform multiplication and addition calculations, wherein during the multiplication and addition calculations, a normalized calculation of the intermediate cardinality and the product of the learnable parameter is performed, and the product is added to the normalized calculation auxiliary item number;

[0130] Step S13: Determine whether the source data of the normalization calculation has been sent. If it has been sent, go to step S14, otherwise go to step S8;

[0131] Step S14: perform an inverse data format conversion on the normalized operation result value to obtain a normalized calculation result value in the original data format; of course, it can be seen from the above description that for the normalized operation result value, when the requirements such as the accuracy of the normalized operation are met, the normalized operation result value can be directly output, that is, the data format inverse conversion is not required here, or the normalized operation result value is converted into other types required subsequently, such as converting the normalized operation result value into data in an integer format. The specific situation can be selected according to actual needs and will not be listed here one by one.

[0132] S15: End this calculation, that is, complete the normalization calculation.

[0133] Depend on Figure 2As can be seen from the above description, when pipeline concurrent calculation is used, an embodiment is shown in which the intermediate number of the denominator is first calculated and the target number of the denominator is calculated, and then the first intermediate number of the numerator is calculated. Other pipeline concurrent calculation methods can be selected according to actual needs, and the corresponding descriptions above can be referred to, and no more examples are given one by one.

[0134] From the above description, it can be seen that an accelerated operation device suitable for normalized calculation of a neural network can be obtained. In one embodiment of the present invention, it includes an accelerated operation processor, wherein at least a neural network is deployed in the accelerated operation processor, and when normalized calculation is performed in the neural network, for any source data in the normalized calculation source data set, the accelerated operation processor uses the above-mentioned accelerated operation method to perform normalized calculation and generates a normalized operation result value corresponding to the current source data.

[0135] Specifically, the accelerated operation processor can use an existing commonly used processing device, such as a computing device that can deploy and run a neural network. The type of accelerated operation processor can be selected according to actual needs. When performing normalization calculation on the source data in the normalized calculation source data set, refer to the above description, which will not be repeated here.

Claims

1. A method for accelerating normalized computation of a neural network, characterized in that: The accelerated operation method comprises: When performing normalization calculation in the neural network, N data channels are configured to load channel data groups respectively, and a normalized calculation source data set is generated based on the N channel data groups; When normalizing any source data in the normalized calculation source data set, a pipeline concurrent calculation method is used to calculate and generate a normalized calculation intermediate array, and a normalized calculation main item number is calculated based on the normalized calculation intermediate array, where: The normalized calculation intermediate array includes the intermediate numbers of the numerator and the intermediate numbers of the denominator. The numerator item intermediate number includes a numerator item first intermediate number or a numerator item second intermediate number corresponding to the current source data, wherein the numerator item first intermediate number is calculated and generated based on the mean difference variant, and the numerator item second intermediate number is consistent with the product result of the numerator item first intermediate number and the learnable parameter; The target number of the denominator item is calculated based on the intermediate number of the denominator item, wherein when calculating and generating the target number of the denominator item, at least the inverse square root operation is performed on the intermediate number of the denominator item; Multiply the intermediate number of the numerator item and the target number of the denominator item to generate a normalized calculation main item base after the multiplication operation; Based on the normalized calculation main item cardinality, the normalized calculation main item number is calculated and generated; The normalized calculation main item number and the normalized calculation auxiliary item number generated by the calculation are added, and the result value of the addition operation is used as the normalized calculation result value of the current source data.

2. The accelerated operation method suitable for normalized calculation of a neural network according to claim 1, characterized in that: When using pipeline concurrent computing to generate normalized computing intermediate arrays, at least two channel data group loading processes are configured, where: Each time a channel data set is loaded, a corresponding normalized calculation source data set is generated; A normalized calculation source data set is used to calculate and generate the corresponding intermediate number of the numerator term, and another normalized calculation source data set is used to calculate and generate the corresponding intermediate number of the denominator term.

3. The accelerated operation method suitable for normalized calculation of a neural network according to claim 2, characterized in that: When calculating the middle number of the denominator, include: For the channel data groups loaded respectively into the N data channels, the channel data cumulative sum and the channel data square cumulative sum of the channel data group corresponding to each data channel are calculated; Performing an addition operation on the channel data accumulation sums corresponding to the N data channels to calculate and generate the source data accumulation sum; Performing an addition operation on the square cumulative sums of the channel data corresponding to the N data channels to calculate and generate the square cumulative sum of the source data; Based on the source data cumulative sum and the source data square cumulative sum, the denominator intermediate number is calculated and generated, where the denominator intermediate number is is the cumulative sum of the source data, is the cumulative sum of the squares of the source data, n is the number of source data in the normalized calculation source data set, x i To normalize the i-th source data in the source data set, ε is a constant.

4. The accelerated operation method suitable for normalized calculation of a neural network according to claim 3, characterized in that: When the inverse square root operation is performed on the intermediate number of the denominator, the interpolation fitting method is used to iteratively operate the intermediate number of the denominator to generate the target number of the denominator after the iterative operation.

5. The accelerated operation method suitable for normalized calculation of a neural network according to claim 2, characterized in that: Each time a channel data group is loaded, N data channels are configured to load the corresponding channel data groups in parallel.

6. The accelerated operation method suitable for normalized calculation of a neural network according to any one of claims 1 to 5, characterized in that: When the middle of the numerator is the first middle of the numerator, for the first middle of the numerator calculated based on the variation of the mean difference, we have: Among them, n is the number of source data in the normalized calculation source data set, x a is the current source data, Cumulative sum of source data.

7. The accelerated operation method suitable for normalized calculation of a neural network according to claim 6, characterized in that: When the middle number of the numerator is the first middle number of the numerator, and the normalized number of principal terms is calculated, then: When the intermediate number of the numerator item is multiplied by the target number of the denominator item, the first intermediate number of the numerator item is multiplied by the target number of the denominator item to generate a normalized calculation main item base number, and then the normalized calculation main item base number is multiplied by the learnable parameter to generate a normalized calculation main item number; or, When the intermediate number of the numerator item is multiplied by the target number of the denominator item, the first intermediate number of the numerator item is first multiplied by the learnable parameter, and the multiplication result is configured as the second intermediate number of the numerator item; thereafter, the second intermediate number of the numerator item is multiplied by the target number of the denominator item to generate a normalized calculation main item base, and the normalized calculation main item base is used as the normalized calculation main item number.

8. The accelerated operation method suitable for normalized calculation of a neural network according to claim 6, characterized in that: When the middle number of the numerator is the second middle number of the numerator, the calculation to generate the second middle number of the numerator includes: First, the first intermediate number of the numerator is calculated based on the mean difference variation, and the calculated first intermediate number of the numerator is multiplied by the learnable parameter, and the result of the multiplication is configured as the second intermediate number of the numerator; or, First, the learnable parameter is multiplied by each source data, and then the second intermediate number of the numerator is calculated based on the mean difference variant; When calculating the normalized number of main items, we have: The second intermediate number of the numerator item is multiplied by the target number of the denominator item to generate a normalized calculation main item base number, and the normalized calculation main item base number is used as the normalized calculation main item number.

9. The accelerated operation method suitable for normalized calculation of a neural network according to any one of claims 1 to 5, characterized in that: It also includes data format conversion processing, in which: Data format conversion processing includes: Convert all source data in the normalized computing source data set to a target format corresponding to each source data, and generate a normalized computing target format data set based on all the target format data, wherein the data bit width of the target format data is consistent with the data bit width of the source data; When generating the corresponding normalized calculation intermediate array, normalized calculation main item number and normalized calculation result value based on the normalized calculation target format data set, the data bit widths of the numerator intermediate number, the denominator intermediate number, the normalized calculation main item number and the normalized calculation result value are consistent with the data bit widths of the target format data; The normalized calculation result values ​​obtained above are subjected to data format output processing to generate normalized calculation result values ​​in a target output format after the data format output processing.

10. An accelerated operation device suitable for normalized calculation of a neural network, characterized in that: It includes an accelerated operation processor, wherein at least a neural network is deployed in the accelerated operation processor, and when normalization calculation is performed in the neural network, for any source data in the normalized calculation source data set, the accelerated operation processor adopts the accelerated operation method described in any one of claims 1 to 9 to perform normalization calculation and generate a normalized operation result value corresponding to the current source data.

Images