A low-power chip, a data processing method thereof and a storage medium

Abstract

This invention discloses a low-power chip, its data processing method, and a storage medium. The method includes: evaluating whether the bias will overflow after quantization based on a first quantization factor of a block image, a second quantization factor of the weight matrix of the block image, and the bias corresponding to the block image; if it is determined that the bias will overflow after quantization, then at least one of the first quantization factor and the second quantization factor is corrected; quantizing the block image, the weight matrix, and the bias using the corrected quantization factor and the remaining uncorrected quantization factors; performing fixed-point operations on the quantized block image, the quantized weight matrix, and the quantized bias using a fixed-point arithmetic unit for the corresponding network layer to obtain the fixed-point operation result; and performing inverse quantization on the fixed-point operation result to obtain the block features of the block image.

Description

Technical Field

[0001] This invention relates to the field of neural network quantization, and in particular to a low-power chip, its data processing method, and storage medium. Background Technology

[0002] In image recognition, neural networks are typically used to extract features from images and then identify them.

[0003] However, neural networks have a very large number of parameters and computational load, and these data are all floating-point data. The processing speed of floating-point operation instructions in low-power chips is usually slow. Therefore, it is difficult to process the large amount of data generated by each network layer in the neural network in a timely manner. This makes it difficult to deploy neural networks on low-power chips, or even if they can be deployed, the processing speed is extremely slow and cannot meet user needs.

[0004] In existing technologies, to improve the processing speed of low-power chips with deployed neural networks, the floating-point data received by each network layer in the neural network is often quantized into integer data (also known as fixed-point data). Then, the fixed-point arithmetic unit in the low-power chip performs fixed-point calculations on the quantized data for the corresponding network layer. Since the processing speed of fixed-point arithmetic instructions is much faster than that of floating-point instructions, the processing speed of low-power chips with deployed neural networks can be improved.

[0005] Typically, neural networks contain convolutional layers and fully connected layers. During the quantization process in these layers, if the activation values ​​and their weights have extremely small distribution extremes, the bias during quantization will overflow. This causes partial loss of the quantized data received by the fixed-point arithmetic unit, resulting in inaccurate calculations and reduced precision of the image features extracted by the convolutional and fully connected layers. These low-precision features accumulate across the neural network layers, leading to inaccurate image recognition even when using low-power chips with deployed neural networks.

[0006] Therefore, how to improve the image recognition accuracy of low-power chips with deployed neural networks has become a pressing technical problem. Summary of the Invention

[0007] This invention provides a low-power chip, its data processing method, and a storage medium to solve the technical problem of low image recognition accuracy of low-power chips with deployed neural networks in the prior art.

[0008] Firstly, to address the aforementioned technical problems, embodiments of the present invention provide a data processing method for a low-power chip, wherein the low-power chip includes a fixed-point arithmetic unit, and the technical solution of the data processing method is as follows:

[0009] Based on the first quantization factor of the received block image, the second quantization factor of the weight matrix of the block image, and the bias value corresponding to the block image, it is evaluated whether the bias value will overflow after being quantized; wherein, the data types corresponding to the block image, the weight matrix, and the bias value are floating-point data;

[0010] If it is determined that the bias will overflow after being quantized, then at least one of the first quantization factor and the second quantization factor is corrected.

[0011] The block image, the weight matrix, and the bias are quantized using the corrected quantization factor and the remaining uncorrected quantization factors;

[0012] The fixed-point arithmetic unit is used to perform fixed-point arithmetic on the quantized block image, the quantized weight matrix, and the quantized bias in the corresponding network layer to obtain the fixed-point arithmetic result; wherein, the data types of the quantized block image, the quantized weight matrix, and the quantized bias are all integer data;

[0013] The fixed-point operation result is dequantized to obtain the block features of the block image.

[0014] One possible implementation involves evaluating whether the bias will overflow after quantization, based on a first quantization factor of the received block image, a second quantization factor of the weight matrix of the block image, and the bias corresponding to the block image, including:

[0015] The first quantization factor is determined based on the extreme values ​​of the data distribution in the block image; the second quantization factor is determined based on the extreme values ​​of the data distribution in the weight matrix.

[0016] The bias is prequantized using the first quantization factor and the second quantization factor to obtain the prequantized bias.

[0017] Determine whether the prequantized bias is greater than or equal to the maximum value of the data range represented by the largest integer data that the fixed-point arithmetic unit can process;

[0018] If yes, it is determined that the bias amount will overflow after being quantized; otherwise, it is determined that the bias amount will not overflow after being quantized.

[0019] One possible implementation involves modifying at least one of the first quantization factor and the second quantization factor, including:

[0020] Determine whether at least one of the first quantization factor and the second quantization factor is invalid data; wherein, the invalid data is data whose value is greater than a preset value;

[0021] If at least one of the first quantization factor and the second quantization factor is invalid data, then the quantization factor determined to be invalid data is corrected.

[0022] If neither the first quantization factor nor the second quantization factor is invalid data, then either the first quantization factor or the second quantization factor shall be corrected.

[0023] One possible implementation involves correcting the quantization factor determined to be invalid data, including:

[0024] Based on the reverse processing of the prequantized bias, the maximum value, the absolute value of the bias, and the data not identified as invalid are reverse processed to obtain the pre-correction value of the quantization factor corresponding to the invalid data.

[0025] Determine whether the pre-correction value is greater than the maximum value;

[0026] If the pre-correction value is greater than or equal to the maximum value, the quantization factor corresponding to the invalid data is finally corrected to the difference between the maximum value and 1.

[0027] If the pre-correction value is less than the maximum value, the quantization factor corresponding to the invalid data will eventually be corrected to the pre-correction value.

[0028] One possible implementation involves, based on the reverse processing of the pre-quantized bias, performing reverse processing on the maximum value, the absolute value of the bias, and the data not identified as invalid, to obtain a pre-correction value for the quantization factor corresponding to the invalid data, including:

[0029] If both the first quantization factor and the second quantization factor are invalid data, then the ratio of the difference between the maximum value and 1 to the absolute value of the bias is determined as the pre-correction value of the first quantization factor and the second quantization factor.

[0030] If one of the first quantization factor and the second quantization factor is invalid data, then the difference between the maximum value and 1, the absolute value of the bias, and the ratio of the quantization factor corresponding to the invalid data are determined as the pre-correction value of the quantization factor corresponding to the invalid data.

[0031] One possible implementation involves modifying either the first quantization factor or the second quantization factor, including:

[0032] Calculate the multiple of the pre-quantized bias value to the difference between the maximum value and 1;

[0033] Either the first quantization factor or the second quantization factor is modified to the ratio of the original value to the multiple.

[0034] One possible implementation involves using the fixed-point arithmetic unit to perform fixed-point arithmetic on the quantized block image, the quantized weight matrix, and the quantized bias value for the corresponding network layer, obtaining the fixed-point arithmetic result, including:

[0035] The fixed-point arithmetic unit is used to perform logic circuit-level multiplication operations on the quantized block image and the quantized weight matrix.

[0036] The result of the multiplication operation is added to the quantized bias value at the logic circuit level to obtain the fixed-point operation result.

[0037] Secondly, embodiments of the present invention provide an apparatus for preventing neural network bias quantization overflow, comprising:

[0038] An evaluation unit is used to evaluate whether the bias will overflow after being quantized, based on a first quantization factor of the received block image, a second quantization factor of the weight matrix of the block image, and a bias corresponding to the block image; wherein the data types of the block image, the weight matrix, and the bias are floating-point data.

[0039] A correction unit is configured to correct at least one of the first quantization factor and the second quantization factor if it is determined that an overflow will occur after the bias is quantized.

[0040] A quantization unit is used to quantize the block image, the weight matrix, and the bias using a corrected quantization factor and the remaining uncorrected quantization factors.

[0041] A fixed-point arithmetic unit is used to perform fixed-point arithmetic on the quantized block image, the quantized weight matrix, and the quantized bias amount in the corresponding network layer to obtain the fixed-point arithmetic result; wherein, the data types of the quantized block image, the quantized weight matrix, and the quantized bias amount are all integer data;

[0042] The dequantization unit is used to perform dequantization on the fixed-point operation result to obtain the block features of the block image.

[0043] In one possible implementation, the evaluation unit is specifically used for:

[0044] The first quantization factor is determined based on the extreme values ​​of the data distribution in the block image; the second quantization factor is determined based on the extreme values ​​of the data distribution in the weight matrix.

[0045] The bias is prequantized using the first quantization factor and the second quantization factor to obtain the prequantized bias.

[0046] Determine whether the prequantized bias is greater than or equal to the maximum value of the data range represented by the largest integer data that the fixed-point arithmetic unit can process;

[0047] If yes, it is determined that the bias amount will overflow after being quantized; otherwise, it is determined that the bias amount will not overflow after being quantized.

[0048] In one possible implementation, the correction unit is used for:

[0049] Determine whether at least one of the first quantization factor and the second quantization factor is invalid data; wherein, the invalid data is data whose value is greater than a preset value;

[0050] If at least one of the first quantization factor and the second quantization factor is invalid data, then the quantization factor determined to be invalid data is corrected.

[0051] If neither the first quantization factor nor the second quantization factor is invalid data, then either the first quantization factor or the second quantization factor shall be corrected.

[0052] In one possible implementation, the correction unit is further configured to:

[0053] Based on the reverse processing of the prequantized bias, the maximum value, the absolute value of the bias, and the data not identified as invalid are reverse processed to obtain the pre-correction value of the quantization factor corresponding to the invalid data.

[0054] Determine whether the pre-correction value is greater than the maximum value;

[0055] If the pre-correction value is greater than or equal to the maximum value, the quantization factor corresponding to the invalid data is finally corrected to the difference between the maximum value and 1;

[0056] If the pre-correction value is less than the maximum value, the quantization factor corresponding to the invalid data will eventually be corrected to the pre-correction value.

[0057] In one possible implementation, the correction unit is further configured to:

[0058] If both the first quantization factor and the second quantization factor are invalid data, then the ratio of the difference between the maximum value and 1 to the absolute value of the bias is determined as the pre-correction value of the first quantization factor and the second quantization factor.

[0059] If one of the first quantization factor and the second quantization factor is invalid data, then the difference between the maximum value and 1, the absolute value of the bias, and the ratio of the quantization factor corresponding to the invalid data are determined as the pre-correction value of the quantization factor corresponding to the invalid data.

[0060] In one possible implementation, the correction unit is further configured to:

[0061] Calculate the multiple of the pre-quantized bias value to the difference between the maximum value and 1;

[0062] Either the first quantization factor or the second quantization factor is modified to the ratio of the original value to the multiple.

[0063] In one possible implementation, the fixed-point arithmetic unit is further used for:

[0064] The fixed-point arithmetic unit is used to perform logic circuit-level multiplication operations on the quantized block image and the quantized weight matrix.

[0065] The result of the multiplication operation is added to the quantized bias value at the logic circuit level to obtain the fixed-point operation result.

[0066] Thirdly, embodiments of the present invention also provide an apparatus for preventing neural network bias quantization overflow, comprising:

[0067] At least one processor, and

[0068] Memory connected to the at least one processor;

[0069] The memory stores instructions that can be executed by the at least one processor, which executes the instructions stored in the memory to perform the method described in the first aspect above.

[0070] Fourthly, embodiments of the present invention also provide a readable storage medium, comprising:

[0071] memory,

[0072] The memory is used to store instructions that, when executed by a processor, cause the apparatus including the readable storage medium to perform the method described in the first aspect above.

[0073] Through the technical solutions in one or more of the above embodiments of the present invention, the embodiments of the present invention have at least the following technical effects:

[0074] In the embodiments provided by this invention, before quantizing the received block image and its corresponding weight matrix and bias, the system assesses whether the bias will overflow after quantization based on the first quantization factor of the received block image, the second quantization factor of the weight matrix of the block image, and the bias corresponding to the block image. If it is determined that the bias will overflow after quantization, at least one of the first quantization factor and the second quantization factor is corrected. Then, the block image, weight matrix, and bias are quantized using the corrected quantization factor and the remaining uncorrected quantization factors, thereby preventing the bias from overflowing during the quantization process and improving the accuracy of the quantized bias. In this way, when the fixed-point arithmetic unit performs fixed-point arithmetic on the quantized block image, the quantized weight matrix, and the quantized bias for the corresponding network layer, the calculated fixed-point arithmetic result will not be inaccurate due to overflow. Thus, the block features of the block image obtained by inverse quantization of the fixed-point arithmetic result are more accurate, ultimately achieving the goal of effectively improving the image recognition accuracy of low-power chips with deployed neural networks. Attached Figure Description

[0075] Figure 1 This is a schematic diagram illustrating how a network layer in a low-power chip with a deployed neural network performs quantization after receiving data.

[0076] Figure 2 A schematic diagram illustrating the dequantization process of using a fixed-point arithmetic unit to calculate the quantized data for the network layers of a neural network to obtain the result;

[0077] Figure 3 A schematic diagram illustrating the conventional method of processing data for network layers in a neural network;

[0078] Figure 4 A schematic diagram illustrating the inverse quantization of data processed by the network layers of a neural network deployed in a low-power chip;

[0079] Figure 5 A flowchart illustrating a data processing method for a low-power chip provided in an embodiment of the present invention;

[0080] Figure 6 This is a schematic diagram of a low-power chip provided in an embodiment of the present invention. Detailed Implementation

[0081] This invention provides a low-power chip, its data processing method, and a storage medium to solve the technical problem of low image recognition accuracy in chips with deployed neural networks in the prior art.

[0082] To better understand the above technical solutions, the technical solutions of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solutions of the present invention, rather than limitations on the technical solutions of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0083] Please see Figure 1 This is a schematic diagram illustrating how a network layer in a low-power chip with a deployed neural network performs quantization after receiving data.

[0084] Neural networks consist of convolutional (CONV) layers and fully connected (FP) layers. Their input (X) is typically 32-bit floating-point data (denoted as FLAOT32 or FP32). Taking a convolutional layer as an example, to improve the speed of convolution calculations, a hardware accelerator in a low-power chip is needed to perform calculations using logic circuits. The hardware accelerator includes multiple fixed-point arithmetic units, but fixed-point arithmetic units can only process integer data. Therefore, the floating-point data received by the convolutional layer needs to be quantized into fixed-point data (also called integer data) before being provided to the fixed-point arithmetic units for calculation. The data received by the convolutional layer includes image data (denoted as X), the weights of X (i.e., Weight, which is the convolution kernel for the corresponding channel in the convolutional layer), and the corresponding bias (denoted as Bias). Their quantization process is as follows: By multiplying X by a fixed quantization factor (denoted as scaleX), we can obtain 8-bit integer data (abbreviated as INT8) (denoted as X_Q, i.e., the quantized X). Weight is multiplied by a fixed weight quantization factor (denoted as scaleW) to obtain INT8 data (denoted as W_Q, i.e., the quantized Weight). To prevent data overflow, the matrix product of X_Q and W_Q is usually of INT32 data type. Therefore, Bias is multiplied by (scaleX × scaleW) to obtain INT32 data (denoted as B_Q, i.e., the quantized Bias), which makes it convenient to add B_Q to the matrix product of X_Q and W_Q.

[0085] Please see Figure 2 This diagram illustrates the dequantization process of a neural network layer, where a fixed-point arithmetic unit is used to compute the quantized data to obtain the result. Y_Q is the result of the fixed-point arithmetic unit adding the matrix product of X_Q and W_Q to B_Q. The data type of Y_Q is INT32. Dequantizing Y_Q involves dividing Y_Q by (scaleX × scaleW) to restore the output Y, which has a data type of FP32.

[0086] Please see Figure 3 and Figure 4 , Figure 3 A diagram illustrating the conventional way data is processed by the network layers of a neural network. Figure 4 A schematic diagram illustrating how network layers of a neural network deployed in a low-power chip process data.

[0087] Please see Figure 3 In a neural network, a network layer, such as a convolutional layer, receives the image data X output from the previous layer along with the corresponding weights and biases. It then directly calculates (Weight × X) using floating-point multiplication and finally uses floating-point addition to obtain Y = (Weight × X) + Bias. Because neural networks consist of numerous layers, including multiple convolutional and fully connected layers, each layer processes a massive number of parameters. Directly using floating-point calculations for each layer would require the device supporting the neural network to have extremely powerful data processing capabilities. In image recognition, due to the vast number of images acquired, the recognition function is often moved to the front end to improve speed and reduce data transmission. However, front-end devices are typically low-power devices with limited data processing capabilities. This means that neural networks either cannot be deployed on low-power devices, or even if some relatively high-performance low-power devices can accommodate them, they will run slowly and fail to meet user needs.

[0088] Please see Figure 4 When a neural network is deployed to a low-power chip in a low-power device, to improve its data processing speed, the floating-point values ​​X, Weight, and Bias are first quantized into integer values ​​X_Q, W_Q, and B_Q. These are then provided to the fixed-point arithmetic unit in the low-power chip for calculation, resulting in an integer value Y_Q. Finally, Y_Q is dequantized to restore it to a floating-point value Y for the next network layer. The scaleX and scaleW used in the quantization process are obtained by analyzing the extreme values ​​M of the distributions of X and Weight and taking their reciprocals. Taking an 8-bit (INT8) quantization width as an example, scaleX = (2... 8 -1) / M_X, scaleW = (2 8 -1) / M_Weight, where M_X is the extreme value of the X distribution and M_Weight is the extreme value of the Weight distribution.

[0089] The inventors discovered that if the absolute value of all values ​​of X or Weight is 10... -20 If it is within a certain range, then M will be very small (e.g., less than 10). -20 This leads to a very large scaleX or scaleW, since the range represented by INT32 is -2,147,483,648 to 2,147,483,647 (approximately -2.15 × 10⁻⁶).10 ~2.15×10 10 When quantizing the bias value to INT32, overflow will occur. For example: scaleX is 2.0, scaleW is 10. 20 If the bias is 0.05, then the quantized bias is 0.05 × 2.0 × 10⁻⁶. 20 =10 19 This clearly exceeds the representation range of INT32; the INT32 value corresponding to the quantized bias is truncated to 2,147,483,647 (approximately 2.15 × 10⁻⁶). 10 This is different from the actual value of 10. 19 The difference is huge. After dequantizing the quantized bias, the resulting bias is: 2,147,483,647 / 10 20 / 2.0 = 1.0737418235 × 10 -11 Compared to the initial quantization value of 0.05, the error is extremely large, leading to a sharp decrease in the quantization accuracy of the final network inference. Furthermore, during neural network training, there is a high probability that the learned values ​​of a certain channel or the entire weight in a convolutional layer or fully connected layer will be very close to 0 (e.g., absolute values ​​less than 10). -20 This makes it very easy for bias overflow to occur when using low-power chips with deployed neural networks, resulting in the inability to accurately extract image features and thus reducing the accuracy of image recognition.

[0090] To address the above problems, embodiments of the present invention provide a low-power chip, its data processing method, and a storage medium, as detailed below:

[0091] Please refer to Figure 5 This invention provides a data processing method for a low-power chip, which includes a fixed-point arithmetic unit. The processing procedure of the data processing method is as follows.

[0092] Step 501: Based on the first quantization factor of the received block image, the second quantization factor of the weight matrix of the block image, and the bias value corresponding to the block image, evaluate whether the bias value will overflow after being quantized; wherein, the data types corresponding to the block image, weight matrix, and bias value are floating-point data.

[0093] Neural networks typically consist of multiple layers, such as input layers, convolutional layers, pooling layers, and fully connected layers. Convolutional layers extract minute features from received image patches by performing convolution operations on those patches. Fully connected layers combine the minute features extracted by previous layers to form a larger feature set for the entire image. Regardless of whether it's a convolutional or fully connected layer, the basic operations they use to extract image features are similar. Figure 4The difference in the operation shown is that multiple convolution operations are required in the convolutional layer, and the operation method corresponding to each convolution operation is as follows: Figure 4 The computation method in the fully connected layer is different from that in the fully connected layer, where computation only needs to be performed. Figure 4 One calculation in the process.

[0094] When the network layer is a convolutional layer, the above-mentioned block image is a portion of the image received by the convolutional layer within the sliding window corresponding to the convolutional kernel, and the above-mentioned weight matrix is ​​the convolutional kernel used for the corresponding data channel (such as the R channel) in the image.

[0095] When the network layer is a fully connected layer, the above block image is the image received by the fully connected layer, and the above weight matrix is ​​a matrix composed of the weights of each neuron connected to the fully connected layer.

[0096] It is important to understand that since a block image is composed of pixels arranged in an array, with each pixel corresponding to a pixel value, a block image can be regarded as a matrix.

[0097] After the network layer receives the block image, the corresponding weight matrix, and the bias, if the first quantization factor of the block image and the second quantization factor of the weight matrix are already determined, the first and second quantization factors and the bias can be directly used to evaluate whether overflow will occur after the bias is quantized. If the first quantization factor of the block image and the second quantization factor of the weight matrix are not yet determined, the first and second quantization factors can be determined in the following ways:

[0098] The first quantization factor is determined based on the extreme values ​​of the data distribution in the block image; the second quantization factor is determined based on the extreme values ​​of the data distribution in the weight matrix.

[0099] For example, by analyzing the data distribution in a block image, the maximum and minimum values ​​of the block image can be determined. The one with the largest absolute value between the maximum and minimum values ​​is identified as the extreme value of the block image, and the reciprocal of the extreme value is used as the first quantization factor. Similarly, the second quantization factor can be determined.

[0100] After obtaining the first quantization factor and the second quantization factor, it is possible to assess whether the quantized bias will result in an output. This can be achieved in the following ways:

[0101] The bias is prequantized using the first quantization factor and the second quantization factor to obtain the prequantized bias. It is then determined whether the prequantized bias is greater than the maximum value of the data range represented by the largest integer data that the fixed-point arithmetic unit can process. If it is, it is determined that the bias will overflow after being quantized; otherwise, it is determined that the bias will not overflow after being quantized.

[0102] For example, suppose that the data in the block image, the data in the weight matrix, and the bias are all of type FP32, the maximum integer data that the fixed-point arithmetic unit in the low-power chip can process is INT32, the maximum value of the data range represented by INT32 is 2147483647, the value of the first quantization factor is a, the value of the second quantization factor is b, and the value of the bias is c.

[0103] First, prequantize c using a and b. The value of the prequantized bias is a×b×c. Then, determine whether a×b×c is greater than or equal to 2147483647. If it is, it is determined that the bias will overflow after being quantized; otherwise, it is determined that the bias will not overflow after being quantized.

[0104] It is important to understand that the pre-quantization mentioned above is still quantization, but the result of pre-quantization is not to provide a given point arithmetic unit. Instead, it is used to determine whether the bias will overflow after being quantized, and then to decide whether to correct the first quantization factor and / or the second quantization factor.

[0105] If it is determined that the bias will not overflow, the block image is quantized using the first quantization factor, the weight matrix is ​​quantized using the second quantization factor, and the bias is quantized using the first and second quantization factors. After quantization, the quantized values ​​are provided to the fixed-point arithmetic unit, which performs fixed-point arithmetic on the corresponding network layer. Finally, the fixed-point arithmetic result is dequantized and UN is calculated to obtain the block features of the block image.

[0106] If it is determined that the bias will overflow after being quantized, then proceed to step 502.

[0107] Step 502: If it is determined that the bias will overflow after being quantized, then at least one of the first quantization factor and the second quantization factor is corrected.

[0108] In the embodiments provided by the present invention, by pre-quantizing the bias and determining whether the pre-quantized bias overflows, and by correcting at least one of the first quantization factor and the second quantization factor when it is determined that overflow will occur, the inaccurate data after quantization due to the overflow of the bias during the quantization process can be prevented, thereby improving the accuracy of the corresponding image recognition.

[0109] In some embodiments, modifying at least one of the first quantization factor and the second quantization factor can be achieved in the following ways:

[0110] Determine whether at least one of the first quantization factor and the second quantization factor is invalid data; where invalid data is data whose value is greater than a preset value; if at least one of the first quantization factor and the second quantization factor is invalid data, then the quantization factor determined to be invalid data is corrected; if neither the first quantization factor nor the second quantization factor is invalid data, then either the first quantization factor or the second quantization factor is corrected.

[0111] In some embodiments, the preset value can be the order of magnitude of the maximum value of the data range represented by the largest integer data that the fixed-point arithmetic unit can process. For example, if the largest integer data that the fixed-point arithmetic unit can process is INT32, its maximum value is 2147483647 ≈ 2.1 × 10⁻⁶. 10 Its order of magnitude is 10 10 Therefore, the default value can be set to 10. 10 .

[0112] By determining whether the first quantization factor and the second quantization factor are greater than the preset value, it is possible to quickly identify which of them is invalid data, that is, to identify which is the main cause of the overflow after the bias is quantized, and then correct it.

[0113] If both the first quantization factor and the second quantization factor are less than the preset value, then modifying either of them can prevent the bias from overflowing after quantization.

[0114] In the embodiments provided by the present invention, by determining whether the first quantization factor and the second quantization factor are invalid data, the quantization factor that causes the bias to overflow can be corrected, thereby preventing the bias from overflowing after quantization.

[0115] In some embodiments, the correction of quantization factors determined to be invalid data can be achieved in the following ways:

[0116] Based on the reverse processing of the pre-quantized bias, the maximum value, the absolute value of the bias, and the data not identified as invalid are processed in reverse to obtain the pre-correction value of the quantization factor corresponding to the invalid data; it is then determined whether the pre-correction value is greater than the maximum value; if the pre-correction value is greater than or equal to the maximum value, the quantization factor corresponding to the invalid data is finally corrected to the difference between the maximum value and 1; if the pre-correction value is less than the maximum value, the quantization factor corresponding to the invalid data is finally corrected to the pre-correction value.

[0117] In some embodiments, based on the reverse processing of the pre-quantized bias, the maximum value, the absolute value of the bias, and the data not determined to be invalid are reverse-processed to obtain the pre-correction value of the quantization factor corresponding to the invalid data. This can be achieved in the following ways:

[0118] If both the first quantization factor and the second quantization factor are invalid data, then the ratio of the difference between the maximum value and 1 to the absolute value of the bias is determined as the pre-correction value of the first quantization factor and the second quantization factor.

[0119] If one of the first quantization factor and the second quantization factor is invalid data, then the difference between the maximum value and 1, the absolute value of the bias, and the ratio of the quantization factor corresponding to the non-invalid data are determined as the pre-correction value of the quantization factor corresponding to the invalid data.

[0120] For example, if the value of the first quantization factor is greater than a preset value, while the value of the second quantization factor is less than a preset value, then the value of the first quantization factor is determined to be invalid data, and the value of the second quantization factor is determined to be valid data (i.e., not invalid data). In this case, according to the reverse processing of the quantized bias, the maximum value of the data range represented by the largest integer data that the fixed-point arithmetic unit can process, the bias, and the second quantization factor are reverse processed to obtain the pre-correction value of the first quantization factor, which can be expressed by the following formula:

[0121] The pre-correction value of the first quantization factor = (maximum value - 1) / |bias | / second quantization factor.

[0122] Then, it is determined whether the pre-correction value of the first quantization factor is greater than or equal to the maximum value. If it is, the first quantization factor is finally corrected to (maximum value - 1); otherwise, the first quantization factor is finally corrected to the pre-correction value of the first quantization factor.

[0123] If the value of the first quantization factor is less than the preset value, while the value of the second quantization factor is greater than or equal to the preset value, then the value of the first quantization factor is determined to be valid data, and the value of the second quantization factor is determined to be invalid data. In this case, according to the reverse processing of the quantized bias, the maximum value of the data range represented by the largest integer data that the fixed-point arithmetic unit can process, the bias, and the first quantization factor are reverse processed to obtain the pre-correction value of the second quantization factor, which can be expressed by the following formula:

[0124] The pre-correction value of the second quantization factor = (maximum value - 1) / |bias| / first quantization factor.

[0125] For example, if the maximum integer data that a fixed-point arithmetic unit in a low-power chip can process is INT32, the maximum value of the corresponding data range is 2147483647. The first quantization factor (ScaleX) = 2.0, and the second quantization factor (ScaleW) = 10. 20 Bias = 0.05, default value is 10. 10For example, the above method can be used to determine that ScaleW is invalid data and needs to be corrected. According to the above solution, ScaleW was finally corrected to 2147483646.0. Quantizing the bias using ScaleX and the corrected ScaleW yields the quantized bias as: 2.0 × 0.05 × 2147483646.0 = 214748364.60000002. Rounding this to INT32, the bias is quantized to 214748365. Then, dequantizing (214748365) gives the bias as: 214748365 / 2.0 / 2147483646.0 = 0.050000000093132255. This is very close to the original bias (0.05), indicating that the quantization error of the bias is very small, within 0.00001. This effectively ensures the accuracy of the calculation results of the subsequent fixed-point arithmetic unit and improves the image recognition accuracy of the low-power chip with deployed neural network.

[0126] Then, it is determined whether the pre-correction value of the second quantization factor is greater than or equal to the maximum value. If it is, the second quantization factor is finally corrected to (maximum value - 1); otherwise, the second quantization factor is finally corrected to the pre-correction value of the second quantization factor.

[0127] If the values ​​of both the first quantization factor and the second quantization factor are greater than or equal to the preset values, then they are both invalid data. In this case, according to the reverse processing of the quantized bias, the maximum value of the data range represented by the largest integer data that the fixed-point arithmetic unit can process and the bias are reverse processed to obtain the pre-corrected values ​​of the first quantization factor and the second quantization factor, which can be expressed by the following formula:

[0128] The pre-correction value of the first quantization factor and the second quantization factor = (maximum value - 1) / |bias|.

[0129] Then, it is determined whether the pre-correction values ​​of the first quantization factor and the second quantization factor are greater than or equal to the maximum value. If yes, the first quantization factor and the second quantization factor are finally corrected to (maximum value - 1). Otherwise, the first quantization factor and the second quantization factor are finally corrected to the square root (positive value) of the pre-correction values ​​of the first quantization factor and the second quantization factor.

[0130] In some embodiments, overflow may occur after the bias is determined to be quantized, but both the first quantization factor and the second quantization factor are valid data. In this case, correction of either the first quantization factor or the second quantization factor can be achieved in the following ways:

[0131] Calculate the multiple of the difference between the prequantized bias and the maximum value and 1; either the first quantization factor or the second quantization factor is corrected to the ratio of the original value to the multiple.

[0132] For example, if the aforementioned estimation method determines that the bias will overflow after quantization, but the relationship between the first quantization factor and the second quantization factor and the preset value determines that they are all valid data, then the multiple of the prequantized bias and (maximum value - 1) (i.e., k) can be calculated, and then the first quantization factor or the second quantization factor can be reduced by a factor of k.

[0133] After completing the correction of at least one of the first quantization factor and the second quantization factor, the border executes step 503.

[0134] Step 503: Quantize the block image, weight matrix, and bias using the corrected quantization factor pair and the remaining uncorrected quantization factors.

[0135] For example, if only the first quantization factor is corrected while the second quantization factor is not corrected, then the image data is quantized using the corrected first quantization factor, the weight matrix is ​​quantized using the second quantization factor, and the bias is corrected using the corrected first quantization factor and the second quantization factor to obtain the quantized block image, the quantized weight matrix, and the quantized bias.

[0136] For example, if both the first quantization factor and the second quantization factor are modified, then the modified first quantization factor is used to quantize the image data, the modified second quantization factor is used to quantize the weight matrix, and the modified first quantization factor and the modified second quantization factor are used to modify the bias to obtain the quantized block image, the quantized weight matrix, and the quantized bias.

[0137] For the specific quantitative calculation methods mentioned above, please refer to... Figure 1 .

[0138] In some embodiments, the quantization bit width used for the quantization block image and the weight matrix is ​​the same, and is at least half the quantization bit width used for the quantization bias.

[0139] If the quantization bit width used for the block image and weight matrix is ​​8 bits, then the quantization bit width used for the quantization bias can be 16 bits or 32 bits, or higher, but less than or equal to the bit width occupied by the largest integer data that the fixed-point arithmetic unit can process. If the bit width occupied by the largest integer data that the fixed-point arithmetic unit can process is 32 bits, then the highest bit of the quantization bit width used for the quantization bias is 32 bits.

[0140] After obtaining the quantized block image, the quantized weight matrix, and the quantized bias, the bounding box is processed through steps 504-505.

[0141] Step 504: Use a fixed-point arithmetic unit to perform fixed-point arithmetic on the quantized block image, the quantized weight matrix, and the quantized bias in the corresponding network layer to obtain the fixed-point arithmetic results; wherein, the data types of the quantized block image, the quantized weight matrix, and the quantized bias are all integer data.

[0142] Step 505: Perform inverse quantization on the fixed-point operation result to obtain the block features of the block image.

[0143] In some embodiments, the fixed-point arithmetic unit performs fixed-point arithmetic on the quantized block image, the quantized weight matrix, and the quantized bias to obtain the fixed-point arithmetic result. This can be achieved in the following ways:

[0144] A fixed-point arithmetic unit is used to perform logic circuit-level multiplication operations on the quantized block image and the quantized weight matrix.

[0145] The result of the multiplication operation is added to the quantized bias value at the logic circuit level to obtain the fixed-point arithmetic result. This fixed-point arithmetic result is the block image represented by integer data.

[0146] Let X_Q represent the quantized block image, W_Q represent the quantized weight matrix, B_Q represent the quantized bias, and Y_Q represent the fixed-point operation result. The calculation process of the above fixed-point arithmetic unit can be represented by the following formula:

[0147] Y_Q=X_Q×W_Q+B_Q.

[0148] Then, by performing inverse quantization on the fixed-point operation results, the block features of the block image (in floating-point data) can be obtained.

[0149] It is important to understand that fixed-point arithmetic units are composed of logic circuits, meaning that the above calculation process is completed purely in hardware. This makes the calculation speed faster than using software, thus effectively improving data processing speed.

[0150] In the embodiments provided by this invention, before quantizing the received block image and its corresponding weight matrix and bias, the system assesses whether the bias will overflow after quantization based on the first quantization factor of the received block image, the second quantization factor of the weight matrix of the block image, and the bias corresponding to the block image. If it is determined that the bias will overflow after quantization, at least one of the first quantization factor and the second quantization factor is corrected. Then, the block image, weight matrix, and bias are quantized using the corrected quantization factor and the remaining uncorrected quantization factors, thereby preventing the bias from overflowing during the quantization process and improving the accuracy of the quantized bias. In this way, when the fixed-point arithmetic unit performs fixed-point arithmetic on the quantized block image, the quantized weight matrix, and the quantized bias for the corresponding network layer, the calculated fixed-point arithmetic result will not be inaccurate due to overflow. Thus, the block features of the block image obtained by inverse quantization of the fixed-point arithmetic result are more accurate, ultimately achieving the goal of effectively improving the image recognition accuracy of low-power chips with deployed neural networks.

[0151] Based on the same inventive concept, one embodiment of the present invention provides a low-power chip. Specific implementation methods for the data processing of this low-power chip can be found in the description of the method embodiments section; repeated details will not be repeated here. Figure 6 The device includes:

[0152] Evaluation unit 601 is used to evaluate whether the bias amount will overflow after being quantized, based on the first quantization factor of the received block image, the second quantization factor of the weight matrix of the block image, and the bias amount corresponding to the block image; wherein, the data types corresponding to the block image, the weight matrix, and the bias amount are floating-point data.

[0153] The correction unit 602 is used to correct at least one of the first quantization factor and the second quantization factor if it is determined that the bias amount will overflow after quantization.

[0154] The quantization unit 603 is used to quantize the block image, the weight matrix, and the bias using the corrected quantization factor and the remaining uncorrected quantization factors.

[0155] The fixed-point arithmetic unit 604 is used to perform fixed-point arithmetic on the quantized block image, the quantized weight matrix, and the quantized bias amount corresponding to the network layer to obtain the fixed-point arithmetic result; wherein, the data types of the quantized block image, the quantized weight matrix, and the quantized bias amount are all integer data.

[0156] The dequantization unit 605 is used to perform dequantization on the fixed-point operation result to obtain the block features of the block image.

[0157] In one possible implementation, the evaluation unit 601 is specifically used for:

[0158] The first quantization factor is determined based on the extreme values ​​of the data distribution in the block image; the second quantization factor is determined based on the extreme values ​​of the data distribution in the weight matrix.

[0159] The bias is prequantized using the first quantization factor and the second quantization factor to obtain the prequantized bias.

[0160] Determine whether the prequantized bias is greater than or equal to the maximum value of the data range represented by the largest integer data that the fixed-point arithmetic unit can process;

[0161] If yes, it is determined that the bias amount will overflow after being quantized; otherwise, it is determined that the bias amount will not overflow after being quantized.

[0162] In one possible implementation, the correction unit 602 is used for:

[0163] Determine whether at least one of the first quantization factor and the second quantization factor is invalid data; wherein, the invalid data is data whose value is greater than a preset value;

[0164] If at least one of the first quantization factor and the second quantization factor is invalid data, then the quantization factor determined to be invalid data is corrected.

[0165] If neither the first quantization factor nor the second quantization factor is invalid data, then either the first quantization factor or the second quantization factor shall be corrected.

[0166] In one possible implementation, the correction unit 602 is further configured to:

[0167] Based on the reverse processing of the prequantized bias, the maximum value, the absolute value of the bias, and the data not identified as invalid are reverse processed to obtain the pre-correction value of the quantization factor corresponding to the invalid data.

[0168] Determine whether the pre-correction value is greater than the maximum value;

[0169] If the pre-correction value is greater than or equal to the maximum value, the quantization factor corresponding to the invalid data is finally corrected to the difference between the maximum value and 1.

[0170] If the pre-correction value is less than the maximum value, the quantization factor corresponding to the invalid data will eventually be corrected to the pre-correction value.

[0171] In one possible implementation, the correction unit 602 is further configured to:

[0172] If both the first quantization factor and the second quantization factor are invalid data, then the ratio of the difference between the maximum value and 1 to the absolute value of the bias is determined as the pre-correction value of the first quantization factor and the second quantization factor.

[0173] If one of the first quantization factor and the second quantization factor is invalid data, then the difference between the maximum value and 1, the absolute value of the bias, and the ratio of the quantization factor corresponding to the invalid data are determined as the pre-correction value of the quantization factor corresponding to the invalid data.

[0174] In one possible implementation, the correction unit 602 is further configured to:

[0175] Calculate the multiple of the pre-quantized bias value to the difference between the maximum value and 1;

[0176] Either the first quantization factor or the second quantization factor is modified to the ratio of the original value to the multiple.

[0177] In one possible implementation, the fixed-point arithmetic unit 604 is further configured to:

[0178] The fixed-point arithmetic unit is used to perform logic circuit-level multiplication operations on the quantized block image and the quantized weight matrix.

[0179] The result of the multiplication operation is added to the quantized bias value at the logic circuit level to obtain the fixed-point operation result.

[0180] Based on the same inventive concept, this embodiment of the invention provides a low-power chip, comprising: at least one processor, and

[0181] Memory connected to the at least one processor;

[0182] The memory stores instructions that can be executed by the at least one processor, and the at least one processor executes the data processing method described above by executing the instructions stored in the memory.

[0183] Based on the same inventive concept, embodiments of the present invention also provide a readable storage medium, comprising:

[0184] memory,

[0185] The memory is used to store instructions that, when executed by a processor, cause the apparatus including the readable storage medium to perform the data processing method described above.

[0186] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0187] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0188] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0189] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0190] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A data processing method for a low-power chip, the low-power chip comprising a fixed-point arithmetic unit, characterized in that, The data processing method includes: Based on the first quantization factor of the received block image, the second quantization factor of the weight matrix of the block image, and the bias value corresponding to the block image, it is evaluated whether the bias value will overflow after being quantized; wherein, the data types corresponding to the block image, the weight matrix, and the bias value are floating-point data; If it is determined that the bias will overflow after being quantized, then at least one of the first quantization factor and the second quantization factor is corrected. The block image, the weight matrix, and the bias are quantized using the corrected quantization factor and the remaining uncorrected quantization factors; The fixed-point arithmetic unit is used to perform fixed-point arithmetic on the quantized block image, the quantized weight matrix, and the quantized bias in the corresponding network layer to obtain the fixed-point arithmetic result; wherein, the data types of the quantized block image, the quantized weight matrix, and the quantized bias are all integer data; The fixed-point operation result is dequantized to obtain the block features of the block image; Specifically, the evaluation of whether the bias will overflow after quantization, based on the first quantization factor of the received block image, the second quantization factor of the weight matrix of the block image, and the bias corresponding to the block image, includes: The first quantization factor is determined based on the extreme values ​​of the data distribution in the block image; the second quantization factor is determined based on the extreme values ​​of the data distribution in the weight matrix. The bias is prequantized using the first quantization factor and the second quantization factor to obtain the prequantized bias. Determine whether the prequantized bias is greater than or equal to the maximum value of the data range represented by the largest integer data that the fixed-point arithmetic unit can process; If yes, it is determined that the bias amount will overflow after being quantized; otherwise, it is determined that the bias amount will not overflow after being quantized.

2. The data processing method as described in claim 1, characterized in that, Modifying at least one of the first quantization factor and the second quantization factor includes: Determine whether at least one of the first quantization factor and the second quantization factor is invalid data; wherein, the invalid data is data whose value is greater than a preset value; If at least one of the first quantization factor and the second quantization factor is invalid data, then the quantization factor determined to be invalid data is corrected. If neither the first quantization factor nor the second quantization factor is invalid data, then either the first quantization factor or the second quantization factor shall be corrected.

3. The data processing method as described in claim 2, characterized in that, Correcting the quantization factors identified as invalid data includes: Based on the reverse processing of the prequantized bias, the maximum value, the absolute value of the bias, and the data not identified as invalid are reverse processed to obtain the pre-correction value of the quantization factor corresponding to the invalid data. Determine whether the pre-correction value is greater than the maximum value; If the pre-correction value is greater than or equal to the maximum value, the quantization factor corresponding to the invalid data is finally corrected to the difference between the maximum value and 1; If the pre-correction value is less than the maximum value, the quantization factor corresponding to the invalid data will eventually be corrected to the pre-correction value.

4. The data processing method as described in claim 3, characterized in that, Based on the reverse processing of the pre-quantized bias, the maximum value, the absolute value of the bias, and the data not identified as invalid are reverse-processed to obtain the pre-correction value of the quantization factor corresponding to the invalid data, including: If both the first quantization factor and the second quantization factor are invalid data, then the ratio of the difference between the maximum value and 1 to the absolute value of the bias is determined as the pre-correction value of the first quantization factor and the second quantization factor. If one of the first quantization factor and the second quantization factor is invalid data, then the difference between the maximum value and 1, the absolute value of the bias, and the ratio of the quantization factor corresponding to the invalid data are determined as the pre-correction value of the quantization factor corresponding to the invalid data.

5. The data processing method as described in claim 2, characterized in that, Modifying either the first quantization factor or the second quantization factor includes: Calculate the multiple of the pre-quantized bias value to the difference between the maximum value and 1; Either the first quantization factor or the second quantization factor is modified to the ratio of the original value to the multiple.

6. The data processing method according to any one of claims 1-5, characterized in that, The fixed-point arithmetic unit is used to perform fixed-point arithmetic on the quantized block image, the quantized weight matrix, and the quantized bias value for the corresponding network layer to obtain the fixed-point arithmetic results, including: The fixed-point arithmetic unit is used to perform logic circuit-level multiplication operations on the quantized block image and the quantized weight matrix. The result of the multiplication operation is added to the quantized bias value at the logic circuit level to obtain the fixed-point operation result.

7. A low-power chip, characterized in that, include: An evaluation unit is used to evaluate whether the bias will overflow after being quantized, based on a first quantization factor of the received block image, a second quantization factor of the weight matrix of the block image, and a bias corresponding to the block image; wherein the data types of the block image, the weight matrix, and the bias are floating-point data. A correction unit is configured to correct at least one of the first quantization factor and the second quantization factor if it is determined that an overflow will occur after the bias is quantized. A quantization unit is used to quantize the block image, the weight matrix, and the bias using a corrected quantization factor and the remaining uncorrected quantization factors. A fixed-point arithmetic unit is used to perform fixed-point arithmetic on the quantized block image, the quantized weight matrix, and the quantized bias amount for the corresponding network layer to obtain the fixed-point arithmetic result; wherein, the data types of the quantized block image, the quantized weight matrix, and the quantized bias amount are all integer data; The dequantization unit is used to perform dequantization on the fixed-point arithmetic result to obtain the block features of the block image; The evaluation unit is further configured to determine the first quantization factor based on the extreme values ​​of the data distribution in the block image; determine the second quantization factor based on the extreme values ​​of the data distribution in the weight matrix; prequantize the bias amount using the first quantization factor and the second quantization factor to obtain the prequantized bias amount; determine whether the prequantized bias amount is greater than or equal to the maximum value of the data range represented by the largest integer data that the fixed-point arithmetic unit can process; if yes, determine that the bias amount will overflow after quantization, otherwise determine that the bias amount will not overflow after quantization.

8. A low-power chip, characterized in that, include: At least one processor, and Memory connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor executes the data processing method as described in any one of claims 1-6 by executing the instructions stored in the memory.

9. A readable storage medium, characterized in that, Including memory, The memory is used to store instructions that, when executed by a processor, cause the apparatus including the readable storage medium to perform the data processing method as described in any one of claims 1 to 6.

Images