Training method and device of target detection neural network, chip, board and equipment

Abstract

Embodiments of the present disclosure provide a target detection neural network training method and device, a chip, a board card, an equipment and a storage medium. In the process of training the neural network, the data of the original data type participating in the calculation can be quantized into data of different data types with different precisions before being calculated for different stages in the neural network training process. In the forward propagation process, the data of the original data type participating in the calculation can be quantized into data of a first data type with lower precision because the requirement for data precision is lower. In the backward propagation process, the data of the original data type participating in the calculation can be quantized into data of a second data type with higher precision because the requirement for data precision is higher. Through this targeted and differentiated quantization method, the resource consumption in the neural network training process can be maximally reduced while ensuring that the performance of the trained neural network is not greatly affected.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a training method, apparatus, chip, board, device and storage medium for an object detection neural network. Background Technology

[0002] Neural networks are applied in various fields. As the tasks processed by neural networks become increasingly diverse and complex, their scale and structure also increase. This leads to the consumption of significant computational and storage resources during training, placing high demands on the performance of the chips used for training and resulting in lengthy training times. Therefore, a solution is needed to reduce resource consumption without significantly impacting the accuracy of the trained neural network. Summary of the Invention

[0003] This disclosure provides a training method, apparatus, chip, board, device, and storage medium for an object detection neural network.

[0004] According to a first aspect of the present disclosure, a method for quantizing a neural network is provided, the method comprising: During the forward propagation process of training a neural network using sample data, for any layer of the neural network, the original data type data participating in the calculation in that layer is quantized into data of the first data type before the relevant calculation is performed. During backpropagation, for any layer of the neural network, the data of the original data type participating in the calculation in that layer is quantized into data of the second data type before the relevant calculation is performed. The precision of both the first data type and the second data type is lower than the precision of the original data type, and the precision of the first data type is lower than the precision of the second data type.

[0005] In some embodiments, the first data type and the second data type are determined based on the type of task performed by the neural network, and the precision of the first data type and the precision of the second data type are positively correlated with the precision required for the task performed by the neural network.

[0006] In some embodiments, the first data type and the second data type are determined based on the type of the operator in either layer, and the precision of the first data type and the precision of the second data type are negatively correlated with the complexity of the operator.

[0007] In some embodiments, for any layer of the neural network, quantizing the original data type data participating in the computation in that layer into data of a first data type or data of a second data type includes: If the layer is determined to include an operator of the target type, the original data type data participating in the calculation in the layer is quantized into data of the first data type or data of the second data type, wherein the target type operator is selected from various operators based on the computational cost of each type of operator.

[0008] In some embodiments, the target type operator includes one or more of the following: convolution, fully connected, and matrix multiplication.

[0009] In some embodiments, for any layer of the neural network, the original data type data participating in the calculation in that layer is quantized into data of a first data type or data of a second data type, and after performing the relevant calculations, the method further includes: The calculation result is dequantized into data of the original data type, and the dequantized result is used as the input data for the next layer.

[0010] In some embodiments, for any layer of the neural network, quantizing the original data type data participating in the computation in that layer into data of a first data type or data of a second data type includes: Obtain the distribution of the original data types involved in the calculation within this layer; Based on the distribution, a quantization scheme is determined for quantizing the original data, wherein the quantization scheme includes quantization type and quantization parameters; Based on the quantization scheme, the original data type is quantized into either the first data type or the second data type.

[0011] In some embodiments, the sample data is divided into multiple data blocks, and each data block is input into the neural network one by one to complete one round of training of the neural network. After each preset number of training rounds of the neural network, an operation is performed to obtain the distribution of the original data types involved in the calculation in that layer.

[0012] In some embodiments, the original data type is a 32-bit floating-point type, the first data type is an 8-bit integer type, and the second data type is a 16-bit integer type.

[0013] According to a second aspect of the present disclosure, a neural network quantization apparatus is provided, the apparatus comprising: The first processing module is used to, during the forward propagation process of training the neural network using sample data, quantize the original data type data participating in the calculation of any layer of the neural network into the first data type data, and then perform the relevant calculations. The second processing module is used to, during the backpropagation process, quantize the original data type data participating in the calculation of any layer of the neural network into the second data type data, and then perform the relevant calculations. The precision of the first data type and the precision of the second data type are both lower than the precision of the original data type, and the precision of the first data type is lower than the precision of the second data type.

[0014] According to a third aspect of the present disclosure, a chip is provided, the chip including a processor, a memory, and computer instructions stored in the memory that are executable by the processor, wherein when the processor executes the computer instructions, it can implement the method mentioned in the first aspect above.

[0015] According to a fourth aspect of the present disclosure, a board is provided, the board being packaged with at least one chip mentioned in the third aspect above.

[0016] According to a fifth aspect of the present disclosure, an electronic device is provided, the electronic device including the board card mentioned in the fourth aspect above or including the chip provided in the third aspect above.

[0017] According to a sixth aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing computer instructions that, when executed, implement the method mentioned in the first aspect above.

[0018] In this embodiment of the disclosure, during the training of the neural network, the original data types used in the computation can be quantized into data of lower precision before computation, thereby reducing resource overhead during training. Furthermore, during quantization, the original data types used in the computation can be quantized into data of different precisions for different stages of the neural network training process. For example, during forward propagation, where data precision requirements are lower, the original data types used in the computation at this stage can be quantized into a first data type with lower precision. During backpropagation, where data precision requirements are higher, the original data types used in the computation at this stage can be quantized into a second data type with higher precision. Through this targeted and differentiated quantization method, resource overhead during neural network training can be minimized while ensuring that the performance of the trained neural network is not significantly affected.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0021] Figure 1 This is a schematic diagram of a neural network training process according to an embodiment of the present disclosure.

[0022] Figure 2 This is a flowchart of a neural network quantization method according to an embodiment of this disclosure.

[0023] Figure 3 This is a schematic diagram of forward propagation during neural network training according to an embodiment of the present disclosure.

[0024] Figure 4 This is a schematic diagram of the logical structure of a neural network training device according to an embodiment of the present disclosure.

[0025] Figure 5 This is a schematic diagram of the logic structure of a chip according to an embodiment of this disclosure.

[0026] Figure 6 This is a schematic diagram of the logical structure of a board according to an embodiment of the present disclosure. Detailed Implementation

[0027] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0028] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items. Additionally, the term “at least one” herein means any combination of at least two of any one or more of a plurality.

[0029] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0030] To enable those skilled in the art to better understand the technical solutions in the embodiments of this disclosure, and to make the above-mentioned objectives, features and advantages of the embodiments of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be further described in detail below with reference to the accompanying drawings.

[0031] Neural networks are applied across various fields. As the diversity and complexity of tasks processed by neural networks increase—such as computer vision, natural language processing, and speech recognition—the demands on the accuracy and real-time performance of neural networks are constantly rising. Consequently, neural networks are becoming larger and more complex, requiring significant computational and storage resources for training. This places increasingly higher demands on the performance of the chips used for training, and the training process is also more time-consuming. For example, currently, the computational data used in each stage of neural network training uses a uniform data type; most neural networks use 32-bit floating-point numbers for training. Because floating-point operations are computationally intensive, they require more storage space and computational resources on the chip, and the entire training process is time-consuming.

[0032] In neural network training, higher precision of the data type used for computation is not always better. In some scenarios, even with high-precision data (such as 32-bit floating-point), the performance improvement is minimal, while significantly increasing resource consumption and reducing training efficiency. Furthermore, the precision requirements for computational data vary across different stages of neural network training. For instance, the forward propagation stage has lower precision requirements, while the backpropagation stage, involving error propagation and gradient calculation, is more sensitive to data precision and thus requires higher accuracy.

[0033] Based on this, embodiments of this disclosure provide a neural network quantization method. During the training of a neural network, the original data types used in the computation can be quantized into data of lower precision before computation, thereby reducing resource overhead during training. Furthermore, during quantization, the original data types used in the computation can be quantized into data of different precisions for different stages of neural network training. For example, in the forward propagation stage, where data precision requirements are lower, the original data types used in the computation can be quantized into a first data type with lower precision. In the backpropagation stage, where data precision requirements are higher, the original data types used in the computation can be quantized into a second data type with higher precision. Through this targeted and differentiated quantization method, resource overhead during neural network training can be minimized while ensuring that the performance of the trained neural network is not significantly affected.

[0034] The neural network quantization method provided in this embodiment can be executed by various electronic devices, such as computers, servers, or server clusters.

[0035] The neural network in this embodiment can be a neural network for processing various tasks, such as a classification network for classifying images and text, or a target detection network for detecting the presence of targets in images. The neural network can also be various types of neural networks, such as convolutional neural networks, recurrent neural networks, generative adversarial networks, etc. For example, in some scenarios, the neural network can be an image classification network, and the sample data used to train the neural network can be sample images carrying classification labels. In some scenarios, the neural network can be a target detection network, and the sample data can be sample images labeled with detection boxes corresponding to targets.

[0036] The data type mentioned in the embodiments of this disclosure can refer to whether the data is of integer or floating-point type, and the bit width of the data, such as 8 bits, 16 bits, 32 bits, etc. For example, the data type can be 32-bit floating-point, 16-bit floating-point, 32-bit integer, 16-bit integer, 8-bit integer, etc.

[0037] like Figure 1 The diagram shown is a schematic representation of the neural network quantization method provided in this embodiment of the present disclosure. Figure 2 The diagram shows a flowchart of the neural network quantization method, which may include the following steps: S202. During the forward propagation process of training the neural network using sample data, for any layer of the neural network, the data of the original data type participating in the calculation in that layer is quantized into data of the first data type before the relevant calculation is performed. The forward propagation process during neural network training involves inputting sample data into the neural network, processing it through each layer, and obtaining the corresponding computational result. Each layer's output becomes the input of the next layer. Within each layer, a series of operations are performed on the input based on pre-defined operators and network parameters (such as weights, biases, or convolutional kernels) to obtain the layer's output. During the neural network design phase, the original data type of the computational data is pre-defined, typically as 32-bit floating-point. However, the high precision of the initially set original data type leads to significant resource overhead and low training efficiency during training.

[0038] Therefore, in step S202, during the forward propagation process of training the neural network using sample data, for any layer in the neural network, the original data type data participating in the calculation in that layer can first be quantized into the first data type data before performing the relevant calculations. For any layer of the neural network, the original data participating in the calculation in that layer includes the input data of that layer (i.e., the output of the previous layer) and the network parameters of that layer, such as weights, biases, and convolution kernels. To save computational resources, the precision of the first data type can be lower than the precision of the original data type.

[0039] S204. During the backpropagation process, for any layer of the neural network, the data of the original data type participating in the calculation in that layer is quantized into data of the second data type, and then the relevant calculation is performed. The precision of the first data type and the precision of the second data type are both lower than the precision of the original data type, and the precision of the first data type is lower than the precision of the second data type.

[0040] The backpropagation process of a neural network is based on the difference between the calculation results obtained from the forward propagation process and the labels of the sample data to obtain the overall error. This error is then propagated forward from the last layer to obtain the error corresponding to each layer, and the network parameters of each layer are adjusted based on this error. In step S204, during the backpropagation process, for any layer of the neural network, the original data type data participating in the calculation in that layer can be quantized into a second data type before performing the relevant calculations. During backpropagation, the original data type data participating in the calculation of each layer can include the error determined by the previous layer, the network parameters of each layer, the pre-set learning rate, and the output of each layer during the forward propagation process. To reduce the resource overhead of the calculation process at this layer, the precision of the second data type can be lower than that of the original data type. Furthermore, since backpropagation is quite sensitive to data precision, meaning that data precision has a significant impact on the calculation results during backpropagation, the precision of the second data type can be higher than that of the first data type to ensure that the performance of the trained neural network is not significantly affected.

[0041] For example, assuming the original data type in the neural network is 32-bit floating-point, during forward propagation, before performing any calculation in any layer, the data involved in the operation can be quantized into 8-bit integers. Similarly, during backpropagation, before performing any calculation in any layer, the data involved can be quantized into 16-bit integers. This approach allows for flexible adjustment of the quantization scheme based on the data precision requirements of each stage, significantly reducing resource consumption during computation without substantially impacting the performance of the trained neural network.

[0042] The above quantization process can be applied to any layer in the neural network, any multiple layers in the neural network, or all layers of the neural network. The specific method can be set based on actual needs.

[0043] In both forward and backward propagation processes, when quantizing the raw data types involved in the computation, the quantization scheme can be determined based on the characteristics of the chip used to train the neural network and the distribution of the raw data types. The quantization scheme includes the quantization type and quantization parameters used to quantize the raw data types. The quantization type can include uniform quantization and non-uniform quantization, with uniform quantization further subdivided into symmetric quantization and asymmetric quantization. The quantization parameters can be one or more of zero-point values ​​and quantization step sizes, and the type of quantization parameters varies depending on the quantization type.

[0044] In some embodiments, regardless of whether it's the forward or backward propagation process, quantization of the neural network can be performed by quantizing the original data of all layers involved in the computation. For example, all layers can be quantized using the same quantization scheme, and after quantization into data of the same type, computation can be performed layer by layer. For instance, in forward propagation, the input sample data and the network parameters of each layer of the neural network can be obtained. Based on the distribution of the input data and the network parameters of each layer, a quantization scheme is determined, and the input sample data and network parameters are quantized. Then, the quantized data is used to perform computation layer by layer to obtain the computation result. Backpropagation can also adopt a similar approach.

[0045] In some embodiments, whether in the forward or backward propagation process, quantization of the neural network can be performed layer by layer. Each layer can be quantized independently, and the quantization scheme for each layer can be determined based on the distribution characteristics of the original data types involved in the computation of that layer. That is, the quantization scheme used for each layer can be different. In this way, a more suitable quantization scheme can be adapted for the computational data of each layer.

[0046] For scenarios involving layer-by-layer quantization, whether in the forward or backward propagation process, the original data types involved in the computation at each layer can be quantized to obtain either first or second data types. Then, the corresponding computation is performed to obtain the result. The computation result is then dequantized back into the original data types, and this dequantized result is used as the input data for the next layer. Before performing computation on the next layer, the original data types involved in the computation of that layer are requantized back into first data types before being used in the computation.

[0047] For example, such as Figure 3 As shown, taking forward propagation as an example, for the Nth layer in a neural network, the original 32-bit floating-point input data and network parameters of that layer can be obtained. These are then quantized into 8-bit integer data before being used for calculations at that layer to obtain the result. Since multiplication and other operations may occur during the calculation, the bit width of the result may be larger than the bit width of the calculated data. For example, the result may be a 32-bit integer. This result can then be dequantized back to 32-bit floating-point data. This 32-bit floating-point result is then used as the input for the (N+1)th layer, and the above steps are repeated to quantize the data for the (N+1)th layer, completing the calculation for that layer. A similar approach can be used for backpropagation.

[0048] In some embodiments, if layer-by-layer quantization is used, during the forward or backward propagation process, for any layer of the neural network, when quantizing the original data type data participating in the computation in that layer into data of a first data type or a second data type, the distribution of the original data type data participating in the computation in that layer can be obtained first. For example, the maximum and minimum values ​​of the data participating in the computation in that layer can be obtained. Then, a quantization scheme can be determined based on the distribution of the data participating in the computation. The quantization scheme includes a quantization type (e.g., symmetric quantization or asymmetric quantization) and quantization parameters (e.g., zero-point values ​​and quantization step size). Then, based on the determined quantization scheme, the original data type data can be quantized into data of a first data type or a second data type.

[0049] In training a neural network using sample data, the sample data is typically divided into multiple data batches. One batch is input into the neural network at a time, completing one round of forward and backward propagation, updating the network parameters. This process is called completing one round of training. For example, using images as sample data, all sample images can be divided into N groups, with one group input at a time, completing one round of forward and backward propagation. In some embodiments, considering that the network parameters are updated after each data batch is used to train the neural network, before quantizing the original data types used in the computation of each layer when training the neural network with the next data batch, it is necessary to re-statistically analyze the current network parameters of each layer to determine the data distribution (e.g., maximum and minimum values), and then determine the quantization scheme based on the data distribution.

[0050] While real-time statistics of data distribution across layers offer greater accuracy, they consume significant computational resources and time, leading to higher resource overhead and lower training efficiency. Considering that the network parameters of a neural network don't change drastically after several training rounds (meaning the maximum and minimum values ​​in the data used for computation are relatively similar), it's unnecessary to perform real-time statistics on the data distribution of each layer. Instead, statistics can be collected intermittently. For example, after a preset number of training rounds, the distribution of the original data types used in the computation of that layer can be retrieved again. This approach further conserves resources during training and improves processing efficiency.

[0051] For example, taking forward propagation as an example, after completing one round of training on the neural network using 5 data blocks (batches), the data distribution of each layer of the current neural network can be re-statistically analyzed, and the quantization scheme can be re-determined based on the data distribution. When training the neural network using the aforementioned 5 data blocks, the quantization scheme can be determined based on the previously analyzed data distribution, quantizing the original data type data used in the calculation of that layer into data of the first data type.

[0052] Different application scenarios have different accuracy requirements for neural network predictions. For example, neural networks used for classification tasks may have lower accuracy requirements, while those used for object detection tasks may have higher accuracy requirements. To adapt to the accuracy requirements of different application scenarios as much as possible, the trained neural network can meet the needs of different applications while minimizing the resource overhead of the training process. When training the neural network, the first and second data types can be determined based on the type of task the neural network performs. For example, the precision of the first and second data types can be positively correlated with the precision required by the task. For instance, whether the first and second data types should be set to floating-point or integer, and their bit width should be set to 32 bits, 16 bits, or 8 bits, can be adjusted based on the adaptability of the task the neural network performs. The higher the accuracy requirement of the task, the higher the precision of the first and second data types can be selected.

[0053] Different types of neural networks contain different types of operators, and the types of operators also vary between different layers of the neural network. Generally, for simple operators, the computational load is relatively low, and even if the data type used for computation is of high precision (e.g., 32-bit floating-point), the computational load will not increase significantly. However, for more complex operators, the computational load is much higher, and if the data type used for computation is of high precision, the computational load will increase significantly, resulting in greater resource consumption. Considering this factor, in some embodiments, the first and second data types can be determined based on the type of operators in any layer. For example, the precision of the first and second data types can be negatively correlated with the complexity of the operators included in that layer. If the operator complexity is high, the first and second data types should be set to lower precision data types, and vice versa. By determining the data type of the quantized data based on the operator type, resource overhead during training can be minimized.

[0054] Of course, quantizing the data used in computations at each layer of a neural network also consumes computational resources. Therefore, a layer-by-layer quantization approach can be adopted, where each layer is quantized independently. For each layer, the need for quantization can be determined based on the specific characteristics of the operators at that layer. For example, if the layer contains simple operators, such as simple addition and subtraction, these operators do not incur significant computational overhead. Even with quantization, the increase in computational overhead and training time is not substantial; in fact, the quantization process itself consumes resources and time. Therefore, in this case, quantization can be omitted, and computation can proceed directly. However, if the layer contains more complex operators, such as matrix multiplication and convolution, these operators are computationally intensive and consume significant computational resources. Therefore, they can be quantized into a first or second data type before performing the corresponding computations.

[0055] Therefore, in some embodiments, for any layer of the neural network, before quantizing the original data type involved in the computation of that layer into data of a first data type or a second data type, it can be first determined whether the operators in that layer include operators of the target type. The target type operators are selected from various operators based on their computational complexity; for example, they could be computationally intensive or complex operators. If the operators in that layer include operators of the target type, then the original data type involved in the computation of that layer is quantized into data of the first data type or a second data type, and then the relevant computation is performed. If not, quantization is not required, and computation can be performed directly.

[0056] In some embodiments, the target type operator includes one or more of the following: convolution, fully connected, and matrix multiplication. This type of operator is computationally intensive and requires a lot of resources. Therefore, for this type of operator, the data involved in the computation can be quantized before the computation is performed.

[0057] In this way, during the neural network training process, it can be determined whether each layer needs to be quantized based on the complexity of the operators contained in each layer, thereby maximizing the saving of computing resources and shortening the training time.

[0058] In some embodiments, neural networks can be trained using a chip. The following example uses an AI chip: Typically, an AI chip includes a processing unit and on-chip storage. Since neural networks have numerous model parameters and training data, these parameters and data are usually stored in external memory connected to the AI ​​chip. Taking the forward propagation process as an example, when the AI ​​chip performs operations at layer N, it typically needs to read the relevant operation data for several layers after layer N (e.g., N+1 to N+3) from external memory beforehand and store them in the on-chip storage for later use. If the operation data is of type Float32, more storage space needs to be used in internal memory to cache it. However, if the operation data is first quantized into type int8 after being retrieved from external memory before caching, the storage space occupied in the on-chip storage can be significantly reduced. Furthermore, during computation, the processing unit inevitably generates intermediate data, which requires memory on the AI ​​chip for storage. If this intermediate data is quantized into low-precision data before computation, the resulting intermediate data will also be low-precision, thus reducing the storage space required on the AI ​​chip. Moreover, compared to computation on Float32 data, computation on low-precision int8 data can improve the processing unit's computation speed, thereby enhancing the overall execution efficiency of the chip.

[0059] This disclosure embodiment quantizes the data used in the neural network computation into low-precision data, and quantizes the data into different precision levels to meet the different precision requirements of forward and backward propagation. This saves resources for the AI ​​chip during neural network computation. For example, quantizing the data into low-precision data allows the AI ​​chip to use less storage resources to store the data to be computed, thus saving storage resources. Furthermore, quantizing the data into low-precision data allows the AI ​​chip to use fewer computational resources to perform calculations on the data, further saving computational resources.

[0060] In some embodiments, the neural network can be an object detection neural network used to detect the location of a target object in an image. The sample data can be a labeled sample image, with the label indicating the location of the target object within the sample image. The original data type refers to the data type of the pixel data of the sample image and the network parameters of each layer in the initial neural network, for example, it can be of type float32. When training the initial neural network using the sample image to obtain the object detection neural network, during the forward propagation process, for the first layer of the initial neural network, the model parameters involved in the first layer and the pixel data of the sample image can be quantized into int8 type data. Then, the quantized int8 type data is used for corresponding calculations to obtain the output of the first layer. Since there may be multiplication and other operations during the calculation process, the bit width of the calculation result may be larger than the bit width of the input data; for example, the output may be of type int32 type data. The output can then be dequantized to float32 type data. For the second layer of the neural network, the output of the first layer and the network parameters of the second layer can be quantized into int8 type data before calculation. The above scheme is repeated for each layer of the initial neural network to obtain the output of the last layer. The output of the last layer is then dequantized into 32-bit float data and compared with the label of the sample image to obtain the error.

[0061] During backpropagation, due to the high precision requirements of the data, the initial neural network layers can be quantized first. This involves processing the overall 32-bit error (obtained from forward propagation), the 32-bit model parameters for each layer, and the pre-set 32-bit learning rate into 16-bit int data. Then, corresponding calculations are performed to obtain the error for each layer, and the model parameters are updated using the error of each layer. Through this method, a target detection neural network can be trained to detect objects in an image and determine their location. To further explain the neural network quantization method in this embodiment, a specific example is provided below.

[0062] During neural network training, data of type float32 is typically used for computation. To reduce memory and computational resource overhead and improve training efficiency, lower-precision data types can be used instead of float32, such as Int8 and Int16. These two data types can significantly save hardware resources compared to float32.

[0063] The network training process is divided into forward propagation and backward propagation. Generally, backward propagation requires higher precision than forward propagation. Considering that for most tasks, using int8 type data is sufficient to meet the precision requirements during forward propagation, the original float32 type data can be quantized into int8 type data. However, since backward propagation is more sensitive to precision, using int8 type data for computation may cause some loss of precision. Therefore, the backward propagation process can use int16 type data for computation, that is, quantizing the original float32 type data into int16 type data.

[0064] The specific quantification process of the forward propagation is as follows: Taking convolution as an example, if the input data and network parameters are of type float32, they first need to be quantized into type int8. For example, symmetric quantization can be used, meaning the range of the quantized input and network parameters is [-127, 127]. Assuming the quantized data is Q, then Q = input / scale, where input is the true input value, and scale is the quantization coefficient. Scale can be determined using the following formula: max(abs(min),abs(max)) / 127, Here, `max` and `min` represent the maximum and minimum values ​​of the input to this layer. The `max` and `min` values ​​can be calculated online, in real-time before each forward pass of this layer. Since real-time calculation of `max` and `min` consumes time and computational resources, optimization methods can be employed, such as calculating them every few iterations (each iteration is defined as one round of training the neural network using a batch).

[0065] After quantization, the input and network parameters are converted to int8 type data, denoted as Qa and Qw respectively, with quantization coefficients scaleA and scaleW. Qa and Qw participate in hardware computation; the result of convolution multiplication and accumulation is Qout, with a bit width of int32 type. When outputting from this layer, Qout*scaleA*scaleB needs to be converted to float32 type data for output, as this value approximates the actual floating-point operation output. The above process is repeated for each layer to complete the forward propagation process.

[0066] The specific quantification process of backpropagation is as follows: The quantization process in backpropagation is similar to that in forward propagation. First, the maximum and minimum values ​​of the current layer's input are calculated. Then, the input and network parameters are quantized into int16 data, where Q = input / scale, and scale = max(abs(min), abs(max)) / 32767. After the backpropagation of this layer is completed, the output also needs to be dequantized, converting the int32 output to float32 before proceeding to the next layer. This process is repeated for each layer, thus completing the backpropagation process.

[0067] By using Int8 and Int16 data types for computation during training, compared to float32, hardware resources can be significantly saved, thereby reducing resource overhead during training and improving processing efficiency.

[0068] It is easy to understand that the solutions described in the above embodiments can be combined when there is no conflict, and not all of them will be listed in this disclosure.

[0069] Accordingly, embodiments of this disclosure also provide a neural network quantization device, such as... Figure 4 As shown, the device includes: The first processing module 41 is used to, during the forward propagation process of training the neural network using sample data, quantize the original data type data participating in the calculation in any layer of the neural network into the first data type data, and then perform the relevant calculations. The second processing module 42 is used to, during the backpropagation process, for any layer of the neural network, quantize the data of the original data type involved in the calculation of that layer into data of the second data type, and then perform the relevant calculations. The precision of the first data type and the precision of the second data type are both lower than the precision of the original data type, and the precision of the first data type is lower than the precision of the second data type.

[0070] In some embodiments, the first data type and the second data type are determined based on the type of task performed by the neural network, and the precision of the first data type and the precision of the second data type are positively correlated with the precision required for the task performed by the neural network.

[0071] In some embodiments, the first data type and the second data type are determined based on the type of the operator in either layer, and the precision of the first data type and the precision of the second data type are negatively correlated with the complexity of the operator.

[0072] In some embodiments, for any layer of the neural network, when the device quantizes the original data type data participating in the computation in that layer into data of a first data type or data of a second data type, it is specifically used for: If the layer is determined to include an operator of the target type, the original data type data participating in the calculation in the layer is quantized into data of the first data type or data of the second data type, wherein the target type operator is selected from various operators based on the computational cost of each type of operator.

[0073] In some embodiments, the target type operator includes one or more of the following: convolution, fully connected, and matrix multiplication.

[0074] In some embodiments, for any layer of the neural network, after quantizing the original data type data participating in the calculation in that layer into data of a first data type or data of a second data type, and then performing the relevant calculations, the apparatus is further configured to: The calculation result is dequantized into data of the original data type, and the dequantized result is used as the input data for the next layer.

[0075] In some embodiments, for any layer of the neural network, when the device quantizes the original data type data participating in the computation in that layer into data of a first data type or data of a second data type, it is specifically used for: Obtain the distribution of the original data types involved in the calculation within this layer; Based on the distribution, a quantization scheme is determined for quantizing the original data, wherein the quantization scheme includes quantization type and quantization parameters; Based on the quantization scheme, the original data type is quantized into either the first data type or the second data type.

[0076] In some embodiments, the sample data is divided into multiple data blocks, and each data block is input into the neural network one by one to complete one round of training of the neural network. After each preset number of training rounds of the neural network, an operation is performed to obtain the distribution of the original data types involved in the calculation in that layer.

[0077] In some embodiments, the original data type is a 32-bit floating-point type, the first data type is an 8-bit integer type, and the second data type is a 16-bit integer type. The specific steps of the above-described device in executing the neural network quantization method can be found in the description of the above method embodiments, and will not be repeated here.

[0078] Furthermore, embodiments of this disclosure also provide a chip, such as... Figure 5As shown, the chip includes a processor 51, a memory 52, and computer instructions stored in the memory 52 that can be executed by the processor 51. When the processor 51 executes the computer instructions, it implements the method described in any of the above embodiments.

[0079] Furthermore, this disclosure also provides a circuit board that includes a package structure encapsulating at least one chip mentioned in the foregoing embodiments. See also... Figure 6 The present invention provides an exemplary board, which includes the chip 700 and may also include other components, including but not limited to: a storage device 702, an interface device 704 and a control device 706.

[0080] The storage device 702 is connected to the chip 700 within the chip package structure via a bus and is used to store data. The storage device 702 may include multiple sets of storage cells 7021, such as DDR SDRAM (Double Data Rate SDRAM). Each set of storage cells 7021 is connected to the image processing chip 700 via a bus.

[0081] The interface device 704 is electrically connected to the chip 700 within the chip package structure. The interface device 704 is used to enable data transmission between the chip and an external device 800 (e.g., a terminal, server, camera, etc.). In one embodiment, the interface device 704 may include a PCIe interface, a network interface, or other interfaces; this disclosure does not impose any limitations.

[0082] The controller 706 is electrically connected to the chip 700 within the chip package structure. The controller 706 may be an MCU, etc., and this disclosure does not impose any limitations.

[0083] This disclosure also provides an electronic device, which includes the board or chip mentioned in the above embodiments.

[0084] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the foregoing embodiments.

[0085] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0086] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the embodiments of this disclosure can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of the embodiments of this disclosure, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this disclosure.

[0087] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0088] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. When implementing the embodiments of this disclosure, the functions of each module can be implemented in one or more software and / or hardware. Alternatively, some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0089] The above description is merely a specific implementation of the embodiments of this disclosure. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles of the embodiments of this disclosure, and these improvements and modifications should also be considered within the protection scope of the embodiments of this disclosure.

Claims

1. A training method for an object detection neural network, characterized in that, The method includes: Acquire a sample image, the sample image carrying a label, the label being used to indicate the location of the target object in the sample image; During the forward propagation process of training the target detection neural network using the sample images, for any layer of the target detection neural network, the original data type data participating in the calculation in that layer is quantized into first data type data, and then the relevant calculations are performed to obtain the prediction result of the first data type of the target detection neural network. The prediction result of the first data type is then dequantized back into the prediction result of the original data type, and the overall error is obtained based on the prediction result of the original data type and the label. Wherein, if the layer is the first layer of the target detection neural network, the original data type data participating in the calculation in that layer includes the pixel data of the sample image and the network parameters of that layer; if the layer is not the first layer, the original data type data participating in the calculation in that layer includes the output result of the previous layer and the network parameters of that layer. During backpropagation, for any layer of the target detection neural network, the original data type data participating in the calculation in that layer is quantized into the second data type data, and then the relevant calculations are performed to obtain the error of that layer. The network parameters of that layer are then updated using the error of that layer. The original data type data participating in the calculation in that layer includes the overall error, the network parameters of that layer, and the learning rate. The precision of both the first data type and the second data type is lower than the precision of the original data type, and the precision of the first data type is lower than the precision of the second data type. The first data type and the second data type are determined based on the type of task performed by the target detection neural network and / or based on the type of operator in any layer.

2. The method according to claim 1, characterized in that, The first data type and the second data type are determined based on the type of task performed by the target detection neural network, and the accuracy of the first data type and the accuracy of the second data type are positively correlated with the accuracy required for the task performed by the target detection neural network.

3. The method according to claim 1, characterized in that, The first data type and the second data type are determined based on the type of the operator in either layer, and the precision of the first data type and the precision of the second data type are negatively correlated with the complexity of the operator.

4. The method according to any one of claims 1-3, characterized in that, For any layer of the target detection neural network, the original data type data participating in the calculation in that layer is quantized into data of a first data type or data of a second data type, including: If the layer is determined to include an operator of the target type, the original data type data participating in the calculation in the layer is quantized into data of the first data type or data of the second data type, wherein the target type operator is selected from various operators based on the computational cost of each type of operator.

5. The method according to claim 4, characterized in that, The target type of operator includes one or more of the following: convolution, fully connected, and matrix multiplication.

6. The method according to any one of claims 1-3, characterized in that, For any layer of the target detection neural network, after quantizing the original data type data participating in the calculation in that layer into data of a first data type or data of a second data type, and then performing the relevant calculations, the method further includes: The calculation result is dequantized into data of the original data type, and the dequantized result is used as the input data for the next layer.

7. The method according to any one of claims 1-3, characterized in that, For any layer of the target detection neural network, the original data type data participating in the calculation in that layer is quantized into data of a first data type or data of a second data type, including: Obtain the distribution of the original data types involved in the calculation within this layer; Based on the distribution, a quantization scheme is determined for quantizing the original data, wherein the quantization scheme includes quantization type and quantization parameters; Based on the quantization scheme, the original data type is quantized into either the first data type or the second data type.

8. The method according to claim 7, characterized in that, The sample image is divided into multiple data blocks, and each data block is input into the target detection neural network one by one to complete one round of training of the target detection neural network. After each preset number of training rounds of the target detection neural network, an operation is performed to obtain the distribution of the original data types involved in the calculation in that layer.

9. The method according to claim 1, characterized in that, The original data type is a 32-bit floating-point type, the first data type is an 8-bit integer type, and the second data type is a 16-bit integer type.

10. A training device for an object detection neural network, characterized in that, The device includes: The acquisition module is used to acquire sample images, the sample images carrying labels, the labels being used to indicate the location of the target object in the sample images; The first processing module is configured to, during the forward propagation process of training the target detection neural network using the sample images, quantize the original data type data participating in the calculation of any layer of the target detection neural network into first data type data, and then perform relevant calculations to obtain the prediction result of the first data type of the target detection neural network. The prediction result of the first data type is then dequantized back into the prediction result of the original data type, and the overall error is obtained based on the prediction result of the original data type and the label. Wherein, if the layer is the first layer of the target detection neural network, the original data type data participating in the calculation of this layer includes the pixel data of the sample image and the network parameters of this layer; if the layer is not the first layer, the original data type data participating in the calculation of this layer includes the output result of the previous layer and the network parameters of this layer. The first processing module is configured to, during backpropagation, quantize the original data type data participating in the calculation of any layer of the target detection neural network into the second data type data, perform relevant calculations to obtain the error of that layer, and update the network parameters of that layer using the error of that layer. The original data type data participating in the calculation of that layer includes the overall error, the network parameters of that layer, and the learning rate. The precision of both the first data type and the second data type is lower than the precision of the original data type, and the precision of the first data type is lower than the precision of the second data type. The first data type and the second data type are determined based on the type of task performed by the target detection neural network and / or based on the type of operator in any layer.

11. A chip, characterized in that, The chip includes a processor and a memory, the memory storing computer instructions executable by the processor, and the processor executing the computer instructions to implement the method as described in any one of claims 1-9.

12. A circuit board, characterized in that, The board includes a package structure that encapsulates at least one chip as described in claim 11.

13. An electronic device, characterized in that, The electronic device includes the board as described in claim 12 or the chip as described in claim 11.

14. A computer-readable storage medium, characterized in that, The storage medium stores computer instructions that, when executed, implement the method as described in any one of claims 1-9.

Images