Data processing methods and related products
By clustering the data to be quantized in the neural network model, quantizing the quantization parameters, quantizing the activation data into a lower number of bits, the problem of inability to effectively reduce the consumption of neural network computing resources in the existing technology is solved, and more efficient Quantitative and computing resource savings.
Patent Information
- Application Number
- CN202010068866.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-01-21
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2040-01-21
AI Technical Summary
The existing neural network quantization scheme cannot effectively reduce the model computing volume and computing resource overhead, especially in lower bit quantization schemes, which are difficult to improve quantization accuracy and reduce computing resource consumption.
By clustering the data to be quantized for each layer in the neural network model, the maximum and minimum values of quantization are determined, the quantization parameters are determined based on these values, and the activation data is quantized into a lower bit number representation, thereby improving the quantization accuracy and Save computing resources.
More precise quantization is achieved, the number of quantized data bits is reduced, computing and storage resources are saved, computing efficiency is improved, floating-point operations are avoided, and fixed-point operations are adopted to improve processing efficiency.
Smart Images

Figure CN113222098B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of neural network technology, and in particular to a data processing method and related products. Background Art
[0002] With the development of neural network technology, neural network quantization technology has emerged, which mainly compresses the weights and activation values (i.e., the output feature maps or input feature maps of each layer) in the neural network (such as convolutional layers and fully connected layers), reduces the bit width of weights, the bit width of activation values, etc., thereby achieving the purpose of compressing the data volume of the neural network model and reducing the computing resource requirements of the neural network model during the prediction process.
[0003] At present, the commonly used neural network quantization scheme is to directly compress the model parameter values and activation values of the neural network model to a fixed bit width. For example, the model parameter values and activation values in the residual network ResNet-18 are compressed to 8 bits. However, this neural network quantization scheme cannot significantly reduce the model operation amount and computing resource overhead. In order to further reduce the model operation amount and computing resource overhead of the neural network model, a neural network quantization scheme with lower bits is needed. Summary of the invention
[0004] The embodiments of the present application disclose a data processing method and related products.
[0005] In a first aspect, an embodiment of the present application provides a data processing method, the method comprising: performing clustering processing on multiple numerical values included in the data to be quantized output by the Lth layer in a neural network model to obtain a clustering result, wherein L is an integer greater than 0; based on the clustering result, obtaining a maximum quantization value and a minimum quantization value; based on the maximum quantization value and the minimum quantization value, determining a quantization parameter corresponding to a first quantization processing of the Lth layer, wherein the first quantization processing is used to quantize the numerical value in the activation data of the Lth layer into an integer represented by M bits.
[0006] Optionally, the data to be quantified is a test image or is obtained by processing the test image or sample image through the first (L-1) network layers of the neural network model. The above-mentioned quantization maximum value is less than the maximum value of each value included in the data to be quantified, and the above-mentioned minimum value is greater than the minimum value of each value included in the data to be quantified.
[0007] In an embodiment of the present application, the various values included in the data to be quantized are clustered to obtain the maximum quantization value and the minimum quantization value corresponding to the data to be quantized, and then the quantization parameter corresponding to the data to be quantized is determined based on the maximum quantization value and the minimum quantization value; so that the distribution of the data to be quantized can be more accurately expressed with a limited quantization order, thereby improving the quantization accuracy.
[0008] In an optional implementation, obtaining a quantization maximum value and a quantization minimum value based on the clustering result includes: taking the maximum value of at least two cluster centers included in the clustering result as the quantization maximum value, and taking the minimum value of the at least two cluster centers as the quantization minimum value.
[0009] In this implementation, the quantization maximum value and the quantization minimum value can be accurately and quickly obtained according to the at least two cluster centers included in the clustering result.
[0010] In an optional implementation, the quantization parameter corresponding to the first quantization processing of the Lth layer is determined based on the quantization maximum value and the quantization minimum value, including: taking the quantization minimum value as the first item and the quantization maximum value as the last item to obtain a first arithmetic progression including N numerical values; determining the quantization parameter based on the N numerical values contained in the first arithmetic progression.
[0011] In this implementation, the common factors of the numerical values contained in the first arithmetic progression are used as quantization parameters, so that the activation data of the Lth layer can be quantized into integers using the quantization parameters.
[0012] In an optional implementation, the method further includes: determining the number of bits M corresponding to the first quantization process of the Lth layer based on the distribution discreteness of the multiple values included in the data to be quantized. Optionally, the discreteness of the multiple values included in the data to be quantized is positively correlated with the number of bits M.
[0013] In this implementation, the number of bits M corresponding to the first quantization process of the Lth layer is determined based on the distribution discreteness of multiple values included in the data to be quantized; the number of bits occupied by the quantized data is reduced, thereby saving computing resources.
[0014] In an optional implementation, the number of bits M corresponding to the first quantization process is preset. That is, the number of bits M corresponding to the first quantization process is determined in advance, and is not determined by the distribution discreteness of multiple values included in the data to be quantized.
[0015] In an optional implementation, after determining the quantization parameter corresponding to the first quantization processing of the Lth layer based on the quantization maximum value and the quantization minimum value, the method further includes: obtaining activation data of the Lth layer in the process of predicting the first image using the neural network model; performing a first quantization processing on the activation data based on the quantization parameter to obtain a processing result of the Lth layer.
[0016] In this implementation, the activation data of the Lth layer is quantized based on the quantized data, which can save computing resources and storage resources.
[0017] In an optional implementation, after determining the quantization parameter corresponding to the first quantization processing of the Lth layer based on the maximum quantization value and the minimum quantization value, the method further includes: converting the first operation parameter into a fixed-point number to obtain a second operation parameter; the first operation parameter is a parameter required to perform the first operation processing, the first operation parameter includes the quantization parameter, and the first operation processing is obtained by merging the network model processing performed by at least one network layer including the Lth layer and the first quantization processing; based on the second operation parameter, performing the first operation processing on the input data of the at least one network layer to obtain the processing result of the at least one network layer. Optionally, the input data of the at least one network layer is the image to be predicted, or is obtained by processing the image to be predicted by at least one second network layer located before the at least one network layer in the neural network model.
[0018] In this implementation, only fixed-point operations are performed in the process of executing the first operation processing, and no floating-point operations are performed, which can improve calculation efficiency.
[0019] In an optional implementation, converting the first operation parameter into a fixed-point number to obtain the second operation parameter includes: expanding each value included in the first operation parameter by K times to obtain an expanded value, where K is an integer greater than 1; and converting the expanded value into a fixed-point number to obtain the second operation parameter. Optionally, the expanded value is converted into a fixed-point number by rounding.
[0020] In this implementation, the floating-point number in the first operation parameter can be quickly and accurately converted into a fixed-point number.
[0021] In an optional implementation, the first operation processing is obtained by merging the network model processing and the first quantization processing performed by at least two adjacent network layers including the Lth layer.
[0022] In an optional implementation, the network model processing performed by at least two adjacent network layers including the Lth layer is linear operation processing.
[0023] In a second aspect, an embodiment of the present application provides a data processing device, comprising: a clustering unit, for clustering a plurality of numerical values included in the data to be quantized outputted by the Lth layer of a neural network model, to obtain a clustering result, wherein L is an integer greater than 0; a determination unit, for obtaining a maximum quantization value and a minimum quantization value based on the clustering result; the determination unit, further for determining a quantization parameter corresponding to a first quantization process of the Lth layer based on the maximum quantization value and the minimum quantization value, wherein the first quantization process is used to quantize the numerical values in the activation data of the Lth layer into an integer represented by M bits. Optionally, the data to be quantized is a test image or is obtained by processing a test image or a sample image through the first (L-1) network layers of the neural network model.
[0024] In an optional implementation, the determining unit is specifically configured to use a maximum value among at least two cluster centers included in the clustering result as the quantized maximum value, and use a minimum value among the at least two cluster centers as the quantized minimum value.
[0025] In an optional implementation, the determination unit is specifically used to obtain a first arithmetic progression including N numerical values with the minimum quantization value as the first item and the maximum quantization value as the last item; and determine the quantization parameter based on the N numerical values contained in the first arithmetic progression.
[0026] In an optional implementation, the determination unit is further used to determine the number of bits M corresponding to the first quantization processing of the Lth layer based on the distribution discreteness of multiple numerical values included in the data to be quantized.
[0027] In an optional implementation, the number of bits M corresponding to the first quantization process is preset. That is, the number of bits M corresponding to the first quantization process is determined in advance, and is not determined by the distribution discreteness of multiple values included in the data to be quantized.
[0028] In an optional implementation, the device also includes: a first processing unit, used to obtain the activation data of the Lth layer in the process of predicting the first image using the neural network model; a quantization unit, used to perform a first quantization processing on the activation data based on the quantization parameter to obtain the processing result of the Lth layer.
[0029] In an optional implementation, the device further includes: a first conversion unit, configured to convert the first operation parameter into a fixed-point number to obtain a second operation parameter; the first operation parameter is a parameter required to perform a first operation process, the first operation parameter includes the quantization parameter, and the first operation process is obtained by merging the network model process performed by at least one network layer including the Lth layer and the first quantization operation process; a second processing unit, configured to perform the first operation process on the input data of the at least one network layer based on the second operation parameter to obtain the at least one network layer. Optionally, the input data of the at least one network layer is an image to be predicted, or is a processing result obtained by processing the image to be predicted by at least one second network layer located before the at least one network layer in the neural network model.
[0030] In an optional implementation, the first conversion unit is specifically used to expand each numerical value included in the first operation parameter by K times to obtain an expanded numerical value, where K is an integer greater than 1; and convert the expanded numerical value into a fixed-point number to obtain the second operation parameter.
[0031] In an optional implementation, the first operation processing is obtained by merging the network model processing and the first quantization processing performed by at least two adjacent network layers including the Lth layer.
[0032] In an optional implementation, the network model processing performed by at least two adjacent network layers including the Lth layer is linear operation processing.
[0033] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory for storing programs; a processor for executing the programs stored in the memory, and when the program is executed, the processor is used to execute the method as described in the first aspect and any optional implementation method mentioned above.
[0034] In a fourth aspect, an embodiment of the present application provides a chip, which includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface to execute the method of the first aspect and any optional implementation method described above.
[0035] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program includes program instructions, which, when executed by a processor, enable the processor to execute the method of the above-mentioned first aspect and any optional implementation method.
[0036] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes program instructions, and when the program instructions are executed by a processor, the processor executes the method of the above-mentioned first aspect and any optional implementation method. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.
[0038] Figure 1 A flow chart of a data processing method provided in an embodiment of the present application;
[0039] Figure 2 A flow chart of another data processing method provided in an embodiment of the present application;
[0040] Figure 3 A schematic diagram of a quantization order provided in an embodiment of the present application;
[0041] Figure 4 A flow chart of another data processing method provided in an embodiment of the present application;
[0042] Figure 5 A flow chart of another data processing method provided in an embodiment of the present application;
[0043] Figure 6 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;
[0044] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;
[0045] Figure 8 A schematic diagram of the structure of a server provided in an embodiment of the present application. DETAILED DESCRIPTION
[0046] The terms "first", "second", and "third" etc. in the specification embodiments and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, including a series of steps or units. Methods, systems, products or equipment are not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment. "And / or" is used to represent the selection of one or all between the two objects connected thereto. For example, "A and / or B" means A, B or A+B.
[0047] Neural network quantization technology mainly quantizes the weights or activation values (activation) of the network layers (such as convolutional layers, pooling layers, fully connected layers, etc.) in the neural network into quantities represented by a specific number of bits, so as to achieve the purpose of compressing the data volume of the neural network model and reducing the computing resource requirements of the neural network model in the prediction process. For example, the weights in the neural network model are quantized into an integer represented by 8 bits. Quantizing the neural network model mainly includes: quantizing the weights in the neural network model, and / or quantizing the activation values (i.e., activation data) obtained from the network layers of the neural network model. The activation value of a network layer in the neural network model can refer to the output data (i.e., output feature map) of the network layer, and the quantized output data can be used as the input data (i.e., input feature map) of the next network layer. In practical applications, the quantization of the weights in the neural network model can be performed during the training process, such as quantization training of the neural network model, or the neural network model can be trained first, and the weights in the trained neural network model can be quantized. Quantizing the activation values in the neural network model can be performed in the process of using the neural network model to perform prediction processing tasks. It can be understood that the quantization of the weights in the neural network model and the quantization of the activation values obtained by the neural network model during the prediction process are performed separately. In practical applications, only the weights can be quantized, only the activation values can be quantized, or both the weights and the activation values can be quantized.
[0048] In the disclosed embodiments, the quantization parameters of each layer (e.g., the convolutional layer) of the neural network model can be determined based on the distribution of numerical values included in the output feature graph of each layer, so that the activation data output by each layer can be quantized using the quantization parameters of each layer when the neural network model is used for prediction processing, thereby improving the quantization efficiency.
[0049] See also Figure 1 , Figure 1 A flow chart of a data processing method provided in an embodiment of the present application. Figure 1 As shown, the method may include:
[0050] 101. A data processing device performs clustering processing on a plurality of numerical values included in the data to be quantized in the Lth layer of the neural network model to obtain a clustering result.
[0051] The data processing device can be a terminal device such as a mobile phone, a tablet computer, a wearable device, a laptop computer, a desktop computer, or a server. The above L is an integer greater than 0, that is, the Lth layer is the first layer, a middle layer, or the last layer of the neural network model. The above data to be quantized includes at least one value represented by more than M bits, and the above M is an integer greater than 1. The data to be quantized of the Lth layer can be a feature map (also referred to as an output feature map) or data obtained by the Lth layer processing the input data. Exemplarily, the data to be quantized is a test image or is obtained by processing the test image or sample image through the first (L-1) network layers of the neural network model. In some embodiments, the above data to be quantized can be a feature map output by any layer except the last layer in the above neural network model, that is, an output feature map. The feature map output by any layer in the neural network model (that is, the output feature map) is the output data (that is, the activation data) of any layer, and the feature map input by any layer (that is, the input feature map) is the input data of any layer. The feature map can be understood as one or more matrices (two dimensions or more). In some embodiments, the input data of the first layer of the neural network model may be preprocessed image data, and the numerical values in the image data may be represented by 8 bits.
[0052] The neural network model can be a deep neural network model, such as any one of a convolutional neural network, a recurrent neural network, a long short-term memory network (Long Short-Term Memory, LSTM), etc., which is not limited in this application. In some embodiments, the neural network model is obtained by training. For example, the neural network model is a prediction model obtained by training using training data. The weights in the neural network model can be unquantized weights, or at least a portion of the weights in the neural network model are quantized. For example, the weights in the neural network model are all quantized to integers represented by less than 8 bits.
[0053] In some embodiments, the data processing device may obtain a trained neural network model; input the training sample into the neural network model for prediction processing, and obtain the data to be quantified output by the Lth layer of the neural network model; cluster the multiple values included in the data to be quantified to obtain a clustering result. It can be understood that the data processing device can obtain the feature map (corresponding to the data to be quantified) output by any layer of the neural network model in a similar manner.
[0054] In some embodiments, a data processing device may obtain a trained neural network model; quantize the weights in the neural network model to obtain a quantized neural network model; input the training samples into the quantized neural network model for prediction processing, and obtain the data to be quantized output by the Lth layer of the quantized neural network model; cluster the multiple numerical values included in the data to be quantized to obtain a clustering result.
[0055] Optionally, the data processing device uses a clustering algorithm to cluster the values included in the above-mentioned data to be quantified to obtain a clustering result, and the clustering result can be at least two cluster centers obtained by clustering the values included in the above-mentioned data to be quantified. The clustering algorithm can be any one of the clustering algorithms such as K-means clustering algorithm, hierarchical clustering algorithm (such as birch algorithm), density-based clustering algorithm, graph theory clustering method, etc., and this application is not limited. Exemplarily, the data processing device uses K-means clustering algorithm to cluster the data to be quantified to obtain at least two cluster centers, that is, clustering results.
[0056] 102. The data processing device obtains a quantization maximum value and a quantization minimum value based on the above clustering results.
[0057] Optionally, the above-mentioned quantization maximum value is smaller than the maximum value of the above-mentioned data to be quantized, and the above-mentioned quantization minimum value is larger than the minimum value of the above-mentioned data to be quantized.
[0058] In some embodiments, the data processing device obtains the quantization maximum value and the quantization minimum value based on the above clustering results in the following manner: the maximum value of at least two cluster centers included in the above clustering results is used as the above quantization maximum value, and the minimum value of the at least two cluster centers is used as the above quantization minimum value. Exemplarily, the data processing device uses the K-means clustering algorithm to cluster the above data to be quantified to obtain at least two cluster centers; the maximum value of the at least two cluster centers is used as the above quantization maximum value; the minimum value of the at least two cluster centers is used as the above quantization minimum value. For example, the data processing device uses the K-means clustering algorithm to cluster the data to be quantified to obtain 5 cluster centers; the maximum value of the 5 cluster centers is used as the quantization maximum value; the minimum value of the 5 cluster centers is used as the quantization minimum value. It should be understood that the data processing device can set the number of clusters, that is, the number of cluster centers, according to actual needs.
[0059] 103. Determine a quantization parameter corresponding to the first quantization process of the Lth layer based on the maximum quantization value and the minimum quantization value.
[0060] The first quantization process is used to quantize the numerical value in the activation data of the Lth layer into an integer represented by M bits.
[0061] Specifically, a quantization sequence can be obtained based on the quantization maximum value and the quantization minimum value, and a quantization parameter can be obtained based on the quantization sequence. In some embodiments, based on the above-mentioned quantization maximum value and the above-mentioned quantization minimum value, an implementation method for determining the quantization parameter corresponding to the first quantization process of the above-mentioned L-th layer is as follows: taking the above-mentioned quantization minimum value as the first item and taking the above-mentioned quantization maximum value as the last item, a first arithmetic progression including N values is obtained; and the common factors of the above-mentioned N values contained in the above-mentioned first arithmetic progression are used as the above-mentioned quantization parameter. For example, the data processing device uses a K-means clustering algorithm to cluster the feature map output by the Lth layer to obtain the maximum value (i.e., max) 1.3 and the minimum value (min) -0.2 corresponding to the feature map; determines a first arithmetic progression {-0.2, -0.1, 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8. 0.9, 1.0, 1.1, 1.2, 1.3} with -0.2 as the first item and 1.3 as the last item, and includes 16 numerical values; and uses the common factor 0.1 of each item in the first arithmetic progression as the quantization parameter of this layer, i.e., the quantization parameter corresponding to the first quantization processing.
[0062] In some embodiments, before executing step 103, the data processing device may perform the following operation: obtain the number M of bits to be used to quantize the numerical values included in the above-mentioned data to be quantized. The number of bits M may be preset, for example, the data processing device or other device may pre-store the number of bits to be used to quantize the numerical values included in the above-mentioned data to be quantized. Accordingly, when it is necessary to determine the quantization parameter, the stored number is obtained from the memory. In other embodiments, the number of bits M may also be determined based on the data to be quantized, for example, based on the distribution discreteness of multiple numerical values included in the above-mentioned data to be quantized, the number of bits M corresponding to the first quantization processing of the above-mentioned Lth layer is determined. Alternatively, the number of bits M may also be determined based on other distribution characteristics or parameters of the data to be quantized.
[0063] Optionally, N is 2 to the power of M. For example, the data processing device determines the feature map (corresponding to the data to be quantized) output by a certain layer of the neural network model to be quantized using 4 bits, and clusters the feature map output by the layer using the K-means clustering algorithm to obtain the maximum value (i.e., max) 1.3 and the minimum value (min) -0.2 corresponding to the feature map; determines the first arithmetic sequence {-0.2, -0.1, 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8. 0.9, 1.0, 1.1, 1.2, 1.3} with -0.2 as the first item and 1.3 as the last item, and includes 16 values. It can be understood that after extracting the above-mentioned common factors from the above-mentioned first arithmetic sequence, a second arithmetic sequence can be obtained, and the values included in the second arithmetic sequence are all integers, and the output data of the Lth layer can be quantized to a certain value in the second arithmetic sequence represented by M bits.
[0064] In an embodiment of the present application, the various values included in the data to be quantized are clustered to obtain the maximum quantization value and the minimum quantization value corresponding to the data to be quantized, and then the quantization parameter corresponding to the data to be quantized is determined based on the maximum quantization value and the minimum quantization value; so that the distribution of the data to be quantized can be more accurately expressed with a limited quantization order, thereby improving the quantization accuracy.
[0065] Some optional implementation methods for determining the number of bits M corresponding to the first quantization processing of the Lth layer based on the data to be quantized are introduced below.
[0066] Exemplarily, the data processing device determines the number of bits M corresponding to the first quantization processing of the Lth layer based on the variance of each value included in the data to be quantized; the number of bits M corresponding to the first quantization processing of the Lth layer is positively correlated with the variance of each value included in the data to be quantized. For example, the data processing device may compare the variance with one or more thresholds to determine the number of bits M, and when the variance of each value included in the data to be quantized is less than the first threshold, determine to use 3 bits to quantize each value included in the data to be quantized; when the variance of each value included in the data to be quantized is not less than the first threshold, determine to use 4 bits to quantize each value included in the data to be quantized. The first threshold may be 0.016, 0.2, 1.2, 10, 100, etc., and is not limited in this application. For another example, multiple segmented intervals may be set. When the variance of each value included in the data to be quantized is less than the second threshold and greater than the third threshold, the data processing device determines to use 4 bits to quantize each value included in the data to be quantized; when the variance of each value included in the data to be quantized is not less than the second threshold, it determines to use 6 bits to quantize each value included in the data to be quantized; when the variance of each value included in the data to be quantized is not greater than the third threshold, it determines to use 2 bits to quantize each value included in the data to be quantized. The third threshold may be 0.016, 0.2, 1.2, etc., and the second threshold may be 10, 25, etc., which are not limited in this application.
[0067] Exemplarily, the data processing device determines the number of bits M corresponding to the first quantization processing of the Lth layer based on the range of each value included in the above-mentioned data to be quantized; the number of bits M corresponding to the first quantization processing of the Lth layer is positively correlated with the range of each value included in the above-mentioned data to be quantized. The range of each value included in the data to be quantized is the difference between the maximum value and the minimum value included in the data to be quantized. For example, when the range of each value included in the data to be quantized is less than the fourth threshold, the data processing device determines to use 3 bits (corresponding to M) to quantize each value included in the data to be quantized; when the variance of each value included in the data to be quantized is not less than the fourth threshold, it determines to use 4 bits to quantize each value included in the data to be quantized. The fourth threshold can be 1.2, 10, 100, etc., and this application is not limited. For another example, when the range of each value included in the data to be quantized is less than the fifth threshold and greater than the sixth threshold, the data processing device determines to use 4 bits to quantize each value included in the data to be quantized; when the range of each value included in the data to be quantized is not less than the fifth threshold, it determines to use 6 bits to quantize each value included in the data to be quantized; when the range of each value included in the data to be quantized is not greater than the sixth threshold, it determines to use 2 bits to quantize each value included in the data to be quantized. The sixth threshold can be 0.016, 0.2, 1.2, etc., and the fifth threshold can be 10, 25, etc., which are not limited in this application.
[0068] Exemplarily, the data processing device determines the number of bits M corresponding to the first quantization processing of the Lth layer based on the standard deviation of each value included in the data to be quantized; the number of bits M corresponding to the first quantization processing of the Lth layer is positively correlated with the standard deviation of each value included in the data to be quantized. For example, when the standard deviation of each value included in the data to be quantized is less than the seventh threshold, the data processing device determines to use 3 bits to quantize each value included in the data to be quantized; when the standard deviation of each value included in the data to be quantized is not less than the seventh threshold, the data processing device determines to use 4 bits to quantize each value included in the data to be quantized. The seventh threshold can be 0.04, 0.2, 1.2, 10, etc., and is not limited in this application.
[0069] In this implementation, the data processing device determines the number of bits M corresponding to the first quantization processing of the above-mentioned Lth layer according to the distribution of each numerical value included in the data to be quantized, that is, the number of bits to be used to quantize each numerical value included in the data to be quantized; it can reduce the bits used for quantization, thereby reducing the computing resources required for prediction processing using the neural network model.
[0070] Figure 2 A flow chart of another data processing method provided in an embodiment of the present application. Figure 2 The method flow in is Figure 1 Further refinement and improvement of the method flow in Figure 2 As shown, the method may include:
[0071] 201. A data processing device obtains the data to be quantized outputted by the Lth layer of the neural network model.
[0072] The data to be quantized includes at least one value represented by M or more bits, L is an integer greater than 0, and M is an integer greater than 1. The implementation of step 201 is similar to that of step 101 and will not be described in detail here.
[0073] 202. Cluster the values included in the data to be quantified to obtain at least two cluster centers.
[0074] 203. Taking a maximum value among the at least two cluster centers as the quantized maximum value, and taking a minimum value among the at least two cluster centers as the quantized minimum value.
[0075] 204. Determine a first arithmetic progression that has the minimum quantization value as the first item, the maximum quantization value as the last item, and includes N values.
[0076] The above N is 2 to the Mth power, and M is the number of bits corresponding to the first quantization process of the Lth layer. In some embodiments, the data processing device may pre-store the number of bits to be used to quantize the numerical value included in the above-mentioned data to be quantized. Accordingly, when it is necessary to determine the quantization parameter, the number of bits M corresponding to the stored quantization process is obtained from the memory. In some embodiments, before executing step 204, the data processing device may perform the following operations: determine the number of bits M corresponding to the first quantization process of the Lth layer.
[0077] 205. Use the common factors of each item in the first arithmetic progression as the quantization parameters of the Lth layer of the neural network model, and store the quantization parameters.
[0078] It can be understood that steps 201 to 205 are a method flow for a data processing device to determine the quantization parameters of the Lth layer in the neural network model. In practical applications, the data processing device may adopt a similar method flow to determine the quantization parameters of each layer in the neural network model, and store the quantization parameters of each layer, so that when the neural network model is used to perform a prediction processing task, the quantization parameters of each layer are used to quantize the feature maps (corresponding to the data to be quantized) output by each layer. Optionally, the data processing device stores the quantization parameters of the Lth layer, the minimum quantization value, and the maximum quantization value. In some embodiments, the data processing device may store the quantization parameters, maximum values, and minimum values of each layer; wherein the maximum value and minimum value of any layer are obtained by clustering the numerical values included in the output feature map of any layer.
[0079] 206. Obtain activation data output by the Lth layer in the process of predicting the first data to be processed using the neural network model.
[0080] Optionally, the first data to be processed may be input image data, or data generated by preprocessing the input image through a neural network model, or other data. The Lth layer may be a convolutional layer, a pooling layer, a batch normalization layer, etc. The activation data output by the Lth layer is the output feature map obtained by the Lth layer performing a certain operation (e.g., convolution) on its input feature map.
[0081] 207. Based on the quantization parameter, use M bits to perform a first quantization process on the activation data to obtain the processing result of the Lth layer.
[0082] It can be understood that steps 206 to 207 are for the data processing device to quantize the first intermediate data output by the Lth layer using the quantization parameter of the Lth layer. In practical applications, the data processing device can use similar steps to quantize the data (i.e., feature maps) output by each layer (except the last layer) in the neural network model, so that each layer outputs a quantized feature map. In some embodiments, the activation data of the last layer of the neural network model is generally not low-bit quantized, but the floating-point output is retained. In some embodiments, when the data processing device uses the quantized neural network model for prediction processing, the input data of the first layer of the neural network model is generally pre-processed image data (e.g., 8 bits represent one pixel), that is, the input of the first layer is not low-bit quantized; the last layer of the neural network model is not low-bit quantized, but the floating-point output is retained; the quantization bit width (corresponding to the above M) of the output data (i.e., activation data) of each intermediate network layer of the neural network model is consistent and the quantization bit width of the weight of each intermediate network layer is consistent. For example, the output data of each intermediate network layer is quantized by 4 bits, and the weight of each intermediate network layer is 2 bits.
[0083] In an embodiment of the present application, when a data processing device uses a neural network model to perform a prediction processing task, the data output by each layer in the neural network model is quantized, which can effectively reduce the amount of calculation.
[0084] The following uses the data output by the Lth layer in the quantized neural network model as an example to introduce the method of quantizing the data output by each layer of the neural network model by the data processing device.
[0085] In some embodiments, the data processing device may store the quantization parameter of the Lth layer of the neural network model, the maximum quantization value, the minimum quantization value, and the number M of bits to be used to quantize the data output by the Lth layer; the data processing device may quantize the above-mentioned first intermediate data based on the maximum quantization value, the minimum quantization value, the number M of bits to be used to quantize the Lth layer, and the quantization parameter. Exemplarily, the data processing device determines a first arithmetic sequence with the minimum quantization value as the first item and the maximum quantization value as the last item, and includes N numerical values, N being 2 to the Mth power; extracts the common factors (i.e., quantization parameters) of each item in the first arithmetic sequence to obtain a second arithmetic sequence, each item in the second arithmetic sequence is an integer; based on the first arithmetic sequence and the second arithmetic sequence, uses M bits to quantize the first intermediate data. For example, the maximum quantization value (i.e., max) is 1.3, the minimum quantization value (min) is -0.2, the quantization parameter is 0.1, and M is 4; determine a first arithmetic progression {-0.2, -0.1, 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8. 0.9, 1.0, 1.1, 1.2, 1.3} with -0.2 as the first item and 1.3 as the last item, and including 16 values; extract the common factor 0.1 (i.e., the quantization parameter) of each item in the first arithmetic progression to obtain a second arithmetic progression {-2, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8.9, 10, 11, 12, 13}; based on the first arithmetic progression and the second arithmetic progression, use 4 bits to quantize the data to be quantized. In this example, the common factor Q (i.e., quantization parameter) corresponding to the first intermediate data is 0.1, and the offset value shift is 2. In this example, the general expression of the quantized first intermediate data is: activation∈Q*{0-shift, 1-shift, .., 15-shift}, shift is expressed in 4 bits, and activation represents each value in the first intermediate data. In other words, each value in the first intermediate data can be quantized as the product of the quantization parameter and a value in the second arithmetic progression. For example, 0.71 can be quantized as Q*7, that is, the quantization value of 0.71 is 7. The following introduces an implementation method for quantizing the first intermediate data using M bits based on the first arithmetic progression and the second arithmetic progression.
[0086] Optionally, the data processing device quantizes the first intermediate data using M bits based on the first arithmetic progression and the second arithmetic progression as follows: When the first value in the first intermediate data is greater than the first average value and not less than the second average value, the first value is quantized into the i-th term in the second arithmetic progression using M bits; wherein, the first average value is the average value of the (i-1)-th term and the i-th term in the first arithmetic progression, the second average value is the average value of the i-th term and the (i+1)-th term in the first arithmetic progression, and i is an integer greater than 1 and less than N; When the second value in the first intermediate data is not greater than the third average value, the second value is quantized into the first term in the second arithmetic progression using M bits; the third average value is the average value of the first term and the second term in the first arithmetic progression; When the third value in the first intermediate data is greater than the fourth average value, the third value is quantized into the last term in the second arithmetic progression using M bits; the fourth average value is the average value of the (N-1)-th term and the N-th term in the first arithmetic progression.
[0087] Optionally, each value in the second arithmetic progression is a quantization step. Figure 3 This is a schematic diagram of a quantization step provided by an embodiment of the present application. Exemplarily, the second arithmetic progression includes 16 integers, Figure 3 where q0, q1, …, q15 in it are the first term to the last term in the second arithmetic progression in sequence. For example, the second arithmetic progression includes 16 integers, and the data processing device can generate 15 thresholds according to the second arithmetic progression, denoted as thr0, thr1, …, thr14 respectively. The following relationship exists between these 15 thresholds and the quantization step: thri = (qi + q(i+1)) / 2, i ∈ [1, 14]. In some embodiments, the data processing device can determine the quantization step corresponding to each value to be quantized in the following manner:
[0088] When Qin <= (q0 + q1) / 2, that is, when Qin <= thr0, Qout = q0 / Q;
[0089] When Qin > (q14 + q15) / 2, that is, when Qin > thr14, Qout = q15 / Q;
[0090] When (q(i-1) + qi) / 2 < Qin <= (qi + q(i+1)) / 2, that is, when thr(i-1) < Qin < thri, Qout = qi / Q, i ∈ [1, 14];
[0091] Among them, Qin represents the value to be quantized, that is, the quantization input; Qout represents the value after quantization, that is, the quantization output; Q is the above-mentioned common factor (that is, the quantization parameter). The above-mentioned values to be quantized can be the values included in the first intermediate data. It can be understood that in this embodiment, (q0+q1) / 2 corresponds to the above-mentioned third average value, q0 / Q corresponds to the first item in the above-mentioned second arithmetic progression; (q14+q15) / 2 corresponds to the above-mentioned fourth average value, q15 / Q corresponds to the last item in the above-mentioned second arithmetic progression; (q(i-1)+qi) / 2 corresponds to the average value of the (i-1)th item and the i-th item in the above-mentioned first arithmetic progression, (qi+q(i+1)) / 2 corresponds to the average value of the i-th item and the (i+1)th item in the above-mentioned first arithmetic progression, and qi / Q corresponds to the i-th item in the above-mentioned second arithmetic progression.
[0092] In some embodiments, the data processing device may store the quantization parameter of the Lth layer of the neural network model and the number M of bits to be used to quantize the data output by the Lth layer; the data processing device may quantize the above-mentioned first intermediate data based on the number M of bits to be used to quantize the data output by the Lth layer and the quantization parameter. Exemplarily, the data processing device uses M bits to quantize the above-mentioned first intermediate data based on the above-mentioned quantization parameter. The corresponding quantization formula can be expressed as: Qout = ceil [(Qin-Q / 2) / Q], where Qin represents the value to be quantized, that is, the quantization input; Qout represents the quantized value, that is, the quantization output, and Q represents the quantization parameter. ceil is a function name used to return the smallest integer greater than or equal to the specified expression. ceil [(Qin-Q / 2) / Q] is the smallest integer greater than or equal to [(Qin-Q / 2) / Q]. For example, ceil (0.54) is equal to 1.
[0093] In an embodiment of the present application, when a data processing device uses a neural network model to perform a prediction processing task, the data output by each layer in the neural network model is quantized, which can effectively reduce the amount of calculation.
[0094] The above embodiments introduce the way in which the data processing device quantizes the data output by each layer of the neural network model. In some embodiments, the data processing device can merge the operation processing performed by each layer and the quantization operation processing performed on the data output by each layer into the same operation processing, and convert the merged operation processing into a fixed-point operation processing. The following is an introduction taking the Lth layer in the neural network model as an example.
[0095] Figure 4 Another data processing method flow chart provided in the embodiment of the present application. Figure 4 As shown, the method may include:
[0096] 401. In the process of using a neural network model to perform predictive processing on second data to be processed, the data processing device obtains the first operation parameter required for the Lth layer of the neural network model to perform the first operation processing.
[0097] The second data to be processed may be input image data, or data generated by preprocessing the input image through a neural network model, or other data. The first operation parameter is a parameter required to perform the first operation processing, and the first operation parameter includes the quantization parameter. The first operation processing is an operation processing obtained by combining the operation processing performed by the Lth layer and the quantization operation processing corresponding to the Lth layer.
[0098] 402. Convert the first operation parameter into a fixed-point number to obtain a second operation parameter.
[0099] Fixed-point numbers are a form of data in computers, as opposed to floating-point numbers. In operations, the decimal point never moves. The fixed-point numbers in the embodiments of the present application can be integers or non-integers.
[0100] Optionally, an implementation method of converting the first operation parameter into a fixed-point number to obtain the second operation parameter is as follows: each floating-point number included in the first operation parameter is enlarged by K times to obtain an enlarged value, where K is an integer greater than 1; and the enlarged value is converted into a fixed-point number to obtain the second operation parameter.
[0101] For example, the operation performed by the Lth layer (Eltwise layer) in the neural network model is: Out = x*A+y*B, where x and y are both floating-point parameters, and A and B are two input feature maps of the Eltwise layer; the quantization operation performed on the activation data output by the Lth layer is as follows: Qout = ceil[(Qin-Q / 2) / Q], where Qin represents the data output by the Lth layer, that is, the quantized input; Qout represents the quantized value, that is, the quantized output, and Q represents the quantization parameter; the first operation performed by the Lth layer is: Qout = ceil[(x*A+y*BQ / 2) / Q]. Exemplarily, the data processing device can execute the following calculation formula to convert the floating-point numbers in the data required for the Lth layer to perform the first operation (that is, the first operation parameter) into fixed-point numbers and perform the first operation: Out = ceil[(round(x*K)*A+round(y*K)-round(Q / 2*K)) / round(Q*K)]. The round function rounds a number to a specified number of digits. The function formula is: round(number to be rounded, number of digits to be retained). For example, round(3.1415926,2)=3.14, round(3.1415926,3)=3.142. In some embodiments, the round function can be used to round a floating point number to an integer. For example, round(3.1415926,0)=3, round(7.76)=8.
[0102] For another example, the convolution operation performed by the Lth layer (convolution layer) in the neural network model is: Conv_out = W × A, where W is the weight matrix (weight) of the Lth layer, and A is the input feature map (i.e., activation) of the Lth layer. Assume that the current quantization adopts 2W4A, that is, the bit width of each element in the weight matrix W after quantization is 2 bits, and the bit width of each element in the input feature map A after quantization is 4 bits. W = alpha*B, A = Qparam*M, where B = {-3, -1, 1, 3}, M = {0-shift, 1-shift, ..., 15-shift}, alpha (floating point number) represents the common factor corresponding to the weight matrix, B represents the quantized weight matrix, Qparam (floating point number) represents the common factor corresponding to the input feature map, and M represents the quantized input feature map. Conv_out=W×A=alpha*Qparam*(B×M)=scale*(B*M), so the convolution operation can extract the common factor scale (i.e., alpha*Qparam), and fixed-point convolution operation is performed by B and M during hardware operation. The calculation formula for the quantization operation of the feature map output by the data processing device on the Lth layer is as follows: Qout=ceil[(Qin-Q / 2) / Q], where Q represents the quantization parameter corresponding to the activation data output by the Lth layer. It should be understood that the formula for the operation processing performed by the Lth layer (corresponding to the above-mentioned first operation processing) is expressed as follows: Qout=ceil[(scale*(B*M)-Q / 2) / Q]. Exemplarily, the data processing device can execute the following calculation formula to convert the floating-point numbers in the data required for the Lth layer to perform the first operation processing (i.e., the first operation parameter) into fixed-point numbers, and perform the first operation processing: Qout=ceil[(round(scale*K)*(B*M)-round(Q / 2*K)) / round(Q / 2*K)].
[0103] 403. Based on the second operation parameter, execute the first operation to obtain the processing result of the Lth layer.
[0104] It should be understood that the data processing device can perform the operation processing and quantization operation processing corresponding to each layer in the neural network model in a manner similar to steps 401 to 403. In some embodiments, the operation processing performed by one or more layers of the neural network model can be combined with the quantization operation processing performed on its output data into one operation processing, and the combined operation processing is converted into a fixed-point operation processing, referring to the operation processing performed by the Eltwise layer above.
[0105] The method proposed in the embodiment of the present application can eliminate all floating-point parameters and floating-point operations in the neural network model, and the entire process adopts fixed-point operations, which can improve computing efficiency.
[0106] Figure 5 Another data processing method flow chart provided in the embodiment of the present application. Figure 5 As shown, the method may include:
[0107] 501. The data processing device obtains the third operation parameters required for executing the second operation in the process of predicting the third data to be processed by using the above-mentioned neural network model.
[0108] The third operation reference is a parameter required to perform the second operation processing, the third operation parameter includes the quantization parameter, and the second operation processing is an operation processing obtained by merging the operation processing performed by at least two network layers including the operation processing performed by the Lth layer and the quantization operation processing corresponding to the Lth layer. Optionally, the operation processing performed by the at least two network layers are all linear operation processing. Linear operations are addition and scalar multiplication. In the field of real numbers, linear equations of two variables that only contain addition and scalar multiplication belong to linear operations, such as y=3x+5. If it is matrix addition and scalar multiplication operations, it is called matrix linear operation; if it is vector addition and scalar multiplication operations, they are collectively referred to as vector linear operations.
[0109] 502. Convert the third operation parameter into a fixed-point number to obtain a fourth operation parameter.
[0110] Optionally, an implementation method of quantizing the floating-point numbers in the above-mentioned third intermediate data into fixed-point numbers is as follows: each floating-point number in the above-mentioned third operation parameter is expanded by K times and converted into a fixed-point number respectively; the above-mentioned K is an integer greater than 1.
[0111] 503. Based on the fourth operation parameter, execute the second operation to obtain the processing result of the Lth layer.
[0112] For example, the convolution operation performed by the Lth layer (convolution layer) in the neural network model is: Conv_out = W × A, where W is the weight matrix (weight) of the Lth layer, A is the input feature map (i.e., activation) of the Lth layer, W = alpha*B, A = Qparam*M, alpha (floating point number) represents the common factor corresponding to the weight matrix, B represents the quantized weight matrix, Qparam (floating point number) represents the common factor corresponding to the input feature map, and M represents the quantized input feature map. Conv_out = W × A = alpha*Qparam*(B×M) = scale*(B*M). Assume that part of the network structure in the neural network model is convolution layer (i.e., Lth layer, conv) -> batch normalization layer (i.e., BN layer) -> activation layer (i.e., relu layer). The operation processing performed by the convolution layer, the operation processing (linear operation) performed by the batch normalization layer, and the operation processing (linear operation) performed by the activation layer can be simplified to: BNout = scale1*A+bias after parameter merging, where scale1 is the floating point parameter after merging (including alpha, Qparam, etc.), and A is the output result of the convolution layer. Next, BNout is quantized. The quantization function can be: Qout = ceil[(Qin-Q / 2) / Q], Qin is BNout, Q / 2 can be merged into bias, and finally the operation processing performed by conv->BN->relu (corresponding to the second operation processing) can be simplified to: Qout = ceil[(scale1*A+bias) / Q], where scale1, bias, and Q are all floating point numbers. Next, floating point numbers need to be eliminated. Exemplarily, the data processing device multiplies scale1, bias and Q by a magnification factor K at the same time, and converts the floating point number to a fixed point through the round function, Qout = ceil[(round(N*scale)*A+round(N*bias)) / round(N*Q)] = ceil[(scale_new*A+bias_new) / Q_new], where scale_new, bias_new and Q_new are all fixed-point numbers.
[0113] When the data processing device uses the neural network model to perform prediction processing, the calculations performed will have floating-point parameters, such as batch normalization, scaling operations, etc. In addition, the quantization process will also introduce floating-point numbers, such as the parameter alpha in weight quantization and the quantization parameter Q in activation quantization. Taking into account the complexity, area, computing resources and other factors of hardware implementation, the method proposed in the embodiment of the present application can eliminate all floating-point parameters and floating-point operations in the neural network model, and the entire process uses fixed-point operations to improve computing efficiency.
[0114] Figure 6 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application. Figure 6 As shown, the data processing device includes:
[0115] A clustering unit 601 is used to perform clustering processing on multiple values included in the data to be quantized output by the Lth layer of the neural network model to obtain a clustering result, where L is an integer greater than 0, and the data to be quantized is a test image or is obtained by processing the test image or sample image through the first (L-1) network layers of the neural network model;
[0116] A determination unit 602 is used to obtain a quantization maximum value and a quantization minimum value based on the above clustering result;
[0117] The determination unit 602 is also used to determine the quantization parameter corresponding to the first quantization processing of the above-mentioned Lth layer based on the above-mentioned quantization maximum value and the above-mentioned quantization minimum value, wherein the above-mentioned first quantization processing is used to quantize the numerical value in the activation data of the above-mentioned Lth layer into an integer represented by M bits.
[0118] In an optional implementation, the determination unit 602 is specifically configured to use the maximum value of at least two cluster centers included in the clustering result as the quantized maximum value, and use the minimum value of the at least two cluster centers as the quantized minimum value.
[0119] In an optional implementation, the determination unit 602 is specifically used to obtain a first arithmetic progression including N numerical values with the above-mentioned minimum quantization value as the first item and the above-mentioned maximum quantization value as the last item; and determine the quantization parameter based on the N numerical values contained in the first arithmetic progression.
[0120] In an optional implementation, the determination unit 602 is further configured to determine the number of bits M corresponding to the first quantization process of the Lth layer based on the distribution discreteness of multiple values included in the data to be quantized.
[0121] In an optional implementation, the above device further includes:
[0122] The first processing unit 603 is used to obtain the activation data of the Lth layer in the process of predicting the first image using the neural network model;
[0123] The quantization unit 604 is used to perform a first quantization process on the activation data based on the quantization parameter to obtain the processing result of the Lth layer.
[0124] In an optional implementation, the above device further includes:
[0125] A first conversion unit 605 is used to convert a first operation parameter into a fixed-point number to obtain a second operation parameter; the first operation parameter is a parameter required to perform a first operation process, the first operation parameter includes the quantization parameter, and the first operation process is obtained by combining the network model process performed by at least one network layer including the Lth layer and the first quantization operation process;
[0126] The second processing unit 606 is used to perform the first operation processing on the input data of the at least one network layer based on the second operation parameter to obtain the at least one network layer; the input data of the at least one network layer is the image to be predicted, or is a processing result obtained by processing the image to be predicted by at least one second network layer located before the at least one network layer in the neural network model.
[0127] In an optional implementation, the first conversion unit 605 is specifically used to expand each floating-point number included in the above-mentioned first operation parameter by K times to obtain an expanded value, where the above-mentioned K is an integer greater than 1; and convert the above-mentioned expanded value into a fixed-point number to obtain the above-mentioned second operation parameter.
[0128] In an optional implementation, the first operation processing is obtained by merging the network model processing and the first quantization processing performed by at least two adjacent network layers including the Lth layer.
[0129] It should be understood that the division of the various units of the above data processing device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. For example, the above units can be separately established processing elements, or they can be integrated into the same chip for implementation. In addition, they can also be stored in the storage element of the controller in the form of program code, and called and executed by a certain processing element of the processor. The functions of the above units. In addition, the various units can be integrated together or implemented independently. The processing element here can be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the above method or the above units can be completed by hardware integrated logic circuits in the processor element or software instructions. The processing element can be a general-purpose processor, such as a central processing unit (CPU), or one or more integrated circuits configured to implement the above method, such as one or more application-specific integrated circuits (ASICs), or one or more microprocessors (DSPs), or one or more field programmable gate arrays (FPGAs).
[0130] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device 70 includes a processor 701, a memory 702 and a communication interface 703; the processor 701, the memory 702 and the communication interface 703 are interconnected via a bus. Figure 7 The electronic device in the embodiment may be the data processing device in the foregoing embodiment.
[0131] The memory 702 includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a portable read-only memory (CDROM), and is used for related instructions and data. The communication interface 703 is used to receive and send data.
[0132] The processor 701 may be one or more central processing units (CPUs). When the processor 701 is a CPU, the CPU may be a single-core CPU or a multi-core CPU. Figure 7 Specifically, the processor 701 can implement Figure 6 The functions of each unit in it.
[0133] The processor 701 in the electronic device 70 is used to read the program code stored in the memory 702 and execute the data processing method in the above-mentioned embodiment.
[0134] Figure 8 : is a structural diagram of a server 800 provided in an embodiment of the present application. The server 800 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 822 (for example, one or more processors) and memory 832, and one or more storage media 830 (for example, one or more mass storage devices) storing application programs 842 or data 844. Among them, the memory 832 and the storage medium 830 can be short-term storage or permanent storage. The program stored in the storage medium 830 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 822 can be configured to communicate with the storage medium 830 and execute a series of instruction operations in the storage medium 830 on the server 800. The server 800 can be a data processing device provided in the present application.
[0135] The server 800 may also include one or more power supplies 826, one or more wired or wireless network interfaces 850, one or more input and output interfaces 858, and / or one or more operating systems 841, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
[0136] The steps performed by the data processing device in the above embodiment can be based on the Figure 8 Specifically, the central processor 822 can implement Figure 6 The functions of each unit in it.
[0137] In an embodiment of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the data processing method provided in the above embodiment is implemented.
[0138] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A data processing method, characterized in that: include: Performing clustering processing on multiple values included in the data to be quantized in the Lth layer of the neural network model to obtain a clustering result, where L is an integer greater than 0, and the data to be quantized is a test image or is obtained by processing the test image or sample image through the first (L-1) network layers of the neural network model; Based on the clustering result, obtaining a quantization maximum value and a quantization minimum value; Determining a quantization parameter corresponding to a first quantization process of the Lth layer based on the quantization maximum value and the quantization minimum value, wherein the first quantization process is used to quantize a numerical value in the activation data of the Lth layer into an integer represented by M bits; After determining the quantization parameter corresponding to the first quantization process of the Lth layer based on the quantization maximum value and the quantization minimum value, the method further includes: Obtaining activation data of the Lth layer during a process of predicting the first image using the neural network model; The first quantization processing is performed on the activation data based on the quantization parameter to obtain a processing result of the Lth layer.
2. The method according to claim 1, characterized in that The obtaining of the maximum quantization value and the minimum quantization value based on the clustering result includes: The maximum value of at least two cluster centers included in the clustering result is used as the quantized maximum value, and the minimum value of the at least two cluster centers is used as the quantized minimum value.
3. The method according to claim 1 or 2, characterized in that: The determining, based on the maximum quantization value and the minimum quantization value, a quantization parameter corresponding to the first quantization process of the Lth layer includes: Taking the minimum quantization value as the first item and the maximum quantization value as the last item, a first arithmetic progression including N values is obtained; The quantization parameter is determined based on the N numerical values included in the first arithmetic progression.
4. The method according to claim 1 or 2, characterized in that: The method further comprises: The number of bits M corresponding to the first quantization process of the Lth layer is determined based on the distribution discreteness of multiple values included in the data to be quantized.
5. A data processing method, characterized in that: include: Performing clustering processing on multiple values included in the data to be quantized in the Lth layer of the neural network model to obtain a clustering result, where L is an integer greater than 0; Based on the clustering result, obtaining a quantization maximum value and a quantization minimum value; Determining a quantization parameter corresponding to a first quantization process of the Lth layer based on the quantization maximum value and the quantization minimum value, wherein the first quantization process is used to quantize a numerical value in the activation data of the Lth layer into an integer represented by M bits; In the process of using the neural network model to perform prediction processing on second data to be processed, the first operation parameter required for the L-th layer to perform the first operation processing is obtained, and the second data to be processed is input picture data or data generated by preprocessing the input picture through the neural network model; Convert the first operation parameter into a fixed-point number to obtain a second operation parameter; the first operation parameter is a parameter required to perform a first operation process, the first operation parameter includes the quantization parameter, and the first operation process is obtained by combining the network model process performed by at least one network layer including the Lth layer and the first quantization process; Based on the second operation parameter, the first operation processing is performed on the input data of the at least one network layer to obtain the processing result of the at least one network layer.
6. The method according to claim 5, characterized in that The converting the first operation parameter into a fixed-point number to obtain a second operation parameter includes: Expand each floating-point number included in the first operation parameter by K times to obtain an expanded value, where K is an integer greater than 1; The expanded value is converted into a fixed-point number to obtain the second operation parameter.
7. The method according to claim 5 or 6, characterized in that: The first operation processing is obtained by merging the network model processing and the first quantization processing performed by at least two adjacent network layers including the Lth layer.
8. A data processing device, characterized in that: include: A clustering unit is used to perform clustering processing on multiple values included in the data to be quantized outputted by the Lth layer of the neural network model to obtain a clustering result, wherein L is an integer greater than 0, and the data to be quantized is a test image or is obtained by processing the test image or sample image through the first (L-1) network layers of the neural network model; A determination unit, configured to obtain a quantization maximum value and a quantization minimum value based on the clustering result; The determining unit is further used to determine a quantization parameter corresponding to a first quantization process of the Lth layer based on the quantization maximum value and the quantization minimum value, wherein the first quantization process is used to quantize a numerical value in the activation data of the Lth layer into an integer represented by M bits; the device also includes: A first processing unit, used for obtaining activation data of the Lth layer in the process of predicting the first image using the neural network model; A quantization unit is used to perform the first quantization processing on the activation data based on the quantization parameter to obtain a processing result of the Lth layer.
9. The device according to claim 8, characterized in that The determining unit is specifically configured to use the maximum value of at least two cluster centers included in the clustering result as the quantized maximum value, and use the minimum value of the at least two cluster centers as the quantized minimum value.
10. The device according to claim 8 or 9, characterized in that The determination unit is specifically used to obtain a first arithmetic progression including N values with the minimum quantization value as the first item and the maximum quantization value as the last item; and determine the quantization parameter based on the N values included in the first arithmetic progression.
11. The device according to claim 8 or 9, characterized in that The determining unit is further configured to determine the number of bits M corresponding to the first quantization process of the Lth layer based on the distribution discreteness of multiple numerical values included in the data to be quantized.
12. A data processing device, characterized in that: include: A clustering unit, used for clustering a plurality of numerical values included in the data to be quantized outputted by the Lth layer of the neural network model to obtain a clustering result, where L is an integer greater than 0; A determination unit, configured to obtain a quantization maximum value and a quantization minimum value based on the clustering result; The determining unit is further used to determine a quantization parameter corresponding to a first quantization process of the Lth layer based on the quantization maximum value and the quantization minimum value, wherein the first quantization process is used to quantize a numerical value in the activation data of the Lth layer into an integer represented by M bits; the device also includes: A first conversion unit is used to convert a first operation parameter into a fixed-point number to obtain a second operation parameter; the first operation parameter is a parameter required to perform a first operation process, the first operation parameter includes the quantization parameter, and the first operation process is obtained by combining a network model process performed by at least one network layer including the Lth layer and the first quantization process; The second processing unit is used to perform the first operation processing on the input data of the at least one network layer based on the second operation parameter to obtain the at least one network layer.
13. The device according to claim 12, characterized in that The first conversion unit is specifically used to expand each floating-point number included in the first operation parameter by K times to obtain an expanded value, where K is an integer greater than 1; and convert the expanded value into a fixed-point number to obtain the second operation parameter.
14. The device according to claim 12 or 13, characterized in that The first operation processing is obtained by merging the network model processing and the first quantization processing performed by at least two adjacent network layers including the Lth layer.
15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the method according to any one of claims 1 to 7.
16. An electronic device, characterized in that: include: Memory, used to store programs; A processor, configured to execute the program stored in the memory; when the program is executed, the processor is configured to execute the method according to any one of claims 1 to 7.
17. A chip, characterized in that: include: A processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface to execute the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Method and apparatus for quantizing parameters of neural network
CN109389219A
Convolutional neural network low bit width quantization method based on weight distribution
CN110222821A
Neural network quantization method and device, image recognition method and device and computer equipment
CN110443165A
Compression and acceleration method of neural network model, data processing method and device
CN110659725A
Data processing method and related product
CN113222097A
