Calibration data selection method for deep learning model post-training uniform quantization

Abstract

The invention discloses a calibration data selection method for training uniform quantization after a deep learning model. The method comprises four steps of clustering centroid initialization, clustering centroid updating, training set data sorting and calibration set data selection. Firstly, random data conforming to uniform distribution are sent to a network for forward reasoning, and a plurality of clustering centroids are determined; then, reasoning is performed through training set data, a knowledge distance is calculated, a clustering centroid is updated, and an activation distribution space accurately reflecting training set data distribution is constructed; then, training set data are sent to the network for reasoning again, knowledge distances are calculated, summation is carried out, and scores of all data samples are obtained and sorted; and finally, according to the size of a specified calibration set, selecting training set data with relatively high scores as the calibration set from high to low. According to the method, through reasonably selecting calibration data, the effect of training uniform quantification after the deep learning model can be improved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning model quantization methods, and in particular to a calibration data selection method for uniform quantization of deep learning model post-training. Background Art

[0002] With the rapid development of deep learning technology, deep neural network (DNN) models have been widely used in many fields such as image recognition and speech processing. However, these models usually contain millions of parameters, resulting in a large amount of computing resources and energy consumption in practical applications. To solve this problem, model quantization technology has emerged, aiming to reduce the memory usage and computing latency of the model by using low-bit representation. In the post-training quantization process, a set of calibration data needs to be selected to determine the quantization parameters to minimize the performance loss caused by quantization. Traditional random selection methods may lead to unstable performance of the quantized model because they cannot guarantee the representativeness of the calibration data. Therefore, how to select appropriate calibration data has become the key to improving the performance of the quantized model. Summary of the invention

[0003] In view of the problems existing in the prior art, the present invention provides a novel calibration data selection method, which selects a calibration data set that can represent the overall data distribution from the training data set through dynamic clustering and knowledge distance measurement, so as to improve the accuracy and stability of the quantitative model.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is:

[0005] The calibration data selection method for uniform quantization of post-training deep learning models includes the following steps:

[0006] Step 1: Initialize cluster centroids: First, random data with uniform distribution and the same size as the original input is fed into the deep neural network. The deep neural network performs forward reasoning through these random data and records the distribution of activations at each layer. Then, according to the activation distribution of each layer, several cluster centroids are counted and determined.

[0007] Step 2: Cluster centroid update: During the cluster centroid update process, all data in the training set are fed into the deep neural network for inference. During this process, the activation statistics of each layer are calculated, and the knowledge distance with all cluster centroids of the current layer is calculated and sorted. Then, the cluster centroid with the closest knowledge distance to the current activation statistics is updated. After traversing all the data, an activation distribution space that accurately reflects the distribution of the training set data is constructed.

[0008] Step 3, sorting the training set data: After updating the cluster centroids, fix all cluster centroids and send each data sample of the training set to the deep neural network for inference again to obtain the activation statistics corresponding to each data sample; then, calculate the knowledge distance between the activation statistics of each layer and the nearest cluster centroid of the current layer, and sum up all the calculated knowledge distances to obtain the score of each data sample;

[0009] Step 4: Calibration set data selection: Based on the specified calibration set size, select the training set data with higher scores from high to low as the calibration data in the final specified calibration set.

[0010] Furthermore, the cluster centroids mentioned in step 1 consist of the mean and variance of the activations.

[0011] Furthermore, the knowledge distance mentioned in step 2 and step 3 refers to the weighted sum of the Euclidean distance between means and the Euclidean distance between variances.

[0012] The present invention uses the activation statistics of the model to select calibration data for post-training uniform quantization. First, the initial cluster centroids of each layer are determined by initializing the cluster centroids in step 1; in step 2, the cluster centroids are updated based on the training data to accurately construct the activation space that reflects the distribution of the training set data; then in step 3, the training set data is scored and sorted according to the knowledge distance from the cluster centroids; finally, in step 4, the data with the highest score is selected as the calibration set to ensure that the range of activation is more accurately determined during the uniform quantization process. This method can effectively reduce the quantization error, thereby improving the inference performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 Schematic diagram of the method for selecting a uniform quantization calibration set for post-training of a neural network disclosed in an embodiment of the present invention

[0014] Figure 2 Schematic diagram of dynamic clustering and data scoring principles for selecting calibration sets disclosed in the present invention. DETAILED DESCRIPTION

[0015] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0016] This embodiment discloses a calibration data selection method for post-training uniform quantization of a deep learning model, which traverses the entire training set and applies dynamic clustering on activation distribution statistics. During the entire dynamic clustering process, the parameters of the batch normalization layer are fixed. The specific process is as follows:

[0017] Step 1: Initialize cluster centroids:

[0018] First, random data with uniform distribution and the same size as the original input are sent to the deep neural network. The deep neural network performs forward reasoning through these random data, and then generates a corresponding number of activations in each layer of the deep neural network, and records the distribution of activations in each layer.

[0019] Then, according to the activation distribution of each layer, multiple activation distribution statistics of each layer are used to initialize multiple cluster centroids, thereby counting and determining several cluster centroids. Taking the case of a single layer in a deep neural network as an example, in the i-th layer, each cluster centroid C i By the mean of an activation distribution and standard deviation Input some random data that conform to uniform distribution, and generate some cluster centroids in each activation layer of deep neural network.

[0020] Step 2: Cluster centroid update: During the cluster centroid update process, all data in the training set are fed into the deep neural network for inference. During this process, the activation statistics of each layer are calculated, and the knowledge distance with all cluster centroids of the current layer is calculated and sorted. Then, the cluster centroid with the closest knowledge distance to the current activation statistics is updated. After traversing all the data, an activation distribution space that accurately reflects the distribution of the training set data is constructed.

[0021] Specifically, in order to update the cluster centroid and score the training set image data, this embodiment proposes a metric for clustering using activation distribution information: knowledge distance. The definition is as follows: for the i-th layer with mean and standard deviation Each cluster centroid C i , calculate the activation statistics μ of the i-th layer during the inference process of a single data sample i and σ i and the cluster centroid C i The Euclidean distance D(C i ), as follows:

[0022]

[0023] Where γ is a hyperparameter for the Euclidean distance of weighted mean and standard deviation, which describes the consistency level of activation distribution from the perspective of activation statistical characteristics. By counting activation data layer by layer, a large amount of computational cost in cluster centroid update and data selection is avoided.

[0024] In this embodiment, the training set data samples are fed into the deep neural network one by one for inference, and dynamic clustering is performed to update the cluster centroid of each layer. First, the activation distribution statistics corresponding to each data are extracted, that is, the mean μ is calculated. i and standard deviation σ i. After that, the knowledge distances between the obtained statistics and all cluster centroids in the same layer are calculated and sorted. For the cluster centroid closest to the current statistics, it is assumed that it best represents the current distribution. In order to learn from the current distribution, the update formula of the cluster centroid closest to the current statistics is as follows:

[0025]

[0026] in: and is the statistics updated within the cluster centroid, λ t is the update step size of dynamic clustering.

[0027] In order to adjust the cluster centroid offset during the update process, this embodiment updates the step length λ of the dynamic clustering t Perform cosine annealing as follows:

[0028]

[0029] Where: t is the current batch number, t max is the total number of batches into which the entire training set is divided, λ min and λ max Represent the start and end update steps respectively. t It changes continuously during the entire training set traversal.

[0030] Step 3, sorting the training set data: After updating the cluster centroids, fix all cluster centroids and send each data sample of the training set to the deep neural network for inference again to obtain the activation statistics corresponding to each data sample; then, calculate the knowledge distance between the activation statistics of each layer and the nearest cluster centroid of the current layer, and sum up all the calculated knowledge distances to obtain the score of each data sample.

[0031] Specifically, after updating the cluster centroids, all cluster centroids are fixed and the training set is traversed again to sort all samples. Figure 2 As shown in the figure, each data sample in the training set is fed into the network for inference in turn, and the distance between each layer activation and all cluster centroids of the layer is calculated. For each layer, the shortest distance to the cluster centroid is selected, and then the shortest distances of each layer are accumulated to obtain the final score of the training data. The calculation formula is as follows:

[0032]

[0033] Where: S represents the score of the current data sample, S i is the score of the activation of the i-th layer of the current data sample, l represents the number of network layers, and N represents the number of cluster centroids of each layer of the network. represents the jth cluster centroid of the i-th layer of the network, represents the knowledge distance between the activation of the i-th layer of the current data sample and the centroid of the j-th cluster in the current layer. D(·) is the calculation function of the knowledge distance. The smallest one among them is the score S of the current layer. i , accumulate the scores of each layer to get the score S of the current data sample.

[0034] Step 4, calibration set data selection: According to the specified calibration set size, select the data in the training set with higher score S from high to low as the calibration data in the final specified calibration set.

[0035] Specific as Figure 1 As shown, after the calibration set is prepared, calibration can be performed, and the calibration set is sent to the network for inference to determine the clipping range of activations of each layer. For activation, a common operation is to input a small amount of unlabeled data into the model for inference, and use the minimum / maximum value of the activation as the boundary of the clipping range [α, β]. The process of determining the activation clipping range is called calibration, and the unlabeled data set used in this process is called the calibration set. Then, in order to represent floating-point numbers with n-bit integers, this embodiment uniformly maps the original floating-point values ​​to the integer interval [ -2 N-1 ,2 N-1 -1 ] ,This process is called uniform quantization, and is defined as follows:

[0036]

[0037] Where: x Q and x FP Represent the quantized value and the full-precision value respectively, Δ is the scaling factor of the uniform mapping, Z is the integer offset used for asymmetric quantization, α is the lower bound of the clipping range, and β is the upper bound of the clipping range. After quantization, during model inference, the activation value will be truncated by the determined clipping range, and the value outside the clipping range will be replaced by α or β, resulting in truncation error. In addition, if the clipping range is too large, the original value will be represented with low precision, resulting in rounding error.

[0038] The method of this embodiment is further described below in conjunction with experiments.

[0039] This example performs image classification task evaluation on the ImageNet dataset. Compared with the calibration set obtained by random sampling under 4-bit quantization, the Top-1 accuracy of ResNet18 is improved by more than 15%. The quantized performance comparison is shown in Table 1:

[0040] Table 1 Comparison of the effects of typical quantization methods and the method of this embodiment

[0041]

[0042]

[0043] The present invention has been extensively experimentally verified on the ImageNet dataset, and the results show that the SelectQ method of this embodiment improves the Top-1 accuracy of the ResNet18 model by more than 15% in 4-bit quantization, showing a significant performance advantage over randomly sampled calibration data. The method of the present invention not only improves the accuracy of the quantization model, but also has less computing resource requirements and is suitable for various deep learning models and hardware deployment environments.

[0044] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. The embodiments described in the present invention are merely descriptions of the preferred embodiments of the present invention, and do not limit the concept and scope of the present invention. The various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction, and such combinations should also be regarded as the contents disclosed in the present disclosure as long as they do not violate the concept of the present invention. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

[0045] The present invention is not limited to the specific details of the above-mentioned embodiments. Within the technical concept of the present invention and without departing from the design concept of the present invention, various modifications and improvements made to the technical solution of the present invention by technical personnel in this field should fall within the protection scope of the present invention. The technical contents for which protection is sought in the present invention have been fully recorded in the claims.

Claims

1. A calibration data selection method for uniform quantization of post-training deep learning models, characterized in that: The following steps are involved: Step 1: Initialize cluster centroids: First, random data with uniform distribution and the same size as the original input is fed into the deep neural network. The deep neural network performs forward reasoning through these random data and records the distribution of activations at each layer. Then, according to the activation distribution of each layer, several cluster centroids are counted and determined. Step 2: Cluster centroid update: During the cluster centroid update process, all data in the training set are fed into the deep neural network for inference. During this process, the activation statistics of each layer are calculated, and the knowledge distance with all cluster centroids of the current layer is calculated and sorted. Then, the cluster centroid with the closest knowledge distance to the current activation statistics is updated. After traversing all the data, an activation distribution space that accurately reflects the distribution of the training set data is constructed; Step 3, sorting the training set data: After updating the cluster centroids, fix all cluster centroids and send each data sample of the training set to the deep neural network for inference again to obtain the activation statistics corresponding to each data sample; then, calculate the knowledge distance between the activation statistics of each layer and the nearest cluster centroid of the current layer, and sum up all the calculated knowledge distances to obtain the score of each data sample; Step 4: Calibration set data selection: Based on the specified calibration set size, select the training set data with higher scores from high to low as the calibration data in the final specified calibration set.

2. The calibration data selection method for uniform quantization of deep learning model post-training according to claim 1, characterized in that: The cluster centroids mentioned in step 1 consist of the mean and variance of the activations.

3. The calibration data selection method for uniform quantization of deep learning model post-training according to claim 1, characterized in that: The knowledge distance mentioned in step 2 and step 3 refers to the weighted sum of the Euclidean distance between means and the Euclidean distance between variances.

Images