A fast method for quantizing visual self-attention models
By constructing the ridge regression model and reset rounding method, the problem of large quantization error of ViTs is solved, and the visual self-attention model is quickly quantified, which improves the model performance and is suitable for resource-constrained environments.
Patent Information
- Application Number
- CN202410793310.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-19
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-06-19
AI Technical Summary
Existing visual self-attention models (ViTs) are difficult to deploy in resource-constrained environments due to their complex architectures, and existing quantization methods ignore the interdependence between weights and activation quantization. , resulting in large quantization errors.
By constructing a ridge regression model, reduce the activation value quantization error, adjust the weight to reduce the quantization error, and determine the weight quantization parameters through reset rounding, and execute the loop until all weights are quantized, making full use of the relationship between weights and activation values, and quickly quantization Visual self-attention model.
It significantly reduces quantization errors and improves the performance of the quantization model, especially at low bit widths, which is suitable for image classification, object detection and instance segmentation tasks.
Smart Images

Figure CN118378672B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to compression and acceleration of artificial neural networks, and in particular to a method for rapidly quantizing a visual self-attention model. Background Art
[0002] The visual self-attention model Vision Transformers (ViTs) has surpassed convolutional neural networks (CNNs) in various visual tasks and has become a new paradigm in the field of computer vision. ViTs uses the multi-head self-attention (MHSA) mechanism to capture the long-range relationship between image patches and has demonstrated impressive performance in various visual tasks (object detection, instance segmentation, image super-resolution, etc.).
[0003] However, along with the great performance comes considerable complexity. The inherent architectural complexity of ViTs leads to high computational requirements and large memory requirements, which poses challenges for deployment in resource-constrained environments. To alleviate this problem, model quantization has attracted continuous attention from both industry and academia. Quantization reduces model complexity by reducing the bit representation of weights and activations, providing a feasible way for efficient deployment. Recently, researchers have begun to favor post-training quantization (PTQ) of ViTs, an approach that aims to quantize models with a small calibration dataset and smaller cost.
[0004] Many scholars have explored various PTQ methods to adapt to the unique structure of ViTs. For example, in order to deal with the activation values after Softmax that present a long-tail distribution, log2 quantizers and double uniform quantizers were introduced. In order to target activation value distributions with large variance, some methods use reparameterization techniques to better quantize activation values. In addition, evolutionary search methods are used to determine and optimize unstable quantization parameters. However, existing methods usually ignore the intricate interdependence between weight and activation quantization, resulting in considerable quantization errors in weight activation quantization. Summary of the invention
[0005] In view of the problems existing in the prior art, the purpose of the present invention is to provide a method for quickly quantizing a visual self-attention model, which can quickly quantize the visual self-attention model and ensure the model performance by relying on the relationship between the total weight and the activation value.
[0006] To achieve the above object, the technical solution adopted by the present invention is:
[0007] A method for quickly quantizing a visual self-attention model, comprising the following steps:
[0008] Step 1: Based on the activation value quantization error reduction strategy, the activation value quantization error reduction problem is constructed as a ridge regression model, and the weight adjustment value is obtained by analyzing the ridge regression model;
[0009] Step 2: Based on the weight quantization error reduction strategy, the quantization parameters of some weights are obtained by resetting the rounding method, and the adjustment values of the remaining weights are determined by ridge regression modeling;
[0010] Step 3: Repeat step 2 until all weights are quantized;
[0011] The step 2 is specifically as follows:
[0012] Using the activation value quantization error reduction strategy, the weight is recorded as , the activation value is , the quantization of the activation value is regarded as a perturbation , so the quantization error Note as:
[0013] ;
[0014] To this end, add disturbance to the weight , build a ridge regression model:
[0015] ;
[0016] in, is a hyperparameter that controls the strength of regularization, represents the activation value after quantization, Indicates the desired operation;
[0017] Solving the above ridge regression model Minimum, can get disturbance , that is, the size of the weight adjustment value is:
[0018] ;
[0019] in, Represents the activation value after quantization Perform transposition; represents the identity matrix;
[0020] The step 2 is specifically as follows:
[0021] Using the weight quantization error reduction strategy, the quantization of the weight W is regarded as adding a related disturbance, that is, the quantization error of the weight , the quantization error generated at this time is:
[0022] ;
[0023] The above formula outputs the weight of the i-th channel of the channel Divided into two parts and ,in, The output dimension representing the weights; Representation Quantization The error disturbance caused by represents the weight to be quantized, represents the full-precision weight; accordingly, the corresponding activation value is divided into and , at this time, the quantization error of the i-th channel is:
[0024] ;
[0025] in, Representation Quantization The error disturbance caused by
[0026] First, the quantization error is perturbed using the rounding strategy. Perform the rounding operation and then reverse the rounding operation as follows:
[0027] ;
[0028] in, represents the error disturbance of the jth column of the ith channel, Express Round down, Express Round up; S is a set whose elements contain elements with the same gradient direction and perturbation direction;
[0029] Next, by adjusting the weights of the full precision , so that the following quantization error term is minimized:
[0030] ;
[0031] in, represents the hyperparameter that controls the regularization strength, It means that the quantization error term is minimized of disturbance;
[0032] Solving the above ridge regression model, we get The sizes are as follows:
[0033] ;
[0034] express The transpose of Represents the activation value after quantization of the nth calibration data ;use and To obtain the above formula and The value of , where n represents the nth calibration data, N represents the Nth calibration data, and is the calibration set size;
[0035] Then the adjusted value of the remaining weight is obtained.
[0036] After adopting the above scheme, the present invention proposes a strategy for reducing the quantization error of the activation value, models the quantization error reduction problem of the activation value as ridge regression, and obtains the adjustment value of the weight through analytical solution; at the same time, the present invention proposes a strategy for reducing the quantization error of the model weight, obtains the quantization parameter of the weight by resetting the rounding method, and obtains the adjustment value of the remaining weight by ridge regression modeling. This method makes full use of the relationship between weights and activation values, can quickly determine the quantization parameters that are adapted to the activation value and weight, obtain the quantization model, and greatly improve the performance of the quantization model. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 A block diagram of the method of the present invention. DETAILED DESCRIPTION
[0038] like Figure 1 As shown, the present invention discloses a method for quickly quantizing a visual self-attention model, which includes an activation value quantization error reduction strategy and a model weight quantization error reduction strategy, wherein the activation value quantization error reduction strategy models the activation value quantization error reduction problem as a ridge regression, and obtains the adjusted value of the weight through an analytical solution; the model weight quantization error reduction strategy obtains the quantization parameter of the weight by resetting the rounding method, and obtains the adjusted value of the remaining weight through ridge regression modeling.
[0039] Assuming full precision activation values Satisfies the distribution P, that is ; Weight , and represents the dimensions of the input and output data, and Represent the input dimension and output dimension of the weight respectively.
[0040] Then, the quantization error of the weight for:
[0041]
[0042] Quantization error of activation values for:
[0043]
[0044] For each layer, the MSE (mean square error) is set as the quantization error:
[0045]
[0046] The quantization goal is to minimize the quantization error .
[0047] In the above formula, Indicates the desired operation. Indicates the weight value after quantization; Represents the activation value after quantization, which is expressed as:
[0048]
[0049] Among them, the quantization bit width , is a uniform quantizer, s and z represent quantization scaling and displacement parameters, represents a rounding operation, The function represents the truncation function, which truncates m between p and q.
[0050] Typically, s is determined using a search strategy, and the size of z is determined by the following formula:
[0051]
[0052] The method of the present invention specifically comprises the following steps:
[0053] Step 1: Based on the activation quantization error reduction strategy (AQER), the activation value quantization error reduction problem is constructed as a ridge regression model, and the adjustment value of the weight is obtained by analyzing the ridge regression model.
[0054] Specifically, the activation quantization error reduction strategy (AQER) is used, and the weight is recorded as , the activation value is , the quantization error of the activation value is , so the quantization error It can be written as:
[0055]
[0056] To this end, add disturbance to the weight , build a ridge regression model:
[0057]
[0058] in, is a hyperparameter that controls the strength of regularization;
[0059] Solving the above ridge regression model, we can get the perturbation The size of (i.e. the adjusted value of the weight) is:
[0060] ;
[0061] in, Represents the quantized activation value Transpose, represents the identity matrix;
[0062] Step 2: Based on the weight quantization error reduction strategy (WQER), the quantization parameters of the weights are obtained by resetting the rounding method, and the adjustment values of the remaining weights are determined by ridge regression modeling.
[0063] Specifically, using the weight quantization error reduction strategy (WQER), for the quantization of weight W, it can be considered that the relevant perturbation is added That is, the quantization error of the weight. The quantization error generated at this time is:
[0064]
[0065] The above formula outputs the weight of the i-th channel of the channel Divided into two parts and ,in, Representation Quantization The error disturbance caused by represents the weight to be quantized, represents the full-precision weight. Correspondingly, the corresponding activation value Divide and , at this time, the quantization error of the i-th channel is:
[0066]
[0067] in, Representation Quantization The error disturbance caused.
[0068] In order to minimize the above quantization error disturbance As small as possible, use the reset rounding method to quantize. Specifically, first use the rounding strategy to perturb the quantization error Perform the rounding operation and then reverse the rounding operation as follows:
[0069]
[0070] Among them, S is a set whose elements contain elements with the same gradient direction and perturbation direction; represents the error disturbance of the ith channel, represents the error disturbance of the jth column of the i-th channel; Express Round down, Express Round up.
[0071] Next, build a ridge regression model:
[0072]
[0073] in, represents the hyperparameter that controls the regularization strength, It means that the above goal can be minimized of disturbance.
[0074] Solve the above ridge regression model to minimize the quantization error term and get the weight adjustment value The sizes are as follows:
[0075]
[0076] Where I represents the identity matrix, Representation and The corresponding activation value, Representation and The corresponding activation value, Indicated Transpose; Represents the activation value after quantization of the nth calibration data .
[0077] In actual operation, use and To obtain the above formula and The value of , where n represents the nth calibration data and the default value is 1, and N represents the Nth calibration data and is the calibration set size.
[0078] Step 3: Repeat step 2 until all weights are quantized.
[0079] As mentioned above, the present invention proposes a strategy for reducing the quantization error of activation values, models the quantization error reduction problem of activation values as ridge regression, and obtains the adjusted value of the weight through analytical solution; at the same time, the present invention proposes a strategy for reducing the quantization error of model weights, obtains the quantization parameters of the weights by resetting the rounding method, and obtains the adjusted value of the remaining weights by ridge regression modeling. This method makes full use of the relationship between weights and activation values, can quickly determine the quantization parameters that are adapted to the activation values and weights, obtain the quantization model, and greatly improve the performance of the quantization model.
[0080] We conduct extensive experiments on image classification, object detection, and instance segmentation.
[0081] For the image classification task, different ViT (a basic model of computer vision, the DeiT and Swin models mentioned below are also basic models of the same type as ViT, only with differences in structural design) types of models are used, including ViT, DeiT and Swin, and evaluated on the ImageNet (a public dataset for computer vision competitions) dataset.
[0082] For object detection and instance segmentation tasks, Mask R-CNN (an object detection model) and Cascade Mask R-CNN (an object detection model) are used on the COCO (a commonly used object detection dataset) dataset, both using Swin as the backbone network.
[0083] The specific experiment is as follows: 32 images are randomly selected from the ImageNet dataset and 1 image is selected from the COCO dataset. The quantization parameters are determined by forward propagation of the calibration dataset, and the activation quantizer is initialized using the reparameterization technique. In the present invention, the maximum number of iterations for resetting rounding is set to 100.
[0084] For image classification tasks, the present invention sets ViT = = 1e4, DeiT-T (T indicates that the number of parameters of the model is small) = = 1e3, DeiT-S (indicates that the model parameters are small) and DeiT-B (indicates that the model parameters are basic) = = 1e4, and Swin's = = 1e4.
[0085] For detection and segmentation tasks, all models = = 1e5. All experiments were implemented using the PyTorch framework on a single NVIDIA 3090 GPU and Intel Xeon 4214R CPU.
[0086] At the same time, using the same 32 images as the calibration dataset, the comparative methods BRECQ (Brecq: Improving post-training quantization performance by block-by-block quantization. Conference on Learning Representations (ICLR), 2021.), QDrop (Qdrop: Random post-training quantization at very low bits. Conference on Learning Representations (ICLR), 2022.), PD-Quant (Pd-quant: Post-training quantization based on prediction difference Computer Vision and Pattern Recognition (CVPR), pages 24427–24437, 2023.), GPTQ (GPTQ: Post-training accurate compression for production models. Preprint website: 2210.17323, 2022.) and their official codes are implemented. The initial implementation of GPTQ does not involve activation quantization. Therefore, the present invention uses the same quantizer as the present method itself to quantize their activations, including reparameterization techniques and Quantizer. At the same time, the comparison methods include PSAQ-ViT (Data-free Quantization Method for Visual Transfer Models. European Conference on Computer Vision, pages 154–170. Springer, 2022.), Ranking (Advances in Neural Information Processing Systems (NeurIPS), pages 28092–28103, 2021.), EasyQuant (Easyquant: Post-training Quantization by Scale Optimization. CoRR, abs / 2006.16669, 2020.), PTQ4ViT (Ptq4vit: Visual Transfer Model Based on Symmetric Uniform Quantization. European Conference on Computer Vision, pages 191–207. Springer, 2022.), APQ-ViT (Post-training Quantization for Visual Transfer Models. Proceedings of the International Conference on Multimedia (ACMMM), pages 5380–5388, 2022.), and NoisyQuant (Noisyquant: Noise Bias Enhanced Post-training Activation Quantization for Visual Transfer Models). Quantization methods for other ViT methods including RepQ-ViT (Repq-vit: Scale Reparameterization for Post-training Quantization of Visual Transformation Models. International Conference on Computer Vision, pp. 17227–17236, 2023.) (the above author names are transliterated).
[0087] The present invention can be applied to the quantization of a visual self-attention model (ViTs) to achieve compression and acceleration of the visual self-attention model (ViTs).
[0088] Table 1 and Table 2 show the quantitative results of the method of the present invention and the existing methods on different data sets. It can be seen that on these quantization models with different bit widths, the present invention (ERQ) outperforms all the comparison methods in most cases. In Table 1 and Table 2, w / o Opt indicates whether there is an optimization algorithm; W / A indicates the quantization bit width of weights and activation values.
[0089] Table 1: Quantitative results on ImageNet
[0090]
[0091] Table 2: Results of object detection and instance segmentation
[0092]
[0093] Table 1 is the quantitative results of ImageNet, showing the comparison between ERQ and other quantization methods of ViTs models. It can be seen that the proposed ERQ shows advantages in all bit width settings, especially in the low bit case. Specifically, due to the small number of calibration datasets, methods involving optimizers usually suffer from overfitting problems and show unstable performance. For example, for the W3A4 case, QDrop and PD-Quant achieved accuracies of 9.77% and 4.56% on ViT-S, respectively. In contrast, the ERQ proposed in the present invention shows stable improvements on all variants. It is worth noting that ERQ shows performance gains of 22.36% and 9.25% on ViT-S and ViT-B, respectively, gains of 1.84%, 8.68% and 8.58% on DeiT-T, DeiT-S and DeiT-B, and gains of 1.61% and 1.45% on Swin-S and Swin-B, respectively. As for the W4A4 case, ERQ achieved 1.32% and 1.51% performance gains on ViT-S and ViT-B, 1.33%, 1.71%, and 2.13% on DeiT-T, DeiT-S, and DeiT-B, and 0.57% and 1.36% on Swin-S and Swin-B. ERQ also showed the best performance for the W5A5 case. For example, ERQ improved the accuracy by 1.28% on Swin-B compared to the unoptimized method.
[0094] Table 2 shows the results of object detection and instance segmentation. It can be seen that ERQ improves the performance in most cases. For example, for W4A4 Mask R-CNN with Swin-T as the backbone network, ERQ improves the box AP and mask AP by 0.5 and 0.3, respectively. In addition, for W4A4 Cascade Mask R-CNN with Swin-T as the backbone network, ERQ improves the box AP and mask AP by 0.8 and 0.6. The above results further demonstrate the effectiveness and generalization ability of the proposed ERQ.
[0095] The above description is merely an embodiment of the present invention and does not limit the technical scope of the present invention. Therefore, any minor modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of the technical solution of the present invention.
Claims
1. A method for fast quantization of visual self-attention models for image classification, object detection or instance segmentation, characterized in that The following steps are involved: Step 1: Obtain the target to be processed for image classification, object detection or instance segmentation; Step 2: Using a fast quantized visual self-attention model to process the target to be processed for image classification, target detection or instance segmentation, and obtaining an image classification result, a target detection result or an instance segmentation result; The construction of the fast quantized visual self-attention model is as follows: Step 1: Based on the activation value quantization error reduction strategy, the activation value quantization error reduction problem is constructed as a ridge regression model, and the weight adjustment value is obtained by analyzing the ridge regression model; Step 2: Based on the weight quantization error reduction strategy, the quantization parameters of some weights are obtained by resetting the rounding method, and the adjustment values of the remaining weights are determined by ridge regression modeling; Step 3: Repeat step 2 until all weights are quantized; The step 1 is specifically as follows: Using the activation value quantization error reduction strategy, the weight is recorded as , the activation value is , the quantization of the activation value is regarded as a perturbation , so the quantization error Note as: ; To this end, add disturbance to the weight , build a ridge regression model: ; in, is a hyperparameter that controls the strength of regularization, represents the activation value after quantization, Indicates the desired operation; Solving the above ridge regression model Minimum, can get disturbance , that is, the size of the weight adjustment value is: ; in, Represents the activation value after quantization Perform transposition; represents the identity matrix; The step 2 is specifically as follows: Using the weight quantization error reduction strategy, the quantization of the weight W is regarded as adding a related disturbance, that is, the quantization error of the weight , the quantization error generated at this time is: ; The above formula outputs the weight of the i-th channel of the channel Divided into two parts and ,in, The output dimension representing the weights; Representation Quantization The error disturbance caused by represents the weight to be quantized, represents the full-precision weight; accordingly, the corresponding activation value is divided into and , at this time, the quantization error of the i-th channel is: ; in, Representation Quantization The error disturbance caused by First, the quantization error is perturbed using the rounding strategy. Perform the rounding operation and then reverse the rounding operation as follows: ; in, represents the error disturbance of the jth column of the ith channel, Express Round down, Express Round up; S is a set whose elements contain elements with the same gradient direction and perturbation direction; Next, by adjusting the weights of the full precision , so that the following quantization error term is minimized: ; in, represents the hyperparameter that controls the regularization strength, It means that the quantization error term is minimized of disturbance; Solving the above ridge regression model, we get The sizes are as follows: ; express The transpose of Represents the activation value after quantization of the nth calibration data ;use and To obtain the above formula and The value of , where n represents the nth calibration data, N represents the Nth calibration data, and is the calibration set size; Then the adjusted value of the remaining weight is obtained.
