Vision Transform model compression method based on two-step hybrid quantization strategy

Abstract

The invention discloses a Vision Transform model compression method based on a two-step hybrid quantization strategy, which comprises the following steps of: firstly, performing quantization sensitivity analysis on a pre-trained Vision Transform model, and proposing the two-step hybrid quantization strategy: in the first step, performing pre-screening on each linear layer and each convolutional layer in the model through cosine similarity, quickly positioning a network layer sensitive to low-bit quantization, and performing compression on the network layer sensitive to low-bit quantization; and 2, based on a fine analysis method of Hessian second-order information, carrying out quantization error evaluation on the sensitive layer, distributing differentiated quantization bit widths for different network layers, realizing mixed precision quantization, then calculating quantization parameters by adopting a channel-by-channel asymmetric quantization mode, and meanwhile, designing a customized quantization scheme for a special operator in a Vision Transform model. According to the method disclosed by the invention, on the premise of ensuring that the reasoning precision of the model is receivable, the storage overhead and the calculation complexity of the Vision Transform model are remarkably reduced, and the method is suitable for efficient deployment on edge equipment with limited resources and is suitable for quantization after training of a large-scale Transform model using a self-attention mechanism.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and neural network compression technology, specifically relating to a Vision Transformer model compression method based on a two-step hybrid quantization strategy. Background Technology

[0002] With the widespread application of the Vision Transformer (ViT) model in the field of computer vision, it has achieved excellent performance in tasks such as image classification and image segmentation. However, the ViT model has a huge number of parameters (e.g., ViT-Base contains 86M parameters, and ViT-Huge has more than 600M parameters), making it difficult to deploy on edge devices and in resource-constrained environments, which seriously restricts its use in mobile devices, embedded systems and real-time applications.

[0003] Model quantization, as an effective model compression technique, converts model weights and activation values ​​from high-precision floating-point numbers (such as FP32) to low-precision integers (such as INT8 and INT4), significantly reducing model deployment time, model size, accelerating inference speed, and reducing energy consumption. However, existing quantization methods for Vision Transformer models suffer from the following problems when applied:

[0004] 1. Significant differences between channels lead to severe accuracy loss: The self-attention mechanism and LayerNorm operation of Vision Transformer result in significant differences in the numerical distribution between different channels. Existing per-Tensor quantization methods using uniform quantization parameters will cause serious quantization errors, which will lead to a relatively serious loss of accuracy.

[0005] 2. Low efficiency in sensitive layer identification: Most existing methods use a single metric (such as only based on weight gradients or only based on the Hessian matrix) to identify quantization sensitive layers, but single-metric methods have limitations in terms of insufficient identification accuracy. Gradient-based methods are computationally simple but not accurate enough, while Hessian-based methods are accurate but computationally expensive, making them impractical for large-scale model quantization deployments.

[0006] 3. Quantization difficulties of special operators: Vision Transformer contains a large number of special operators (such as Softmax, GELU, and LayerNorm in the model). The output distribution of these operators is complex (such as the bimodal distribution of Softmax output). Existing uniform quantization methods cannot effectively handle these distribution characteristics, so special processing is required for these special operators.

[0007] In summary, there is a need for a Vision Transformer quantization method that can efficiently identify sensitive layers, adapt to differences between channels, handle special operators, and has good hardware compatibility. Summary of the Invention

[0008] To address the aforementioned technical issues, this invention proposes a VisionTransformer model compression method based on a two-step hybrid quantization strategy. This method efficiently identifies sensitive layers through a two-step strategy that combines rapid initial screening using cosine similarity with fine-grained Hessian analysis. It utilizes channel-by-channel quantization to adapt to the inter-channel differences in VisionTransformer and integrates multiple advanced quantization techniques to handle special operators. This approach achieves reduced model memory usage and accelerated model inference overhead while maintaining high accuracy.

[0009] The technical solution adopted in this invention is: a Vision Transformer model compression method based on a two-step hybrid quantization strategy, the specific steps of which are as follows:

[0010] S1. Prepare the pre-trained model and calibration dataset;

[0011] First, a full-precision Vision Transformer model trained on a large-scale dataset is prepared. The Vision Transformer model includes an embedding layer, a multi-layer Transformer encoding module, and an output header module.

[0012] The Vision Transformer model includes: ViT-Base, ViT-Large, DeiT-Base, and SAM-Huge.

[0013] Then, the model structure is confirmed, and the quantitative configuration parameters are initialized. This involves analyzing the type and parameter scale of each layer in the model, identifying all Linear layers, Conv2d layers, LayerNorm layers, and Softmax layers, and clarifying the core structure of the loaded model, including the Patch embedding layer, multiple Transformer encoding modules, and the output header module.

[0014] Finally, calibration dataset sampling is performed, which involves sampling several batches from a subset of the training set of the pre-trained model based on the actual situation; and performing the same preprocessing operations as in the model training phase on the sampled calibration data.

[0015] The preprocessing includes: resize, normalization, and image patch partitioning based on patch embedding.

[0016] S2. Based on step S1, execute a two-step hybrid quantization strategy;

[0017] First, the target network layers in the model are pre-screened based on cosine similarity: the cosine similarity between the pseudo-quantization weights and the full-precision weights of each layer under the baseline low-bit quantization is calculated, and the network layers with similarity below a preset threshold are marked as "layers sensitive to low-bit quantization", thus quickly constructing a candidate set of quantization-sensitive layers; then, for the candidate sensitive layers, the Hessian second-order information analysis method based on Hutchinson random estimation is introduced to finely evaluate the loss sensitivity of each layer under quantization perturbation, and accordingly, differentiated mixed quantization bit widths are assigned to different network layers, i.e., a mixed precision bit width configuration table is generated.

[0018] S3. Based on step S2, calculate the quantization parameters for each channel;

[0019] Based on the quantization bit widths determined in step S2, independent quantization scaling factors and zero-point offsets are calculated for weight parameters and activation channels at the output channel granularity to mitigate the impact of differences in numerical distribution between channels on quantization accuracy. Specifically, for each Linear and Conv2d layer, the quantization parameters scale and zero-point are calculated at the output channel granularity. ,calculate Each is an independent quantization parameter.

[0020] in, Represents the real number field. These represent the number of output channels and the number of input channels, respectively.

[0021] S4. Based on step S3, perform special operator quantization processing;

[0022] For the special operators in the Vision Transformer model, customized quantization strategies are designed according to their numerical distribution characteristics.

[0023] Specifically, a scaling quantization method based on power factor is used for the LayerNorm operator, replacing multiplication with shift operations; and a quantization method based on integer approximation in the logarithmic field is used for the Softmax operator, implementing approximate exponential calculation using an integer lookup table.

[0024] S5. Based on step S4, generate a quantitative model and evaluate it;

[0025] Based on the quantization configuration results from steps S2 to S4, the original Vision Transformer model is quantized and replaced to generate a quantized model, completing the quantization layer replacement and format conversion of the model; and the inference performance of the quantized model is evaluated using a calibration dataset to obtain model accuracy, inference latency and energy consumption indicators, and the indicators are verified and iteratively adjusted until the indicators meet the standards and a qualified quantized model is output.

[0026] S6. Based on step S5, store and deploy the models that have passed the evaluation.

[0027] The weights, quantization parameters, and operator configurations of the qualified quantization model are stored in a unified manner, and a collaborative architecture of FPGA / embedded hardware is configured. Data interaction is achieved through the AXI protocol to complete the deployment preparation.

[0028] Furthermore, step S2 is specifically as follows:

[0029] S21. Load the pre-trained model and calibration dataset prepared in step S1;

[0030] S22. Based on step S21, set the reference quantization bit width;

[0031] First, initialize the unified quantization configuration, and uniformly set the symmetric quantization reference bit width for all layers to be quantized. Initial cosine similarity threshold .

[0032] The layers to be quantized include: the Linear layer and the Conv2d layer.

[0033] Then, based on the variance theory estimated by Hutchinson, the number of Hessian trace sampling times is set. The sensitivity grading threshold is initially set to , .

[0034] Among them, through , These represent the specific bit widths allocated to the high-sensitivity layer, medium-sensitivity layer, and low-sensitivity layer, respectively.

[0035] S23. Based on step S22, perform pseudo-quantization on the Linear / Conv layer;

[0036] First, the base width Next, quantization-dequantization operations are performed on the weights to simulate real quantization noise. For the first... Layer, with weights of Perform symmetric pseudo-quantization based on the reference bit width to determine the quantization scale factor, as shown in the following expression:

[0037] ;

[0038] in, Represents the real number field. These represent the number of output channels and the number of input channels for this layer, respectively. This represents the scaling factor; then, a quantization operation is performed to map the full-precision weights to the k-bit integer field, as shown in the following expression:

[0039] ;

[0040] Where k-bit represents the quantization bit width, Indicates the first Layer quantization weights, This indicates rounding to the nearest integer. The operation limits the overflow value to the range of k-bit signed integers [- , ],.

[0041] Finally, dequantization is performed to restore the data to the floating-point domain, resulting in the pseudo-quantization weights under k-bit quantization. The expression is as follows:

[0042] ;

[0043] in, Indicates the pseudo-quantized first... Layer weights.

[0044] S24. Based on step S23, calculate the hierarchical quantitative similarity;

[0045] By measuring the vector angle between the original weights and the quantized weights using cosine similarity, layers sensitive to perturbations in the quantization direction are identified, and the original weights are calculated layer by layer. and pseudo-quantization weights The cosine similarity is expressed as follows:

[0046] ;

[0047] in, This indicates that the full-precision weights and pseudo-quantization weights are in the first... line, number The product of elements at column positions. and These represent the original weights. and pseudo-quantization weights The L2 norm.

[0048] The quantization process is defined by the following expression:

[0049] ;

[0050] ;

[0051] ;

[0052] in, Indicates quantization scale. Represents zero_point, This represents the weights after quantization, for symmetric quantization. =0, The calculation expression is as follows:

[0053] ;

[0054] in, Represents the weight tensor Perform the absolute value operation on each element; then determine if the cosine similarity is below the threshold. ,like < This indicates that the layer is sensitive to quantization perturbations. Proceed to step S25 to... Adding a layer to the sensitive layer candidate set ,like ≥ This indicates that the layer is a non-sensitive layer, and proceeds to step S27 to allocate a low bit width.

[0055] S25. Based on step S24, all those that satisfy... Conditional layer index Add to sensitive layer candidate set ;

[0056] S26. Based on step S25, perform a fine-grained sensitivity assessment based on Hessian.

[0057] For candidate layer Using cross-entropy loss function Let Hessian be the objective function. Defined as the matrix of second-order partial derivatives of the loss function with respect to the parameters of this layer, its expression is as follows:

[0058] ;

[0059] in, Indicates the first Candidate layer parameters.

[0060] Then, for the candidate set of sensitive layers For each layer, the Hessian trace is calculated using the Hutchinson stochastic estimation method. The trace definition for the Hessian matrix is ​​as follows:

[0061] ;

[0062] in, Indicates the first Layer Hessian matrix The 1 eigenvalue, Represents the cross-entropy loss function For the Layer parameter vector The The second-order partial derivatives of each element.

[0063] The Hutchinson stochastic estimation algorithm is adopted, and efficient computation is achieved based on the Hutchinson estimator. The approximate calculation based on random sampling is expressed as follows:

[0064] ;

[0065] in, Indicates the number of samples. Indicates an index variable, with a range of values. , Indicates the first The random variable obtained from the sampling, and ; This indicates the transpose operation.

[0066] S27. Based on step S26, generate a mixed precision bit width configuration table;

[0067] After calculating the Hessian trace in step S26, bit width is allocated, i.e., a three-level bit width mapping is performed based on a preset threshold. The expression is as follows:

[0068] ;

[0069] in, Indicates the first The quantization bit width allocated by the layer.

[0070] Furthermore, step S3 is specifically as follows:

[0071] S31. Calculate the quantization parameters of the weight channel;

[0072] For the Layer weight tensor Sliced ​​according to the output channel dimension independent subvectors The scaling factor is calculated channel by channel, as shown in the following expression:

[0073] ;

[0074] in, This indicates the layer quantization bit width allocated in step S2. Indicates the first The weight scaling factor for each output channel is uniformly set to zero for zero-point offset.

[0075] S32. Calculate the quantization parameters of the activation channel;

[0076] Asymmetric quantization is used for the activation channels, and forward propagation is performed on the calibration dataset to collect the output activation values ​​of each layer. The channel-by-channel statistics, i.e., activation statistics using Min-Max accumulation, for each channel. The maximum and minimum values ​​are accumulated over several batches, as expressed below:

[0077] ;

[0078] in, Indicates batch size. Indicates the number of channels. Indicates the feature map height. Indicates the width of the feature map. .

[0079] Then, symmetric quantization is used to avoid the hardware overhead introduced by calculating the zero-point offset. The expression for calculating the symmetric quantization parameters is as follows:

[0080] ;

[0081] in, Indicates the first Activation value scaling factor for each output channel Indicates the first Zero offset of the activation value of each output channel.

[0082] S33, Storage of weight and activation quantization parameters;

[0083] The calculated and Bind to the corresponding channel and update the quantization parameter dictionary for batch reading.

[0084] Furthermore, step S4 is specifically as follows:

[0085] S41. The LayerNorm operator is quantized using a scaling method based on power factor, replacing multiplication with shift operations.

[0086] S411, Load the LN layer input activation tensor;

[0087] Loading the first Input activation tensors of each LayerNorm layer .

[0088] S412. Calculate the basic quantization parameters of the input feature channels;

[0089] For input activation tensor Perform basic quantization preprocessing channel by channel: First, calculate the quantization of each channel. mean and standard deviation Then, the normalized features... Calculate the fundamental quantization parameters and quantize them into integer tensors. .

[0090] Among them, the basic quantization parameters, namely the scaling factors, include: scaling factors And midnight .

[0091] S413. Determine the target quantization bit width With hyperparameters ;

[0092] Set the target quantization bit width according to the target quantization accuracy requirements. And set the displacement constraint hyperparameters. hyperparameters Used to limit the range of values ​​for the exponent.

[0093] S414. Calculate the power exponent for each channel;

[0094] For the Learnable scaling parameters for each Layer Norm layer Search for the closest power of 2 approximation for each channel. The calculation expression is as follows:

[0095] ;

[0096] in, This indicates rounding to the nearest integer. Indicates the first LayerNorm The power index corresponding to each channel.

[0097] Combined with displacement constraint hyperparameters For power exponent To apply range constraints, use the following expression:

[0098] ;

[0099] S415. Generate a channel-by-channel displacement factor table. ;

[0100] Based on constrained channel-wise power exponent Construct a channel-by-channel displacement factor table , Indicates the first LayerNorm The displacement factors corresponding to each channel, and all displacement factors constitute a displacement factor table. .

[0101] S416, Displacement factor table generated based on step S416 Execute the LayerNorm quantification strategy;

[0102] Based on the basic quantization parameters and displacement factor table of the input feature channels Combined, from the displacement factor table Extract the current channel displacement factor Perform LayerNorm calculation on the integer field.

[0103] S417. Replace multiplication operations with channel-by-channel shift operations;

[0104] The original LayerNorm calculation process expression is as follows:

[0105] ;

[0106] in, This represents the input feature vector of the LayerNorm layer. Let represent the mean and standard deviation of the eigenvector, respectively. , This represents the learnable scaling and bias parameters. This represents the output characteristics after normalization, scaling, and biasing.

[0107] If we further modify the original floating-point multiplication to replace integer shift operations, the integer version of the LayerNorm calculation process expression is as follows:

[0108] ;

[0109] in, This represents the integer output of LayerNorm. This represents the quantization bias parameter. Indicates arithmetic right shift. Indicates the current channel The displacement factor can be quickly calculated by looking up the displacement factor table α.

[0110] S418. Output LayerNorm quantization results. ;

[0111] This will generate the LayerNorm output for the integer field. Used for calculations in the Transformer layer.

[0112] S42. A quantization method based on logarithmic field integer approximation is adopted for the Softmax operator, and the approximate exponent calculation is realized by using an integer lookup table.

[0113] S421, Quantitative , , Feature matrix;

[0114] The original floating-point query matrix of the Transformer attention module Key matrix and numerical matrices Fixed-point quantization is performed separately to obtain the corresponding INTk representation. .

[0115] S422. Calculate the attention score matrix based on integer operations;

[0116] First, calculate the quantized attention score matrix, as shown in the following expression:

[0117] ;

[0118] in, This represents the quantized attention score matrix. This represents the quantized query matrix. This represents the transpose of the quantized key matrix. The transpose operation is used to match the dimension of matrix multiplication.

[0119] S423. Construct an index approximate lookup table;

[0120] Offline construction of exponential approximation lookup table This is used to store the fixed-point approximation of the exponential function, and its expression is as follows:

[0121] ;

[0122] in, Indicates the lookup table index; Indicates the exponential offset constant; Indicates the scale parameter; Indicates the index bit width; lookup table entries are stored in a fixed-point format, overwriting... arrive The exponential approximation.

[0123] S424. Perform approximate exponent calculation based on lookup table;

[0124] Input an integer value based on Softmax. The approximate result of the corresponding exponent is obtained by looking up a table. The original Softmax definition expression is as follows:

[0125] ;

[0126] in, This represents the first query vector in the attention score matrix. Each correlation score, This represents other relevance scores corresponding to the same query vector.

[0127] Taking the natural logarithm of both the numerator and denominator, we transform the sum of exponents into a subtraction of the logarithmic field, as shown in the following expression:

[0128] ;

[0129] Attention score Quantized to k-bit integers As the input index to the lookup table, the integer exponent approximation is obtained by looking up the table during model inference, as shown in the following expression:

[0130] ;

[0131] S425, Normalization calculation based on displacement operation;

[0132] Perform a shift-based normalization operation on the exponential approximation result, as shown in the following expression:

[0133] ;

[0134] ;

[0135] in, Indicates the number of bits in the normalized shift. Represents the attention weight matrix in the integer field. This represents quantized to a k-bit integer. .

[0136] S426, based on Calculate and output the weighted sum of the matrices;

[0137] The normalized attention weights and numerical matrix are then compared. To perform integer multiplication and addition operations, the expression is as follows:

[0138] ;

[0139] in, This represents the attention calculation result with k-bit integer precision.

[0140] Furthermore, step S5 is specifically as follows:

[0141] S51, Quantization Model Construction;

[0142] Based on the full-precision Vision Transformer model and quantization configuration, the target network layer is replaced with a quantization layer, and the integer calculation logic of LayerNorm shift quantization and Softmax logarithmic domain approximation special operator is integrated to generate the initial quantization model.

[0143] S52, Quantization Model Export and Format Conversion;

[0144] The completed quantization model is converted into a format that can be directly deployed on hardware, ensuring that the model contains complete quantization parameters and operator logic, and is compatible with the computing architecture of the target hardware.

[0145] S53. Based on the quantization model obtained in steps S51-S52, perform calibration set performance evaluation;

[0146] The quantization model is evaluated for inference using a calibration dataset to obtain multi-dimensional performance metrics, including:

[0147] (1) Accuracy metrics: Calculate Top-1 / Top-5 accuracy and cosine similarity to evaluate the accuracy loss of the model after quantization;

[0148] (2) Performance indicators: statistical inference latency, memory usage, and hardware power consumption, to evaluate the deployment efficiency of the model;

[0149] (3) Stability verification: Test the inference stability of the model under different batch sizes and input resolutions.

[0150] S54. Based on the multi-dimensional performance indicators obtained in step S53, perform quantitative model verification and iterative optimization.

[0151] Compare the indicators obtained from step S53 with the preset target thresholds: if the indicators meet the requirements, proceed to step S6; if the indicators do not meet the requirements, return to steps S3 to S4 to adjust the quantitative configuration and then return to step S5 to rebuild the quantitative model until the indicators meet the requirements.

[0152] The beneficial effects of this invention are as follows: First, the method of this invention performs quantization sensitivity analysis on a pre-trained Vision Transformer model and proposes a two-step hybrid quantization strategy. The first step pre-screens each linear and convolutional layer in the model using cosine similarity to quickly locate network layers sensitive to low-bit quantization. The second step, based on a refined analysis method using Hessian second-order information, evaluates the quantization error of the sensitive layers and assigns differentiated quantization bit widths to different network layers to achieve hybrid precision quantization. Then, a channel-by-channel asymmetric quantization method is adopted, calculating independent quantization parameters for weight parameters and activation channels to improve quantization accuracy. Simultaneously, for special operators in the Vision Transformer model, customized quantization schemes are designed for LayerNorm and Softmax operators, including scaling quantization methods based on power factors and logarithmic domain integer approximation methods, to reduce the accuracy loss of special operators under low-bit quantization. The method of this invention significantly reduces the storage overhead and computational complexity of the Vision Transformer model while ensuring acceptable inference accuracy. It is suitable for efficient deployment on resource-constrained edge devices and for post-training quantization (PTQ) of large-scale Transformer models using self-attention mechanisms. Attached Figure Description

[0153] Figure 1 This is a flowchart of a Vision Transformer model compression method based on a two-step hybrid quantization strategy according to the present invention.

[0154] Figure 2 This is a flowchart of a two-step hybrid quantization strategy in an embodiment of the present invention.

[0155] Figure 3 This is a computational architecture diagram for the LayerNorm special operator in an embodiment of the present invention.

[0156] Figure 4 This is a computational architecture diagram for the Softmax special operator in an embodiment of the present invention.

[0157] Figure 5 This is a flowchart illustrating the overall process of generating, storing, deploying, and verifying the quantized model in this embodiment of the invention. Detailed Implementation

[0158] To facilitate understanding of the technical content of the present invention by those skilled in the art, the method of the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0159] like Figure 1The flowchart of a Vision Transformer model compression method based on a two-step hybrid quantization strategy is shown below. The specific steps are as follows:

[0160] S1. Prepare the pre-trained model and calibration dataset;

[0161] First, a full-precision Vision Transformer model trained on a large-scale dataset is prepared. The Vision Transformer model includes an embedding layer, a multi-layer Transformer encoding module, and an output header module.

[0162] The Vision Transformer models include: ViT-Base, ViT-Large, DeiT-Base, and SAM-Huge. This embodiment uses the ViT-Base model for quantization compression on the ImageNet-1K dataset.

[0163] In this embodiment, the TIMM library is used to load the pre-trained ViT-Base model, and the model is set to the evaluation mode to fix the network parameters and avoid backpropagation interference. At the same time, the model is migrated to the GPU device to improve the efficiency of subsequent calibration and quantization calculations, thereby ensuring the stability of the quantization evaluation process.

[0164] Then, the model structure is confirmed, and the quantization configuration parameters are initialized. This involves analyzing the type and parameter scale of each layer in the model, identifying all Linear layers, Conv2d layers, LayerNorm layers, and Softmax layers, and clarifying the core structure of the loaded model, including the Patch embedding layer (which segments the image into patches and maps them to vectors), multiple Transformer encoding modules (including self-attention mechanisms and feedforward networks), and the output header module, to ensure that the quantization process is adapted to the structural characteristics of each layer of the model.

[0165] Finally, a calibration dataset is sampled. This involves sampling several batches from a subset of the pre-trained model's training set, depending on the actual situation. (The number of batches and the number of images per batch can be adaptively adjusted based on model size, hardware computing resources, and calibration accuracy requirements; a typical configuration is 32 batches, each containing 128 images). The calibration dataset is used to statistically analyze the impact of quantization errors on model behavior without backpropagation. In this embodiment, 32 batches are randomly sampled from the ImageNet-1K training set, each containing 128 images, to achieve a balance between calibration speed and accuracy. The calibration data needs to cover diverse image feature distributions to ensure the statistical representativeness of the quantization parameters. The sampled calibration data undergoes the same preprocessing operations as during model training to ensure that the distribution characteristics of the calibration data are consistent with the actual inference data, allowing the statistical errors during quantization to accurately reflect the model's inference performance.

[0166] The preprocessing includes: resize, normalization, and image patch partitioning based on patch embedding.

[0167] S2. Based on step S1, execute a two-step hybrid quantization strategy;

[0168] First, the target network layers (including all Linear layers and Conv2d layers) in the model are pre-screened based on cosine similarity: the cosine similarity between the pseudo-quantization weights and the full-precision weights of each layer under the baseline low-bit quantization is calculated, and network layers with similarity below a preset threshold are marked as "layers sensitive to low-bit quantization", thus quickly constructing a candidate set of quantization-sensitive layers; then, for the candidate sensitive layers, the Hessian second-order information analysis method based on Hutchinson random estimation is introduced to finely evaluate the loss sensitivity of each layer under quantization perturbation, and accordingly, differentiated mixed quantization bit widths are assigned to different network layers, i.e., a mixed precision bit width configuration table is generated.

[0169] S3. Based on step S2, calculate the quantization parameters for each channel;

[0170] Based on the quantization bit widths determined in step S2, independent quantization scaling factors and zero-point offsets are calculated for weight parameters and activation channels at the output channel granularity to mitigate the impact of differences in numerical distribution between channels on quantization accuracy. Specifically, for each Linear and Conv2d layer, the quantization parameters scale and zero-point are calculated at the output channel granularity. ,calculate Each is an independent quantization parameter.

[0171] in, Represents the real number field. These represent the number of output channels and the number of input channels, respectively.

[0172] This step aims to address the significant differences in numerical distribution among the output channels of the Vision Transformer. Existing per-tensor quantization uses a uniform scaling factor, resulting in excessive quantization errors in high-variance channels and wasted quantization resolution in low-variance channels. This invention implements fine-grained channel-level quantization, treating each model output channel as an independent unit, calculating the quantization scaling factor and zero-point offset for weights and activation values ​​separately, adapting to the differences in data distribution across different channels, and reducing quantization errors.

[0173] S4. Based on step S3, perform special operator quantization processing;

[0174] The LayerNorm and Softmax operators in Vision Transformer exhibit non-uniform and nonlinear numerical distribution characteristics, and traditional uniform quantization leads to a catastrophic decrease in accuracy. This invention designs customized solutions for the specific operators of the model.

[0175] For the special operators in the Vision Transformer model, customized quantization strategies are designed according to their numerical distribution characteristics.

[0176] Specifically, a scaling quantization method based on power factor is adopted for the LayerNorm operator, replacing multiplication with shift operation; and a quantization method based on integer approximation in the logarithmic field is adopted for the Softmax operator, implementing approximate exponential calculation by using an integer lookup table, thereby reducing computational complexity and precision loss.

[0177] S5. Based on step S4, generate a quantitative model and evaluate it;

[0178] Based on the quantization configuration results from steps S2 to S4, the original Vision Transformer model is quantized and replaced to generate a quantized model, completing the quantization layer replacement and format conversion of the model; and the inference performance of the quantized model is evaluated using a calibration dataset to obtain model accuracy, inference latency and energy consumption indicators, and the indicators are verified and iteratively adjusted until the indicators meet the standards and a qualified quantized model is output.

[0179] S6. Based on step S5, store and deploy the models that have passed the evaluation.

[0180] The weights, quantization parameters, and operator configurations of the qualified quantization model are stored in a unified manner, and a collaborative architecture of FPGA / embedded hardware is configured. Data interaction is achieved through the AXI protocol to complete the deployment preparation.

[0181] like Figure 2As shown, in this embodiment, step S2 is specifically as follows:

[0182] S21. Load the pre-trained model and calibration dataset prepared in step S1;

[0183] This embodiment uses the TIMM library to load the ViT-Base model pre-trained on ImageNet-1K. The model structure includes 12 Transformer encoding layers, embedding layers, and a classification head. The model is placed in evaluation mode (model.eval()) and migrated to a GPU device. 32 batches (128 images of 224×224×3 per batch) are randomly sampled from the training set as a calibration dataset to ensure that the data distribution covers diverse visual features.

[0184] S22. Based on step S21, set the reference quantization bit width;

[0185] First, a unified quantization configuration is initialized. In this embodiment, a symmetrical quantization reference bit width is uniformly set for all layers to be quantized. Initial cosine similarity threshold This value can effectively filter out layers that respond strongly to quantization errors, while excluding errors caused by numerical jitter.

[0186] The layers to be quantized include: the Linear layer and the Conv2d layer.

[0187] Then, based on the variance theory estimated by Hutchinson, the number of Hessian trace sampling times is set. . When the value is ≥10, the estimated variance can be controlled within 5%. When the sampling count is 15, the variance further decreases to 3.2%. Increasing the sampling count further provides limited improvement in accuracy but results in a linear increase in computational overhead. Therefore, this embodiment uses a limited number of Hessian trace sampling times. =15. The sensitivity grading threshold is initially set to... =0.001, =0.0002.

[0188] Among them, through , The specific bit widths allocated to the high-sensitivity layer, medium-sensitivity layer, and low-sensitivity layer are determined through a small-scale grid search, ensuring that the high-sensitivity layer (usually corresponding to the Query / Key projection layer in the attention mechanism) is allocated 8 bits, the medium-sensitivity layer (such as the Value projection layer) is allocated 6 bits, and the low-sensitivity layer (such as the linear layer in MLP) is allocated 4 bits.

[0189] S23. Based on step S22, perform pseudo-quantization on the Linear / Conv layer;

[0190] First, the base width The next step involves performing quantization-dequantization on the weights to simulate real quantization noise, providing comparable quantized weights for cosine similarity calculation. For the first... Layer, with weights of Perform symmetric pseudo-quantization based on the reference bit width to determine the quantization scale factor, as shown in the following expression:

[0191] ;

[0192] in, Represents the real number field. These represent the number of output channels and the number of input channels for this layer, respectively. Since this calculation is performed using element-by-element comparison, the computational complexity is O(n log n). , This represents the scaling factor; then a quantization operation is performed to map the full-precision weights to a 4-bit integer field, as shown in the following expression:

[0193] ;

[0194] in, Indicates the first Layer quantization weights, This indicates rounding to the nearest integer. The operation limits the overflow value to the range of 4-bit signed integers [-8, 7].

[0195] Finally, dequantization is performed to restore the data to the floating-point domain, resulting in the pseudo-quantization weights under 4-bit quantization. The expression is as follows:

[0196] ;

[0197] in, Indicates the pseudo-quantized first... The layer weights, this pseudo-quantization result approximates the weights of the real 4-bit quantization in terms of numerical distribution, but retains the floating-point format for subsequent similarity calculations. In this embodiment, step S23 is executed entirely in parallel on the GPU, accelerated using CUDA kernel functions, and takes approximately 0.3 seconds to process all 86M parameters of ViT-Base.

[0198] S24. Based on step S23, calculate the hierarchical quantitative similarity;

[0199] By measuring the vector angle between the original weights and the quantized weights using cosine similarity, layers sensitive to perturbations in the quantization direction are identified, and the original weights are calculated layer by layer. and pseudo-quantization weights The cosine similarity is expressed as follows:

[0200] ;

[0201] in, This indicates that the full-precision weights and pseudo-quantization weights are in the first... line, number The element-wise product of the column positions, and the sum of the products of all positions, constitutes the dot product of the two tensors, representing the "degree of symmetry" between the two weight tensors in the feature space. and These represent the original weights. and pseudo-quantization weights The L2 norm reflects and The overall amplitude.

[0202] The quantization process is defined by the following expression:

[0203] ;

[0204] ;

[0205] ;

[0206] in, Indicates quantization scale. Represents zero_point, This represents the weights after quantization, for symmetric quantization. =0, The calculation expression is as follows:

[0207] ;

[0208] in, Represents the weight tensor Perform the absolute value operation on each element; then determine if the cosine similarity is below the threshold. ,like < This indicates that the layer is sensitive to quantization perturbations. Proceed to step S25 to... Adding a layer to the sensitive layer candidate set ,like ≥ This indicates that the layer is a non-sensitive layer, and proceeds to step S27 to allocate a low bit width.

[0209] S25. Based on step S24, all those that satisfy... Conditional layer index Add to sensitive layer candidate set ;

[0210] In this embodiment The system consists of 35 layers, primarily comprising Query / Key projection layers for their respective attention blocks and the first two MLP layers. Theoretical analysis indicates these layers play a crucial role in feature representation capabilities, therefore their representational power must be preserved. The resulting candidate set of sensitive layers... Then you can proceed to step S26 for calculation.

[0211] S26. Based on step S25, perform a fine-grained sensitivity assessment based on Hessian.

[0212] For candidate layer Using cross-entropy loss function Let Hessian be the objective function. Defined as the matrix of second-order partial derivatives of the loss function with respect to the parameters of this layer, its expression is as follows:

[0213] ;

[0214] in, Indicates the first Candidate layer parameters. The diagonal elements of this matrix. The trace of the Hessian can characterize the curvature of the loss with respect to a single parameter perturbation. As the sum of all eigenvalues, it can reflect the overall sensitivity of the layer's overall parameter perturbation to the loss.

[0215] Then, for the candidate set of sensitive layers For each layer, the Hessian trace is calculated using the Hutchinson stochastic estimation method. The trace definition for the Hessian matrix is ​​as follows:

[0216] ;

[0217] in, Indicates the first Layer Hessian matrix The 1 eigenvalue, Represents the cross-entropy loss function For the Layer parameter vector The The second-order partial derivatives of each element.

[0218] Theoretical analysis shows that, The larger the bit width, the more significant the increase in loss caused by small perturbations introduced by quantization; therefore, a higher bit width is needed to suppress quantization errors. However, directly calculating the trace complexity of the Hessian matrix is ​​extremely high, requiring explicit construction. For a 768-dimensional projection layer of ViT-Base, the computational cost in a single operation can reach [amount missing]. This operation is impractical for model quantization deployment. Therefore, this invention employs the Hutchinson stochastic estimation algorithm (achieving unbiased estimation through Monte Carlo sampling) and performs efficient computation based on the Hutchinson estimator (reducing estimation variance through multiple sampling averaging). The approximate calculation based on random sampling is expressed as follows:

[0219] ;

[0220] in, Indicates the number of samples. Indicates an index variable, with a range of values. , Indicates the first The random variable obtained from the sampling, and ; This indicates a transpose operation. In this embodiment, for each candidate layer... ,generate =15 random vectors The elements are sampled independently and identically distributed from either a Rademacher distribution (with equal probability of ±1) or a standard normal distribution. The Rademacher distribution is preferred in this embodiment due to its simple calculation and small variance. Theoretically, this estimate is unbiased, and its variance is proportional to the number of samples. Inversely proportional. This embodiment is set... =15 reduces the estimated standard deviation to within 5% of the true value. To accelerate computation, a batch processing mode is used to estimate traces of multiple layers simultaneously, and GPU parallelization of vector operations is utilized.

[0221] S27. Based on step S26, generate a mixed precision bit width configuration table;

[0222] After calculating the Hessian trace in step S26, bit width is allocated, i.e., a three-level bit width mapping is performed based on a preset threshold. The expression is as follows:

[0223] ;

[0224] in, Indicates the first The quantization bit width allocated by the layer.

[0225] In this embodiment, step S3 is specifically as follows:

[0226] S31. Calculate the quantization parameters of the weight channel;

[0227] For the Layer weight tensor Sliced ​​according to the output channel dimension independent subvectors The scaling factor is calculated channel by channel, as shown in the following expression:

[0228] ;

[0229] in, This indicates the layer quantization bit width allocated in step S2. Indicates the first The weight scaling factor for each output channel is uniformly set to zero for zero-point offset.

[0230] S32. Calculate the quantization parameters of the activation channel;

[0231] Asymmetric quantization is used for the activation channels (to accommodate the non-negative or offset distribution characteristics of activation values), and forward propagation is performed on the calibration dataset to collect the output activation values ​​of each layer. The channel-by-channel statistics, i.e., activation statistics using Min-Max accumulation, for each channel. In this embodiment, the maximum and minimum values ​​are accumulated over 32 batches, as expressed below:

[0232] ;

[0233] in, Indicates batch size. Indicates the number of channels. Indicates the feature map height. Indicates the width of the feature map. .

[0234] Then, symmetric quantization is used to avoid the hardware overhead introduced by calculating the zero-point offset. The expression for calculating the symmetric quantization parameters is as follows:

[0235] ;

[0236] in, Indicates the first Activation value scaling factor for each output channel Indicates the first Zero offset of the activation value of each output channel.

[0237] S33, Storage of weight and activation quantization parameters;

[0238] The calculated and Bind to the corresponding channel and update the quantization parameter dictionary for batch reading.

[0239] In this embodiment, step S4 is specifically as follows:

[0240] S41, such as Figure 3 As shown, a scaling quantization method based on power factor is used for the LayerNorm operator, replacing multiplication operations with shift operations;

[0241] LayerNorm contains learnable scaling parameters Its dynamic range is relatively large, reaching

[10] -3 10 2 In this embodiment, a scaling quantization method based on power factor is used, replacing multiplication with shift operation.

[0242] S411, Load the LN layer input activation tensor;

[0243] Loading the first Input activation tensors of each LayerNorm layer .

[0244] S412. Calculate the basic quantization parameters of the input feature channels;

[0245] For input activation tensor Perform basic quantization preprocessing channel by channel: First, calculate the quantization of each channel. mean and standard deviation Then, the normalized features... Calculate the fundamental quantization parameters and quantize them into integer tensors. This is used for subsequent fixed-point calculations.

[0246] Among them, the basic quantization parameters, namely the scaling factors, include: scaling factors And midnight .

[0247] S413. Determine the target quantization bit width With hyperparameters ;

[0248] Set the target quantization bit width according to the target quantization accuracy requirements. And set the displacement constraint hyperparameters. In this embodiment, the following is adopted: = 8 is used to limit the range of values ​​for the power exponent to prevent excessive quantization error. This operation constrains the original floating-point scaling factor to... Discrete sets that satisfy the distribution requirements of LayerNorm in the model.

[0249] S414. Calculate the power exponent for each channel;

[0250] For the Learnable scaling parameters for each Layer Norm layer ( The dynamic range is [10 -3 10 2 ]), search for the closest power of 2 approximation for each channel, for each channel The calculation expression is as follows:

[0251] ;

[0252] in, This indicates rounding to the nearest integer. Indicates the first LayerNorm The power exponent corresponding to each channel can be calculated in this embodiment. The theoretical range of values ​​is [-10, 7].

[0253] Combined with displacement constraint hyperparameters For power exponent To ensure that the power approximation factor is within a reasonable dynamic range, a range constraint is imposed, as shown in the following expression:

[0254] ;

[0255] S415. Generate a channel-by-channel displacement factor table. ;

[0256] Based on constrained channel-wise power exponent Construct a channel-by-channel displacement factor table Therefore, the constrained displacement factor can be limited to [1 / 256, 256], which is much larger than the parameter. Dynamic range

[10] -3 10 2 This meets the requirements.

[0257] in, Indicates the first LayerNorm The displacement factors corresponding to each channel, and all displacement factors constitute a displacement factor table. For parameter lookup in subsequent displacement calculations.

[0258] S416, Displacement factor table generated based on step S416 Execute the LayerNorm quantification strategy;

[0259] Based on the basic quantization parameters and displacement factor table of the input feature channels Combined, from the displacement factor table Extract the current channel displacement factor Perform LayerNorm calculation on the integer field.

[0260] S417. Replace multiplication operations with channel-by-channel shift operations;

[0261] The original LayerNorm calculation process expression is as follows:

[0262] ;

[0263] in, This represents the input feature vector of the LayerNorm layer. Let represent the mean and standard deviation of the eigenvector, respectively. , This represents the learnable scaling and bias parameters. This represents the output characteristics after normalization, scaling, and biasing.

[0264] If we further modify the original floating-point multiplication to replace integer shift operations, the integer version of the LayerNorm calculation process expression is as follows:

[0265] ;

[0266] in, This represents the integer output of LayerNorm. This represents the quantization bias parameter. Indicates arithmetic right shift. Indicates the current channel The displacement factor can be quickly calculated by looking up the displacement factor table α.

[0267] S418. Output LayerNorm quantization results. ;

[0268] This will generate the LayerNorm output for the integer field. Used for calculations in subsequent Transformer layers.

[0269] S42, such as Figure 4 As shown, a quantization method based on integer approximation in the logarithmic field is used for the Softmax operator, and the approximate exponent calculation is achieved by using an integer lookup table.

[0270] Softmax involves exponential operations and normalization, and its output exhibits a bimodal distribution. Studies have shown that traditional quantization of Softmax in the VisionTransformer model leads to data invalidation. This embodiment employs a scheme to transform it to the logarithmic domain, using lookup tables and shifting to achieve integer approximation.

[0271] S421, Quantitative , , Feature matrix;

[0272] The original floating-point query matrix of the Transformer attention module Key matrix and numerical matrices Perform fixed-point quantization processing separately to obtain the corresponding INT8 representation. .

[0273] S422. Calculate the attention score matrix based on integer operations;

[0274] First, calculate the quantized attention score matrix, as shown in the following expression:

[0275] ;

[0276] in, This represents the quantized attention score matrix. This represents the quantized query matrix. This represents the transpose of the quantized key matrix. The transpose operation is used to match the dimension of matrix multiplication.

[0277] S423. Construct an index approximate lookup table;

[0278] Offline construction of exponential approximation lookup table This is used to store the fixed-point approximation of the exponential function, and its expression is as follows:

[0279] ;

[0280] in, Indicates the lookup table index; = 128 represents the exponential offset constant; The scale parameter is taken in this embodiment. ; =8 indicates the index bit width, corresponding to a total lookup table length of 256 entries; the lookup table entries are stored in a fixed-point format, covering... arrive The exponential approximation. The lookup table entries described in this embodiment are stored in a 16-bit fixed-point format, through... arrive computational coverage arrive An exponential approximation of the range.

[0281] S424. Perform approximate exponent calculation based on lookup table;

[0282] Input an integer value based on Softmax. The approximate result of the corresponding exponent is obtained by looking up a table. The original Softmax definition expression is as follows:

[0283] ;

[0284] in, This represents the first query vector in the attention score matrix. Each correlation score, This represents other relevance scores corresponding to the same query vector, used for normalization calculations.

[0285] Taking the natural logarithm of both the numerator and denominator transforms the exponential sum into a subtraction over the logarithmic field, reducing the dynamic range. The expression is as follows:

[0286] ;

[0287] This embodiment uses attention scores Quantized to 8-bit integer As the input index to the lookup table, the integer exponent approximation is obtained by looking up the table during model inference, as shown in the following expression:

[0288] ;

[0289] S425, Normalization calculation based on displacement operation;

[0290] Perform a shift-based normalization operation on the exponential approximation result, as shown in the following expression:

[0291] ;

[0292] ;

[0293] in, Indicates the number of bits in the normalized shift. Represents the attention weight matrix in the integer field. This represents quantized to an 8-bit integer. .

[0294] S426, based on Calculate and output the weighted sum of the matrix (this operation can be completed in a single cycle, avoiding floating-point exponents);

[0295] The normalized attention weights and numerical matrix are then compared. To perform integer multiplication and addition operations, the expression is as follows:

[0296] ;

[0297] in, This represents the attention calculation result with 8-bit integer precision, which serves as the input for subsequent Transformer layers.

[0298] like Figure 5 As shown, in this embodiment, step S5 is specifically as follows:

[0299] S51, Quantization Model Construction;

[0300] Based on the full-precision Vision Transformer model and quantization configuration (including mixed quantization bit width, channel-wise quantization parameters, and special operator logic), the target network layer (Linear / Conv2d / LayerNorm / Softmax) is replaced with a quantization layer, and the integer calculation logic of special operators such as LayerNorm shift quantization and Softmax logarithmic field approximation is integrated to generate the initial quantization model.

[0301] S52, Quantization Model Export and Format Conversion;

[0302] The completed quantization model is converted into a format that can be directly deployed on hardware (including ONNX and TensorRTEngine) to ensure that the model contains complete quantization parameters and operator logic and is compatible with the computing architecture of the target hardware.

[0303] S53. Based on the quantization model obtained in steps S51-S52, perform calibration set performance evaluation;

[0304] The quantization model is evaluated for inference using a calibration dataset to obtain multi-dimensional performance metrics, including:

[0305] (1) Accuracy metrics: Calculate Top-1 / Top-5 accuracy and cosine similarity to evaluate the accuracy loss of the model after quantization;

[0306] (2) Performance indicators: statistical inference latency, memory usage, and hardware power consumption, to evaluate the deployment efficiency of the model;

[0307] (3) Stability verification: Test the inference stability of the model under different batch sizes and input resolutions.

[0308] S54. Based on the multi-dimensional performance indicators obtained in step S53, perform quantitative model verification and iterative optimization.

[0309] Compare the indicators obtained from step S53 with the preset target thresholds: if the indicators meet the requirements, proceed to step S6; if the indicators do not meet the requirements, return to steps S3 to S4 to adjust the quantization configuration (adjust the channel-by-channel quantization parameters and improve the quantization logic of special operators) and then return to step S5 to rebuild the quantization model until the indicators meet the requirements.

[0310] In this embodiment, step S6 is specifically as follows:

[0311] In this embodiment, the ONNX quantization model is used for export. The quantization model is exported to ONNX format using torch.onnx.export, and the input dimensions (1,3,224,224) are specified, corresponding to a single 224×224 image. During export, the quantization parameters (scale / zero_point) and bit width information are embedded to ensure that the ONNX model contains complete quantization logic.

[0312] In summary, the method of this invention addresses the pain points of Vision Transformer model deployment at the edge, such as large parameter scale, high storage and computational overhead, and easy loss of accuracy. Through a two-step sensitive layer identification strategy of "cosine similarity pre-screening + Hutchinson random estimation", channel-by-channel quantization adaptation, customized quantization with special operators, and hardware and software co-optimization design, it achieves a compression ratio of 4-6 times and controls the Top-1 accuracy loss to within 1%, while significantly reducing the inference latency of edge devices (60-70% lower than the full-precision model) and energy consumption (50-60% lower), and greatly reducing the occupation of storage resources.

[0313] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of the claims of the invention.

Claims

1. A compression method for Vision Transformer models based on a two-step hybrid quantization strategy, the specific steps of which are as follows: S1. Prepare the pre-trained model and calibration dataset; First, prepare a full-precision Vision Transformer model trained on a large-scale dataset. The Vision Transformer model includes: Embedding layer, multi-layer Transformer encoding module, and output header module; The Vision Transformer model includes: ViT-Base, ViT-Large, DeiT-Base, and SAM-Huge; Then, the model structure is confirmed, and the quantitative configuration parameters are initialized. This involves analyzing the type and parameter scale of each layer in the model, identifying all Linear layers, Conv2d layers, LayerNorm layers, and Softmax layers, and clarifying the core structure of the loaded model, including the Patch embedding layer, multiple Transformer encoding modules, and the output header module. Finally, the calibration dataset is sampled, which means sampling several batches from a subset of the training set of the pre-trained model according to the actual situation; and performing the same preprocessing operations as in the model training stage on the sampled calibration data. The preprocessing includes: resizing, normalization, and image patch partitioning based on patch embedding; S2. Based on step S1, execute a two-step hybrid quantization strategy; First, the target network layers in the model are pre-screened based on cosine similarity: the cosine similarity between the pseudo-quantization weights and the full-precision weights of each layer under the baseline low-bit quantization is calculated, and network layers with similarity below a preset threshold are marked as "layers sensitive to low-bit quantization", thus quickly constructing a candidate set of quantization-sensitive layers; then, for the candidate sensitive layers, the Hessian second-order information analysis method based on Hutchinson random estimation is introduced to finely evaluate the loss sensitivity of each layer under quantization perturbation, and accordingly, differentiated mixed quantization bit widths are assigned to different network layers, i.e., a mixed precision bit width configuration table is generated; S3. Based on step S2, calculate the quantization parameters for each channel; Based on the quantization bit widths determined in step S2, independent quantization scaling factors and zero-point offsets are calculated for weight parameters and activation channels at the output channel granularity to mitigate the impact of differences in numerical distribution between channels on quantization accuracy; that is, for each Linear and Conv2d layer, the quantization parameters scale and zero-point are calculated at the output channel granularity, and for the weight... ,calculate Each independent quantization parameter; in, Represents the real number field. These represent the number of output channels and the number of input channels, respectively. S4. Based on step S3, perform special operator quantization processing; For the special operators in the Vision Transformer model, customized quantization strategies are designed according to their numerical distribution characteristics. Specifically, a scaling quantization method based on power factor is used for the LayerNorm operator, replacing multiplication with shift operations; and a quantization method based on integer approximation in the logarithmic field is used for the Softmax operator, implementing approximate exponential calculation using an integer lookup table. S5. Based on step S4, generate a quantitative model and evaluate it; Based on the quantization configuration results from steps S2 to S4, the original Vision Transformer model is quantized and replaced to generate a quantized model, completing the quantization layer replacement and format conversion of the model; and the inference performance of the quantized model is evaluated using a calibration dataset to obtain model accuracy, inference latency and energy consumption indicators, and the indicators are verified and iteratively adjusted until the indicators meet the standards and a qualified quantized model is output. S6. Based on step S5, store and deploy the models that have passed the evaluation. The weights, quantization parameters, and operator configurations of the qualified quantization model are stored in a unified manner, and a collaborative architecture of FPGA / embedded hardware is configured. Data interaction is achieved through the AXI protocol to complete the deployment preparation.

2. The Vision Transformer model compression method based on a two-step hybrid quantization strategy according to claim 1, characterized in that, Step S2 is as follows: S21. Load the pre-trained model and calibration dataset prepared in step S1; S22. Based on step S21, set the reference quantization bit width; First, initialize the unified quantization configuration, and uniformly set the symmetric quantization reference bit width for all layers to be quantized. Initial cosine similarity threshold ; The layers to be quantized include: Linear layer and Conv2d layer; Then, based on the variance theory estimated by Hutchinson, the number of Hessian trace sampling times is set. The sensitivity grading threshold is initially set to , ; Among them, through , These represent the specific bit widths allocated to the high-sensitivity layer, medium-sensitivity layer, and low-sensitivity layer, respectively. S23. Based on step S22, perform pseudo-quantization on the Linear / Conv layer; First, the base width Next, quantization-dequantization operations are performed on the weights to simulate real quantization noise; for the first... Layer, with weights of Perform symmetric pseudo-quantization based on the reference bit width to determine the quantization scale factor, as shown in the following expression: ； in, Represents the real number field. These represent the number of output channels and the number of input channels for this layer, respectively. This represents the scaling factor; then, a quantization operation is performed to map the full-precision weights to the k-bit integer field, as shown in the following expression: ； Where k-bit represents the quantization bit width, Indicates the first Layer quantization weights, This indicates rounding to the nearest integer. The operation limits the overflow value to the range of k-bit signed integers [- , ],; Finally, dequantization is performed to restore the data to the floating-point domain, resulting in the pseudo-quantization weights under k-bit quantization. The expression is as follows: ； in, Indicates the pseudo-quantized first... Layer weights; S24. Based on step S23, calculate the hierarchical quantitative similarity; By measuring the vector angle between the original weights and the quantized weights using cosine similarity, layers sensitive to perturbations in the quantization direction are identified, and the original weights are calculated layer by layer. and pseudo-quantization weights The cosine similarity is expressed as follows: ； in, This indicates that the full-precision weights and pseudo-quantization weights are in the first... line, number The product of elements at column positions. and These represent the original weights. and pseudo-quantization weights The L2 norm; The quantization process is defined by the following expression: ； ； ； in, Indicates quantization scale. Represents zero_point, This represents the weights after quantization, for symmetric quantization. =0, The calculation expression is as follows: ； in, Represents the weight tensor Perform the absolute value operation on each element; then determine if the cosine similarity is below the threshold. ,like < This indicates that the layer is sensitive to quantization perturbations. Proceed to step S25 to... Adding a layer to the sensitive layer candidate set ,like ≥ This indicates that the layer is a non-sensitive layer, and proceeds to step S27 to allocate a low bit width; S25. Based on step S24, all those that satisfy... Conditional layer index Add to the candidate set of sensitive layers ; S26. Based on step S25, perform a fine-grained sensitivity assessment based on Hessian. For candidate layer Using cross-entropy loss function Let Hessian be the objective function. Defined as the matrix of second-order partial derivatives of the loss function with respect to the parameters of this layer, its expression is as follows: ； in, Indicates the first Candidate layer parameters; Then, for the candidate set of sensitive layers For each layer in the matrix, the Hessian trace is calculated using the Hutchinson stochastic estimation method; the trace definition expression for the Hessian matrix is ​​as follows: ； in, Indicates the first Layer Hessian matrix The 1 eigenvalue, Represents the cross-entropy loss function For the first Layer parameter vector The The second-order partial derivatives of each element; The Hutchinson stochastic estimation algorithm is adopted, and efficient computation is achieved based on the Hutchinson estimator. The approximate calculation based on random sampling is expressed as follows: ； in, Indicates the number of samples. Indicates an index variable, with a range of values. , Indicates the first The random variable obtained from the sampling, and ; Indicates the transpose operation; S27. Based on step S26, generate a mixed precision bit width configuration table; After calculating the Hessian trace in step S26, the bit width is allocated, i.e., a three-level bit width mapping is performed based on a preset threshold; the expression is as follows: ； in, Indicates the first The quantization bit width allocated by the layer.

3. The Vision Transformer model compression method based on a two-step hybrid quantization strategy according to claim 1, characterized in that, Step S3 is as follows: S31. Calculate the weighted channel quantization parameters; For the Layer weight tensor Sliced ​​by output channel dimension independent subvectors The scaling factor is calculated channel by channel, as shown in the following expression: ； in, This indicates the layer quantization bit width allocated in step S2. Indicates the first The weight scaling factor of each output channel is uniformly set to zero for zero point offset; S32. Calculate the quantization parameters of the activation channel; Asymmetric quantization is used for the activation channels, and forward propagation is performed on the calibration dataset to collect the output activation values ​​of each layer. The channel-by-channel statistics, i.e., activation statistics using Min-Max accumulation, for each channel. The maximum and minimum values ​​are accumulated over several batches, as expressed below: ； in, Indicates batch size. Indicates the number of channels. Indicates the feature map height. Indicates the width of the feature map. ; Then, symmetric quantization is used to avoid the hardware overhead introduced by calculating the zero-point offset. The expression for calculating the symmetric quantization parameters is as follows: ； in, Indicates the first Activation value scaling factor for each output channel Indicates the first Zero offset of the activation value of each output channel; S33, Storage of weight and activation quantization parameters; The calculated and Bind to the corresponding channel and update the quantization parameter dictionary for batch reading.

4. The Vision Transformer model compression method based on a two-step hybrid quantization strategy according to claim 1, characterized in that, Step S4 is as follows: S41. The LayerNorm operator is quantized using a scaling method based on power factor, replacing multiplication with shift operations. S411, Load the LN layer input activation tensor; Loading the first Input activation tensors of each LayerNorm layer ; S412. Calculate the basic quantization parameters of the input feature channels; For input activation tensor Perform basic quantization preprocessing channel by channel: First, calculate the quantization of each channel. mean and standard deviation Then, the normalized features... Calculate the fundamental quantization parameters and quantize them into integer tensors. ; Among them, the basic quantization parameters, namely the scaling factors, include: scaling factors And midnight ; S413. Determine the target quantization bit width With hyperparameters ; Set the target quantization bit width according to the target quantization accuracy requirements. And set the displacement constraint hyperparameters. hyperparameters Used to limit the range of values ​​for the exponent; S414. Calculate the power exponent for each channel; For the Learnable scaling parameters for each Layer Norm layer Search for the closest power of 2 approximation for each channel. The calculation expression is as follows: ； in, This indicates rounding to the nearest integer. Indicates the first LayerNorm The power index corresponding to each channel; Combined with displacement constraint hyperparameters For power exponent To apply range constraints, use the following expression: ； S415. Generate a channel-by-channel displacement factor table. ; Based on constrained channel-wise power exponent Construct a channel-by-channel displacement factor table , Indicates the first LayerNorm The displacement factors corresponding to each channel, and all displacement factors constitute a displacement factor table. ; S416, Displacement factor table generated based on step S416 Execute the LayerNorm quantization strategy; Based on the basic quantization parameters and displacement factor table of the input feature channels Combined, from the displacement factor table Extract the current channel displacement factor Perform LayerNorm calculation on the integer field; S417. Replace multiplication operations with channel-by-channel shift operations; The original LayerNorm calculation process expression is as follows: ； in, This represents the input feature vector of the LayerNorm layer. Let represent the mean and standard deviation of the eigenvector, respectively. , This represents the learnable scaling and bias parameters. This represents the output characteristics after normalization, scaling, and biasing. If we further modify the original floating-point multiplication to replace integer shift operations, the integer version of the LayerNorm calculation process expression is as follows: ； in, This represents the integer output of LayerNorm. This represents the quantization bias parameter. Indicates arithmetic right shift. Indicates the current channel The displacement factor can be quickly calculated by looking up the displacement factor table α. S418. Output LayerNorm quantization results. ; This will generate the LayerNorm output for the integer field. Used for calculations in the Transformer layer; S42. A quantization method based on logarithmic field integer approximation is adopted for the Softmax operator, and the approximate exponent calculation is realized by using an integer lookup table. S421, Quantitative , , Feature matrix; The original floating-point query matrix of the Transformer attention module Key matrix and numerical matrices Fixed-point quantization is performed separately to obtain the corresponding INTk representation. ; S422. Calculate the attention score matrix based on integer operations; First, calculate the quantized attention score matrix, as shown in the following expression: ； in, This represents the quantized attention score matrix. This represents the quantized query matrix. This represents the transpose of the quantized key matrix. The transpose operation is used to match the dimension of matrix multiplication. S423. Construct an index approximate lookup table; Offline construction of exponential approximation lookup table This is used to store the fixed-point approximation of the exponential function, and its expression is as follows: ； in, Indicates the lookup table index; Indicates the exponential offset constant; Indicates the scale parameter; Indicates the index bit width; lookup table entries are stored in a fixed-point format, overwriting... arrive The exponential approximation; S424. Perform approximate exponent calculation based on lookup table; Input an integer value based on Softmax. The approximate result of the corresponding exponent is obtained by looking up a table; the original Softmax definition expression is as follows: ； in, This represents the first query vector in the attention score matrix. Each correlation score, This represents other relevance scores corresponding to the same query vector; Taking the natural logarithm of both the numerator and denominator, we transform the sum of exponents into subtraction over the logarithmic field, as shown in the following expression: ； Attention score Quantized to k-bit integers As the input index to the lookup table, the integer exponent approximation is obtained by looking up the table during model inference, as shown in the following expression: ； S425, Normalization calculation based on displacement operation; Perform a shift-based normalization operation on the exponential approximation result, as shown in the following expression: ； ； in, Indicates the number of bits in the normalized shift. Represents the attention weight matrix in the integer field. This represents quantized to a k-bit integer. ; S426, based on Calculate and output the weighted sum of the matrices; The normalized attention weights and numerical matrix are then compared. To perform integer multiplication and addition operations, the expression is as follows: ； in, This represents the attention calculation result with k-bit integer precision.

5. The Vision Transformer model compression method based on a two-step hybrid quantization strategy according to claim 1, characterized in that, Step S5 is as follows: S51. Quantitative model construction; Based on the full-precision Vision Transformer model and quantization configuration, the target network layer is replaced with a quantization layer, and the integer calculation logic of LayerNorm shift quantization and Softmax logarithmic domain approximation special operator is integrated to generate the initial quantization model. S52, Quantization model export and format conversion; The completed quantization model is converted into a format that can be directly deployed on hardware, ensuring that the model contains complete quantization parameters and operator logic and is compatible with the computing architecture of the target hardware. S53. Based on the quantization model obtained in steps S51-S52, perform calibration set performance evaluation; The quantization model is evaluated for inference using a calibration dataset to obtain multi-dimensional performance metrics, including: (1) Accuracy metrics: Calculate Top-1 / Top-5 accuracy and cosine similarity to evaluate the accuracy loss of the model after quantization; (2) Performance indicators: statistical inference latency, memory usage, and hardware power consumption, to evaluate the deployment efficiency of the model; (3) Stability verification: Test the inference stability of the model under different batch sizes and input resolutions; S54. Based on the multi-dimensional performance indicators obtained in step S53, perform quantitative model verification and iterative optimization. Compare the indicators obtained from step S53 with the preset target thresholds: if the indicators meet the requirements, proceed to step S6; if the indicators do not meet the requirements, return to steps S3 to S4 to adjust the quantitative configuration and then return to step S5 to rebuild the quantitative model until the indicators meet the requirements.

Images