Visual converter post-training quantification method based on group awareness cooperation

Abstract

The invention discloses a visual converter post-training quantification method based on group awareness cooperation, and relates to the technical field of visual converters, and the method comprises the following steps: inputting a pre-training visual converter model, preparing a calibration data set, and initializing quantification hyper-parameters; calculating an original quantization parameter of each channel, constructing a feature vector after standardization, and calculating an intra-group unified quantization parameter and implementing uniform quantization by clustering dynamic grouping channels; a trainable parameter is introduced to the Softmax activation layer, a continuous displacement factor is generated through a constraint function, and nonlinear quantization is executed in combination with KL divergence optimization; a quantization error is calculated, and mean value compensation is carried out according to dimension characteristics of a convolutional layer or a full connection layer; a continuous displacement factor is pre-calculated as a fixed-point integer, and integer shift is used to replace exponential operation in inverse quantization. Through a dynamic perception grouping quantization mechanism, the problem that activation values are distributed unevenly after middle-layer normalization of the visual converter is solved.

Description

Technical Field

[0001] This invention belongs to the field of visual transformer technology, and in particular relates to a post-training quantization method for visual transformers based on group perception collaboration. Background Technology

[0002] Post-Training Quantization (PTQ) is a key technology in the field of deep learning model compression. Its core value lies in the fact that it can achieve model size compression and inference efficiency improvement by simply performing low-bit conversion on the weights and activation values ​​of the pre-trained model without retraining the model. It is especially suitable for edge device deployment scenarios with limited computing power and storage resources.

[0003] The activation values ​​of the normalized output layer in the visual transformer exhibit dynamic distribution mismatch between channels, constituting the core bottleneck of ultra-low bit quantization. Their distribution characteristics fluctuate drastically with the input data, leading to a severe mismatch between the fixed range of traditional static quantization schemes (such as layer / channel-level unified quantization) and the true activation range. Especially at quantization below 4 bits, this distribution mismatch causes the loss of key features of the attention mechanism. Existing global calibration or local compensation methods cannot effectively solve the channel-level dynamic coupling problem, resulting in a collapse in model accuracy. To address these issues, the following solutions are proposed. Summary of the Invention

[0004] The purpose of this invention is to provide a post-training quantization method for visual transformers based on group perception collaboration. By performing adaptive clustering based on the similarity of the distribution of quantization parameters between channels, the activation values ​​within a group are kept highly consistent, accurately matching the fluctuation characteristics of different channels. This solves the problem of quantization mismatch caused by the dynamic distribution imbalance of activation values ​​between channels due to layer normalization in existing static quantization schemes.

[0005] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0006] This invention relates to a post-training quantization method for a visual transformer based on group-aware collaboration, the quantization method comprising:

[0007] Step S1, Model and Data Initialization: Input the pre-trained visual transformer model, prepare the calibration dataset, and initialize the quantization hyperparameters;

[0008] Step S2, Dynamic Sensing Group Quantization: Calculate the original quantization parameters for each channel, construct feature vectors after standardization, dynamically group channels by clustering, calculate uniform quantization parameters within the group, and implement uniform quantization;

[0009] Step S3, Adaptive Continuous Displacement Quantization: Introduce trainable parameters to the Softmax activation layer, generate continuous displacement factors through constraint functions, and perform nonlinear quantization by combining KL divergence optimization;

[0010] Step S4, Dynamic Dimension Error Compensation: Calculate the quantization error and perform mean compensation according to the dimensionality characteristics of the convolutional layer or fully connected layer;

[0011] Step S5, Hardware Deployment Optimization: Pre-calculate the continuous displacement factor as a fixed-point integer and replace the exponential operation in dequantization with integer shift.

[0012] Furthermore, step S1, model and data initialization, specifically includes the following steps:

[0013] Step S11: Input the pre-trained full-precision visual transformer model;

[0014] Step S12: Prepare the calibration dataset by randomly sampling 32 images from the ImageNet training set for the image classification task, or sampling 1 image from the COCO dataset for the object detection task.

[0015] Step S13: Initialize key quantization hyperparameters, including setting the number of bits b for weights and activation values, defining the dynamic group size K, and configuring the shift factor learning rate λ. η This establishes the basic configuration for subsequent quantitative operations;

[0016] This step is responsible for preparing the entire quantization process; inputting a pre-trained full-precision visual transformer model (e.g., ViT, DeiT, or Swing Transformer) and preparing a very small calibration dataset (e.g., randomly selecting 32 images from the ImageNet training set for classification tasks); at the same time, setting key quantization hyperparameters, including the number of bits for the target weights and activation values ​​(e.g., 4 bits), the number of channels in each group during dynamic grouping operations (e.g., 16 channels per group), and the learning rate used to optimize subsequent key parameters.

[0017] Furthermore, step S2, dynamic sensing group quantization, specifically includes the following steps:

[0018] Step S21: Extract the activation values ​​after LayerNorm layer by layer, calculate the original quantization parameters for each channel, including the scaling factor determined based on the range of activation values ​​within the channel and the number of quantization bits, and the zero-point value aligned to the minimum value position:

[0019]

[0020] In the formula, s i x is the scaling factor for the i-th channel. i Let be the activation value tensor of the i-th channel, b be the number of quantization bits, and max(x) be the maximum value of the i-th channel. i ) is x i The maximum value, min(x) i) is x i The minimum value of z i Let be the zero point of the i-th channel;

[0021] Step S22: Standardize parameters to eliminate dimensional differences and avoid interference from dimensional differences between different channels with the grouping effect.

[0022]

[0023] In the formula, s norm,i s is the standardized scaling factor. i Let be the original scaling factor for the i-th channel, med(s) be the median of the scaling factors for all channels, max(s) and min(s) be the maximum and minimum scaling factors for all channels, respectively, ∈ be a constant to be divided by zero, and z norm,i Let z be the zero point of the i-th channel after normalization. i Let z be the original zero of the i-th channel, med(z) be the median of all channel zeros, and max(z) and min(z) be the maximum and minimum values ​​of all channel zeros, respectively.

[0024] Step S23: Concatenate the standardized scaling factor and zeros of each channel into a two-dimensional feature vector, which will serve as the basis for clustering.

[0025] The channels are divided into K groups using K-means clustering;

[0026] In the formula, f i Let i be the feature vector of the i-th channel. It is a two-dimensional real number space;

[0027] Step S24: After grouping, calculate the uniform quantization parameters within each group independently:

[0028]

[0029] In the formula, s g Let z be the uniform scaling factor for the g-th group. g S is the unified zero of the g-th group. g Let |S| be the set of channel indices contained in the g-th group. g | represents the number of channels in the g-th group;

[0030] Step S25: Apply (s) to all channels within the group g ,z g Uniform quantization is performed to achieve hardware-friendly grouped collaborative quantization.

[0031]

[0032] In the formula, Quant(x) i) is for input floating-point value x i Perform uniform quantization operation, x i Here, b is the input floating-point activation value, and b is the number of quantization bits. This is the rounding operation, and clip(·) is the truncation function;

[0033] This step first calculates the original quantization scale parameter for each channel; then, to fairly compare parameters with different dimensions, these scale parameters are standardized; next, using the characteristics of the standardized parameters, a clustering algorithm is used to dynamically divide all channels into several groups; finally, for each group, the average value of the quantization parameters of all channels within the group is calculated as the unified quantization parameter for that group, and this unified parameter is applied to quantize the activation values ​​of all channels within that group; in this way, the distribution of activation values ​​within the group is more similar, reducing quantization error.

[0034] Furthermore, step S3, adaptive continuous displacement quantization, specifically includes the following steps:

[0035] Step S31: Define a continuously learnable shift factor and initialize trainable parameters for each softmax post-activation layer. The range of displacement factors constrained by hyperbolic tangent:

[0036] η * =10 tanh(α) η * ∈[10 -1 10 1 Continuously adjustable

[0037] In the formula, α is a trainable parameter, tanh(·) is the hyperbolic tangent function, and η * It is a continuous displacement factor;

[0038] Step S32: Perform nonlinear quantization forward propagation, adjust the shift of the activation value based on η, calculate the dynamic scaling factor, and generate integer output values ​​through logarithmic transformation and uniform quantization.

[0039]

[0040] γ=-log2(min(X)+η * )

[0041] In the formula, X q Let X be the quantized integer tensor, X be the activation tensor after Softmax, γ be the dynamic scaling factor, and min(X) be the minimum value of X.

[0042] Step S33: Optimize the displacement factor by minimizing the KL divergence loss between the original Softmax distribution and the quantized reconstructed distribution, and continuously optimize the α parameter using gradient descent to adapt the displacement factor to the dynamic distribution characteristics of different layers, thereby improving the resolution of high-density regions.

[0043]

[0044] In the formula, Let D be the KL divergence loss function. KL (·||·) represents the KL divergence, and Softmax(·) represents the Softmax probability distribution. Let λ be the dequantized tensor. η The learning rate is the displacement factor. The gradient of the loss with respect to α;

[0045] This step first introduces a trainable parameter for each Softmax layer; this parameter is then mapped to a continuously adjustable shift factor value through a non-linear function, which can smoothly change within a reasonable range; next, this dynamically generated shift factor is used to shift the Softmax activation values, followed by logarithmic transformation and uniform quantization; after quantization, the difference in distribution (loss) between the quantization result and the original Softmax output is calculated; using this loss value, the trainable parameter is updated through a gradient descent algorithm, thereby dynamically adjusting the shift factor to automatically adapt to the actual distribution characteristics of different layers and different input data, achieving more refined and adaptive quantization.

[0046] Furthermore, step S4, dynamic dimensional error compensation, specifically includes the following steps:

[0047] Step S41: Calculate the error tensor generated during quantization, i.e., the difference between the original activation value and the dequantized value:

[0048] ∈=XQ -1 (Q(X))

[0049] In the formula, ∈ is the quantization error tensor, Q(·) is the quantization function, and Q -1 (·) is the inverse quantization function;

[0050] Step S42: The system adopts differentiated compensation strategies based on the characteristics of different network layers, as follows:

[0051] For convolutional layers, the mean error is calculated and compensated along the spatial dimension to ensure the consistency of spatial features:

[0052]

[0053] In the formula, For the final output after compensation, Let H be the dequantized tensor, H be the spatial height dimension, W be the spatial width dimension, and ∈[:,h,w] be the error slice at spatial location (h,w).

[0054] For fully connected layers, the mean error is calculated and compensated along the feature dimension to maintain the accuracy of the feature representation:

[0055]

[0056] In the formula, d is the feature dimension size, ∈ i Let be the error of the i-th feature dimension;

[0057] This step is responsible for compensating for the rounding errors that are inevitably introduced during quantization, in order to mitigate the impact of quantization on model performance. The core is to perform error compensation in specific dimensions based on the different characteristics of network layer types. First, the difference between the values ​​before and after the quantization operation (i.e., the quantization error) is calculated. For convolutional layers, their spatial dimensions (height and width) are identified, the average error at each spatial location is calculated, and this average compensation value is added to the quantization output of all spatial locations of that layer. For fully connected layers, their feature dimensions are identified, the average error at each feature dimension is calculated, and this average compensation value is added to the quantization output of all feature points of that layer. This dimension-aware compensation strategy is more in line with the computational characteristics of different layers and can more effectively correct the distribution shift introduced by quantization.

[0058] Furthermore, step S5, hardware deployment optimization, specifically includes the following steps:

[0059] Step S51: The continuous displacement factor η obtained through adaptive learning * To convert to a fixed-point representation, multiply it by 2 to the power of 8, round it to the nearest integer, and store it as an 8-bit integer to suit hardware processing requirements:

[0060]

[0061] In the formula, η fix This is the displacement factor after localization;

[0062] Step S52: Optimize the dequantization process by simplifying the computation, which originally required complex exponential operations, into efficient integer shift operations. This improves inference speed while maintaining computational accuracy, enabling the entire quantization scheme to be efficiently deployed on various edge computing devices.

[0063]

[0064] In the formula, This is the dequantization output, << is the left shift operator, and μ is the calibration constant;

[0065] This step transforms the parameters or operations generated in the previous steps—which are suitable for the algorithm but may be detrimental to efficient hardware execution—into a hardware-friendly form, ensuring that the final model can run efficiently on real devices. Crucially, the optimized continuous shift factor (a floating-point number) is pre-calculated and converted into a fixed-point representation (e.g., converted to 8-bit integer storage). Simultaneously, during dequantization, complex floating-point exponentiation operations are cleverly replaced with much lower-cost integer shift operations. These transformations ensure that the quantized model maintains accuracy while achieving low-latency, high-efficiency inference in resource-constrained environments such as embedded devices or mobile devices.

[0066] The present invention has the following beneficial effects:

[0067] 1. This invention alleviates the problem of uneven activation value distribution after layer normalization in visual transformers through a dynamic perception grouping quantization mechanism. The grouping strategy performs adaptive clustering based on the similarity of quantization parameters between channels, ensuring the consistency of activation value distribution within a group and reducing the distribution bias introduced by traditional layer quantization or channel quantization. At the same time, the continuous displacement factor can break free from the limitation of discrete candidate sets and adapt to the heavy-tailed distribution characteristics of different layers through learnable displacement factors, achieving fine-grained resolution allocation of high-density regions on key tensors such as soft attention output. Error compensation can further correct accumulated errors along the feature dimension, forming an error control closed loop from local to global, enabling the model to maintain near-full-precision performance even under low-precision conditions, and solving the model collapse problem that occurs in existing methods at extremely low bit quantization.

[0068] 2. This invention avoids the complex exponential operations in traditional logarithmic quantization by using integer shift dequantization; by pre-compiling continuous shift factors as fixed-point constants and replacing floating-point multiplication with shift operations during inference, the entire quantization computation graph can be mapped to edge devices that only support integer operations; the group-level quantization parameters generated by the dynamically perceived grouping strategy reduce storage overhead, and the grouping logic is optimized offline, eliminating the need for online computation; these designs enable the method to run without dedicated hardware acceleration units, achieving low-latency deployment on resource-constrained mobile chips, IoT devices, and edge computing platforms, providing fundamental support for the application of visual converters in diverse real-world scenarios.

[0069] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0070] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1 This is a flowchart illustrating a post-training quantization method for a visual transformer based on group perception collaboration according to the present invention. Detailed Implementation

[0072] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0073] Please see Figure 1 As shown, this invention is a quantization method after training a visual transformer based on group perception collaboration. The quantization method includes:

[0074] Step S1, Model and Data Initialization: Input the pre-trained visual transformer model, prepare the calibration dataset, and initialize the quantization hyperparameters, specifically:

[0075] Step S11: Input the pre-trained full-precision visual transformer model;

[0076] Step S12: Prepare the calibration dataset by randomly sampling 32 images from the ImageNet training set for the image classification task, or sampling 1 image from the COCO dataset for the object detection task.

[0077] Step S13: Initialize key quantization hyperparameters, including setting the number of bits b for weights and activation values, defining the dynamic group size K, and configuring the shift factor learning rate λ. η This establishes the basic configuration for subsequent quantitative operations.

[0078] Step S2, Dynamic Sensing Group Quantization: Calculate the original quantization parameters for each channel, construct a feature vector after standardization, dynamically group the channels by clustering, calculate the uniform quantization parameters within each group, and implement uniform quantization. Specifically:

[0079] Step S21: Extract the activation values ​​after LayerNorm layer by layer, calculate the original quantization parameters for each channel, including the scaling factor determined based on the range of activation values ​​within the channel and the number of quantization bits, and the zero-point value aligned to the minimum value position:

[0080]

[0081] In the formula, s i x is the scaling factor for the i-th channel. i Let be the activation value tensor of the i-th channel, b be the number of quantization bits, and max(x) be the maximum value of the i-th channel. i ) is x i The maximum value, min(x) i ) is x i The minimum value of z i Let be the zero point of the i-th channel;

[0082] Step S22: Standardize parameters to eliminate dimensional differences and avoid interference from dimensional differences between different channels with the grouping effect.

[0083]

[0084] In the formula, s norm,i s is the standardized scaling factor. i Let be the original scaling factor for the i-th channel, med(s) be the median of the scaling factors for all channels, max(s) and min(s) be the maximum and minimum scaling factors for all channels, respectively, ∈ be a constant to be divided by zero, and z norm,i Let z be the zero point of the i-th channel after normalization. i Let z be the original zero of the i-th channel, med(z) be the median of all channel zeros, and max(z) and min(z) be the maximum and minimum values ​​of all channel zeros, respectively.

[0085] Step S23: Concatenate the standardized scaling factor and zeros of each channel into a two-dimensional feature vector, which will serve as the basis for clustering.

[0086] The channels are divided into K groups using K-means clustering;

[0087] In the formula, f i Let i be the feature vector of the i-th channel. It is a two-dimensional real number space;

[0088] Step S24: After grouping, calculate the uniform quantization parameters within each group independently:

[0089]

[0090] In the formula, s g Let z be the uniform scaling factor for the g-th group. g S is the unified zero of the g-th group. g Let |S| be the set of channel indices contained in the g-th group. g | represents the number of channels in the g-th group;

[0091] Step S25: Apply (s) to all channels within the group g ,z g Uniform quantization is performed to achieve hardware-friendly grouped collaborative quantization.

[0092]

[0093] In the formula, Quant(x) i ) is for input floating-point value x i Perform uniform quantization operation, x i Here, b is the input floating-point activation value, and b is the number of quantization bits. This is the rounding operation, and clip(·) is the truncation function.

[0094] Step S3, Adaptive Continuous Displacement Quantization: Trainable parameters are introduced into the Softmax activation layer, a continuous displacement factor is generated through a constraint function, and nonlinear quantization is performed in conjunction with KL divergence optimization. Specifically:

[0095] Step S31: Define a continuously learnable shift factor and initialize trainable parameters for each softmax post-activation layer. The range of displacement factors constrained by hyperbolic tangent:

[0096] η * =10 tanh(α) η * ∈[10 -1 10 1 Continuously adjustable

[0097] In the formula, α is a trainable parameter, tanh(·) is the hyperbolic tangent function, and η * It is a continuous displacement factor;

[0098] Step S32: Perform nonlinear quantization forward propagation, adjust the shift of the activation value based on η, calculate the dynamic scaling factor, and generate integer output values ​​through logarithmic transformation and uniform quantization.

[0099]

[0100] γ=-log2(min(X)+η * )

[0101] In the formula, X q Let X be the quantized integer tensor, X be the activation tensor after Softmax, γ be the dynamic scaling factor, and min(X) be the minimum value of X.

[0102] Step S33: Optimize the displacement factor by minimizing the KL divergence loss between the original Softmax distribution and the quantized reconstructed distribution, and continuously optimize the α parameter using gradient descent to adapt the displacement factor to the dynamic distribution characteristics of different layers, thereby improving the resolution of high-density regions.

[0103]

[0104] In the formula, Let D be the KL divergence loss function. KL (·||·) represents the KL divergence, and Softmax(·) represents the Softmax probability distribution. Let λ be the dequantized tensor. η The learning rate is the displacement factor. Let be the gradient of the loss with respect to α.

[0105] Step S4, Dynamic Dimension Error Compensation: Calculate the quantization error and perform mean compensation based on the dimensionality characteristics of the convolutional or fully connected layers, specifically as follows:

[0106] Step S41: Calculate the error tensor generated during quantization, i.e., the difference between the original activation value and the dequantized value:

[0107] ∈=XQ -1 (Q(X))

[0108] In the formula, ∈ is the quantization error tensor, Q(·) is the quantization function, and Q -1 (·) is the inverse quantization function;

[0109] Step S42: The system adopts differentiated compensation strategies based on the characteristics of different network layers, as follows:

[0110] For convolutional layers, the mean error is calculated and compensated along the spatial dimension to ensure the consistency of spatial features:

[0111]

[0112] In the formula, For the final output after compensation, Let H be the dequantized tensor, H be the spatial height dimension, W be the spatial width dimension, and ∈[:,h,w] be the error slice at spatial location (h,w).

[0113] For fully connected layers, the mean error is calculated and compensated along the feature dimension to maintain the accuracy of the feature representation:

[0114]

[0115] In the formula, d is the feature dimension size, ∈ i Let be the error of the i-th feature dimension.

[0116] Step S5, Hardware Deployment Optimization: Pre-calculate the continuous shift factor as a fixed-point integer, and replace the exponential operation in dequantization with integer shift, specifically:

[0117] Step S51: The continuous displacement factor η obtained through adaptive learning * To convert to a fixed-point representation, multiply it by 2 to the power of 8, round it to the nearest integer, and store it as an 8-bit integer to suit hardware processing requirements:

[0118]

[0119] In the formula, η fix This is the displacement factor after localization;

[0120] Step S52: Optimize the dequantization process by simplifying the computation, which originally required complex exponential operations, into efficient integer shift operations. This improves inference speed while maintaining computational accuracy, enabling the entire quantization scheme to be efficiently deployed on various edge computing devices.

[0121]

[0122] In the formula, This is the dequantization output, << is the left shift operator, and μ is the calibration constant.

[0123] One specific application of this embodiment is:

[0124] Step S1, Model and Data Initialization:

[0125] Load the pre-trained full-precision DeiT-Small model;

[0126] Prepare calibration data: Randomly select 32 images from the ImageNet training set; adjust the image size to 224×224 pixels, and apply standard normalization (mean = [0.485, 0.456, 0.406], variance = [0.229, 0.224, 0.225]);

[0127] Set the quantization parameters: weight / activation bit width b = 4; dynamic block size K = 16; initial shift factor α = 0.0 (corresponding to η). * =10 0 =1); learning rate λ for displacement factor η =0.01;

[0128] Step S2, Dynamic Sensing Group Quantization:

[0129] Extract the activation tensor from the LayerNorm output of the 3rd layer.

[0130] Calculate the quantization parameters for each channel:

[0131] Standardized parameters (∈=10) -7 ):

[0132]

[0133] Construct the feature matrix:

[0134] Perform K-means clustering (number of clusters K = 16):

[0135] Randomly initialize cluster centers

[0136] Iterative optimization until convergence:

[0137] Calculate the quantization parameters of the group (taking group 5, S5, with 24 channels as an example):

[0138]

[0139] Apply uniform quantization to channels within the group:

[0140] Step S3, Adaptive continuous displacement factor quantization:

[0141] Process the output of the 4th layer Softmax (Number of heads × Sequence length);

[0142] Continuity displacement factor calculation: η * =10 tanh(α) =10 tanh(0.0) =1.0 (initial value);

[0143] Perform nonlinear quantization:

[0144] Calculate the dynamic scaling factor: γ = -log2(min(A) + η * ) = -log2(10 -5 +1.0)≈0.0;

[0145] Quantification formula:

[0146] Optimize displacement factor:

[0147] Calculate the KL divergence loss (taking the first head as an example):

[0148]

[0149] Backpropagation to update parameters:

[0150] After three iterations, α = -0.35, yielding the optimized value η. * =10 tanh ( -0.35 )≈0.457;

[0151] Step S4, Dynamic Dimension Perception Error Compensation:

[0152] Calculate the output of the 5th fully connected layer Quantization error: ∈ = FQ -1 (Q(F));

[0153] Compensation along the feature dimension:

[0154] Step S5, Hardware Deployment Optimization:

[0155] Convert the displacement factor to a fixed number: (Stored as an 8-bit integer);

[0156] Implement integer operations for dequantization:

[0157] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0158] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A post-training quantization method for a visual transformer based on group-aware collaboration, characterized in that, Includes the following steps: Step S1, Model and Data Initialization: Input the pre-trained visual transformer model, prepare the calibration dataset, and initialize the quantization hyperparameters; Step S2, Dynamic Sensing Group Quantization: Calculate the original quantization parameters for each channel, construct feature vectors after standardization, dynamically group channels by clustering, calculate uniform quantization parameters within the group, and implement uniform quantization; Step S3, Adaptive Continuous Displacement Quantization: Introduce trainable parameters to the Softmax activation layer, generate continuous displacement factors through constraint functions, and perform nonlinear quantization by combining KL divergence optimization; Step S4, Dynamic Dimension Error Compensation: Calculate the quantization error and perform mean compensation according to the dimensionality characteristics of the convolutional layer or fully connected layer; Step S5, Hardware Deployment Optimization: Pre-calculate the continuous displacement factor as a fixed-point integer and replace the exponential operation in dequantization with integer shift.

2. The post-training quantization method for a visual transformer based on group-aware collaboration according to claim 1, characterized in that, Step S1, model and data initialization, specifically includes the following steps: Step S11: Input the pre-trained full-precision visual transformer model; Step S12: Prepare the calibration dataset by randomly sampling 32 images from the ImageNet training set for the image classification task, or sampling 1 image from the COCO dataset for the object detection task. Step S13: Initialize key quantization hyperparameters, including setting the number of bits b for weights and activation values, defining the dynamic group size K, and configuring the shift factor learning rate λ. η This establishes the basic configuration for subsequent quantitative operations.

3. The post-training quantization method for a visual transformer based on group-aware collaboration according to claim 1, characterized in that, Step S2, dynamic sensing group quantization, specifically includes the following steps: Step S21: Extract the activation values ​​after LayerNorm layer by layer, calculate the original quantization parameters for each channel, including the scaling factor determined based on the range of activation values ​​within the channel and the number of quantization bits, and the zero-point value aligned to the minimum value position: In the formula, s i x is the scaling factor for the i-th channel. i Let be the activation value tensor of the i-th channel, b be the number of quantization bits, and max(x) be the maximum value of the i-th channel. i ) is x i The maximum value, min(x) i ) is x i The minimum value of z i Let be the zero point of the i-th channel; Step S22: Standardize parameters to eliminate dimensional differences and avoid interference from dimensional differences between different channels with the grouping effect. In the formula, s norm,i s is the standardized scaling factor. i Let be the original scaling factor for the i-th channel, med(s) be the median of the scaling factors for all channels, max(s) and min(s) be the maximum and minimum scaling factors for all channels, respectively, ∈ be a constant to be divided by zero, and z norm,i Let z be the zero point of the i-th channel after normalization. i Let z be the original zero of the i-th channel, med(z) be the median of all channel zeros, and max(z) and min(z) be the maximum and minimum values ​​of all channel zeros, respectively. Step S23: Concatenate the standardized scaling factor and zeros of each channel into a two-dimensional feature vector, which will serve as the basis for clustering. The channels are divided into K groups using K-means clustering; In the formula, f i Let i be the feature vector of the i-th channel. It is a two-dimensional real number space; Step S24: After grouping, calculate the uniform quantization parameters within each group independently: In the formula, s g Let z be the uniform scaling factor for the g-th group. g S is the unified zero of the g-th group. g Let |S| be the set of channel indices contained in the g-th group. g | represents the number of channels in the g-th group; Step S25: Apply (s) to all channels within the group g ,z g Uniform quantization is performed to achieve hardware-friendly grouped collaborative quantization. In the formula, Quant(x) i ) is for input floating-point value x i Perform uniform quantization operation, x i Here, b is the input floating-point activation value, and b is the number of quantization bits. This is the rounding operation, and clip(·) is the truncation function.

4. The post-training quantization method for a visual transformer based on group perception collaboration according to claim 1, characterized in that, Step S3, adaptive continuous displacement quantization, specifically includes the following steps: Step S31: Define a continuously learnable shift factor and initialize trainable parameters for each softmax post-activation layer. The range of displacement factors constrained by hyperbolic tangent: η * =10 tanh(α) η * ∈[10 -1 10 1 Continuously adjustable In the formula, α is a trainable parameter, tanh(·) is the hyperbolic tangent function, and η * It is a continuous displacement factor; Step S32: Perform nonlinear quantization forward propagation, adjust the shift of the activation value based on η, calculate the dynamic scaling factor, and generate integer output values ​​through logarithmic transformation and uniform quantization. γ=-log2(min(X)+η * ) In the formula, X q Let X be the quantized integer tensor, X be the activation tensor after Softmax, γ be the dynamic scaling factor, and min(X) be the minimum value of X. Step S33: Optimize the displacement factor by minimizing the KL divergence loss between the original Softmax distribution and the quantized reconstructed distribution, and continuously optimize the α parameter using gradient descent to adapt the displacement factor to the dynamic distribution characteristics of different layers, thereby improving the resolution of high-density regions. In the formula, Let D be the KL divergence loss function. KL (·||·) represents the KL divergence, and Softmax(·) represents the Softmax probability distribution. Let λ be the dequantized tensor. η The learning rate is the displacement factor. Let be the gradient of the loss with respect to α.

5. The post-training quantization method for a visual transformer based on group-aware collaboration according to claim 1, characterized in that, Step S4, dynamic dimension error compensation, specifically includes the following steps: Step S41: Calculate the error tensor generated during quantization, i.e., the difference between the original activation value and the dequantized value: ∈=X-Q -1 (Q(X)) In the formula, ∈ is the quantization error tensor, Q(·) is the quantization function, and Q -1 (·) is the inverse quantization function; Step S42: The system adopts differentiated compensation strategies based on the characteristics of different network layers, as follows: For convolutional layers, the mean error is calculated and compensated along the spatial dimension to ensure the consistency of spatial features: In the formula, For the final output after compensation, Let H be the dequantized tensor, H be the spatial height dimension, W be the spatial width dimension, and ∈[:,h,w] be the error slice at spatial location (h,w). For fully connected layers, the mean error is calculated and compensated along the feature dimension to maintain the accuracy of the feature representation: In the formula, d is the feature dimension size, ∈ i Let be the error of the i-th feature dimension.

6. The post-training quantization method for a visual transformer based on group-aware collaboration according to claim 1, characterized in that, Step S5, hardware deployment optimization, specifically includes the following steps: Step S51: The continuous displacement factor η obtained through adaptive learning * To convert to a fixed-point representation, multiply it by 2 to the power of 8, round it to the nearest integer, and store it as an 8-bit integer to suit hardware processing requirements: In the formula, η fix This is the displacement factor after localization; Step S52: Optimize the dequantization process by simplifying the computation, which originally required complex exponential operations, into efficient integer shift operations. This improves inference speed while maintaining computational accuracy, enabling the entire quantization scheme to be efficiently deployed on various edge computing devices. In the formula, This is the dequantization output, << is the left shift operator, and μ is the calibration constant.

Images