ViT model-oriented post-training static quantification method and related product

Abstract

The invention discloses a ViT model-oriented post-training static quantification method and a related product, and belongs to the technical field of deep learning models. According to the ViT model-oriented post-training static quantization method provided by the invention, the quantized ViT model is generated by positioning and splitting the core components of the ViT model, quantizing a linear layer based on a low-rank compensation strategy, quantizing a normalization layer based on scale re-parameterization, quantizing a nonlinear activation function based on binary shift operation and fusing the quantized core components; according to the method, the key structure module in the ViT network is quantified on the premise that retraining is carried out without relying on original training data, the calculation and storage overhead is reduced while the model accuracy rate is basically unchanged, and an efficient solution is provided for edge device deployment.

Description

Technical Field

[0001] This invention relates to the field of deep learning model technology, specifically to a post-training static quantization method and related products for ViT models. Background Technology

[0002] In the field of railway monitoring, traditional monitoring methods mostly rely on manual inspections or single-device sensing, which suffers from problems such as delayed response, limited multi-hazard identification capabilities, and poor cross-scenario adaptability, making it difficult to meet the needs of accurate early warning. Specifically, the identification accuracy of typical hazards such as retaining wall cracks in complex environments is insufficient, the model's generalization ability is weak in different scenarios, and there is a lack of efficient collaboration between real-time processing by edge devices and deep analysis in the cloud, resulting in limited timeliness and accuracy of disaster monitoring, and an overall passive response state.

[0003] In recent years, attention mechanisms have been widely used in deep neural networks due to their powerful modeling capabilities, significantly improving the model's ability to model long-distance dependencies and complex semantic structures, especially in the field of computer vision. The ViT model, based on the attention mechanism, as a structure utilizing self-attention for image modeling, has achieved breakthroughs in tasks such as image classification, object detection, and semantic segmentation, gradually becoming the next-generation visual backbone network after convolutional neural networks, and providing technical possibilities for high-precision identification of railway disasters. However, the high dependence of the ViT model on computational resources has become a major bottleneck for its practical application: compared to traditional CNN architectures, the ViT model involves a large number of dense matrix multiplications, multi-head attention calculations, and normalization operations in forward inference, leading to a significant increase in computational complexity and memory consumption, making it difficult to deploy directly in edge computing scenarios with low power consumption, low storage, and high real-time response requirements. Summary of the Invention

[0004] The purpose of this invention is to provide a post-training static quantization method and related products for ViT models, so as to overcome the problem that existing CNN model quantization methods cannot be directly applied to ViT models.

[0005] The present invention solves the above-mentioned technical problems through the following technical solution: This invention provides a post-training static quantization method for ViT models, comprising the following steps: The ViT model is trained using an image dataset, which includes images of railway disasters and foreign objects encroaching on railway lines. The core components of the trained ViT model are located and decomposed, including linear layers, normalization layers, and nonlinear activation functions. The core components are quantized: linear layers are quantized based on a low-rank compensation strategy, normalized layers are quantized based on scale reparameterization, and nonlinear activation functions are quantized based on binary shift operations. The core components after fusion and quantization are combined to generate the quantized ViT model.

[0006] A further improvement of this invention is that locating and decomposing the core components of the ViT model specifically includes the following steps: The ViT model divides the input image data samples into several fixed-size image blocks. Each image block is flattened and then converted into a fixed-dimensional vector through a linear mapping layer to form the initial feature representation. The initial feature representation is input into the backbone network composed of multiple Transformer modules. Based on the structural characteristics of the Transformer modules, the core components of the ViT model are located and decomposed.

[0007] A further improvement of the present invention is that the quantization linear layer based on the low-rank compensation strategy is specifically as follows: the weight quantization loss is compensated by the low-rank matrix, the quantization parameters are calculated independently column by column of the weight matrix, the sovereign weights are frozen and the low-rank matrix is ​​optimized to minimize the reconstruction error.

[0008] A further improvement of the present invention is that the scale-based reparameterized quantization normalization layer is specifically: by adjusting the affine factor of the normalization layer and the parameters of the next linear layer, the channel-by-channel quantization is converted into layer-by-layer quantization.

[0009] A further improvement of the present invention is that the nonlinear activation function includes the Softmax function and the GELU function.

[0010] A further improvement of this invention is that the nonlinear activation function based on binary shift operation is specifically: A learnable bias is added to the Softmax function to compensate for quantization error, and a Log quantizer is used for quantization. The exponential operation is split into an integer part and a fractional part, and integer inference is achieved through linear fitting and shift operations. A fixed offset is added to the GELU function to fit the Log quantizer, and the offset effect is eliminated through parameter compensation.

[0011] This invention also provides a post-training static quantization system for ViT models, comprising: The first module is used to locate and decompose the core components of the ViT model, which include linear layers, normalized layers, and nonlinear activation functions. The second module is used to quantize the core components: quantizing linear layers based on low-rank compensation strategy, quantizing normalized layers based on scale reparameterization, and quantizing nonlinear activation functions based on binary shift operations. The third module is used to integrate the core components after quantization processing to generate the quantized ViT model.

[0012] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the post-training static quantization method for the ViT model as described above.

[0013] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the post-training static quantization method for the ViT model as described above.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the post-training static quantization method for the ViT model as described above.

[0015] Compared with the prior art, the positive and progressive effects of the present invention are as follows: The post-training static quantization method for ViT models provided by this invention locates and decomposes the core components of the ViT model, quantizes linear layers based on a low-rank compensation strategy, quantizes normalized layers based on scale reparameterization, and quantizes nonlinear activation functions based on binary shift operations. The quantized core components are then fused to generate a quantized ViT model. This method quantizes key structural modules in the ViT network without relying on the original training data for retraining, reducing computational and storage overhead while maintaining the model's accuracy, thus providing an efficient solution for edge device deployment. Attached Figure Description

[0016] The accompanying drawings are provided to further understand the invention and constitute a part of this invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0017] Figure 1 This is a diagram showing the overall structure of the quantized ViT model of this invention. Figure 2 This is a diagram of the low-rank matrix learning architecture of the present invention; Figure 3 A distribution plot of LayerNorm activation values; Figure 4 This is a distribution map of Softmax activation values; Figure 5 This is a flowchart illustrating a post-training static quantization method for ViT models according to the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] In the description of this invention, it should be understood that the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0020] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0021] It should be understood that although terms such as first, second, third, etc., may be used in the embodiments of the present invention to describe the preset range, these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from one another. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0022] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0023] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This is an explanation of the present invention and not a limitation thereof.

[0024] A post-training static quantization method for ViT models includes the following steps: The ViT model is trained using an image dataset, which includes images of railway disasters and foreign objects encroaching on railway lines. The core components of the trained ViT model are located and decomposed, including linear layers, normalization layers, and nonlinear activation functions. The core components are quantized: linear layers are quantized based on a low-rank compensation strategy, normalized layers are quantized based on scale reparameterization, and nonlinear activation functions are quantized based on binary shift operations. The core components after fusion and quantization are combined to generate the quantized ViT model.

[0025] This method quantizes key structural modules in the ViT network without relying on the original training data for retraining. While maintaining the model accuracy at a relatively constant level, it reduces the computational and storage overhead, providing an efficient solution for edge device deployment.

[0026] In a specific embodiment of the present invention, a post-training static quantization system for ViT models is provided, including a linear layer quantization module, a normalization layer quantization module, an activation function quantization module, and a fusion module. The linear layer quantization module is used to implement linear layer quantization based on a low-rank compensation strategy to correct quantization errors. It includes a weighted quantization unit and a low-rank compensation unit. The weighted quantization unit performs column-by-column quantization and calculates the independent scaling factor and zero point for each column. The low-rank compensation unit compensates for the weighted quantization loss by end-to-end optimizing and learning the low-rank matrices A and B. The normalization layer quantization module is used to implement normalization layer quantization based on scale reparameterization. It accelerates the inference of layer-by-layer quantization while maintaining channel-by-channel accuracy. It includes a channel-by-channel quantization unit and a scale conversion unit. The channel-by-channel quantization unit retains channel-level accuracy in the weighted quantization stage. The scale conversion unit completes the channel-by-channel to layer-by-layer quantization conversion by adjusting the affine factor and the next layer parameter. The activation function quantization module is used to implement approximate quantization based on binary shift operations. During the quantization stage, it is used to compensate for the accuracy loss in the output of nonlinear functions. It includes a Softmax quantization unit and a GELU quantization unit. The Softmax quantization unit realizes integer inference through learnable bias, Log quantizer and binary shift. The GELU quantization unit adapts the Log quantizer with a fixed offset and combines parameter compensation to eliminate the offset effect. The fusion module is used to implement the core components after fusion and quantization processing, and generate the quantized ViT model.

[0027] This system addresses the deployment challenges of ViT models by exploring model quantization techniques adapted to their nonlinear structures (most existing mainstream quantization methods are designed for linear operations in CNNs and are difficult to apply directly to ViT networks containing complex nonlinear modules). It improves operational efficiency without sacrificing accuracy, ultimately solving core technical challenges such as cross-scenario adaptability and intelligent judgment. This forms a complete collaborative monitoring system from real-time edge perception to intelligent decision-making in the cloud, promoting the upgrade of railway flood control monitoring from passive response to proactive early warning, and providing more reliable technical support for railway transportation safety.

[0028] See Figure 1 and Figure 5 This invention discloses a post-training static quantization method for ViT models. Specifically, it performs quantization operations on key components of the ViT model, including linear layers, normalization layers, and nonlinear activation functions, to address the significant performance degradation of ViT models after low-bit quantization. Existing mainstream quantization methods are mostly designed around the linear operation characteristics of CNNs, making them difficult to directly apply to ViT networks containing complex nonlinear modules. The method includes the following steps: Step 1: Module decomposition and key component identification of the pre-trained ViT model The input image data samples are divided into several fixed-size image patches. Each patch is flattened and transformed into a fixed-dimensional vector representation through a linear mapping layer, forming the initial feature representation. Subsequently, the feature representation is input into the backbone network composed of multiple Transformer modules for layer-by-layer modeling and representation enhancement. In the ViT network, attention is calculated through a multi-head self-attention module. This multi-head self-attention mechanism achieves weighted aggregation of features by constructing similarity calculations between queries, keys, and values. Each Transformer module's MLP sublayer typically consists of two linear transformations and a non-linear activation function, GELU. Based on this, the pre-trained ViT model is modularly decomposed, identifying and separating three core components.

[0029] This invention discloses a post-training static quantization method for ViT models. Based on the hierarchical structure of the ViT model's Transformer module, the pre-trained ViT model is modularly decomposed, identifying and separating three core components that determine model performance and efficiency: linear layers responsible for linear feature mapping, normalization layers ensuring feature distribution stability, and activation functions introducing nonlinear expressive power. For linear layers, a low-rank compensation-based quantization strategy is adopted, introducing a low-rank matrix to compensate for weight quantization loss, while quantizing each column of weights independently to adapt to distribution differences. For normalization layers, channel-wise quantization is converted to layer-wise quantization through scale reparameterization, reducing inference complexity while preserving accuracy. For nonlinear activation functions, an approximate quantization based on binary shift is designed, transforming exponential operations into linear transformations and shift operations to achieve pure integer inference. Finally, the quantization results of each module are fused to generate a low-bit quantization model, significantly reducing model storage requirements and improving inference speed while ensuring controllable accuracy loss. This method is applicable to various architectures such as ViT, DeiT, and Swin, providing an efficient solution for edge device deployment.

[0030] Step 2: Linear layer quantization based on low-rank compensation strategy To address the issue of linear layers easily losing weight information during low-bit quantization, a strategy of "low-rank matrix error compensation + column-by-column quantization adaptation" is designed. This strategy compensates for quantization loss using a low-rank matrix while simultaneously adapting to weight distribution differences using column-by-column quantization. The specific steps are as follows: See Figure 2 This invention employs uniform quantization on the weight matrix of each linear layer and defines a quantization operation Quant. The quantization process is as follows: (1) in, — The number of bits in the quantization — Scaling factor — Midnight — Integer function, — Quantization function: snowflake indicates that this part is frozen and the weights are not updated, spark indicates that this part of the weights is updated during training.

[0031] For the linear layer, let the weights be... The dimension is ,enter The dimension is In the weight quantization process, this invention designs low-rank weights to compensate for the loss of weight information caused by quantization, and designs a pair of low-rank matrices. and ,matrix and The sizes are respectively and This makes the compensation item This is approximately represented as the weighting error during the quantization process. To reduce the computational cost of the model, The value is much smaller than and ,Right now In subsequent calculations, the linear layers in ViT are updated as follows: (2) To further improve quantization accuracy, column-by-column quantization is adopted in the weight quantization process, that is, the scaling factor is calculated independently for each column of W. and zero point This strategy can more precisely match the differences in column distributions, improving the overall representation fidelity. This applies to the quantized weight matrix and the low-rank matrix. The updated weight matrix is ​​as follows: (3) To stabilize the learning process and control model complexity, a sovereign recalculation method was adopted. The strategy only optimizes and The matrix, the overall error introduced by quantization, defines the loss function in the form of reconstruction error. : (4) Through end-to-end training, low-rank matrix and Learning the optimal solution to minimize reconstruction error improves the approximate accuracy of the quantization model during the inference phase. This step enhances ViT's performance under low-bit quantization and solves the quantization grid alignment problem through a grouped quantization strategy. While maintaining model structure and inference efficiency, it provides reliable support for deployment in resource-constrained scenarios such as edge devices.

[0032] This invention proposes a customized quantization strategy based on low-rank compensation. A low-rank compensation matrix is ​​designed for each linear layer to correct quantization errors. A set of learnable low-rank weights B and A are designed for each linear layer to compensate for the information loss caused by weight quantization. The ranks of matrices B and A are much smaller than the rank of the original weight matrix, thereby reducing additional computational overhead and preventing overfitting. This compensation term enhances the model's expressive power and improves the accuracy of the quantized model with minimal parameters and computational overhead. Furthermore, a "column-by-column grouping quantization" mechanism is adopted, independently dividing each column of the weight matrix into several groups, assigning each group a dedicated scaling factor and zero point to adapt to the distribution differences of weights in different columns, further improving the representational power and accuracy of the quantized model.

[0033] Step 3: Quantization of normalized layers based on scale reparameterization In the ViT model, LayerNorm normalization is mainly used for the hidden feature vector of each sample. The standardization process is as follows: (5) in, Indicates the sample index. Represents the feature vector of a sample. and Indicates the mean and variance. and Represents the affine transformation factor. This indicates the Hadamard product operation.

[0034] Figure 3The distribution of activation values ​​in the normalized layer between the 300th and 350th channel dimensions is shown. By examining the activation value distribution plot of LayerNorm, it can be seen that the activation values ​​of different channels exhibit significant distribution differences, indicating severe inter-channel variations in the LayerNorm output. Based on this, a scale-reparameterized normalized layer quantization method is proposed. This method converts to lower computational cost layer-by-layer quantization while maintaining channel-by-channel quantization accuracy, thus balancing model performance and inference efficiency.

[0035] For a given activation input of a normalized layer In the model weight quantization stage, the normalization layer is first quantized channel by channel to obtain the scaling factors for different channels. and zero point ; Calculate the scaling factor and zeros for layer-by-layer quantization, and then apply the new scaling factor. and zero point The scaling factor and the average of the zeros for all channels are determined by the following formula: (6) (7) Define two change factors and And these two change factors and The following conditions must be met: (8) (9) Calculate the zeros and scaling factors at the channel level: (10) (11) Will Add to Each channel generates a new zero point ,Will Divide each channel by Generate a new scaling factor These operations can be achieved by adjusting the affine factor of the normalization layer, as shown in the following formula: (12) (13) Through the above operations, the impact of switching from channel-wise quantization to layer-wise quantization is eliminated by adjusting the affine factor, and a new scaling factor for layer-wise quantization is implemented. And the new zero point The reparameterization completed the reconstruction of the quantization parameters.

[0036] However, the above operations will cause a shift in the distribution of activation values ​​in the normalization layer. To avoid affecting the performance of subsequent modules, this shift needs to be compensated for by adjusting the weights and biases of the next linear layer to eliminate the impact of the distribution shift on quantization performance. The specific calculation formula for the adjustment process is as follows: (14) in, Indicates the bias of the linear layer. This represents the weights of the linear layer.

[0037] Through the above operations, the following will be used and and Channel-by-channel quantization and Layer-by-layer quantization reduces model computational overhead while maintaining quantization accuracy.

[0038] The two-stage scale reparameterization method designed in this invention accelerates inference by layer-by-layer quantization while maintaining channel-by-channel accuracy. In the weight quantization stage, channel-by-channel quantization is performed on the activation values ​​of the normalized layer, and the scaling factor and zero point are calculated independently for each channel to preserve channel-level feature details to the greatest extent. In the model inference stage, by readjusting the affine factor and the weights and biases of subsequent layers, channel-by-channel quantization is equivalently converted into layer-by-layer quantization. This design retains the high-precision advantage of channel-by-channel quantization and simplifies the computation path in the inference stage, significantly reducing the hardware resource consumption and inference latency of the model.

[0039] Step 4: Approximate quantization of nonlinear activation functions based on binary shift operations Figure 4 The histogram of the Softmax activation distribution of the first multi-head self-attention module in the ViT model is presented. It is found that most Softmax activation values ​​are within a relatively small range of 0.0 to 0.3, with only a small portion of activation values ​​discretely distributed within a larger range, exhibiting a power-law distribution characteristic. Therefore, this invention uses a quantization resolution higher... The quantizer performs quantization operations to more finely characterize the activation distribution.

[0040] To address the quantization error generated during the Softmax quantization process, a learnable bias is added. This is used to compensate for quantization errors. (For learning...) The value of is used to define the loss function as: (15) in, Represents the original value. express Quantization function, Indicates a learnable bias; Through training, the optimal bias compensation for the Softmax function can be obtained, reducing quantization loss during model quantization. After obtaining the bias compensation, the quantization and dequantization processes of the Softmax function are modified as follows: (16) The inverse quantization formula is: (17) in, Represents the first input vector. Each element.

[0041] For floating-point input The floating-point numbers in the Softmax formula are dequantized using the dequantization formula. Represented by integers, the specific calculation process is as follows: (18) in, Indicates the scaling factor. This represents the quantized value. express The bias compensation value; for easier calculation The value of is used to calculate the exponential function using shift operations. Specifically, because It is an integer, while It is not necessarily an integer, making direct shift operations difficult. Therefore, for subsequent shift operations, ... The fraction is split into an integer part q and a fractional part r. To perform approximately linear calculations, the range of the fractional part r is defined as follows: .

[0042] Shift operations can be performed directly for... This invention transforms the function into a linear function through approximation operations, wherein... and The image in The interval images are very close, therefore this invention uses Approximate replacement .

[0043] Through the above operations, the following was achieved: In the quantizer, shift operations replace exponential calculations, reducing the computational load of the model and improving its computational speed.

[0044] The GELU function is also a non-linear activation function in the ViT network model, and its activation distribution also exhibits a power-law distribution. However, unlike the Softmax distribution, the distribution of GELU activation values ​​is highly asymmetrical in both positive and negative regions. The negative values ​​of GELU are relatively concentrated, mainly distributed in... Within this range. Therefore, a fixed offset is added to the GELU function input. Ensure the input is a positive value to fit. The quantizer uses the same operation as the scale-reparameterized normalized layer quantization method to correct the parameters of the next linear layer to eliminate the offset effect.

[0045] By using The quantizer improves the fitting ability to power-law distributions, and a learnable bias is designed to reduce quantization errors. Simultaneously, integer shifting and linear approximation are used to replace complex exponential calculations, and structural reparameterization methods are combined to eliminate offset errors. This approach reduces computational requirements while maintaining model accuracy.

[0046] This invention proposes an approximate quantization scheme based on binary shift for nonlinear activation function quantization. A learnable bias vector is designed to compensate for accuracy loss in the nonlinear function output during the quantization stage. For the Softmax function, a learnable bias vector is introduced to compensate for quantization errors, while a Log quantizer is used to cover its extreme value distribution range. To eliminate the high complexity of exponential operations, the exponent part is split into integer and fractional parts during dequantization. The fractional part is approximated by linear fitting, and binary shift operations are used to replace exponential calculations, achieving pure integer inference. For GELU activation function quantization, an additional positive offset is designed to ensure that all input values ​​are positive to fit the Log quantizer. The original distribution is then restored through a parametric inverse transform, eliminating the offset and reducing quantization loss.

[0047] Step 5: Use in downstream few-sample image classification algorithms This method decomposes the pre-trained ViT model into modules, identifying and separating three core components: linear layers, normalization layers, and nonlinear activation functions. For the linear layers, a low-rank compensation-based quantization method is employed. This method introduces a low-rank matrix to fit the weight quantization error and combines column-by-column grouping quantization to adapt to weight distribution differences, thus addressing the issue of weight information loss in low-bit quantization. For the normalization layers, a scale-reparameterized quantization method is used. This method preserves channel-by-channel accuracy during weight quantization and converts it to layer-by-layer quantization during inference by adjusting the affine factor and compensating for the next layer's parameters, balancing accuracy and efficiency. For the nonlinear activation functions, an approximate quantization method based on binary shift operations is used. Linear transformations and shift operations replace exponential operations to achieve integer inference. Finally, the quantization results from each module are fused to generate the quantized ViT model. This invention quantizes key structural modules in ViT networks without relying on retraining the original training data. While maintaining the model's accuracy, it reduces computational and storage overhead. It is applicable to various architectures such as ViT, DeiT, and Swin, providing an efficient solution for edge device deployment. It solves the problem that the performance of ViT models degrades significantly after low-bit quantization. Existing mainstream quantization methods are mostly designed around the linear operation characteristics of CNNs, making them difficult to apply directly to ViT networks containing complex nonlinear modules.

[0048] During the experiment, the present invention was tested on the ImageNet dataset and compared with the existing ViT quantization method to evaluate the performance of the proposed model quantization algorithm. The effectiveness of each quantization component was verified through ablation experiments.

[0049] Based on the same inventive concept, this application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a post-training static quantization method for a ViT model. The memory may include main memory, such as high-speed random access memory, or it may also include non-volatile memory, such as at least one disk storage device. The processor, network interface, and memory are interconnected via an internal bus, which may be an industry-standard architecture bus, a peripheral component interconnection standard bus, an extended industry-standard architecture bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory stores the program; specifically, the program may include program code, which includes computer operation instructions. The memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0050] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps of the post-training static quantization method for the ViT model. Specifically, the computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. The volatile memory may include RAM (Random Access Memory) and / or cache memory, etc. The non-volatile memory may include ROM (Read-Only Memory), hard disk, flash memory, optical disk, magnetic disk, etc.

[0051] Based on the same inventive concept, this application provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the post-training static quantization method for the ViT model described above.

[0052] Those skilled in the art will understand that embodiments of the present invention can be provided as methods or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM (Compact Disc Read-Only Memory), optical storage, etc.) containing computer-usable program code.

[0053] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer apparatus or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0054] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer device or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0055] These computer program instructions may also be loaded onto a computer device or other programmable data processing equipment to cause a series of operational steps to be performed on the computer device or other programmable equipment to produce a process implemented by the computer device, thereby providing instructions that execute on the computer device or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0056] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0057] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A post-training static quantization method for ViT models, characterized in that, Includes the following steps: The ViT model is trained using an image dataset, which includes images of railway disasters and foreign objects encroaching on railway lines. The core components of the trained ViT model are located and decomposed, including linear layers, normalization layers, and nonlinear activation functions. The core components are quantized: linear layers are quantized based on a low-rank compensation strategy, normalized layers are quantized based on scale reparameterization, and nonlinear activation functions are quantized based on binary shift operations. The core components after fusion and quantization are combined to generate the quantized ViT model.

2. The post-training static quantization method for ViT models according to claim 1, characterized in that, The process of locating and decomposing the core components of the ViT model specifically includes the following steps: The ViT model divides the input image data samples into several fixed-size image blocks. Each image block is flattened and then converted into a fixed-dimensional vector through a linear mapping layer to form the initial feature representation. The initial feature representation is input into the backbone network composed of multiple Transformer modules. Based on the structural characteristics of the Transformer modules, the core components of the ViT model are located and decomposed.

3. The post-training static quantization method for ViT models according to claim 1, characterized in that, The quantization linear layer based on the low-rank compensation strategy specifically involves: compensating for the weight quantization loss through a low-rank matrix, independently calculating the quantization parameters column by column of the weight matrix, freezing the sovereign weights, and optimizing the low-rank matrix to minimize the reconstruction error.

4. The post-training static quantization method for ViT models according to claim 1, characterized in that, The scale-based reparameterized quantization normalization layer specifically involves: converting channel-by-channel quantization into layer-by-layer quantization by adjusting the affine factor of the normalization layer and the parameters of the next linear layer.

5. The post-training static quantization method for ViT models according to claim 1, characterized in that, The nonlinear activation functions include the Softmax function and the GELU function.

6. The post-training static quantization method for ViT models according to claim 5, characterized in that, The nonlinear activation function based on binary shift operation quantization is specifically as follows: A learnable bias is added to the Softmax function to compensate for quantization error, and a Log quantizer is used for quantization. The exponential operation is split into an integer part and a fractional part, and integer inference is achieved through linear fitting and shift operations. A fixed offset is added to the GELU function to adapt it to the Log quantizer, and the effect of the offset is eliminated through parameter compensation.

7. A post-training static quantization system for ViT models, characterized in that, include: The first module is used to train the ViT model using an image dataset to obtain a trained ViT model. The image dataset includes images of railway disasters and foreign objects encroaching on railway lines. The module locates and decomposes the core components of the trained ViT model, which include linear layers, normalization layers, and nonlinear activation functions. The second module is used to quantize the core components: quantizing linear layers based on low-rank compensation strategy, quantizing normalized layers based on scale reparameterization, and quantizing nonlinear activation functions based on binary shift operations. The third module is used to integrate the core components after quantization processing to generate the quantized ViT model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the post-training static quantization method for ViT models as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the post-training static quantization method for ViT models as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the post-training static quantization method for ViT models as described in any one of claims 1 to 6.

Images