Method and device for post-training quantization of an image super-resolution model based on a corrector route

Abstract

The application provides a method and device for training and post-quantization of an image super-resolution model based on corrector routing, comprising: training a corrector group by using a gradient back propagation algorithm according to the weight increment, pre-training weight and quantization function of each corrector in the corrector group, obtaining optimized corrector parameters and quantizer parameters; based on the optimized corrector parameters and quantizer parameters, performing static weighting coefficient optimization on each quantization module by using a calibration image dataset, calculating an optimal weight increment according to the optimal static weighting coefficient, and generating a static routing table; enhancing the pre-training weight of the image super-resolution model by using the optimal weight increment, and calling each quantization module to pre-quantize the enhanced weight; and inputting a to-be-processed image into the model based on the pre-quantized weight for processing, and restoring a quality-enhanced image. In this way, the information loss caused by low-bit quantization is effectively reduced, and the ability of the quantized image super-resolution model to restore fine image details is significantly improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and image processing technology, and in particular to a method and apparatus for post-training quantization of an image super-resolution model based on corrector routing. Background Technology

[0002] With the rapid development of deep learning, image super-resolution models have achieved significant performance breakthroughs. However, their high computational and storage costs severely limit their deployment on real-world devices. Therefore, achieving efficient inference while maintaining accuracy has become a key research focus, with low-bit quantization being a promising solution. Low-bit quantization compresses the floating-point parameters of neural networks into low-bit representations, thereby reducing model size and latency while maintaining accuracy and achieving hardware acceleration. Quantization methods are generally divided into Quantization Aware Training (QAT) and Post-Training Quantization (PTQ). Although QAT is widely considered to minimize accuracy loss, it typically requires high training costs and long training times, sometimes even more difficult than training the original full-precision model. In contrast, PTQ completes quantization by calibrating weights / activations to adjust quantizer parameters without retraining the model. Therefore, PTQ offers low training costs and rapid deployment, but it often suffers from significant accuracy degradation in low-bit settings.

[0003] Currently, the recurrent states and dynamic gating mechanisms of the Mamba architecture in traditional low-bit quantization methods are prone to error accumulation and numerical sensitivity, leading to a significant decrease in the ability of existing quantization models to recover fine image details. Super-resolution tasks using the Mamba architecture are highly sensitive to pixel-level accuracy and local texture fidelity. Porting the Mamba architecture to the super-resolution domain typically produces unsatisfactory results, failing to meet stringent fidelity requirements, resulting in blurred details and lost textures, and exacerbating the substantial information loss inherent in low-bit quantization. Summary of the Invention

[0004] The purpose of this invention is to provide a method and apparatus for post-training quantization of an image super-resolution model based on corrector routing, which solves the problems of a significant decrease in the ability of existing quantization models to recover fine image details and the large amount of information loss caused by low-bit quantization.

[0005] To address the aforementioned technical problems, the embodiments of the present invention provide the following technical solutions:

[0006] The first aspect of this invention provides a post-training quantization method for an image super-resolution model based on corrector routing, comprising:

[0007] Based on the weight increment, pre-trained weights, and quantization function corresponding to each corrector in the corrector group, the corrector group is trained using the gradient backpropagation algorithm to obtain optimized corrector parameters and optimized quantizer parameters.

[0008] Based on the optimized corrector parameters and optimized quantizer parameters, the static weighting coefficients of each quantization module in the image super-resolution model are optimized using the calibration image dataset. The optimal weight increment corresponding to each quantization module is calculated based on the optimal static weighting coefficients to generate a static routing table. The static routing table is used to indicate the mapping relationship between each quantization module and the optimal weight increment.

[0009] The pre-trained weights of the image super-resolution model are enhanced using the optimal weight increment indicated by the static routing table, and each quantization module is called to pre-quantize the enhanced weights. During inference, the image to be processed is input into the quantized image super-resolution model and processed based on the pre-quantized weights to recover the image with enhanced quality.

[0010] A second aspect of the present invention provides a post-training quantization apparatus for an image super-resolution model based on corrector routing, comprising:

[0011] The corrector group training module is used to train the corrector group using the gradient backpropagation algorithm based on the weight increment, pre-trained weights and quantization function corresponding to each corrector in the corrector group, so as to obtain optimized corrector parameters and optimized quantizer parameters.

[0012] The static routing table generation module is used to optimize the static weighting coefficients of each quantization module in the image super-resolution model based on the optimized corrector parameters and optimized quantizer parameters, using the calibration image dataset, and calculate the optimal weight increment corresponding to each quantization module according to the optimal static weighting coefficients to generate a static routing table. The static routing table is used to indicate the mapping relationship between each quantization module and the optimal weight increment.

[0013] The image enhancement module is used to enhance the pre-trained weights of the image super-resolution model using the optimal weight increment indicated by the static routing table, and calls each quantization module to pre-quantize the enhanced weights. During inference, the image to be processed is input into the quantized image super-resolution model and processed based on the pre-quantized weights to recover the image with enhanced quality.

[0014] Compared to existing technologies, the image super-resolution model training post-quantization method and apparatus based on corrector routing provided by this invention trains the corrector group using the gradient backpropagation algorithm based on the weight increment, pre-trained weights, and quantization function corresponding to each corrector in the corrector group to obtain optimized corrector parameters and optimized quantizer parameters. Based on the optimized corrector parameters and optimized quantizer parameters, the static weighting coefficients of each quantization module in the image super-resolution model are optimized using a calibration image dataset, and the optimal weight increment corresponding to each quantization module is calculated based on the optimal static weighting coefficients to generate a static routing table. The pre-trained weights of the image super-resolution model are enhanced using the optimal weight increments indicated by the static routing table, and each quantization module is called to pre-quantize the enhanced weights. During inference, the image to be processed is input into the quantized image super-resolution model and processed based on the pre-quantized weights to recover the image with enhanced quality. In this way, by determining the weight increment corresponding to each corrector in the corrector group before quantization, which means injecting learnable compensation information into the model before quantization, the model can actively adapt to the quantization process. This can effectively reduce the information loss caused by low-bit quantization while ensuring the lightweight nature of the image super-resolution model. A fixed static routing table is generated through offline calibration to select the optimal weight increment for each quantization module in the image super-resolution model. This allows the optimal weight increment to enhance the weight expression capability of the quantization model, thereby significantly improving the ability of the quantized image super-resolution model to recover fine image details. Attached Figure Description

[0015] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the invention are illustrated by way of example and not limitation, with the same or corresponding reference numerals denoteing the same or corresponding parts, wherein:

[0016] Figure 1 A flowchart illustrating the post-training quantization method for an image super-resolution model based on corrector routing is shown.

[0017] Figure 2 The illustrations show a visual comparison of the various methods in some complex scenarios. Figure 1 ;

[0018] Figure 3 The illustrations show a visual comparison of the various methods in some complex scenarios. Figure 2 ;

[0019] Figure 4 A schematic diagram of the quantization device after training of an image super-resolution model based on corrector routing is shown. Detailed Implementation

[0020] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0021] It should be noted that, unless otherwise stated, the technical or scientific terms used in this invention should have the ordinary meaning as understood by one of ordinary skill in the art.

[0022] The methods described in the embodiments of the present invention will be explained in detail below.

[0023] Figure 1 A flowchart illustrating the post-training quantization method for an image super-resolution model based on corrector routing in an embodiment of the present invention is shown. See [link / reference]. Figure 1 As shown, the post-training quantization method for the image super-resolution model based on corrector routing can include:

[0024] S101. Based on the weight increment, pre-trained weights, and quantization function corresponding to each corrector in the corrector group, the corrector group is trained using the gradient backpropagation algorithm to obtain optimized corrector parameters and optimized quantizer parameters.

[0025] Specifically, before training the corrector group using the gradient backpropagation algorithm based on the weight increment, pre-trained weights, and quantization function corresponding to each corrector in the corrector group to obtain optimized corrector parameters and optimized quantizer parameters, the method further includes:

[0026] Construct an appliance group containing multiple appliances, where the expression for the appliance group is:

[0027] ;

[0028] in, For the orthodontic appliance group, This is the weight increment corresponding to the first corrector. For the first The weight increment corresponding to each corrector For the first The first low-rank matrix corresponding to each corrector. For the first The second low-rank matrix corresponding to each corrector This refers to the number of orthodontic appliances in the orthodontic appliance group.

[0029] Each corrector is a lightweight trainable corrector. Adding a lightweight trainable corrector compensates for the information lost during quantization. Each corrector consists of two low-rank matrices, namely the first low-rank matrix. Second low-rank matrix composition, , For the set of real numbers, It is a low-rank dimension. For input dimensions, For output dimensions, It is the size of A real matrix, It is the size of A real matrix.

[0030] The weight increments corresponding to all correctors are also called compensation information. Injecting this compensation information into the image super-resolution model before quantization effectively mitigates the significant information loss caused by low-bit quantization. Based on this, this invention sets up and learns a set of correctors to provide diverse information compensation for complex detail recovery in the quantized image super-resolution model.

[0031] Specifically, based on the weight increment, pre-trained weights, and quantization function corresponding to each corrector in the corrector group, the corrector group is trained using the gradient backpropagation algorithm to obtain optimized corrector parameters and optimized quantizer parameters, including:

[0032] Step A1: Perform a weighted fusion of the weight increment corresponding to each corrector and the weighting coefficient corresponding to each corrector to obtain the total weight increment.

[0033] In order to integrate the information compensation capabilities of these orthodontic devices, a corresponding weighting coefficient is assigned to each orthodontic device.

[0034] Step A2: Add the total weight increment to the pre-trained weights, and use a quantization function to quantize the result of the addition to obtain the quantized weights.

[0035] The expression for the quantization weight is:

[0036] ;

[0037] in, To quantify weights, Upper bound of quantization parameters and the lower bound of the quantization parameter The defined quantization function, For pre-trained weights, This is the total weight increment. This refers to the number of appliances in an appliance set. For the first The weighting coefficients corresponding to each corrector For the first The weight increment corresponding to each corrector. , It is the size of A real matrix.

[0038] Step A3: Based on the quantization weights, use the gradient backpropagation algorithm to train the corrector group to jointly optimize the corrector parameters and quantizer parameters, and obtain the optimized corrector parameters and optimized quantizer parameters.

[0039] The corrector parameters include the weighting coefficients for each corrector, the first low-rank matrix and the second low-rank matrix for each corrector, and the quantizer parameters include the upper bound of the quantization parameters. and the lower bound of the quantization parameter .

[0040] Specifically, during the training of the corrector group, the gradient backpropagation algorithm is used to jointly optimize all corrector parameters, the required upper and lower bounds of all quantizers, and the weighting coefficients. This joint optimization strategy enables different correctors to fully participate and optimize during training, allowing each corrector to acquire specialized capabilities for processing different information and compensating for different types of quantization errors. The core of this joint optimization strategy lies in the fact that the weight increments generated by the correctors and the quantizer parameters are jointly optimized under a unified objective. This optimization design ensures that weight updates not only compensate for accuracy loss but also fundamentally reduce quantization errors.

[0041] S102. Based on the optimized corrector parameters and optimized quantizer parameters, the static weighting coefficients of each quantization module in the image super-resolution model are optimized using the calibration image dataset. The optimal weight increment of each quantization module is calculated based on the optimal static weighting coefficients to generate a static routing table.

[0042] The static routing table is used to indicate the mapping relationship between each quantization module and the optimal weight increment.

[0043] Specifically, based on the optimized corrector parameters and optimized quantizer parameters, static weighting coefficient optimization is performed on each quantization module in the image super-resolution model using a calibration image dataset, including:

[0044] Step B1: During the optimization process of static weighting coefficients, the optimized corrector parameters and optimized quantizer parameters are kept constant.

[0045] Step B2: Based on the calibration image dataset, optimize the combination of static weighting factors corresponding to each quantization module by minimizing the loss function to obtain the optimal static weighting coefficients.

[0046] Without altering the original super-resolution model structure, this invention aims to better adapt the various compensation capabilities of the correctors learned during training to each quantization module and maximize performance. It optimizes a set of static weighting factor combinations to selectively integrate the compensation capabilities of different correctors. During this process, all parameters except the optimal static weighting coefficients remain unchanged. The specific optimization objective is to find the weighting factor combination that minimizes the loss function.

[0047] The expression for the optimal static weighting coefficients is as follows:

[0048] ;

[0049] in, These are the optimal static weighting coefficients, used to combine multiple orthodontic appliances. It is a static weighted factor combination. It is a static weighted factor weighted combination. For loss function, This refers to the forward computation process of the image super-resolution model. To calibrate the image dataset, Upper bound of quantization parameters and the lower bound of the quantization parameter The defined quantization function, For pre-trained weights, For the first The weighting coefficients corresponding to each corrector For the first The weight increment corresponding to each corrector This represents the number of correctors in the corrector group. The optimal static weighting coefficients for each quantization module can be obtained by calibrating using a small sample of the calibration image dataset.

[0050] Specifically, the optimal weight increment for each quantization module is calculated based on the optimal static weighting coefficients to generate a static routing table, including:

[0051] Step C1: Based on the optimal static weighting coefficients, calculate the optimal weight increment for each quantization module by weighted summation.

[0052] Step C2: Determine the mapping relationship between each quantization module and the corresponding optimal weight increment, and use the mapping relationship as a static routing table.

[0053] The static routing table contains each quantization module and its corresponding optimal weight increment.

[0054] S103. Using the optimal weight increment indicated by the static routing table, the pre-trained weights of the image super-resolution model are enhanced, and each quantization module is called to pre-quantize the enhanced weights. During inference, the image to be processed is input into the quantized image super-resolution model and processed based on the pre-quantized weights to recover the image with enhanced quality.

[0055] The image super-resolution model includes a neural network, a trained corrector group, various quantization modules, and a static routing table.

[0056] Specifically, step S103 includes: inputting the image to be processed into the image super-resolution model to find the optimal weight increment corresponding to each quantization module in the static routing table, fusing the optimal weight increment with the pre-trained weights to obtain enhanced weights, and calling each quantization module to pre-quantize the enhanced weights. During the inference process, the image to be processed is input into the quantized image super-resolution model so that the quantized image super-resolution model quantizes the generated feature activations based on the pre-quantized weights, performs low-bit quantization inference, and performs multi-layer quantization inference calculation to generate an image with enhanced quality.

[0057] The quantized augmentation weights are the final quantized weights used for forward inference. The output of the application process is:

[0058] ;

[0059] in, This indicates the feature activation after processing by the quantization module. For the enhanced weights after quantization, This represents the quantized output feature of this layer. Weight quantization and activation quantization work together to achieve low-bit inference acceleration of the model with almost no additional computational overhead.

[0060] The quantized image super-resolution model in this invention can be deployed on smart devices with limited computing resources, such as mobile phones and cameras, to achieve image quality enhancement. Specifically, the quantized image super-resolution model is deployed to the smart device; when there is a need for image quality enhancement, the image to be processed is input to the smart device, and the smart device calls the quantized image super-resolution model to perform enhancement processing and outputs an enhanced image, i.e., a high-definition image, thereby achieving efficient deployment and image quality enhancement of a large image super-resolution model in resource-constrained environments.

[0061] Compared with existing methods, the present invention has the following advantages:

[0062] First, compared with existing quantization methods, the image super-resolution model training post-quantization method proposed in this invention, by injecting learnable compensation information into the model before quantization, can actively adapt to the quantization process and effectively alleviate the information loss problem caused by extremely low bit compression (such as 4 bits and 2 bits).

[0063] Second, this invention employs a static priority routing mechanism, generating a fixed routing table through offline calibration to select the optimal compensation increment for each layer of the model. During the inference phase, the compensation information is pre-integrated into the model weights, eliminating the need for dynamic routing or introducing any additional computational overhead. This enhances the expressive power of the image super-resolution model while fully preserving the advantages of low-bit quantization in model lightweighting and inference acceleration.

[0064] The implementation environment and parameter settings for the image super-resolution model training post-quantization method based on corrector routing of the present invention are as follows:

[0065] Training Dataset: The DF2K dataset was used during the training process of this invention. This dataset is composed of two high-quality image datasets, DIV2K and Flickr2K, and contains approximately 3450 high-resolution images in total. The images cover a wide range of scenes, including natural landscapes, architecture, people, and everyday objects. This dataset provides the model with a larger scale and more diverse training samples, thereby effectively improving the model's generalization ability and reconstruction quality in different scenes.

[0066] Training Process and Loss Function: In the training process of this invention, the neural network used is the super-resolution model MambaIRv2-light. MambaIRv2-light is used as the base model, and experiments are conducted. This invention performs comprehensive experiments on image super-resolution tasks, testing at magnifications of 2 and 4, and at quantization accuracies of 2 bits and 4 bits. All hyperparameter settings are kept consistent throughout the experiments. The optimizer used is Adam, and the learning rate is set to... Set the first hyperparameter Second hyperparameter for In terms of learning rate scheduling, a cosine annealing strategy is adopted to ensure the stability of the training process. To improve the robustness and generalization ability of the image super-resolution model, this invention performs data augmentation on the training data, including random rotations of 90°, 180°, and 270°, as well as random horizontal flipping. The total number of training iterations is 12,000, and the batch size is 8.

[0067] Loss function for training image super-resolution models as follows:

[0068] ;

[0069] in, Pixel-level loss is used to measure the difference between the output image of the quantized model and the full-precision reference image. Specifically, the pixel reconstruction loss is obtained by calculating the absolute difference between the model output and the reference image at the pixel level and averaging them, thereby guiding the model to preserve image details as much as possible under quantization conditions. This is a feature-level loss used to establish constraints in the intermediate layers of the network, enabling the quantized model to progressively approximate the full-precision model in the feature representations of each computational block. Unlike methods that only perform feature distillation at the stage level, this invention introduces alignment constraints on each computational block, thereby compensating for the errors introduced by quantization at a finer-grained level and improving the overall representation capability and stability. These are weighting coefficients used to balance the loss ratio.

[0070] Training termination and model storage: When the number of iterations of the image super-resolution model reaches the pre-set iteration limit, training stops. Every 100 iterations, the model network structure and corresponding model parameters are saved. Simultaneously, the model is tested on the Set5 benchmark dataset as a validation set to select the best model. Finally, in the application inference testing phase, the best model obtained from the Set5 benchmark dataset is tested on five benchmark sets.

[0071] The model is input with sample images from the benchmark set to obtain the restored high-resolution image. The restored high-resolution image is then compared with the original high-resolution image, and the evaluation metrics Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) are calculated to compare different testing methods.

[0072] Simulation conditions:

[0073] This invention is implemented based on the PaddlePaddle deep learning framework, using Python as the programming language, and was tested and trained on an NVIDIA RTX 4090 GPU. During training, the DF2K dataset was selected as the training data. During testing, five benchmark datasets covering various typical scenarios were used, including Set5, Set14, B100, Urban100, and Manga109, to comprehensively verify the effectiveness and robustness of the proposed method under different image types and application scenarios. The datasets reference existing literature, and the multiple existing quantization methods used are the most advanced methods for Mamba model quantization in the past two years, including PTQ4VM (WACV 2025), Quamba (2024), and MambaQuant (ICLR 2025). Table 1 shows the results of 2x magnification tests and comparisons on the five benchmark sets, and Table 2 shows the results of 4x magnification tests and comparisons on the five benchmark sets. Tables 1 and 2 show that the quantization method based on corrector routing for image super-resolution model training in this invention significantly outperforms existing quantization methods in all evaluation scenarios. At 2x magnification, in the 4-bit test of the Set5 dataset, the PSNR and SSIM of the method in this invention are 0.55 dB higher than the best existing methods. On the more challenging Urban100 dataset, it improves by approximately 1 dB compared to other existing quantitative methods under 4-bit conditions. Even with a reduction in precision to 2 bits, it maintains strong performance, only decreasing by 1.75 dB compared to its 4-bit version, and still showing a significant improvement of 1.31 dB compared to other state-of-the-art methods.

[0074] Table 1. Results of 2x magnification tests and comparisons on five benchmark sets.

[0075]

[0076] Table 2. Results of 4x magnification tests and comparisons on five benchmark sets.

[0077]

[0078] Figure 2 The illustrations show a visual comparison of the various methods in some complex scenarios. Figure 1 , Figure 3 The illustrations show a visual comparison of the various methods in some complex scenarios. Figure 2 , Figure 2 and Figure 3 The selected images, though different, were all from complex scenes. Figure 2 and Figure 3The image shows a visual comparison under 4x magnification, demonstrating that the method of this invention can more clearly restore texture and edge details while maintaining overall structural consistency, resulting in a generated image with richer high-frequency information and a greater sense of visual hierarchy.

[0079] Based on the above Figure 1 As can be seen from the implementation method, the embodiments of the present invention train the corrector group using the gradient backpropagation algorithm based on the weight increment, pre-trained weights, and quantization function corresponding to each corrector in the corrector group to obtain optimized corrector parameters and optimized quantizer parameters; based on the optimized corrector parameters and optimized quantizer parameters, the static weighting coefficients of each quantization module in the image super-resolution model are optimized using the calibration image dataset, and the optimal weight increment corresponding to each quantization module is calculated according to the optimal static weighting coefficients to generate a static routing table; the pre-trained weights of the image super-resolution model are enhanced using the optimal weight increment indicated by the static routing table, and each quantization module is called to pre-quantize the enhanced weights; during the inference process, the image to be processed is input into the quantized image super-resolution model and processed based on the pre-quantized weights to recover the image quality enhanced image. In this way, by determining the weight increment corresponding to each corrector in the corrector group before quantization, which means injecting learnable compensation information into the model before quantization, the model can actively adapt to the quantization process. This can effectively reduce the information loss caused by low-bit quantization while ensuring the lightweight nature of the image super-resolution model. A fixed static routing table is generated through offline calibration to select the optimal weight increment for each quantization module in the image super-resolution model. This allows the optimal weight increment to enhance the weight expression capability of the quantization model, thereby significantly improving the ability of the quantized image super-resolution model to recover fine image details.

[0080] Based on the same inventive concept, as an implementation of the above-mentioned image super-resolution model training post-quantization method based on corrector routing, this embodiment of the invention also provides an image super-resolution model training post-quantization device based on corrector routing. Figure 4 This is a structural diagram of the image super-resolution model training and quantization device based on corrector routing in an embodiment of the present invention. See also... Figure 4 As shown, the post-training quantization device for the image super-resolution model based on corrector routing may include:

[0081] The corrector group training module 401 is used to train the corrector group using the gradient backpropagation algorithm based on the weight increment, pre-trained weights and quantization function corresponding to each corrector in the corrector group, so as to obtain optimized corrector parameters and optimized quantizer parameters.

[0082] The static routing table generation module 402 is used to optimize the static weighting coefficients of each quantization module in the image super-resolution model based on the optimized corrector parameters and optimized quantizer parameters, using the calibration image dataset, and calculate the optimal weight increment corresponding to each quantization module according to the optimal static weighting coefficients to generate a static routing table. The static routing table is used to indicate the mapping relationship between each quantization module and the optimal weight increment.

[0083] The image restoration module 403 is used to enhance the pre-trained weights of the image super-resolution model according to the optimal weight increment indicated by the static routing table, and call each quantization module to pre-quantize the enhanced weights; during the inference process, the image to be processed is input into the quantized image super-resolution model and processed based on the pre-quantized weights to restore the image with enhanced image quality.

[0084] The device may further include: a corrector group construction module, used to construct a corrector group containing multiple correctors before training the corrector group using a gradient backpropagation algorithm based on the weight increment, pre-trained weights, and quantization function corresponding to each corrector in the corrector group to obtain optimized corrector parameters and optimized quantizer parameters, wherein the expression for the corrector group is:

[0085] ;

[0086] in, For the orthodontic appliance group, This is the weight increment corresponding to the first corrector. For the first The weight increment corresponding to each corrector For the first The first low-rank matrix corresponding to each corrector. For the first The second low-rank matrix corresponding to each corrector This refers to the number of orthodontic appliances in the orthodontic appliance group.

[0087] The corrector group training module 401 is specifically used to perform weighted fusion of the weight increment and weighting coefficient corresponding to each corrector to obtain the total weight increment; add the total weight increment to the pre-trained weights, and use a quantization function to quantize the result of the addition to obtain the quantized weights; based on the quantized weights, use the gradient backpropagation algorithm to train the corrector group to jointly optimize the corrector parameters and quantizer parameters, and obtain the optimized corrector parameters and optimized quantizer parameters.

[0088] In the corrector group training module 401, the expression for the quantization weights is:

[0089] ;

[0090] in, To quantify weights, Upper bound of quantization parameters and the lower bound of the quantization parameter The defined quantization function, For pre-trained weights, This is the total weight increment. This refers to the number of appliances in an appliance set. For the first The weighting coefficients corresponding to each corrector For the first The weight increment corresponding to each corrector.

[0091] In the static routing table generation module 402, based on the optimized corrector parameters and optimized quantizer parameters, the static weighting coefficients of each quantization module in the image super-resolution model are optimized using the calibration image dataset. This includes: setting the optimized corrector parameters and optimized quantizer parameters to remain unchanged during the optimization process of the static weighting coefficients; and optimizing the combination of static weighting factors corresponding to each quantization module by minimizing the loss function based on the calibration image dataset to obtain the optimal static weighting coefficients.

[0092] In the static routing table generation module 402, the expression for the optimal static weighting coefficient is:

[0093] ;

[0094] in, These are the optimal static weighting coefficients. It is a static weighted factor combination. It is a static weighted factor weighted combination. For loss function, This refers to the forward computation process of the image super-resolution model. To calibrate the image dataset, Upper bound of quantization parameters and the lower bound of the quantization parameter The defined quantization function, For pre-trained weights, For the first The weighting coefficients corresponding to each corrector For the first The weight increment corresponding to each corrector This refers to the number of orthodontic appliances in the orthodontic appliance group.

[0095] In the static routing table generation module 402, the optimal weight increment corresponding to each quantization module is calculated based on the optimal static weighting coefficient to generate a static routing table. This includes: calculating the optimal weight increment corresponding to each quantization module by weighted summation based on the optimal static weighting coefficient; determining the mapping relationship between each quantization module and its corresponding optimal weight increment; and using the mapping relationship as the static routing table.

[0096] The image restoration module 403 is specifically used to input the image to be processed into the image super-resolution model, to find the optimal weight increment corresponding to each quantization module in the static routing table, to fuse the optimal weight increment with the pre-trained weights to obtain the enhanced weights, and to call each quantization module to pre-quantize the enhanced weights. The image to be processed is then input into the quantized image super-resolution model so that the quantized image super-resolution model quantizes the generated feature activations based on the pre-quantized weights, performs low-bit quantization inference, and performs multi-layer quantization inference calculations to generate an image with enhanced quality.

[0097] It should be noted that the above description of the image super-resolution model training post-quantization apparatus embodiment based on corrector routing is similar to the description of the image super-resolution model training post-quantization method embodiment based on corrector routing, and has similar beneficial effects. For any technical details not disclosed in the embodiments of the image super-resolution model training post-quantization apparatus based on corrector routing of the present invention, please refer to the description of the image super-resolution model training post-quantization method embodiment based on corrector routing of the present invention for understanding.

[0098] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A post-training quantization method for an image super-resolution model based on corrector routing, characterized in that, include: Based on the weight increment, pre-trained weights, and quantization function corresponding to each corrector in the corrector group, the corrector group is trained using the gradient backpropagation algorithm to obtain optimized corrector parameters and optimized quantizer parameters. Based on the optimized corrector parameters and the optimized quantizer parameters, the static weighting coefficients of each quantization module in the image super-resolution model are optimized using the calibration image dataset. The optimal weight increment corresponding to each quantization module is calculated based on the optimal static weighting coefficients to generate a static routing table. The static routing table is used to indicate the mapping relationship between each quantization module and the optimal weight increment. The pre-trained weights of the image super-resolution model are enhanced using the optimal weight increment indicated by the static routing table, and the enhanced weights are pre-quantized by calling each quantization module. During the inference process, the image to be processed is input into the quantized image super-resolution model and processed based on the pre-quantized weights to recover the image with enhanced quality.

2. The image super-resolution model training and post-quantization method based on corrector routing according to claim 1, characterized in that, Before training the corrector group using the gradient backpropagation algorithm based on the weight increment, pre-trained weights, and quantization function corresponding to each corrector in the corrector group to obtain optimized corrector parameters and optimized quantizer parameters, the method further includes: Construct an appliance group comprising multiple appliances, wherein the expression for the appliance group is: ； in, For the aforementioned orthodontic assembly, This is the weight increment corresponding to the first corrector. For the first The weight increment corresponding to each corrector For the first The first low-rank matrix corresponding to each corrector. For the first The second low-rank matrix corresponding to each corrector The number of orthodontic devices in the orthodontic group.

3. The image super-resolution model training and post-quantization method based on corrector routing according to claim 1, characterized in that, The step of training the corrector group using the gradient backpropagation algorithm based on the weight increment, pre-trained weights, and quantization function corresponding to each corrector in the corrector group to obtain optimized corrector parameters and optimized quantizer parameters includes: The weight increment corresponding to each corrector and the weighting coefficient corresponding to each corrector are weighted and fused to obtain the total weight increment; The total weight increment is added to the pre-trained weights, and the result of the addition is quantized using the quantization function to obtain the quantized weights. Based on the quantization weights, the corrector group is trained using the gradient backpropagation algorithm to jointly optimize the corrector parameters and quantizer parameters, thereby obtaining the optimized corrector parameters and the optimized quantizer parameters.

4. The image super-resolution model training and post-quantization method based on corrector routing according to claim 3, characterized in that, The expression for the quantization weight is: ； in, For the quantization weight, Upper bound of quantization parameters and the lower bound of the quantization parameter The defined quantization function, For the pre-trained weights, This is the total weight increment. The number of orthodontic appliances in the orthodontic appliance group. For the first The weighting coefficients corresponding to each corrector For the first The weight increment corresponding to each corrector.

5. The image super-resolution model training and post-quantization method based on corrector routing according to claim 1, characterized in that, Based on the optimized corrector parameters and the optimized quantizer parameters, the static weighting coefficients of each quantization module in the image super-resolution model are optimized using a calibration image dataset, including: During the optimization process of the static weighting coefficients, the optimized corrector parameters and the optimized quantizer parameters are kept constant. Based on the calibration image dataset, the combination of static weighting factors corresponding to each quantization module is optimized by minimizing the loss function to obtain the optimal static weighting coefficient.

6. The image super-resolution model training and post-quantization method based on corrector routing according to claim 5, characterized in that, The expression for the optimal static weighting coefficient is: ； in, The optimal static weighting coefficients are... It is a static weighted factor combination. It is a static weighted factor weighted combination. For loss function, This refers to the forward computation process of the image super-resolution model. For the calibration image dataset, Upper bound of quantization parameters and the lower bound of the quantization parameter The defined quantization function, For the pre-trained weights, For the first The weighting coefficients corresponding to each corrector For the first The weight increment corresponding to each corrector The number of orthodontic devices in the orthodontic group.

7. The image super-resolution model training and post-quantization method based on corrector routing according to claim 1, characterized in that, The step of calculating the optimal weight increment for each quantization module based on the optimal static weighting coefficient to generate a static routing table includes: Based on the optimal static weighting coefficients, the optimal weight increment corresponding to each quantization module is calculated by weighted summation; Determine the mapping relationship between each quantization module and the corresponding optimal weight increment, and use the mapping relationship as the static routing table.

8. The image super-resolution model training and post-quantization method based on corrector routing according to claim 1, characterized in that, The optimal weight increment indicated by the static routing table is used to enhance the pre-trained weights of the image super-resolution model, and the enhanced weights are pre-quantized by calling each quantization module. During the inference process, the image to be processed is input into the quantized image super-resolution model and processed based on pre-quantized weights to recover the image with enhanced quality, including: The image to be processed is input into the image super-resolution model to find the optimal weight increment corresponding to each quantization module in the static routing table, and the optimal weight increment is fused with the pre-trained weight to obtain the enhanced weight. The enhanced weight is then pre-quantized by each quantization module. During the inference process, the image to be processed is input into the quantized image super-resolution model, so that the quantized image super-resolution model quantizes the generated feature activations based on the pre-quantized weights, performs low-bit quantization inference, and performs multi-layer quantization inference calculation to generate the image quality enhancement image.

9. The image super-resolution model training and post-quantization method based on corrector routing according to claim 1, characterized in that, The image super-resolution model includes a neural network, a trained corrector group, the quantization modules, and the static routing table.

10. A quantization device for image super-resolution model training based on corrector routing, characterized in that, include: The corrector group training module is used to train the corrector group using the gradient backpropagation algorithm based on the weight increment, pre-trained weights and quantization function corresponding to each corrector in the corrector group, so as to obtain optimized corrector parameters and optimized quantizer parameters. A static routing table generation module is used to optimize the static weighting coefficients of each quantization module in the image super-resolution model based on the optimized corrector parameters and the optimized quantizer parameters, using a calibration image dataset, and to calculate the optimal weight increment corresponding to each quantization module according to the optimal static weighting coefficients, so as to generate a static routing table. The static routing table is used to indicate the mapping relationship between each quantization module and the optimal weight increment. The image enhancement module is used to enhance the pre-trained weights of the image super-resolution model according to the optimal weight increment indicated by the static routing table, and to call the quantization modules to pre-quantize the enhanced weights. During the inference process, the image to be processed is input into the quantized super-resolution image model and processed based on the pre-quantized weights to recover the image with enhanced quality.

Images