Super-resolution reconstruction method, device and storage medium

Abstract

The application relates to the field of deep learning, and provides a super-resolution reconstruction method, a device and a storage medium. The method comprises the following steps: acquiring a pre-trained floating-point super-resolution model and a calibration data set; based on the calibration data set, performing sorting-based quantization boundary initialization, which comprises the following steps: statistically sorting the weight and activation value data of the floating-point super-resolution model by using the calibration data set, and determining a quantization interval by using a percentile method based on the sorted data distribution; based on the quantization interval, performing bias compensation quantization, which comprises the following steps: calculating a bias compensation mean value, and applying adaptive offset to the weight and activation value data of the floating-point super-resolution model; and using the quantization model optimized by using the calibration data set to perform super-resolution reconstruction on a low-resolution image. According to the technical scheme, the quantization interval is optimized, and the data bias is corrected, so that the reconstruction precision of the super-resolution model after quantization is effectively improved.

Description

Technical Field

[0001] This application relates to the field of deep learning, and in particular to a super-resolution reconstruction method, apparatus, and storage medium. Background Technology

[0002] Image super-resolution technology is a key research direction in computer vision, aiming to reconstruct high-resolution images with high detail from low-resolution images. In recent years, deep learning models based on the Transformer architecture have made significant progress in this field, generating high-quality restored images due to their powerful global dependency modeling capabilities. However, such models typically have a large number of parameters and high computational complexity, severely limiting their practical deployment on edge devices with limited storage space and computing power.

[0003] To overcome the bottleneck of model deployment, model quantization technology is widely used. It significantly reduces model size and improves inference speed by converting high-precision floating-point parameters into low-bit integers. Post-training quantization (PTQ) methods have become a practical solution for super-resolution models due to their advantages of not requiring retraining and their efficient process. In existing technologies, two-stage PTQ algorithms such as 2DQuant mitigate the accuracy loss caused by quantization to some extent through distribution boundary initialization and distillation calibration strategies.

[0004] However, through in-depth analysis, the inventors of this application discovered that existing PTQ methods still have significant shortcomings when dealing with Transformer-based super-resolution models. The activation functions in these models (e.g., GELU, Softmax, etc.) introduce non-linear distribution biases, resulting in an asymmetrical bell-shaped curve in the distribution of activation values, where the mean deviates from the center and one-sided stacking occurs. Existing quantization schemes fail to adequately consider this distribution bias caused by the inherent operation of the model when determining the quantization interval, leading to inaccurate quantization interval estimation. This results in a large number of central region data points being mapped onto too few discrete integers after quantization, thus introducing significant quantization errors and limiting the accuracy preservation capability of super-resolution models under low-bit quantization. Summary of the Invention

[0005] This application provides a super-resolution reconstruction method, apparatus, and storage medium, which effectively improves the reconstruction accuracy of the super-resolution model after quantization by optimizing the quantization interval and correcting the data bias.

[0006] On the one hand, this application provides a super-resolution reconstruction method, the method comprising:

[0007] Obtain the pre-trained floating-point super-resolution model and calibration dataset;

[0008] Based on the calibration dataset, sort-based quantization boundary initialization is performed, including: statistically sorting the weights and activation values ​​of the floating-point super-resolution model using the calibration dataset, and determining the quantization interval based on the percentile method according to the sorted data distribution;

[0009] Based on the quantization interval, bias-compensated quantization is performed, including: calculating the bias compensation mean and applying an adaptive offset to the weights and activation values ​​of the floating-point super-resolution model.

[0010] Super-resolution reconstruction of low-resolution images is performed using a quantization model optimized with the aforementioned calibration dataset.

[0011] On the other hand, this application provides a super-resolution reconstruction apparatus, the apparatus comprising:

[0012] The acquisition module is used to acquire the pre-trained floating-point super-resolution model and calibration dataset;

[0013] An initialization module is used to perform sort-based quantization boundary initialization based on the calibration dataset, including: statistically sorting the weights and activation values ​​of the floating-point super-resolution model using the calibration dataset, and determining the quantization interval based on the percentile method according to the sorted data distribution;

[0014] The quantization module is used to perform bias-compensated quantization based on the quantization interval, including: calculating the bias compensation mean and applying an adaptive offset to the weights and activation values ​​of the floating-point super-resolution model.

[0015] The reconstruction module is used to perform super-resolution reconstruction of low-resolution images using a quantization model optimized with the calibration dataset.

[0016] Thirdly, this application provides an apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the technical solution of the super-resolution reconstruction method described above.

[0017] Fourthly, this application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the technical solution of the super-resolution reconstruction method described above.

[0018] As can be seen from the technical solution provided in this application, on the one hand, by initializing the quantization boundary based on the calibration dataset, the actual distribution characteristics of the model weights and activation values ​​can be more accurately characterized. Compared with the traditional boundary determination method based on extreme values, the use of sorting and percentile methods can eliminate the interference of extreme values, making the quantization interval more closely fit the concentrated area of ​​the data. This feature provides a more reasonable numerical mapping basis for subsequent quantization operations, thereby helping to reduce the quantization error introduced by the distribution estimation bias from the source. On the other hand, by introducing the adaptive offset operation of the bias compensation mean, the distribution asymmetry caused by the nonlinear operation of the model can be actively corrected. This operation makes the data distribution before quantization closer to a symmetrical state by making targeted adjustments to the data that deviates from the data center, improving the balance of the data in the quantization interval. This feature enables the quantization process to treat the data in the positive and negative value regions more fairly, avoiding the loss of quantization accuracy caused by one-sided bias. In summary, the technical solution of this application, based on the synergistic effect of sorting boundary initialization and bias compensation, enables the entire quantization method to more precisely control quantization error when facing complex super-resolution models, and ultimately achieves efficient model compression and acceleration while better maintaining the reconstruction quality of image super-resolution. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of the super-resolution reconstruction method provided in the embodiments of this application;

[0021] Figure 2 This is a schematic diagram of the structure of the super-resolution reconstruction device provided in the embodiments of this application;

[0022] Figure 3 This is a schematic diagram of the device provided in the embodiments of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] In this specification, adjectives such as "first" and "second" are used only to distinguish one element or action from another, without necessarily requiring or implying any actual such relationship or order. Where circumstances permit, reference to an element or component or step (etc.) should not be construed as being limited to only one of the elements, components, or steps, but may be one or more of the elements, components, or steps, etc.

[0025] For ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn to actual scale.

[0026] In recent years, deep learning models based on the Transformer architecture have made significant progress in this field, generating high-quality restored images due to their powerful global dependency modeling capabilities. However, such models typically have a large number of parameters and high computational complexity, severely limiting their practical deployment on edge devices with limited storage space and computing power. To overcome this bottleneck, model quantization techniques have been widely used, which significantly reduce model size and improve inference speed by converting high-precision floating-point parameters into low-bit integers. Post-training quantization (PTQ) methods have become a practical solution for super-resolution models due to their advantages of not requiring retraining and efficient processes. In existing technologies, two-stage PTQ algorithms such as 2DQuant have mitigated the accuracy loss caused by quantization to some extent through distribution boundary initialization and distillation calibration strategies. However, after in-depth analysis, the inventors of this application have found that existing PTQ methods still have significant shortcomings when dealing with Transformer-based super-resolution models. The activation functions in these models (e.g., GELU, Softmax, etc.) introduce non-linear distribution biases, resulting in an asymmetrical bell-shaped curve in the distribution of activation values, i.e., the mean deviates from the center, exhibiting a one-sided accumulation phenomenon. The existing quantization scheme fails to fully consider the distribution bias caused by the inherent operation of the model when determining the quantization interval, resulting in inaccurate estimation of the quantization interval. This causes a large number of central region data to be mapped onto too few discrete integers after quantization, thus introducing a large quantization error and limiting the accuracy preservation capability of the super-resolution model under low-bit quantization.

[0027] To address the aforementioned problems in the prior art, this application proposes a super-resolution reconstruction method, the flowchart of which is attached. Figure 1 As shown, the main steps include S101 to S104, which are detailed below:

[0028] Step S101: Obtain the pre-trained floating-point super-resolution model and calibration dataset.

[0029] Model quantization is a key technology for solving the problem of deployment efficiency in deep learning models. Its core idea is to convert high-precision floating-point parameters (such as 32-bit floating-point numbers FP32) in the model into low-precision integers (e.g., 8-bit integers INT8, 4-bit integers INT4), thereby reducing model storage footprint, lowering memory bandwidth consumption, and improving computational efficiency. The essence of quantization technology is to seek a balance between accuracy loss and performance improvement. Depending on the timing of quantization, it can be divided into two main categories: Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT). QAT effectively alleviates the accuracy degradation caused by quantization by introducing quantization error simulation during model training, but it requires a complete training dataset and training process, which is time-consuming, labor-intensive, and costly. In contrast, PTQ does not require retraining the model; it only requires statistical analysis and parameter calibration of the pre-trained floating-point super-resolution model to complete quantization. It has the advantages of simple process and high deployment efficiency, and is especially suitable for scenarios lacking training data or with limited computing resources. Therefore, this application obtains a pre-trained floating-point super-resolution model and a calibration dataset. The calibration dataset uses DF2K as training data (composed of DIV2K and Flickr2K, the high-resolution part of which is not needed because knowledge distillation training is used).

[0030] Step S102: Based on the calibration dataset, perform sort-based quantization boundary initialization, including: statistically sorting the weights and activation values ​​of the floating-point super-resolution model using the calibration dataset, and determining the quantization interval based on the percentile method according to the sorted data distribution.

[0031] The success of the Transformer architecture in natural language processing has prompted researchers to apply it to visual tasks. SwinIR, a representative work, adapted the Swin Transformer for image restoration tasks, balancing global dependency modeling with computational efficiency through local attention mechanisms and sliding window design. Subsequent studies have further optimized the Transformer architecture. HAT proposed axial deformable attention, combining axial attention with deformable convolution to enhance the modeling ability for long-range dependencies and local details. Swin2SR introduces a dynamic mask attention mechanism, dynamically adjusting the attention window based on the input content to reduce redundant computation.

[0032] Model quantization reduces storage requirements and computational complexity by lowering the numerical precision of neural network weights and activation values ​​(e.g., from 32-bit floating-point numbers to 8-bit integers), making it a key technology for addressing the bottleneck in deploying large super-resolution models. DFSQ unifies activation values ​​to the [-1, 1] range through sample-by-sample, channel-by-channel normalization, eliminating the influence of variance, and employs a hardware-friendly subset quantization method to select the optimal quantization point. PAMS introduces trainable truncation parameters to adaptively explore the upper limit of the quantization range, combining structured knowledge transfer (SKT) loss to fine-tune the quantization network. QuantSR proposes a redistribution-driven learnable quantizer (RLQ), which reduces information loss caused by discretization through learnable quantization intervals and mean shift parameters.

[0033] Commonly used linear quantization methods can be described through a series of transformations: for the value to be quantized, v, firstly, a clipping operation is used. Constrain it within the quantization range Inside (of which, l The lower bound of the range, u The upper bound of the range is usually set as the minimum and maximum values ​​in a set of data.

[0034] Subsequently, through The quantization process (i.e., mapping continuous numerical values ​​to discrete integers) is completed by mapping the cropped values ​​to an N-bit integer field. Here, Round represents the rounding operation. It is the largest integer that can be represented by N bits.

[0035] If it is necessary to recover an approximate original value from the quantization result (i.e., dequantization), then by...

[0036] integer Remapping back to the original numerical range yields an approximate floating-point number. In the PTQ process, dequantization is often performed immediately after quantization, allowing the model to adapt to quantized numbers while still maintaining floating-point inference. The difference from the original number v is used to adjust the quantization parameters and reduce the performance loss during actual quantization.

[0037] Mean shift has been noted in numerous recent publications. QuantSR uses the offset to shift the input globally. x The aim is to adjust the mean to be close to 0 for easier quantization. 2DQuantSR further analyzes the "long tail" phenomenon of parameter distribution in the activation function layer. Although the mean is not adjusted overall, a different acquisition scheme is adopted for the activation function layer than for the non-activation layer. The quantization formulas of the two are as follows: the two sub-formulas of formula (1) represent the quantization formula used by QuantSR, while the three sub-formulas of formula (2) represent the quantization formula used by 2DQuantSR.

[0038] (1)

[0039] (2)

[0040] In the two sub-formulas of formula (1) above, , Represents the learnable quantization range. Its function is to limit the gradient change of parameters that are far from the 0 point. Among the three sub-formulas of formula (2), , These are the learnable upper and lower bounds. In non-activation functions, both change simultaneously to remove extreme large and small values. Due to the "one-sided long-tailed" distribution of activation functions (similar to exponential distributions), only... Make changes to remove extremely large values.

[0041] It's easy to see that both representative methods impose gradient constraints or direct clipping operations on extreme values ​​to prevent them from lengthening the quantization range *s*, which would result in fewer integers mapped to a large number of intermediate data points, thus degrading the quantization performance. However, neither paper addresses or resolves the "biased bell curve" problem, where the overall parameters approximate a bell curve but one side is larger.

[0042] Through observation of a large amount of data and in conjunction with existing research, the inventors of this application have discovered that although the parameters of the non-activated layers are all normally distributed, there is always a "one-sided bias" phenomenon—specifically, the mean of the parameter distribution is not exactly at the center of all the data, but biased to one side. This introduces additional errors into the QuantSR quantization scheme based on the "bell curve" assumption. The inventors believe that the cause of the overall "one-sided bias" phenomenon is the combined effect of the activation function and the input. Taking SwinIR as an example, there are three main activation functions for Transformer-based super-resolution models: GELU, LeakyReLU, and Softmax. Although these three functions differ significantly, they share a common feature as activation functions: they all significantly suppress parameters smaller than the activation value.

[0043] To better illustrate the above points, we can make a simple assumption: the input is X, which is transformed into Y after being processed by the activation function, and then needs to be multiplied by Z. X and Z both satisfy the following four assumptions: 1) They both conform to a standard normal distribution. ;2) They are independent of each other;3) Both are discrete distributions;4) Both have a sufficient number of values ​​and data volume, but are still finite. This normal distribution well simulates the scenario where the model is initialized with the default MSRA, X is activated during forward propagation, and then convolves with Z. Assume that X is activated by the LeakyReLU activation function ( If the negative values ​​of X, which originally followed a normal distribution, are reduced overall, the entire value is "compressed" towards the positive direction of the coordinate axis. After compression, the point of highest probability shifts to the negative half of the coordinate axis, while the mean shifts to the positive half. The probability distribution function of Y (X after activation) is then as follows:

[0044] (3)

[0045] Although Y is highly asymmetric, if Z conforms to a "perfect standard normal distribution," the output XZ will still not exhibit bias and will remain symmetrically distributed. The inventors of this application provide a simple proof I:

[0046] Proof I: First, the expected value is 0. E(YZ) = E(Y)E(Z) = 0, which satisfies the necessary condition for symmetry.

[0047]

[0048] Secondly, if we prove symmetry, as a discrete random variable, we only need to prove that for any k, we have Assume Y has The i-th value is obtained from the given values. but:

[0049]

[0050] and Since Z follows a standard normal distribution, therefore regarding 0

[0051] Symmetrical, therefore

[0052] Therefore, for any i, we have

[0053] ,

[0054] so Q.E.D.

[0055] As can be seen from the above proof, the premise of YZ symmetry is still that Z is symmetric about 0.

[0056] At this point, if Z has a perturbation such that its relation to If symmetric, then only satisfying

[0057] ,

[0058] Not satisfied

[0059] ,

[0060] This results in asymmetry, which in turn leads to bias. When the actual number of parameters in a convolutional layer is small, the randomly obtained initial values ​​cannot perfectly conform to a normal distribution, and thus Δz will very commonly appear.

[0061] Because rigorously proving the magnitude of the bias is too complex, the inventors simulated this scenario. Taking X ~ N(0,1), they used LeakyReLU (… ) to activate and obtain Y, and with Multiplication. The results of simulating 100,000 points are shown in Table 1 below:

[0062] Table 1 Scenario Simulation Results

[0063]

[0064] It is clear from the table above that, A value of only 0.2 is sufficient to induce a significant "mean one-sided bias" and "highest probability one-sided bias" in the product YZ. This is because the compression of the activation function drastically increases the frequency of values ​​near 0, resulting in a very large distribution of data close to 0. Even slight perturbations can cause significant differences. These perturbations continuously disrupt the mean of Z, propagating the disturbance to subsequent layers, up to another normalization layer.

[0065] In summary, the inventors of this application have used a simple example to demonstrate how activation functions affect subsequent quantization processes—compressing the data distribution to the center, resulting in an overabundance of data near the center or uneven data distribution in the extreme value region, causing the mean or the highest probability value to move away from the center, introducing a large bias into the estimation of the quantization process—as demonstrated by the QuantSR scheme, which shifts the data as a whole. No matter how you adjust it, the above deviations cannot be eliminated.

[0066] To eliminate the problems of the existing technology, two points must be achieved: (1) During the quantization and immediate pseudo-quantization process, a portion of the values ​​should be shifted to make the model “balanced” on both sides of the mean, avoiding the unfair phenomenon of one side having more and the other side having less, and tending towards a normal distribution. Although a large amount of data is shifted, each data point changes little and has little impact on the whole; (2) When determining the upper and lower bounds of quantization, percentile sorting should be used instead of extreme values ​​to prevent the accumulation of central values.

[0067] Specifically, as an embodiment of this application, based on the calibration dataset, performing sorting-based quantization boundary initialization can be achieved through steps S1021 to S1023, as detailed below:

[0068] Step S1021: Use the calibration dataset to perform forward propagation on the weights and activation values ​​of the floating-point super-resolution model and collect statistical information.

[0069] As mentioned earlier, the calibration dataset uses DF2K as the training data. Regarding the implementation details of step S1021, SwinIR-light is used as the backbone network, the search steps in the DOBI process are K=100, and activation statistics are obtained from 32 randomly cropped images from DF2K (each image retains only a 3×64×64 portion). The Adam optimizer (learning rate 1e-2, beta values ​​set to 0.9 and 0.999, weight decay to 0, and the learning rate for both parameters in the BQ process is 1e-3) is used, coupled with a cosine annealing learning rate scheduler for stable training, while simultaneously performing random 90°, 180°, and 270° rotations and horizontal flips for data augmentation.

[0070] Step S1022: Sort the collected statistical information and divide it into 2 b There are several intervals, among which... b This represents the number of quantization bits.

[0071] This application innovates upon the original 2DQuant model. The new 2DQuant+ model addresses the aforementioned problems of existing technologies by improving the distribution-oriented boundary initialization (DOBI) preprocessing process. It uses the percentiles of sorted data to determine the quantization interval (ROBI), reducing center clustering, further subdividing regions with more central data, and using more integer mappings; while reducing integer mappings for the smaller portions on either side. The process of determining upper and lower bounds can be changed to first sorting the data, then dividing it sequentially. A quantification scheme for each interval. Specifically, the collected statistical information is sorted and divided into 2... b The interval can be: the accumulated data is divided into 2 intervals using the equal-frequency binning method. b 2 intervals; calculate 2 b The boundary values ​​of each interval are used to refine the quantization interval.

[0072] Step S1023: Determine the upper bound u and lower bound l of the quantization interval by removing extreme values ​​of the preset percentile.

[0073] Determining the upper and lower bounds involves progressively eliminating the largest and smallest extreme values. Therefore, two operations can be performed consecutively based on a single sorting operation. The pseudo-quantized value is directly taken as the average of the two endpoints of the interval, avoiding the influence of extreme values.

[0074] In one embodiment of this application, statistically sorting the weights and activation values ​​of a floating-point super-resolution model using a calibration dataset can be achieved by: collecting the statistical distribution of weights and activation values ​​for each network layer of the floating-point super-resolution model; and independently calculating the quantization interval parameters of each layer based on the forward propagation results of the calibration dataset.

[0075] Step S103: Perform bias-compensated quantization based on the quantization interval, including: calculating the bias compensation mean and applying adaptive offset to the weights and activation values ​​of the floating-point super-resolution model.

[0076] This application innovates upon the original 2DQuant model. The new 2DQuant+ model solves the aforementioned problems of the existing technology, namely, the design scheme for the PTQ process—adaptive offset compensation and mean offset compensation (BC) for non-uniform parameters—to solve the problems of mean offset and frequency offset.

[0077] Specifically, as one embodiment of this application, calculating the bias compensation mean and applying adaptive offset to the weights and activation values ​​of the floating-point super-resolution model can be done by: using Calculate the mean of bias compensation ,in, u This is the upper bound of the quantization interval. l This is the lower bound of the quantization interval. The mean offset parameter is learned based on the calibration dataset; for each data element ,when At that time, Replace with ,in, The offset parameters are learned based on the calibration dataset, when At that time, keep The following remains unchanged:

[0078]

[0079] In this embodiment, the 2DQuant+ model employs a two-stage quantization strategy, moving from coarse to fine: the first stage is Distribution-Oriented Boundary Initialization (DOBI), which uses a fast mean squared error (MSE) search to narrow the symmetric boundary for symmetric distributions (e.g., weights) and fixes the lower bound to the minimum data value for asymmetric distributions (e.g., partial activation values), only traversing the right boundary to efficiently obtain the coarse boundary of the quantizer; the second stage is Distillation Quantization Calibration (DQC), which uses the full-precision model as the teacher and the quantized model as the student, achieving fine-tuning of the quantizer parameters through joint optimization of the output L1 loss and the normalized L2 loss of the intermediate feature layer. With this strategy, 2DQuant can compress the Transformer model to 2-4 bits, achieving a compression ratio of 3.07-3.60 times and an inference speedup of 3.99-5.08 times, resulting in a performance improvement of up to 4.52 dB over the original state-of-the-art PTQ method on datasets such as Set5 (×2). Despite its excellent performance, the PTQ process still has room for optimization. For example, the distribution adaptability of the DOBI stage can be further improved, and the loss weight allocation of DQC has not yet fully incorporated the feature reconstruction requirements of the super-resolution task. This means that improving its PTQ strategy is of significant value. Therefore, in the embodiments of this application, the offset parameter can be adjusted. and mean offset parameter The optimization is performed as follows: Quantization loss is calculated by performing forward propagation on the calibration dataset; offset parameters are updated via backpropagation. and mean offset parameter Iterate and optimize until the loss function converges.

[0080] Step S104: Perform super-resolution reconstruction of the low-resolution image using a quantization model optimized with a calibrated dataset.

[0081] Specifically, as an embodiment of this application, super-resolution reconstruction of a low-resolution image using a quantization model optimized with a calibration dataset can be performed as follows: The low-resolution image is input into the quantization model optimized with the calibration dataset; the quantization model optimized with the calibration dataset obtains a high-resolution image through forward inference; in the YCbCr color space, the Y channel of the high-resolution image is compared with the Y channel of a standard high-resolution image, and the peak signal-to-noise ratio and structural similarity index are calculated. The quantization model optimized with the calibration dataset is the floating-point super-resolution model obtained by performing sorted quantization boundary and bias compensation quantization on the pre-trained floating-point super-resolution model through steps S102 and S103, as described in the aforementioned embodiment.

[0082] This application enables training of floating-point super-resolution models on an NVIDIA GeForce RTX 3090. Due to limitations in GPU memory, the batch size was reduced from 16 to 1, and the iter count was increased from 4000 to 10000. This application only trained a model with x2 4-bit quantization because x2 is not only the most commonly used resolution, but 4-bit quantization is also the most commonly used highest precision. Furthermore, for super-resolution models, smaller quantizations such as 3-bit or 2-bit would result in significant performance loss, leading to poor image reconstruction quality, lower than most small-parameter models. This contradicts the original intention of quantizing large-parameter models.

[0083] Under the experimental conditions described above, the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) of the quantized model optimized with the calibrated dataset for super-resolution reconstruction of low-resolution images in step S104 are compared in Tables 2 and 3 below:

[0084] Table 2 Comparison of Peak Signal-to-Noise Ratio (PSNR) for Super-Resolution Reconstruction

[0085]

[0086] Table 3 Comparison of Structural Similarity Indices (SSIM)

[0087]

[0088] As can be seen from the PSNR and SSIM metrics, the addition of the two core improvements, ROBI and BC, resulted in a comprehensive improvement in both PSNR and SSIM. The highest improvement was observed on the Manga109 dataset, with a PSNR increase of 0.09 dB. Although the numerical value is small, it still holds significant value as a solution that represents a comprehensive improvement over the original two-stage quantization super-resolution state-of-the-art algorithm. Furthermore, since the ROBI and BC processes are only implemented during PTQ, they do not affect the parameters or computational cost of the final quantization result. The 2DQuant+ model represents a more efficient outcome.

[0089] From the above appendix Figure 1As demonstrated by the example of super-resolution reconstruction, on the one hand, initializing the quantization boundary based on a calibrated dataset allows for a more accurate characterization of the actual distribution features of model weights and activation values. Compared to traditional boundary determination methods based on extreme values, using sorting and percentile methods can eliminate interference from extreme values, making the quantization interval more closely match the concentrated areas of the data. This feature provides a more reasonable numerical mapping basis for subsequent quantization operations, thus helping to reduce quantization errors introduced by distribution estimation biases from the outset. On the other hand, by introducing an adaptive offset operation with bias-compensated mean, the distribution asymmetry caused by model nonlinear operations can be actively corrected. This operation, by specifically adjusting data deviating from the data center, makes the data distribution before quantization closer to a symmetrical state, improving the balance of the data within the quantization interval. This characteristic allows the quantization process to treat data in both positive and negative value regions more fairly, avoiding the loss of quantization accuracy caused by one-sided bias. In summary, the technical solution of this application, based on the synergistic effect of sorting boundary initialization and bias compensation, enables the entire quantization method to more precisely control quantization error when facing complex super-resolution models, and ultimately achieves efficient model compression and acceleration while better maintaining the reconstruction quality of image super-resolution.

[0090] Please see the appendix Figure 2 This application provides a super-resolution reconstruction device, which may include an acquisition module 201, an initialization module 202, a quantization module 203, and a reconstruction module 204, as detailed below:

[0091] The acquisition module 201 is used to acquire the pre-trained floating-point super-resolution model and calibration dataset;

[0092] Initialization module 202 is used to perform sort-based quantization boundary initialization based on the calibration dataset, including: statistically sorting the weight and activation value data of the floating-point super-resolution model using the calibration dataset, and determining the quantization interval based on the percentile method according to the sorted data distribution;

[0093] The quantization module 203 is used to perform bias-compensated quantization based on the quantization interval, including: calculating the bias compensation mean and applying an adaptive offset to the weight and activation value data of the floating-point super-resolution model.

[0094] Reconstruction module 204 is used to perform super-resolution reconstruction of low-resolution images using a quantization model optimized with a calibrated dataset.

[0095] From the above appendix Figure 2As demonstrated by the example super-resolution reconstruction device, on the one hand, initializing the quantization boundary based on the calibration dataset allows for a more accurate characterization of the actual distribution features of model weights and activation values. Compared to traditional boundary determination methods based on extreme values, using sorting and percentile methods can eliminate interference from extreme values, making the quantization interval more closely match the concentrated area of ​​the data. This feature provides a more reasonable numerical mapping basis for subsequent quantization operations, thus helping to reduce quantization errors introduced by distribution estimation bias from the source. On the other hand, by introducing an adaptive offset operation with bias compensation mean, the distribution asymmetry caused by model nonlinear operations can be actively corrected. This operation, by specifically adjusting data deviating from the data center, makes the data distribution before quantization closer to a symmetrical state, improving the balance of the data within the quantization interval. This characteristic allows the quantization process to treat data in both positive and negative value regions more fairly, avoiding the loss of quantization accuracy caused by one-sided bias. In summary, the technical solution of this application, based on the synergistic effect of sorting boundary initialization and bias compensation, enables the entire quantization method to more precisely control quantization error when facing complex super-resolution models, and ultimately achieves efficient model compression and acceleration while better maintaining the reconstruction quality of image super-resolution.

[0096] Figure 3 This is a schematic diagram of the structure of a device provided in one embodiment of this application. For example... Figure 3 As shown, the device 3 in this embodiment mainly includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a program for a super-resolution reconstruction method. When the processor 30 executes the computer program 32, it implements the steps in the above-described super-resolution reconstruction method embodiment, for example... Figure 1 The steps S101 to S104 are shown. Alternatively, when the processor 30 executes the computer program 32, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 2 The functions of the acquisition module 201, initialization module 202, quantization module 203, and reconstruction module 204 are shown.

[0097] Exemplarily, the computer program 32 of the super-resolution reconstruction method mainly includes: acquiring a pre-trained floating-point super-resolution model and a calibration dataset; performing sort-based quantization boundary initialization based on the calibration dataset, including: statistically sorting the weights and activation values ​​of the floating-point super-resolution model using the calibration dataset, and determining the quantization interval based on the percentile method according to the sorted data distribution; performing bias-compensated quantization based on the quantization interval, including: calculating the bias compensation mean and applying an adaptive offset to the weights and activation values ​​of the floating-point super-resolution model; and performing super-resolution reconstruction of the low-resolution image using the quantization model optimized by the calibration dataset. The computer program 32 can be divided into one or more modules / units, which are stored in memory 31 and executed by processor 30 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 32 in device 3. For example, computer program 32 can be divided into the functions of acquisition module 201, initialization module 202, quantization module 203, and reconstruction module 204 (modules in the virtual device). The specific functions of each module are as follows: Acquisition module 201 is used to acquire a pre-trained floating-point super-resolution model and a calibration dataset; Initialization module 202 is used to perform sort-based quantization boundary initialization based on the calibration dataset, including: statistically sorting the weights and activation values ​​of the floating-point super-resolution model using the calibration dataset, and determining the quantization interval based on the percentile method according to the sorted data distribution; Quantization module 203 is used to perform bias-compensated quantization based on the quantization interval, including: calculating the bias compensation mean and applying an adaptive offset to the weights and activation values ​​of the floating-point super-resolution model; Reconstruction module 204 is used to perform super-resolution reconstruction of the low-resolution image using the quantization model optimized by the calibration dataset.

[0098] Device 3 may include, but is not limited to, processor 30 and memory 31. Those skilled in the art will understand that... Figure 3 This is merely an example of device 3 and does not constitute a limitation on device 3. It may include more or fewer components than shown, or combine certain components, or different components. For example, the device may also include input / output devices, network access devices, buses, etc.

[0099] The processor 30 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0100] The memory 31 can be an internal storage unit of the device 3, such as a hard disk or RAM of the device 3. The memory 31 can also be an external storage device of the device 3, such as a plug-in hard disk, Smart MediaCard (SMC), Secure Digital (SD) card, or Flash Card equipped on the device 3. Furthermore, the memory 31 can include both internal and external storage units of the device 3. The memory 31 is used to store computer programs and other programs and data required by the device. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0101] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed. That is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above-described device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0102] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0103] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0104] In the embodiments provided in this application, it should be understood that the disclosed apparatus / device and method can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0105] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0106] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0107] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program for the super-resolution reconstruction method can be stored in a storage medium. When the computer program is executed by a processor, it can implement the steps of the various method embodiments described above, namely, obtaining a pre-trained floating-point super-resolution model and a calibration dataset; performing sorted quantization boundary initialization based on the calibration dataset, including: statistically sorting the weights and activation values ​​of the floating-point super-resolution model using the calibration dataset, and determining the quantization interval based on the percentile method according to the sorted data distribution; performing bias-compensated quantization based on the quantization interval, including: calculating the bias compensation mean and applying an adaptive offset to the weights and activation values ​​of the floating-point super-resolution model; and performing super-resolution reconstruction of the low-resolution image using the quantization model optimized by the calibration dataset. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate form. Storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the contents of storage media can be appropriately added to or removed according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, storage media may not include electrical carrier signals and telecommunication signals.

[0108] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application. The specific embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the protection scope of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A super-resolution reconstruction method, characterized in that, The method includes: Obtain the pre-trained floating-point super-resolution model and calibration dataset; Based on the calibration dataset, sort-based quantization boundary initialization is performed, including: statistically sorting the weights and activation values ​​of the floating-point super-resolution model using the calibration dataset, and determining the quantization interval based on the percentile method according to the sorted data distribution; Based on the quantization interval, bias-compensated quantization is performed, including: calculating the bias compensation mean and applying an adaptive offset to the weights and activation values ​​of the floating-point super-resolution model. The calculation of the bias compensation mean and the application of the adaptive offset to the weights and activation values ​​of the floating-point super-resolution model includes: using... Calculate the mean of the bias compensation The As the upper bound of the quantization interval, the As the lower bound of the quantization interval, the The mean offset parameter is learned based on the calibration dataset; for each data element ,when At that time, Replace with The The offset parameters are learned based on the calibration dataset; Super-resolution reconstruction of low-resolution images is performed using a quantization model optimized with the aforementioned calibration dataset.

2. The super-resolution reconstruction method according to claim 1, characterized in that, The process of performing sort-based quantization boundary initialization based on the calibration dataset includes: The weights and activation values ​​of the floating-point super-resolution model are forward-propagated using the calibration dataset to collect statistical information; The collected statistical information was sorted and divided into 2. b The interval, the b The number of quantization bits; The upper bound of the quantization interval is determined by removing extreme values ​​at a preset percentile. and the lower realm .

3. The super-resolution reconstruction method according to claim 2, characterized in that, The step of statistically sorting the weights and activation values ​​of the floating-point super-resolution model using the calibration dataset includes: For each network layer of the floating-point super-resolution model, the statistical distribution of weights and activation values ​​is collected. Based on the forward propagation results of the calibration dataset, the quantization interval parameters of each layer are calculated independently.

4. The super-resolution reconstruction method according to claim 3, characterized in that, The collected statistical information is sorted and divided into 2. b The intervals include: The accumulated data is divided into 2 using the equal-frequency binning method. b A range; Calculate the 2 b The boundary values ​​of each interval are used to refine the quantization interval.

5. The super-resolution reconstruction method according to claim 1, characterized in that, The offset parameter and mean offset parameter Optimize in the following ways: Perform forward propagation on the calibration dataset to compute the quantization loss; The offset parameter is updated via backpropagation. and the mean offset parameter ; Iterate and optimize until the loss function converges.

6. The super-resolution reconstruction method according to claim 1, characterized in that, The step of using the quantization model optimized with the calibration dataset to perform super-resolution reconstruction of low-resolution images includes: The low-resolution image is input into the quantization model optimized by the calibration dataset. The quantization model optimized using the calibration dataset obtains high-resolution images through forward inference. In the YCbCr color space, the Y channel of the high-resolution image is compared with the Y channel of a standard high-resolution image to calculate the peak signal-to-noise ratio and the structural similarity index.

7. A super-resolution reconstruction device, characterized in that, The device includes: The acquisition module is used to acquire the pre-trained floating-point super-resolution model and calibration dataset; An initialization module is used to perform sort-based quantization boundary initialization based on the calibration dataset, including: statistically sorting the weights and activation values ​​of the floating-point super-resolution model using the calibration dataset, and determining the quantization interval based on the percentile method according to the sorted data distribution; A quantization module is used to perform bias-compensated quantization based on the quantization interval, including: calculating the bias compensation mean and applying an adaptive offset to the weight and activation value data of the floating-point super-resolution model. The calculation of the bias compensation mean and the application of the adaptive offset to the weight and activation value data of the floating-point super-resolution model includes: using... Calculate the mean of the bias compensation The As the upper bound of the quantization interval, the As the lower bound of the quantization interval, the The mean offset parameter is learned based on the calibration dataset; for each data element ,when At that time, Replace with The The offset parameters are learned based on the calibration dataset; The reconstruction module is used to perform super-resolution reconstruction of low-resolution images using a quantization model optimized with the calibration dataset.

8. An apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

9. A storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images