An image super-resolution method based on binary classification and uncertainty elimination

Abstract

The application relates to the technical field of image processing, in particular to an image super-resolution method based on binary classification and uncertainty elimination, which comprises the following steps: acquiring a low-resolution image; inputting the low-resolution image into a trained image super-resolution model to obtain a high-resolution super-resolution image; the image super-resolution model comprises an encoder, a binary decoder and an iterative uncertainty elimination module; the application adopts a binary method to convert an integer pixel value into an 8-bit label, thereby applying a BCE loss to image super-resolution and solving the boundary blur problem caused by the L1 loss distribution hypothesis error; the uncertainty of outputs of different branches is evaluated and gradually eliminated, the high-resolution image is iteratively optimized in a binary space, the model can adaptively suppress unreliable prediction and strengthen reliable information when reconstructing pixel values, and the defects of excessive smoothing and detail loss of a traditional method under a single loss constraint are overcome.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to an image super-resolution method based on binary classification and uncertainty elimination. Background Technology

[0002] Image super-resolution is a key technology in the field of image processing. It refers to the process of recovering the original or high-quality image by processing and analyzing observed images or image data. In real-world scenarios, due to limitations of imaging equipment, interference during transmission, and the complexity of the shooting environment, the acquired images often suffer from problems such as blurriness, noise, and missing data. The mission of image super-resolution is to revitalize these damaged or low-quality images, restoring their original clarity and rich details to the greatest extent possible.

[0003] In existing technologies, loss functions used to train deep neural networks are divided into two categories: classification-based loss functions and regression-based loss functions, used for classification tasks and regression tasks, respectively. Since pixel values ​​are continuous, most methods treat image super-resolution as a regression task and apply regression loss functions such as L1 loss to minimize the super-resolution error. However, due to the computational characteristics of L1 loss, it still presents some significant challenges for image inpainting tasks. On the one hand, from a statistical perspective, L1 loss is a maximum likelihood estimate of high-quality pixels with a Laplacian prior distribution, which conflicts with integer pixel values ​​in digital images. Typically, for super-resolution of complex textures, a simple high-quality image distribution prior is insufficient, potentially leading to poor performance in image boundary regions. On the other hand, the gradient of L1 loss is fixed at 1, resulting in slow convergence speeds for most image inpainting methods.

[0004] To overcome the limitations of L1 loss in image inpainting tasks, perceptual loss utilizes pre-trained networks (such as VGG) to compute differences in the feature space, thereby improving the perceptual performance of super-resolution textures. Furthermore, adversarial loss employs a training method based on generative adversarial networks (GANs), making the distribution of the inpainted image closer to that of a natural image, effectively mitigating the problem of overly smoothed super-resolution boundaries.

[0005] Therefore, existing technologies ignore the inherent defects of regression loss functions in statistical prior matching and gradient characteristics, making it urgent to solve the technical problem of poor performance in complex texture reconstruction and boundary restoration. Summary of the Invention

[0006] In view of this, this application discloses an image super-resolution method based on binary classification and uncertainty elimination to solve the problems in the prior art, including:

[0007] A low-resolution image is acquired and input into a trained image super-resolution model to obtain a high-resolution super-resolution image. The image super-resolution model includes: an encoder, a dichotomy decoder, and an iterative uncertainty elimination module (IUE). The training process of the image super-resolution model includes:

[0008] S1. Obtain the image super-resolution dataset and preprocess it to obtain the preprocessed dataset; the image super-resolution dataset includes low-resolution images and their ground truth (GT) images.

[0009] S2. Encode the images in the preprocessed dataset using an encoder to obtain feature embeddings. ;

[0010] S3, using a binary decoder (Dichotomy Decoder) Decode;

[0011] The Dichotomy Decoder is a dual-branch binary decoder used to decode a decimal image into eight-bit binary, splitting the image into channels. and The calculation is divided into two parts, L1 loss and BCE loss respectively.

[0012] S4. Based on the Iterative Uncertainty Elimination Module (IUE), eliminate... and The uncertainty is used to obtain the super-resolution image; including:

[0013] S41, respectively and Considering both the true and approximate distributions, uncertainty calculations are performed.

[0014] S42. Iteratively update the high-resolution super-resolution image in binary representation based on uncertainty;

[0015] S43. Perform a binary number conversion on the updated binary high-resolution super-resolution image to obtain the super-resolution image;

[0016] S5. Calculate the total loss function value based on the super-resolution image and the ground truth image. Update the parameters of the image super-resolution model based on the total loss function value. When the total loss function value is minimized, the trained image super-resolution model is obtained.

[0017] This application uses a binary search method to convert integer pixel values ​​into 8-bit labels, thereby applying the BCE loss to image super-resolution and solving the boundary blurring problem caused by the incorrect L1 loss distribution assumption.

[0018] By designing an iterative uncertainty elimination module, the uncertainty of different branch outputs is evaluated and gradually eliminated. The high-resolution image is iteratively optimized in binary space, enabling the model to adaptively suppress unreliable predictions and enhance credible information when reconstructing pixel values. This significantly improves the detail expression and edge clarity of the reconstruction results, overcoming the shortcomings of traditional methods that are prone to oversmoothing and loss of detail under a single loss constraint.

[0019] The method designed in this application can be flexibly embedded into any currently popular image super-resolution method, and has transferability. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the training process of the image super-resolution model in the embodiments of this application;

[0021] Figure 2 This is a schematic diagram illustrating the image processing flow of the image super-resolution model in this embodiment of the application;

[0022] Figure 3 This is a schematic diagram of the image super-resolution model in the embodiments of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, features, and advantages of this application clearer and to facilitate a better understanding of the technical solutions of this application by those skilled in the art, the following detailed description of this application is provided in conjunction with the accompanying drawings and embodiments.

[0024] Example 1:

[0025] This embodiment includes an image super-resolution method based on binary classification and uncertainty elimination, comprising: acquiring a low-resolution image; inputting the low-resolution image into a trained image super-resolution model to obtain a high-resolution super-resolution image; the image super-resolution model includes: an encoder, a binary decoder, and an iterative uncertainty elimination module (IUE); a schematic diagram of the image super-resolution model processing the image is shown below. Figure 2 As shown.

[0026] The training process of an image super-resolution model is as follows: Figure 1 As shown, it includes:

[0027] S1. Obtain the image super-resolution dataset X and preprocess it to obtain the preprocessed dataset. The image super-resolution dataset includes low-resolution images and their ground truth (GT) images, and a preprocessing dataset. It includes a training set and a test set, used for training and testing the model, respectively. The preprocessing includes: normalization, random rotation, and random cropping. After preprocessing, the low-resolution image size is... GT image size is .

[0028] S2. Encode the images in the preprocessed dataset using an encoder to obtain feature embeddings. .

[0029] In this embodiment, the encoder uses the Mamba backbone network. The encoder based on the Mamba backbone network processes the preprocessed dataset. The data processing includes: performing preliminary feature transformation on the input low-resolution image through a linear mapping layer to obtain an initial feature representation; and inputting the initial feature representation into multiple cascaded Mamba modules for feature modeling.

[0030] The Mamba module, based on the Selective State Space Model, achieves long-distance dependency modeling of image spatial information through linear recursive modeling of sequence data. Within each Mamba module, a gating mechanism dynamically weights input features to enhance key feature responses and suppress redundant information. A residual connection structure is used to progressively fuse the outputs of each module layer to alleviate the gradient vanishing problem during deep network training and improve feature representation capabilities. The features processed by multiple Mamba modules are normalized and channel-integrated to obtain the final feature embedding. .

[0031] Through the above encoding process, the encoder can effectively capture the global context information and local detail features of the image with low computational complexity, providing high-quality feature representation for the high-precision reconstruction of the subsequent binary decoder. In some embodiments, encoders with other structures can also be used.

[0032] S3, using a binary decoder (Dichotomy Decoder) Decoding is performed, and the L1 loss and BCE loss are calculated respectively; the binary decoder, such as... Figure 3 As shown in (a), a binary decoder employing a dual-branch architecture is used to decode an image represented in decimal into eight bits, splitting the image into two parts along the channel dimension, and calculating the L1 loss and BCE loss respectively; in this embodiment, the image size before decoding is [size missing]. The size of the decoded image is .

[0033] In this embodiment, the binary decoder includes a B2D branch and a D2B branch, and the binary decoder processes the data as follows:

[0034] S31, to Upsampling is performed to obtain an upsampled image. .

[0035] S32, Upsample the image Convolution is performed to obtain a high-resolution super-resolution image in binary representation. and , respectively and Input the B2D branch and the D2B branch.

[0036] B2D branch will Convert to decimal, use the ground truth image as the true value, and calculate... The L1 loss; the D2B branch converts the GT image to binary and calculates... The BCE loss is used. The L1 loss and BCE loss are then used to train the model subsequently via backpropagation and gradient descent.

[0037] S4. Based on the Iterative Uncertainty Elimination Module (IUE), eliminate... and The uncertainty is used to obtain the super-resolution image; including:

[0038] S41, respectively and The uncertainty is calculated by treating both the true and approximate distributions. This embodiment will... and KL divergence as a measure of uncertainty ; The calculation formula is:

[0039]

[0040] S42. Based on the uncertainty, iteratively update the high-resolution super-resolution image in binary representation to eliminate the uncertainty and obtain the updated binary high-resolution super-resolution image; the calculation formula for the iterative update is as follows:

[0041]

[0042]

[0043]

[0044] in, Indicates the first The binary representation of the recovered image at the next iteration. The variance represents the uncertainty. Indicates the compensation deviation term. Represents a learnable neural network, initial values , , .

[0045] As a preferred embodiment, in order to ensure numerical stability during the backpropagation process, the calculation formula uses... replace That is, gradient-stable Replace the gradient unstable The more stable update formula is obtained as follows:

[0046]

[0047] Furthermore, considering It can be defined as a sequence model, possessing efficient sequence modeling capabilities, and employs the state-space equations of the Mamba model. The formula for prediction is:

[0048]

[0049]

[0050] in, This indicates the updated compensation deviation. The parameter is The mapping function, Indicates time step The hidden states are represented by A, B, and C, which represent learnable parameters in the Mamba model, and cat represents the concatenation operation.

[0051] In this embodiment, after two iterations, an updated binary high-resolution super-resolution image is obtained.

[0052] S43. Perform a binary number conversion on the updated binary high-resolution super-resolution image to obtain the super-resolution image.

[0053] Specifically, the iteratively updated binary high-resolution image is first reorganized along the channel dimension, arranging the 8 binary bits corresponding to each pixel in a preset order to form a corresponding binary encoding sequence; then, the binary encoding sequence at each pixel position is weighted and summed, and the k-th binary value is associated with its corresponding weight. Multiply the values ​​and sum all bits to obtain the decimal representation of the corresponding pixel; then constrain the numerical range of the obtained decimal pixel values ​​to make them fall within the preset pixel value range; finally, after completing the above transformation for all pixels, reconstruct a three-channel high-resolution image as the final super-resolution image.

[0054] S5. Calculate the total loss function value based on the super-resolution image and the ground truth image. Update the parameters of the image super-resolution model based on the total loss function value. When the total loss function value is minimized, the trained image super-resolution model is obtained. The total loss function includes L1 loss and BCE loss. In this embodiment, the total loss function... The formula is:

[0055]

[0056]

[0057] in, Represents the total loss function. Indicates L1 loss, Represents a super-resolution image. Represents a GT image. This represents the loss function of the binary decoder. and Represents the weight parameters. This indicates the loss of BCE.

[0058] Example 2:

[0059] This embodiment includes an image super-resolution method based on binary classification and uncertainty elimination, comprising: acquiring a low-resolution image, inputting the low-resolution image into a trained image super-resolution model to obtain a high-resolution super-resolution image; the image super-resolution model includes: an encoder, a binary decoder, and an iterative uncertainty elimination module (IUE).

[0060] The difference from Example 1 is that this example will explain the image super-resolution method in conjunction with a specific image super-resolution model, such as... Figure 3 As shown, after the input image enters the image super-resolution model, it is encoded by the encoder into... The features are then input into the Dichotomy Decoder, and after upsampling and convolution mapping, a binary representation of the regression branch output is generated. and category branch output .in, Used for pixel-level regression, after B2D to decimal conversion, it is used for subsequent calculation of L1 loss with GT. Used for pixel-level classification, and subsequently for calculating BCE loss with GT. It should be noted that the GT here is the GT converted to binary representation by D2B.

[0061] Then and The input is fed into the IUE iterative uncertainty elimination module. First, the uncertainty between the two is calculated based on the KL divergence, and then initialization is performed. , , and Then, based on the above initial parameters, the results are updated multiple times through the iterative uncertainty elimination formula to reduce the prediction error.

[0062] Finally, the optimized binary representation is converted to a different number system to reconstruct a high-resolution image output. Through bi-branch modeling and uncertainty-guided iterative optimization, the image detail recovery capability and reconstruction accuracy are improved.

[0063] Finally, it should be noted that the above description only depicts some embodiments of this application. For those skilled in the art, various changes, modifications, substitutions, and variations can be conceived of these embodiments without departing from the principles and spirit of this application. The scope of protection of this application is defined by the appended claims and their equivalents, and all the above-mentioned behaviors should be covered within the scope of protection of this application.

Claims

1. An image super-resolution method based on binary classification and uncertainty elimination, characterized in that, include: Acquire a low-resolution image, input the low-resolution image into a trained image super-resolution model, and obtain a high-resolution super-resolution image; The image super-resolution model includes: an encoder, a binary decoder, and an iterative uncertainty elimination module (IUE); the training process of the image super-resolution model includes: S1. Obtain the image super-resolution dataset and preprocess it to obtain the preprocessed dataset; the image super-resolution dataset includes low-resolution images and their ground truth (GT) images. S2. Encode the images in the preprocessed dataset using an encoder to obtain feature embeddings. ; S3, using a binary decoder (Dichotomy Decoder) Decode; The Dichotomy Decoder is a dual-branch binary decoder used to decode a decimal image into eight-bit binary, splitting the image into channels. and The problem is divided into two parts, and the L1 loss and BCE loss are calculated separately. S4. Based on the Iterative Uncertainty Elimination Module (IUE), eliminate... and The uncertainty is used to obtain the super-resolution image; including: S41, respectively and Considering both the true and approximate distributions, uncertainty calculations are performed. S42. Iteratively update the high-resolution super-resolution image in binary representation based on uncertainty; S43. Perform a binary number conversion on the updated binary high-resolution super-resolution image to obtain the super-resolution image; S5. Calculate the total loss function value based on the super-resolution image and the ground truth image. Update the parameters of the image super-resolution model based on the total loss function value. When the total loss function value is minimized, the trained image super-resolution model is obtained.

2. The image super-resolution method based on binary classification and uncertainty elimination according to claim 1, characterized in that, The binary decoder includes a B2D branch and a D2B branch; The binary decoder processes data including: S31, to Upsampling is performed to obtain an upsampled image. ; S32, Upsample the image Convolution is performed to obtain a high-resolution super-resolution image in binary representation. and , respectively and Input B2D branch and D2B branch; B2D branch will Convert to decimal, use the ground truth image as the true value, and calculate... L1 loss; The D2B branch converts the GT image to binary and calculates... BCE loss.

3. The image super-resolution method based on binary classification and uncertainty elimination according to claim 1, characterized in that, The uncertainty calculation is performed, and KL divergence is used as a measure of uncertainty.

4. The image super-resolution method based on binary classification and uncertainty elimination according to claim 1, characterized in that, The high-resolution super-resolution image based on uncertainty is iteratively updated using the following formula: ;in, Indicates the first The binary representation of the recovered image at the next iteration. The variance represents the uncertainty. This indicates the compensation deviation term.

5. The image super-resolution method based on binary classification and uncertainty elimination according to claim 4, characterized in that, The initial value for iterative updates is , , .

6. The image super-resolution method based on binary classification and uncertainty elimination according to claim 4, characterized in that, The uncertainty variance is updated using the following formula: ;in, This represents the KL divergence.

7. The image super-resolution method based on binary classification and uncertainty elimination according to claim 4, characterized in that, The update formula for the compensation deviation term is: ;in, This represents a learnable neural network.

8. The image super-resolution method based on binary classification and uncertainty elimination according to claim 4, characterized in that, In the updated formula, the following is adopted: replace .

9. The image super-resolution method based on binary classification and uncertainty elimination according to claim 4, characterized in that, The state-space equations of the Mamba model are used to... Make predictions.

10. The image super-resolution method based on binary classification and uncertainty elimination according to claim 1, characterized in that, The parameters of the image super-resolution model are updated based on the loss function value. The formula for the loss function is: ； ； in, Represents the total loss function. Indicates L1 loss, Represents a super-resolution image. Represents a GT image. This represents the loss function of the binary decoder. and Represents the weight parameters. This indicates the loss of BCE.

Images