Image super-resolution reconstruction method based on multi-scale exposure normalization and double-branch feature aggregation

Abstract

The invention discloses an image super-resolution reconstruction method based on multi-scale exposure normalization and double-branch feature aggregation, and relates to the technical field of image reconstruction, and the method comprises the steps: obtaining a to-be-processed image; constructing a super-resolution network model based on a Swin Transform architecture, performing feature reconstruction on the to-be-processed image through the super-resolution network model, and outputting a super-resolution image; wherein the super-resolution network model extracts shallow feature information of the to-be-processed image, exposure normalization processing is performed on the to-be-processed image based on the shallow feature information, and after color correction and feature enhancement are performed on exposure normalization features, the super-resolution image is mapped and reconstructed. According to the method, the problems of important detail missing, color distortion, structure distortion and the like in the image exposure correction process in an overexposure scene are solved, so that high-definition and high-resolution image reconstruction is realized.

Description

Technical Field

[0001] This invention relates to the field of image reconstruction technology, and more specifically to an image super-resolution reconstruction method based on multi-scale exposure normalization and bi-branch feature aggregation. Background Technology

[0002] Image super-resolution refers to the process of reconstructing a high-resolution image from a low-resolution image. In recent years, image super-resolution has attracted significant attention in the field of computer vision, playing a crucial role in real-world applications such as security monitoring, authentication, gaze estimation, and social media. The performance of image recognition largely depends on the quality of the input image. However, in many cases, images suffer from various types of degradation. For example, surveillance cameras operating in high-intensity lighting environments may introduce unknown noise due to limited light sensitivity. Mobile devices performing face unlock may lose facial details due to camera shake or device limitations. Drones or remote imaging systems may produce blurry or unevenly exposed images due to atmospheric turbulence or lens limitations. Furthermore, images shared on social media platforms may suffer from quality degradation due to compression, beautification, or filtering. Especially in excessively bright outdoor environments, facial images often appear pale and lack facial contours.

[0003] Previous studies on image super-resolution have largely relied on prior image information, such as geometric priors, for image reconstruction. Some methods have explored using facial thermal images to locate facial features and improve the quality of image super-resolution. Other methods utilize facial landmarks to reconstruct facial details. However, when dealing with overexposed images, the images recovered by these methods generally suffer from limitations such as low brightness and missing textures.

[0004] Previous work on general image exposure correction has typically relied on Retinex. SICE proposed a network based on Retinex to recover reflectance and illuminance maps separately, and then combined these components to generate the final output. Guo divided the input image into luminance and chrominance components, applying different smoothing operators to each for correction. However, directly applying these exposure correction methods to image super-resolution often leads to loss of texture details or color inconsistencies in the reconstructed image.

[0005] In conclusion, simply integrating exposure correction and image super-resolution reconstruction together is unlikely to produce satisfactory visual quality. This is because single-task methods focus on specific types of degradation, while simple combinations are susceptible to coupled degradation. In this case, the processing in the preceding stage inevitably introduces uncertain interference to the subsequent stages, which accumulates and amplifies over time, ultimately leading to the loss of important information.

[0006] Therefore, how to provide an image super-resolution reconstruction method based on multi-scale exposure normalization and bi-branch feature aggregation to achieve high-definition, high-resolution image reconstruction is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides an image super-resolution reconstruction method based on multi-scale exposure normalization and bi-branch feature aggregation, which addresses the problems of loss of important details, color distortion and structural distortion in the image exposure correction process in overexposed scenes, so as to achieve high-definition and high-resolution image reconstruction.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: an image super-resolution reconstruction method based on multi-scale exposure normalization and bi-branch feature aggregation, comprising:

[0009] Obtain the image to be processed;

[0010] A super-resolution network model based on the Swin Transformer architecture is constructed. The super-resolution network model is used to reconstruct features of the image to be processed and output a super-resolution image.

[0011] The super-resolution network model extracts shallow feature information of the image to be processed, performs exposure normalization processing on the image to be processed based on the shallow feature information, and performs color correction and feature enhancement on the exposure normalized features before mapping and reconstructing them into a super-resolution image.

[0012] Preferably, shallow feature information of the image to be processed is extracted by sequentially connecting a normalization layer, a 1×1 convolutional layer, and a ReLU activation layer;

[0013] The multi-scale exposure normalization module is used to perform exposure normalization and correction on the image to be processed, restoring it to an image under normal lighting conditions;

[0014] By combining a color space conversion module and a CNN-Transformer architecture, color correction and feature enhancement are performed on exposure-normalized features.

[0015] The feature map is reconstructed into a super-resolution image by using sequentially connected 3×3 convolutional layers, pixel rearrangement layers, and ReLU activation layers.

[0016] Preferably, the multi-scale exposure normalization module is implemented by merging the multi-scale feature extraction unit and the exposure normalization unit step by step;

[0017] The multi-scale feature extraction unit is used to extract feature information of different scales of the image to be processed; the exposure normalization module is used to map features of different exposure levels to an exposure-invariant feature space.

[0018] Preferably, the multi-scale feature extraction unit targets large-scale shallow features by first adjusting the size through a 1×1 convolution, then extracting shallow features through a 5×5 convolution, then mapping two sets of dense residual convolutional layers to fuse local and global features, and finally performing feature fusion by connecting a 5×5 convolutional layer with the residual.

[0019] Preferably, the exposure normalization module uses an instance normalization layer to coarsely align the extracted feature information at different scales to obtain normalized features;

[0020] Specifically, the correlation between normalized features and unnormalized original multi-scale features is extracted in both spatial and channel dimensions through compensation.

[0021] Preferably, in the spatial dimension, the normalized features are... Unnormalized raw multiscale features With attention maps A and A respectively n Integrate;

[0022] Among them, A and A n Representation of features and characteristics The correlation between them;

[0023] The formula derived from spatial attention is:

[0024]

[0025] Where W0 represents the weight matrix of the convolution kernel, * represents the convolution operation, and [,] represents the concatenation operation;

[0026] Spatial features I′ and I n ′ is calculated using the following formula:

[0027]

[0028] At the channel dimension, through attention weight A f For spatial features I′ and I n Adaptive integration is performed on the connection features of ′ to obtain the integrated result.

[0029] Preferably, the color space conversion module converts the RGB information of the image into the YCbCr color space, where Y represents the luminance channel, and the color channels Cb and Cr represent saturation and hue, respectively.

[0030] The three components of brightness, saturation, and hue are separated, and color correction and adjustment are performed on the color channels, followed by deep feature extraction.

[0031] Preferably, in the CNN-Transformer combined architecture, the CNN branch uses multiple spatial and channel reconstruction convolutional layers for convolutional processing to obtain local features;

[0032] The Swin Transformer branch includes multiple Aggregate CNNs and Transformer blocks to capture globally dependent features in an image and aggregate local features provided by the CNN branch.

[0033] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses an image super-resolution reconstruction method based on multi-scale exposure normalization and dual-branch feature aggregation. The beneficial effects of the present invention are as follows: The core of the super-resolution network model proposed in the present invention lies in the multi-scale exposure normalization module, the color space conversion module, and the dual-branch feature aggregation stage, which can effectively process face images in overexposed scenes and achieve high-quality super-resolution reconstruction, thus making it more suitable for practical applications of super-resolution reconstruction of overexposed face images; The image super-resolution reconstruction method in overexposed scenes of the present invention can be used for face image super-resolution reconstruction, thereby contributing to downstream tasks such as facial expression recognition and identity verification, and enhancing the practical application capabilities of downstream tasks. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0035] Figure 1 The diagram shows the super-resolution network model and the structures of MENM and ACTB provided in the embodiments of the present invention.

[0036] Figure 2(a) is a schematic diagram of the first layer structure of the MFEM module provided in an embodiment of the present invention.

[0037] Figure 2(b) is a schematic diagram of the second layer structure of the MFEM module provided in the embodiment of the present invention.

[0038] Figure 2(c) is a schematic diagram of the third layer structure of the MFEM module provided in the embodiment of the present invention.

[0039] Figure 3 This is a schematic diagram of the CCCM module provided in an embodiment of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] This invention discloses an image super-resolution reconstruction method based on multi-scale exposure normalization and bi-branch feature aggregation, comprising: acquiring an image to be processed;

[0042] Construct an EFSRNet super-resolution network model based on the Swing Transformer architecture, such as Figure 1 As shown, the super-resolution network model is used to reconstruct features of the image to be processed, and outputs a super-resolution image.

[0043] The super-resolution network model extracts shallow feature information of the image to be processed, performs exposure normalization processing on the image to be processed based on the shallow feature information, and performs color correction and feature enhancement on the exposure normalized features before mapping and reconstructing them into a super-resolution image.

[0044] Specifically, this invention is applied to super-resolution reconstruction of face images. LR-HR face image data pairs were obtained through overexposure simulation and degradation model processing for experiments. The widely used CelebA-HQ and FFHQ datasets were used in the experiments. Specifically, 8000 face images were randomly selected as the training dataset, and 1000 face images were used as the validation dataset. For the test dataset, 500 face images were randomly selected from CelebA-HQ, FFHQ, Helen, and SCUT-FBP for quantitative and qualitative evaluation. Regarding evaluation metrics, three of the most commonly used metrics were selected: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS).

[0045] Specifically, shallow feature information of the image to be processed is extracted by sequentially connecting a normalization layer, a 1×1 convolutional layer, and a ReLU activation layer;

[0046] The multi-scale exposure normalization module is used to perform exposure normalization and correction on the image to be processed, restoring it to an image under normal lighting conditions;

[0047] By combining a color space conversion module and a CNN-Transformer architecture, color correction and feature enhancement are performed on exposure-normalized features.

[0048] The feature map is reconstructed into a super-resolution image by using sequentially connected 3×3 convolutional layers, pixel rearrangement layers, and ReLU activation layers.

[0049] Specifically, the processed LR-HR face image data is trained and inferred using the EFSRNet network. The EFSRNet architecture is based on the Swing Transformer network. First, a shallow feature extraction stage is used to extract shallow feature information from the input image, which involves applying a LayerNorm (normalization) operation to the input image. This is followed by 1×1 convolutional operations and ReLU activation layers. This process reduces the model's complexity and minimizes computational resource consumption in the subsequent exposure normalization and bi-branch color space feature aggregation stages.

[0050] I SFE =ReLu(Conv(LN(I LR )));

[0051] Among them, I LR This represents a low-resolution input image, LN represents normalization, CONV represents convolution, and ReLU represents an activation layer.

[0052] Subsequently, after the shallow feature extraction stage, shallow feature I SFE The image is fed into the Multi-Scale Exposure Normalization (MENM) module, which mainly performs exposure normalization and correction on the face image to effectively restore it to normal lighting conditions.

[0053] I MENM =MENM(I SFE ).

[0054] Next, in the dual-branch color space feature aggregation stage, the color space transformation module (CCCM) and the CNN-Transformer combined architecture are used to process the exposure-normalized face features I. MENM Color correction and feature enhancement are performed. This process ensures natural skin tone, defined facial contours, and clear texture details.

[0055] Finally, in the image reconstruction stage, the facial feature maps are reconstructed into an HR image. Specifically, 3×3 convolutions, pixel rearrangement layers, and ReLU activation layers are used to obtain the final output of a standard super-resolution image.

[0056]

[0057] in, This indicates facial features after color correction and feature enhancement; HRM stands for High Resolution Reconstruction Module.

[0058] Specifically, the multi-scale exposure normalization module is implemented by the step-by-step fusion of the multi-scale feature extraction unit and the exposure normalization unit;

[0059] The multi-scale feature extraction unit is used to extract feature information of different scales of the image to be processed; the exposure normalization module is used to map features of different exposure levels to an exposure-invariant feature space.

[0060] Specifically, the entire Multi-Scale Exposure Normalization (MENM) module is implemented by the step-by-step fusion of the multi-scale feature extraction module and the exposure normalization module, such as... Figure 1 The diagram shown is the structure of EFSRNet. LR First, shallow feature extraction is performed to obtain I. SFE Then it enters the multi-scale exposure normalization module to obtain I. MENM Then, it enters the dual-branch color space feature aggregation stage to obtain Finally, I was reconstructed using the HR reconstruction module. SR image.

[0061] Specifically: the deepest features at the bottom layer are first summed element-wise with the features at the second layer, and then added to the feature maps of the shallowest features at the top layer. Considering the balance between complexity and accuracy, two Exposure Normalization and Compensation (ENC) modules are selected for each multi-scale exposure feature processing branch. This architecture enables the model to represent feature information of different exposure levels at different scales, while also ensuring the model's inference efficiency, avoiding complex computational redundancy, and significantly improving the model's ability to correct exposure in facial images. The specific formula can be expressed as:

[0062]

[0063] Where I′1, I′2, and I′3 represent multi-scale facial features, ENC represents the exposure normalization and compensation module, and MFEM represents the multi-scale feature extraction module. MENM This represents the normalized facial features after exposure.

[0064] Specifically, the multi-scale feature extraction unit targets large-scale shallow features by first adjusting the size through a 1×1 convolution, then extracting shallow features through a 5×5 convolution, then mapping two sets of dense residual convolutional layers to fuse local and global features, and finally performing feature fusion by connecting a 5×5 convolutional layer with the residual.

[0065] Specifically, the exposure normalization module uses an instance normalization layer to coarsely align the extracted feature information at different scales to obtain normalized features;

[0066] Specifically, the correlation between normalized features and unnormalized original multi-scale features is extracted in both spatial and channel dimensions through compensation.

[0067] Specifically, for illumination normalization, both large-scale and small-scale features should be used simultaneously. Therefore, this embodiment of the invention designs a Multiscale Feature Extraction (MFEM) module, which extracts facial image features at different scales before normalizing for different exposure levels. Specifically, this embodiment of the invention's MFEM designs different architectures for feature extraction at different scales. The structure of the MFEM is as follows... Figures 2(a)-2(c) The diagram shows the structure of the multi-scale feature extraction module. Figure 2(a) shows the structure of the first layer in MENM, i.e., MFEM1; Figure 2(b) shows the structure of the second layer, i.e., MFEM2; and Figure 2(c) shows the structure of the bottom layer, i.e., MFEM3. Residual connections have proven effective in image restoration tasks. For large-scale shallow features, the first 1×1 convolution resizes the image and is followed by a 5×5 convolution to extract shallow features. Then, two sets of dense residual convolutional layers are mapped onto the shallow features, where the fusion of local and global features makes training more stable. The last 5×5 convolutional layer is connected to the previous residuals to further fuse the features, giving it rich facial feature information. ReLU layers are added after some convolutional layers to accelerate convergence.

[0068] Specifically, facial images with different exposure levels exhibit different appearances, and the processes for correcting them to normal exposure vary significantly. Therefore, this embodiment of the invention introduces a simplified exposure normalization module to map features of different exposure levels to an exposure-invariant feature space. The compensation portion of this module integrates features that have not undergone normalization. Specifically, in the multi-exposure normalization part, an instance normalization layer is first used to coarsely align the extracted multi-scale information. Assuming the multi-scale information is I, instance normalization is performed as follows:

[0069]

[0070] Here, μ() and σ() represent the mean and standard deviation computed across spatial dimensions for each channel and each sample, while γ and β are parameters learnable from the data. Instance normalization is performed in the multi-scale information space, thereby reducing representational discrepancies between them.

[0071] Normalization of discriminative information inevitably leads to the loss of illumination information. To address this issue, the compensation section of this invention integrates the unprocessed multi-scale features of the instance after normalization to ensure information integrity. The compensation section comprehensively extracts the correlation between multi-scale features and normalized features in both spatial and channel dimensions. These correlations help guide the integration of lost information from the multi-scale features.

[0072] Specifically, in the spatial dimension, normalized features will be... Unnormalized raw multiscale features With attention maps A and A respectively n Integrate;

[0073] Among them, A and A n Representation of features and characteristics The correlation between them;

[0074] The formula derived from spatial attention is:

[0075]

[0076] Where W0 represents the weight matrix of the convolution kernel, * represents the convolution operation, and [,] represents the connection operation. Similarly, A can also be obtained through the spatial attention derivation formula. n ;

[0077] Spatial features I′ and I n ′ is calculated using the following formula:

[0078]

[0079] After calculating the spatial features, these two features are integrated in the spatial dimension;

[0080] At the channel dimension, through attention weight A f For spatial features I′ and I n Adaptive integration is performed on the connection features of ′ to obtain the integrated result.

[0081] Specifically, spatial features I′ and I are further integrated at the channel dimension. n ′, through attention weight A f For spatial features I′ and I n Adaptive integration is performed on the connection features of ′, and A is derived through a channel attention mechanism similar to SE (compression and activation). f ;

[0082] A f The derivation formula can be expressed as:

[0083] A f=sigmoid(W2·relu(W1*pool([I,I n ]))).

[0084] Specifically, the color space conversion module converts the RGB information of the image into the YCbCr color space, where Y represents the luminance channel, and the color channels Cb and Cr represent saturation and hue, respectively.

[0085] The three components of brightness, saturation, and hue are separated, and color correction and adjustment are performed on the color channels, followed by deep feature extraction.

[0086] In the dual-branch color space feature aggregation stage, specifically, the color space transformation convolution converts the RGB information of the image into the YCbCr color space, where luminance (Y) represents the luminance channel, and the color components (Cb and Cr) correspond to saturation and hue. This embodiment of the invention separates these three components, processing only luminance (Y) to suppress luminance information while preserving the color components (Cb, Cr) to prevent unexpected luminance changes. In the dual-branch feature extraction stage, color correction and adjustment are performed on the CbCr channel to continuously reduce color distortion caused by the previous stage. This method isolates the luminance issue, focusing on depth feature extraction to ensure structural integrity and facial detail restoration. The structure of CCCM is as follows: Figure 3 The diagram shows the structure of the color space conversion module. The curved curve in the diagram represents the suppression of the Y channel.

[0087] Specifically, in the CNN-Transformer combined architecture, the CNN branch uses multiple spatial and channel reconstruction convolutional layers to perform convolutional processing and obtain local features;

[0088] The Swin Transformer branch includes multiple CNNs and Transformer blocks to capture globally dependent features in an image and aggregate local features provided by the CNN branch.

[0089] Specifically, after mitigating the unavoidable color distortion problem introduced by previous methods, this invention found that a single CNN in the SR network can only process local regions of an image and is limited by the size of the convolutional kernel. It struggles to achieve satisfactory efficiency in modeling long-range dependencies. However, Liu has demonstrated that a moving window architecture based on the Swin Transformer can capture long-range dependencies. Therefore, combining these two methods, this architecture is more suitable for solving complex degradation recovery super-resolution tasks. Thus, this invention combines CNN and Transformer, including a Swin Transformer branch and a CNN branch. This efficient hybrid dual-branch architecture aggregates rich features, combining local features from the CNN with long-range dependency features captured by the Swin Transformer. During feature aggregation, the two branches complement each other. Specifically, the CNN branch performs convolutions using multiple spatial and channel reconstruction convolutions (SCConv), while the Swin Transformer branch consists of multiple CNNs and Transformer blocks (ACTBs) that capture global dependency features in facial images and aggregate local features provided by the CNN branch.

[0090]

[0091] Among them, I up I represents the facial features entering the convolution branch. down This represents the face features entering the Transformer branch. LN represents the normalization operation, CAB represents the channel attention module, LINB represents the local instance normalization module, and α and β represent small constants.

[0092] This invention provides a super-resolution method for overexposed face images based on multi-scale exposure normalization and bi-branch aggregation. Currently, no publicly available dataset is specifically designed for face super-resolution tasks under overexposed conditions. Therefore, to verify the effectiveness of this invention, overexposed simulation experiments were conducted on an existing publicly available HR face image dataset. When simulating overexposed conditions in real-world scenes, the RGB values ​​of each face image were adjusted pixel-by-pixel. The RGB value of each pixel was multiplied by the exposure value (EV) to obtain the required overexposed face image. To avoid a single overexposed condition and better simulate changes in lighting conditions, the EV was randomly adjusted within the range of [1.1, 2.0], meaning the original image was overexposed to 110% to 200%. To further simulate degradation in real-world scenes and obtain realistic LR face images, this invention adds noise to the face images using various degradation methods, including Gaussian noise, JPEG compression, camera sensor noise, motion blur, and downsampling. A random shuffling order consisting of seven numbers was designed. Considering that downsampling and motion blur are the most common degradation types, these operations were each performed twice. Since the shuffling is based on random numbers, different images will undergo different degradation sequences during processing, and all degradation methods will be applied. This approach helps increase data diversity, thereby improving the robustness of the model. Therefore, through the above operations, LR-HR image pairs in complex overexposed scenes were successfully obtained for subsequent experiments.

[0093] This invention employs the EFSRNet super-resolution network model, which utilizes a multi-scale exposure normalization module to effectively reduce the distance between different exposure levels, significantly lowering facial brightness. The color space conversion module resolves color distortion caused by exposure normalization in the previous process, ensuring accurate color correction. Simultaneously, the dual-branch feature aggregation stage effectively preserves the integrity of the facial structure and ensures the clarity of texture details after illumination adjustment, ultimately achieving excellent face image reconstruction results.

[0094] Specifically, the quantitative analysis of the experimental results is shown in Table 1. The embodiments of this invention present the results of each model tested on three evaluation metrics on four datasets (CelebAHQ, FFHQ, SCUT-FBP, and Helen). The method of the embodiments of this invention achieved the best PSNR, SSIM, and LPIPS scores on most evaluation datasets, demonstrating the strong competitive performance of the models. It is worth acknowledging that joint models generally outperform models designed for specific subtasks, even achieving best results on a smaller number of evaluation metrics. However, this does not necessarily indicate that such joint models effectively mitigate interference introduced by the preceding process. Furthermore, the embodiments of this invention demonstrate superior capabilities in reducing overexposure, preserving image details, and maintaining facial structure, further highlighting the advantages of the embodiments of this invention.

[0095] Specifically, qualitative analysis of the experimental results shows that the embodiments of the present invention effectively mitigate structural distortion caused by varying degrees of overexposure while preserving fine-grained texture details. In contrast, FSRNet introduces noticeable white patches in the facial region, while RepRFN struggles to maintain the integrity of the facial structure. Overall, the embodiments of the present invention achieve convincing results in restoring realistic and visually believable facial images. The proposed EFSRNet effectively restores fine textures and facial details while maintaining color consistency, achieving excellent visual quality.

[0096] In addition, ablation experiments were conducted on the CelebAHQ dataset in this embodiment of the invention. The ablation experiment results of EFSRNet can be seen in Table 2, which verifies the effectiveness of each component.

[0097] Specifically, to further demonstrate the effectiveness of the embodiments of the present invention, a downstream task of facial expression recognition was used for further evaluation. Specifically, DeepFace, one of the most representative facial expression recognition models, was selected to evaluate its recognition performance. KDEF was chosen as the test dataset. KDEF consisted of 35 male and 35 female volunteers, each displaying seven different facial expressions: happiness, sadness, surprise, fear, disgust, anger, and neutral. The specific evaluation process is as follows: First, the same overexposure processing technique was applied to KDEF. Then, each model used in the previous comparative experiments was used to reconstruct the image. Finally, the reconstructed image was input into DeepFace for expression recognition. As shown in Table 3, DeepFace achieved a recognition accuracy of 31.76% in the original KDEF (without overexposure). After overexposure, the recognition accuracy dropped to 23.67%. Among all models, the embodiments of the present invention showed the highest improvement in expression recognition accuracy in the restored KDEF, an improvement of 6.41%. This is sufficient to demonstrate the advantage of the embodiments of the present invention in restoring overexposed face images in the downstream task of facial expression recognition.

[0098] Table 1 shows the results of each model tested on the three evaluation metrics across the four datasets.

[0099]

[0100]

[0101] Table 2 Ablation Experiment Results of EFSRNet

[0102] MENM CCCM SC ACTB PSNR SSIM LPIPS √ √ √ 29.6554 0.9078 0.1487 √ √ √ 30.1265 0.9110 0.1479 √ √ √ 30.1539 0.9113 0.1466 √ √ √ 30.0047 0.9103 0.1455 √ √ √ √ 30.3741 0.9114 0.1460

[0103] Table 3. Comparison Experiment Results of Downstream Task Facial Recognition

[0104]

[0105] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0106] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image super-resolution reconstruction method based on multi-scale exposure normalization and bi-branch feature aggregation, characterized in that, include: Obtain the image to be processed; A super-resolution network model based on the Swin Transformer architecture is constructed. The super-resolution network model is used to reconstruct features of the image to be processed and output a super-resolution image. The super-resolution network model extracts shallow feature information of the image to be processed, performs exposure normalization processing on the image to be processed based on the shallow feature information, and performs color correction and feature enhancement on the exposure normalized features before mapping and reconstructing them into a super-resolution image.

2. The image super-resolution reconstruction method based on multi-scale exposure normalization and bi-branch feature aggregation according to claim 1, characterized in that, Shallow feature information of the image to be processed is extracted by sequentially connecting a normalization layer, a 1×1 convolutional layer, and a ReLU activation layer. The multi-scale exposure normalization module is used to perform exposure normalization and correction on the image to be processed, restoring it to an image under normal lighting conditions; By combining a color space conversion module and a CNN-Transformer architecture, color correction and feature enhancement are performed on exposure-normalized features. The feature map is reconstructed into a super-resolution image by using sequentially connected 3×3 convolutional layers, pixel rearrangement layers, and ReLU activation layers.

3. The image super-resolution reconstruction method based on multi-scale exposure normalization and bi-branch feature aggregation according to claim 2, characterized in that, The multi-scale exposure normalization module is implemented by the step-by-step fusion of the multi-scale feature extraction unit and the exposure normalization unit; The multi-scale feature extraction unit is used to extract feature information of different scales of the image to be processed; the exposure normalization module is used to map features of different exposure levels to an exposure-invariant feature space.

4. The image super-resolution reconstruction method based on multi-scale exposure normalization and bi-branch feature aggregation according to claim 3, characterized in that, The multi-scale feature extraction unit targets large-scale shallow features by first adjusting the size through a 1×1 convolution, then extracting shallow features through a 5×5 convolution, then mapping two sets of dense residual convolutional layers to fuse local and global features, and finally performing feature fusion by connecting a 5×5 convolutional layer with the residual.

5. The image super-resolution reconstruction method based on multi-scale exposure normalization and bi-branch feature aggregation according to claim 3, characterized in that, The exposure normalization module uses an instance normalization layer to coarsely align the extracted feature information at different scales to obtain normalized features; Specifically, the correlation between normalized features and unnormalized original multi-scale features is extracted in both spatial and channel dimensions through compensation.

6. The image super-resolution reconstruction method based on multi-scale exposure normalization and bi-branch feature aggregation according to claim 5, characterized in that, In the spatial dimension, normalized features Unnormalized raw multiscale features With attention maps A and A respectively n Integrate; Among them, A and A n Representation of features and characteristics The correlation between them; The formula derived from spatial attention is: Where W0 represents the weight matrix of the convolution kernel, * represents the convolution operation, and [,] represents the concatenation operation; Spatial features I′ and I n ′ is calculated using the following formula: At the channel dimension, through attention weight A f For spatial features I′ and I′ n Adaptive integration is performed on the connectivity features to obtain the integrated result.

7. The image super-resolution reconstruction method based on multi-scale exposure normalization and dual-branch feature aggregation according to claim 2, characterized in that, The color space conversion module converts the RGB information of the image into the YCbCr color space, where Y represents the luminance channel and Cb and Cr represent saturation and hue, respectively. The three components of brightness, saturation, and hue are separated, and color correction and adjustment are performed on the color channels, followed by deep feature extraction.

8. The image super-resolution reconstruction method based on multi-scale exposure normalization and bi-branch feature aggregation according to claim 2, characterized in that, In the CNN-Transformer combined architecture, the CNN branch uses multiple spatial and channel reconstruction convolutional layers to perform convolutional processing and obtain local features; The Swin Transformer branch includes multiple Aggregate CNN and Transformer blocks to capture globally dependent features in an image and aggregate local features provided by the CNN branch.

Images