A method for image super-resolution reconstruction based on asymptotic restoration of detail information

By utilizing the Sobel operator to extract edge features in image super-resolution reconstruction and combining it with a context feature enhancement module and a feature refinement module, the problems of low reconstruction accuracy and high computational complexity in existing technologies are solved, achieving efficient image detail restoration and texture clarity enhancement.

CN115272066BActive Publication Date: 2026-11-17KUNMING UNIV OF SCI & TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210715960.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-23
Publication Date
2026-11-17
Estimated Expiration
2042-06-23

AI Technical Summary

Technical Problem

Existing image super-resolution reconstruction methods suffer from problems such as low reconstruction algorithm accuracy, insufficient utilization of feature information, and loss of texture details. Furthermore, methods based on deep neural networks can only achieve good reconstruction performance at extremely high network depths, resulting in high computational complexity and poor scene adaptability.

Method used

We adopt a method based on asymptotic restoration of detail information. We use the Sobel operator to extract edge features and combine a context feature enhancement module and a feature refinement module. Through feature alignment, attention module, feature fusion and adaptive upsampling mechanism, we gradually restore image content and context details. We use L1 function and gradient loss to optimize image reconstruction.

Benefits of technology

It improves the performance of image super-resolution networks, clearly restores image texture and edge features, reduces computational complexity, and enhances the ability to recover image detail information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115272066B_ABST
    Figure CN115272066B_ABST
Patent Text Reader

Abstract

This invention relates to an image super-resolution reconstruction method based on asymptotic restoration of detail information, belonging to the field of image super-resolution reconstruction technology. The method addresses problems such as edge information loss and visual artifacts in learning-based image super-resolution reconstruction algorithms. First, edge features in the SR network are transferred to the SR network through a feature alignment module and an attention module, respectively, preserving features with high information content while suppressing features with low information content. Then, a context feature enhancement module and a feature refinement module fuse features from different network depths. The residual feature refinement module effectively helps the network retain richer image content and texture details while enhancing lost detail and context information. Finally, a super-resolution image result is obtained. Compared with existing super-resolution methods, this method significantly improves the performance of super-resolution image detail information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to an image super-resolution reconstruction method based on asymptotic restoration of detail information, belonging to the field of image processing technology. Background Technology

[0002] Image super-resolution is an important topic in digital image processing. Simply put, image super-resolution represents the amount of information, which can be understood as the number of pixels in an inch of image. Single-image super-resolution reconstruction (SISR) aims to obtain a high-resolution reconstructed image output from a low-resolution image input, maximizing the recovery of image details and texture information while increasing its spatial resolution. From the perspective of signal entropy, this is a process of increasing entropy. Without reference or prior knowledge, this is an ill-posed problem of reconstructing the original true signal from a lossy, degraded signal. From early interpolation-based methods to current learning-based methods, because the human eye is not sensitive to high-frequency details, these algorithms only optimize image reconstruction from the perspective of visual features, ignoring the rich information and high-frequency details in the real image, and failing to consider the loss of detail information in deep networks. Therefore, SISR tasks often suffer from low reconstruction accuracy, insufficient utilization of data features, and loss of texture details. High-precision single-image super-resolution reconstruction methods have significant application value in medical imaging, surveillance, satellite aerial imagery, image processing, pattern recognition, and computational sub-visual neighborhoods.

[0003] To address the issues of low accuracy, insufficient utilization of feature information, and loss of details in the aforementioned reconstruction algorithms, with the continuous development of deep learning in recent years, methods based on deep neural networks have been introduced into the SISR problem to optimize it with high quality, reduce the computational complexity of reconstruction, and improve the scene adaptability of the algorithm. However, simple neural networks require extremely deep networks to achieve good reconstruction performance, and convolutional methods are only effective for the local receptive field of the image. Therefore, methods that improve performance by deepening the network can only reconstruct optimized results that conform to human visual perception, and their performance is often poor for some advanced pattern recognition tasks. Summary of the Invention

[0004] This invention provides an image super-resolution reconstruction method based on asymptotic restoration of detail information. It utilizes the Sobel operator to extract edge features and transfers them to an image super-resolution network to enhance the high-frequency information of the image. Within the super-resolution network, a progressive feature restoration module, consisting of a context feature enhancement module and a feature refinement module, is used to help the network recover image content and contextual detail information to generate a super-resolution image.

[0005] The technical solution adopted in this invention is: An image super-resolution reconstruction method based on asymptotic restoration of detail information is characterized by the following steps: Step 1: Obtain the training dataset: Preprocess the original high-resolution images to obtain low-resolution image datasets at different scales; Step 2: Extract edge features from the image: Obtain the low-resolution images from the low-resolution image dataset in Step 1 using the Sobel method. Edge features are features that enhance image edge information. ; Step 3: Feature extraction from the training dataset in Step 1: Use a feature extractor consisting of two 3×3 convolutions and BN+ReLU to obtain shallow features. ; Step 4: Feature Matching: Design a feature alignment module to match the edge features from Step 2. Shallow features of Step 3 To perform matching, a graph matching operation is performed on the feature groups of any two images, and similar features are weighted to generate features. ; Step 5: Obtain informative features: Use an attention module to process the edge features from Step 2. and shallow features in Step 3 The process involves retaining features with high information content and suppressing features with low information content to obtain the desired features. ; Step 6: Obtain deep features: First, fuse the features from Step 4 using the feature fusion module. and the features of Step 5 Generate fusion features Then, a 3×3 convolution is used to extract the fused features. That is, generating deep features ; Step 7: Obtain contextual features: First, enhance the deep features from Step 6 using the first contextual feature enhancement module. Generate first-level detail features Then, the first residual feature refinement module gradually recovers the first layer of detailed features from the first context feature enhancement module. and the shallow features of Step 3 To generate contextual features ; Step 8: Obtain detailed contextual features: Gradually recover the contextual features from Step 7 using a feature recovery module consisting of multiple interconnected contextual feature enhancement modules and residual feature refinement modules. and the shallow features of Step 3 This generates the final contextual detail features. ; Step 9: Obtain super-resolution images: Use an adaptive upsampling mechanism consisting of two 3×3 convolutions, ReLU, and sub-pixel convolutions to process the low-resolution images at different scales from Step 1 and the contextual detail features from Step 8. Generate super-resolution images ; Step 10: Perform loss operation on the image: Apply the L1 function to the original high-resolution image from Step 1. Super-resolution images with Step9 The reconstruction loss and the original high-resolution image from Step 1 are calculated. Super-resolution image with Step 11 Edge loss is used to improve the performance of super-resolution networks.

[0006] Specifically, the Step 3 shallow features The specific steps for extraction are as follows: Shallow features The extraction is shown in the following formula:

[0007] In the formula This represents a shallow feature extraction function, consisting of two convolutional layers with a kernel size of 3×3 and a BN+ReLU layer. This indicates a low-resolution image.

[0008] Specifically, in Step 4, the edge features from Step 2 are transferred using a feature alignment module. This feature alignment module adjusts the edge features and the shallow features from Step 3. The process involves stitching together images, measuring similarity, and assigning weights to these similar features. These similar features are then transferred to the lower-resolution image. The specific steps are as follows:

[0009]

[0010]

[0011]

[0012]

[0013]

[0014]

[0015] In the formula, These represent edge features, edge feature extractor, shallow features, shallow feature extractor, feature fusion operation, fused edge features and shallow features, fused alignment features, and activation function, respectively. The weights of the alignment features are represented. The weights represent the shallow features. Both are represented as alignment features.

[0016] Specifically, the specific steps of the attention module in Step 5 are as follows:

[0017]

[0018]

[0019] In the formula, `Conv` and `Conv` represent 1x1 and 3x3 convolution operations, respectively. A represents the rich features of information, and A represents the attention weight map.

[0020] Specifically, the steps of Step 6 are as follows: Generation of fusion features:

[0021] Generation of deep features: .

[0022] Specifically, the steps in Step 7 to enrich the context features through the first context feature enhancement module and the first residual feature refinement module are as follows: The information enhanced by the first context feature enhancement module is as follows:

[0023] In the formula, This represents the detailed features of the first layer of the network. This represents a context feature enhancement function.

[0024] The first residual feature refinement module restores the context information, and the specific steps are as follows:

[0025]

[0026] In the formula, and ( (These represent the size of the patch block in the first layer of the network, respectively) and both represent the intermediate feature maps of the first layer of the network. and These represent the detailed features and shallow features of the first layer of the network, respectively. and This represents the weight graph of the first layer of the network. This indicates a channel-level connection operation. It is the ReLU activation function. This represents element-wise multiplication. This indicates a broadcast operation, where weights are broadcast and replicated along the channel dimension.

[0027] Contextual features of the network's first layer obtained from the feature complementation module composition. The specific steps are as follows: .

[0028] Specifically, the specific steps of Step 8 in restoring the context information through the feature-based step-by-step recovery module are as follows: The context feature enhancement module enhances the information in the first layer of the network, and the specific operations are as follows:

[0029] In the formula, Indicates the network number Detailed features of the layer Indicates network depth. Indicates the network number Contextual feature enhancement function of the layer; The residual feature refinement module operates as follows:

[0030]

[0031] In the formula, and ( Indicates the network number The size of the layer patch block represents the network's first layer. Intermediate feature map of the layer, and They represent the network's first and second generations respectively. Contextual features and shallow features of a layer and Indicates the network number Layer weight graph This indicates a channel-level connection operation. It is the ReLU activation function. This represents element-wise multiplication. This indicates a broadcast operation, where the weights are broadcast and replicated along the channel dimension. The output obtained from the feature complementation module composition is the first... Feature map of layer The specific steps are as follows:

[0032] The final enhanced detail features of the context enhancement module for

[0033] In the formula, Indicates the network number Contextual feature enhancement function of the layer Indicates the network number Detailed features of the layer.

[0034] The generation context features of the final output of the residual refinement module :

[0035] In the formula, and All represent the network's first Intermediate feature map of the layer The feature recovery module ultimately outputs contextual detail features. for:

[0036] In the formula, .

[0037] Specifically, the adaptive upsampling mechanism in Step 9 operates as follows:

[0038] In the formula, I SR Super-resolution images, Representing contextual details, This indicates a 1x1 convolution operation, and ↑ indicates upsampling.

[0039] Specifically, the steps of the loss function in Step 10 are as follows: Total loss function:

[0040] In the formula, and Let represent the hyperparameters of the reconstruction loss and the hyperparameters of the edge loss, respectively. These represent reconstruction loss and edge loss, respectively.

[0041] Reconstruction Loss: The first loss is the reconstruction loss.

[0042] In the formula For high-resolution images The length, width, and height of the object. Using L1 loss, it has been shown to be more effective and converge more easily than L2 loss; Gradient Loss: To generate reliable high-resolution images, image gradient maps are introduced into the image super-resolution method. The image gradient map specifies the directional changes in color within the image, corresponding to high-frequency details. Furthermore, the gradient map represents the location and size of image details. Super-resolution images can explicitly recover these high-frequency components guided by the gradient map. Regularization methods introduce additional gradient-based constraints to aid in the recovery of high-frequency details. For accurate image recovery, edges and textures should appear particularly realistic. The image gradient is used as the regularization term in the loss function.

[0043] In the formula, For gradient detectors, The gradient map of the super-resolution image. This is the gradient map of a high-resolution image.

[0044] The beneficial effects of this invention are: 1. This invention is a single-image super-resolution method based on convolutional neural networks (CNNs). This method utilizes a feature alignment module to measure the similarity between high-frequency features and low-resolution image features, fusing highly similar features into the super-resolution network to enhance the network's detail information; simultaneously, it utilizes a feature progressive recovery module to progressively recover information from networks of different depths. By using these two modules as the main components of the super-resolution network, the performance of the image super-resolution network is improved, resulting in visually clearer image texture and edge features. Attached Figure Description

[0045] Figure 1 This is a flowchart of an image super-resolution reconstruction method based on asymptotic restoration of detail information.

[0046] Figure 2 This is a schematic diagram of an image super-resolution network. Figure 3 A schematic diagram of the context feature enhancement module; Figure 4 This is a schematic diagram of the feature refinement module. Detailed Implementation

[0047] To better illustrate the technical solution of the present invention, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0048] Example 1: As Figure 1 This is a schematic diagram of the process of the method of the present invention. Step 1: Obtain the training dataset: Preprocess the original high-resolution images to obtain low-resolution image datasets at different scales; Step 2: Extract edge features from the image: Obtain the low-resolution images from the low-resolution image dataset in Step 1 using the Sobel method. Edge features are features that enhance image edge information. ; Step 3: Feature extraction from the training dataset in Step 1: Use a feature extractor consisting of two 3×3 convolutions and BN+ReLU to obtain shallow features. ; Step 4: Feature Matching: Design a feature alignment module to match the edge features from Step 2. Shallow features of Step 3 To perform matching, a graph matching operation is performed on the feature groups of any two images, and similar features are weighted to generate features. ; Step 5: Obtain informative features: Use an attention module to process the edge features from Step 2. and shallow features in Step 3 The process involves retaining features with high information content and suppressing features with low information content to obtain the desired features. ; Step 6: Obtain deep features: First, fuse the features from Step 4 using the feature fusion module. and the features of Step 5 Generate fusion features Then, a 3×3 convolution is used to extract the fused features. That is, generating deep features ; Step 7: Obtain contextual features: First, enhance the deep features from Step 6 using the first contextual feature enhancement module. Generate first-level detail features Then, the first residual feature refinement module gradually recovers the first layer of detailed features from the first context feature enhancement module. and the shallow features of Step 3 To generate contextual features ; Step 8: Obtain detailed contextual features: Gradually recover the contextual features from Step 7 using a feature recovery module consisting of multiple interconnected contextual feature enhancement modules and residual feature refinement modules. and the shallow features of Step 3 This generates the final contextual detail features. ; Step 9: Obtain super-resolution images: Use an adaptive upsampling mechanism consisting of two 3×3 convolutions, ReLU, and sub-pixel convolutions to process the low-resolution images at different scales from Step 1 and the contextual detail features from Step 8. Generate super-resolution images ; Step 10: Perform loss operation on the image: Apply the L1 function to the original high-resolution image from Step 1. Super-resolution images with Step9 The reconstruction loss and the original high-resolution image from Step 1 are calculated. Super-resolution image with Step 11 Edge loss is used to improve the performance of super-resolution networks.

[0049] Furthermore, the Step 3 shallow features The specific steps for extraction are as follows: Shallow features The extraction is shown in the following formula:

[0050] In the formula This represents a shallow feature extraction function, consisting of two convolutional layers with a kernel size of 3×3 and a BN+ReLU layer. This indicates a low-resolution image.

[0051] Furthermore, in Step 4, the edge features from Step 2 are transferred using a feature alignment module. Specifically, the feature alignment module adjusts the edge features and the shallow features from Step 3. The process involves stitching together images, measuring similarity, and assigning weights to these similar features. These similar features are then transferred to the lower-resolution image. The specific steps are as follows:

[0052]

[0053]

[0054]

[0055]

[0056]

[0057]

[0058] In the formula, These represent edge features, edge feature extractor, shallow features, shallow feature extractor, feature fusion operation, fused edge features and shallow features, fused alignment features, and activation function, respectively. The weights of the alignment features are represented. The weights represent the shallow features. Both are represented as alignment features.

[0059] Furthermore, the specific steps of the attention module in Step 5 are as follows:

[0060]

[0061]

[0062] In the formula, `Conv` and `Conv` represent 1x1 and 3x3 convolution operations, respectively. A represents the rich features of information, and A represents the attention weight map.

[0063] Furthermore, the specific steps of Step 6 are as follows: Generation of fusion features:

[0064] Generation of deep features:

[0065] Furthermore, the specific steps of Step 7 in enriching the context features through the first context feature enhancement module and the first residual feature refinement module are as follows: The information enhanced by the first context feature enhancement module is as follows:

[0066] In the formula, This represents the detailed features of the first layer of the network. This represents a context feature enhancement function.

[0067] The first residual feature refinement module restores the context information, and the specific steps are as follows:

[0068]

[0069] In the formula, and ( (These represent the size of the patch block in the first layer of the network, respectively) and both represent the intermediate feature maps of the first layer of the network. and These represent the detailed features and shallow features of the first layer of the network, respectively. and This represents the weight graph of the first layer of the network. This indicates a channel-level connection operation. It is the ReLU activation function. This represents element-wise multiplication. This indicates a broadcast operation, where weights are broadcast and replicated along the channel dimension.

[0070] Contextual features of the network's first layer obtained from the feature complementation module composition. The specific steps are as follows: .

[0071] Furthermore, the specific steps of Step 8 in restoring the context information through the feature-based step-by-step recovery module are as follows: The context feature enhancement module enhances the information in the first layer of the network, and the specific operations are as follows:

[0072] In the formula, Indicates the network number Detailed features of the layer Indicates network depth. Indicates the network number Contextual feature enhancement function of the layer; The residual feature refinement module operates as follows:

[0073]

[0074] In the formula, and ( Indicates the network number The size of the layer patch block represents the network's first layer. Intermediate feature map of the layer, and They represent the network's first and second generations respectively. Contextual features and shallow features of a layer and Indicates the network number Layer weight graph This indicates a channel-level connection operation. It is the ReLU activation function. This represents element-wise multiplication. This indicates a broadcast operation, where the weights are broadcast and replicated along the channel dimension. The output obtained from the feature complementation module composition is the first... Feature map of layer The specific steps are as follows:

[0075] The final enhanced detail features of the context enhancement module for

[0076] In the formula, Indicates the network number Contextual feature enhancement function of the layer Indicates the network number Detailed features of the layer.

[0077] The generation context features of the final output of the residual refinement module :

[0078] In the formula, and All represent the network's first Intermediate feature map of the layer The feature recovery module ultimately outputs contextual detail features. for:

[0079] In the formula, .

[0080] Furthermore, the adaptive upsampling mechanism in Step 9 operates as follows:

[0081] In the formula, I SR Super-resolution images, Representing contextual details, This indicates a 1x1 convolution operation, and ↑ indicates upsampling.

[0082] Furthermore, the specific steps of the loss function in Step 10 are as follows: Total loss function:

[0083] In the formula, and Let represent the hyperparameters of the reconstruction loss and the hyperparameters of the edge loss, respectively. These represent reconstruction loss and edge loss, respectively.

[0084] Reconstruction Loss: The first loss is the reconstruction loss.

[0085] In the formula For high-resolution images The length, width, and height of the object. Using L1 loss, it has been shown to be more effective and converge more easily than L2 loss; Gradient Loss: To generate reliable high-resolution images, image gradient maps are introduced into the image super-resolution method. The image gradient map specifies the directional changes in color within the image, corresponding to high-frequency details. Furthermore, the gradient map represents the location and size of image details. Super-resolution images can explicitly recover these high-frequency components guided by the gradient map. Regularization methods introduce additional gradient-based constraints to aid in the recovery of high-frequency details. For accurate image recovery, edges and textures should appear particularly realistic. The image gradient is used as the regularization term in the loss function.

[0086] In the formula, For gradient detectors, This is the gradient map of the super-resolution image. This is the gradient map of a high-resolution image.

[0087] In this embodiment, experiments were conducted on the DIV2K dataset, and PSNR and SSIM were compared with state-of-the-art super-resolution reconstruction methods.

[0088] In our experiments, we used the DIV2K dataset, which contains 800 high-quality RGB training images and is widely used for image restoration tasks. For evaluation, we used five widely used benchmark datasets: Set5, Set14, BSD100, Urban100, and MANGO109. The training set consisted of 800 pairs, the validation set consisted of 200 pairs, and a scaling factor of ×2 was considered in the experiments.

[0089] Implementation Details: To obtain the LR DIV2K training images, this invention uses double-edge interpolation in STSR and downscales the HR images using a scaling factor of 2. HR image patches of size 192×192 are randomly cropped from the HR images and used as input to the model of this invention, with the mini-batch size set to 16. For data augmentation, this invention performs random horizontal flipping and 90-degree rotation. and The weighting coefficients are 1, , and The batch size is set to 16. The model of this invention is based on the ADAM optimizer and momentum parameters. They were trained together. The initial learning rate was set to... ,Every The performance is halved in each iteration. The proposed method, STSR, is implemented in PyTorch 1.8.0 and runs on a GTX 3060 GPU. The proposed network outperforms existing super-resolution reconstruction networks.

[0090] Table 1. Quantitative Analysis of Double Upsampling Results Table 1 Quantitative analysis of double upsampling results

[0091] Note: Bold text represents the optimal value for each line. This invention proposes an image super-resolution reconstruction method based on asymptotic restoration of detail information. This method uses the Sobel operator to extract edge features from the image for detail information in the image super-resolution network. A feature alignment module is designed in the super-resolution network to match edge features with shallow features. Graph matching is performed on feature groups of any two images, assigning weights to similar features and using an attention module to retain patches with more information and suppress patches with less information. A feature progressive restoration module, a context feature enhancement module, and a feature refinement module fuse features from different depths of the network. To avoid information loss, a residual feature refinement module is designed, where deep information is processed by a residual feature refinement module composed of a context feature fusion module, a weight learning module, and a feature complementation module. This module helps the network retain richer image content and texture details, enhancing detail and context information. The weight learning module adaptively adjusts the weights of the feature complementation module. An adaptive upsampling mechanism utilizes a flexible valve to handle LR images at different scales, realizing a flexible SR model. Compared with existing super-resolution methods, this method significantly improves the performance of super-resolution image detail information.

[0092] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. An image super-resolution reconstruction method based on detail information asymptotic recovery, characterized in that: Includes the following steps: Step 1: Obtain the training dataset: For the original high-resolution images Preprocessing is performed to obtain low-resolution image datasets at different scales; Step 2: Extract edge features from the image: Obtain the low-resolution images from the low-resolution image dataset in Step 1 using the Sobel method. Structural features are used as features to enhance image edge information, i.e., edge features. ; Step 3: Feature extraction from the training dataset in Step 1: Use a feature extractor consisting of two 3×3 convolutions and BN+ReLU to obtain shallow features. ; Step 4: Feature Matching The design feature alignment module applies to the edge features of Step 2. Shallow features of Step 3 To perform matching, a graph matching operation is performed on the feature groups of any two images, and similar features are weighted to generate features. ; Step 5: Obtain informative features: Use an attention module to process the edge features from Step 2. and shallow features in Step 3 The process involves retaining features with high information content and suppressing features with low information content to obtain the desired features. ; Step 6: Obtain deep features: First, fuse the features from Step 4 using the feature fusion module. and the features of Step 5 Generate fusion features Then, a 3×3 convolution is used to extract the fused features. That is, generating deep features ; Step 7: Obtain contextual features: First, enhance the deep features from Step 6 using the first contextual feature enhancement module. Generate first-level detail features Then, the first residual feature refinement module gradually recovers the first layer of detailed features from the first context feature enhancement module. and the shallow features of Step 3 To generate contextual features ; Step 8: Obtain detailed contextual features: Gradually recover the contextual features from Step 7 using a feature recovery module consisting of multiple interconnected contextual feature enhancement modules and residual feature refinement modules. and the shallow features of Step 3 This generates the final contextual detail features. ; Step 9: Obtain super-resolution images: Use an adaptive upsampling mechanism consisting of two 3×3 convolutions, ReLU, and sub-pixel convolutions to process the low-resolution images at different scales from Step 1 and the contextual detail features from Step 8. Generate super-resolution images ; Step 10: Perform loss operation on the image: Apply the L1 function to the original high-resolution image from Step 1. Super-resolution images with Step9 The reconstruction loss and the original high-resolution image from Step 1 are calculated. Super-resolution images with Step9 Use edge loss to improve the performance of super-resolution networks; In Step 4, the edge features from Step 2 are transferred using a feature alignment module. This feature alignment module adjusts the edge features and the shallow features from Step 3. The process involves stitching together images, measuring similarity, and assigning weights to these similar features. These similar features are then transferred to the lower-resolution image. The specific steps are as follows: ; ; ; ; ; ; In the formula, These represent edge features, edge feature extractor, shallow features, feature fusion operation, fused edge features and shallow features, fused aligned features, and activation function, respectively. The weights of the alignment features are represented. Both are represented as alignment features.

2. The image super-resolution reconstruction method based on asymptotic restoration of detail information according to claim 1, characterized in that: The shallow features of Step 3 The specific steps for extraction are as follows: Shallow features The extraction is shown in the following formula: ; In the formula This represents a shallow feature extraction function, which consists of two convolutional layers with a kernel size of 3×3 and a BN+ReLU layer.

3. The image super-resolution reconstruction method based on asymptotic restoration of detail information according to claim 1, characterized in that: The specific steps of the Step 5 attention module are as follows: ; ; ; In the formula, `Conv` and `Conv` represent 1x1 and 3x3 convolution operations, respectively. A represents information-rich features, and A represents the attention weight map.

4. The image super-resolution reconstruction method based on asymptotic restoration of detail information according to claim 3, characterized in that, The specific steps of Step 6 are as follows: Generation of fusion features: ; Generation of deep features: 。 5. The image super-resolution reconstruction method based on asymptotic restoration of detail information according to claim 1, characterized in that: The specific steps of Step 7, which enriches the context features through the first context feature enhancement module and the first residual feature refinement module, are as follows: The information enhanced by the first context feature enhancement module is as follows: ; In the formula, This represents the detailed features of the first layer of the network. This represents a context-feature enhancement function; The first residual feature refinement module restores the context information, and the specific steps are as follows: ; ; In the formula, and They all represent intermediate feature maps from the first layer of the network. These represent the sizes of the first-layer patch block network, and These represent the detailed features and shallow features of the first layer of the network, respectively. and This represents the weight graph of the first layer of the network. This indicates a channel-level connection operation. It is the ReLU activation function. This represents element-wise multiplication. This indicates a broadcast operation, where the weights are broadcast and replicated along the channel dimension. Contextual features of the network's first layer obtained from the feature complementation module composition. The specific steps are as follows: 。 6. The image super-resolution reconstruction method based on asymptotic restoration of detail information according to claim 1, characterized in that, The specific steps of Step 8, which involves restoring the context information through the feature-based stepwise recovery module, are as follows: The residual feature refinement module operates as follows: ; ; In the formula, and All represent the network's first Intermediate feature map of the layer, Indicates the network number The size of the layer patch block, and They represent the network's first and second generations respectively. Contextual features and shallow features of a layer and Indicates the network number Layer weight graph This indicates a channel-level connection operation. It is the ReLU activation function. This represents element-wise multiplication. This indicates a broadcast operation, where the weights are broadcast and replicated along the channel dimension. The output obtained from the feature complementation module composition is the first... Feature map of layer The specific steps are as follows: ; The final enhanced detail features of the context enhancement module for ; In the formula, Indicates the network number Context feature enhancement function of layer Indicates the network number Detailed features of the layer; The generation context features of the final output of the residual refinement module : ; In the formula, and All represent the network's first Intermediate feature map of the layer, The feature recovery module ultimately outputs contextual detail features. for: ; In the formula, .

7. The image super-resolution reconstruction method based on asymptotic restoration of detail information according to claim 1, characterized in that, The adaptive upsampling mechanism in Step 9 operates as follows: ; In the formula, Indicates super-resolution image, Indicates contextual details. This indicates a 1x1 convolution operation, and ↑ indicates upsampling.

8. The image super-resolution reconstruction method based on asymptotic restoration of detail information according to claim 1, characterized in that, The specific steps of the loss function in Step 10 are as follows: Total loss function: ; In the formula, and Let represent the hyperparameters of the reconstruction loss and the hyperparameters of the edge loss, respectively. These represent reconstruction loss and edge loss, respectively. Reconstruction Loss: The first loss is the reconstruction loss. ; In the formula For high-resolution images The length, width, and height of the image are determined using L1 loss, which has been shown to be more effective and convergent than L2 loss. Gradient Loss: To generate reliable high-resolution images, image gradient maps are introduced into the image super-resolution method. The image gradient map specifies the directional changes in color within the image, corresponding to high-frequency image details. Furthermore, the gradient map represents the location and size of image details. Guided by the gradient map, the super-resolution image explicitly recovers these high-frequency components. A regularization method introduces additional gradient-based constraints to aid in the recovery of high-frequency details and ensure correct image restoration. The image gradient is used as the regularization term in the loss function. ; In the formula, For gradient detectors, The gradient map of the super-resolution image. This is the gradient map of a high-resolution image.

Citation Information

Patent Citations

  • Image super-resolution reconstruction method and system and computer storage medium

    CN110458758A

  • Super-resolution reconstruction method for single image from rough to fine

    CN111161146A