Image super-resolution reconstruction method and system

By introducing a deformable window attention module in the image super-resolution reconstruction method, the window position is dynamically adjusted to capture long-distance features, solving the problem of poor performance of existing methods when processing important long-distance features, significantly improving image weight The effect of the structure.

CN119399027APending Publication Date: 2025-02-07GUANGDONG UNIV OF TECH
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411485946.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-23
Publication Date
2025-02-07

AI Technical Summary

Technical Problem

Existing Transformer-based image super-resolution methods do not perform well when processing important features from a distance, resulting in limited image reconstruction effects.

Method used

A super-resolution reconstruction method is proposed, using the Deformable Window Attention Block (DWAB) to dynamically adjust the window position and automatically locate important information areas in the image, so as to effectively capture the far away while maintaining the calculation efficiency. Distance-related feature information.

Benefits of technology

During the image reconstruction process, the key information at different locations in the image is fully utilized, avoiding the omission of important features due to the fixed window size, and significantly improving the effect of image reconstruction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119399027A_ABST
    Figure CN119399027A_ABST
Patent Text Reader

Abstract

The invention discloses an image super-resolution reconstruction method and system, and relates to the technical field of image processing. The method comprises the following steps: acquiring image data, and preprocessing the image data; constructing a super-resolution network model, dividing the preprocessed image data into a training set and a test set, performing image super-resolution reconstruction training on the super-resolution network model by using the training set, and testing the trained super-resolution network model by using the test set, after the test is passed, a trained super-resolution network model is obtained; and performing super-resolution reconstruction on the image data to be super-divided by using the trained super-resolution network model, and outputting a super-resolution image obtained by reconstruction. According to the method and the system, key information of different positions in the image can be fully utilized in the image reconstruction process, the situation that important features are missed due to the fixed window size is avoided, and the image reconstruction effect is effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image super-resolution reconstruction method and system. Background Art

[0002] Image super-resolution technology is a method of generating high-resolution images from low-resolution images. It has important research value and broad application prospects in the field of computer vision. With the continuous improvement of image and video quality requirements in modern society, image super-resolution technology is in urgent need in many practical scenarios. For example, in medical imaging, super-resolution technology can be used to improve the details of low-resolution medical images, helping doctors to make more accurate diagnoses; in the field of remote sensing, super-resolution can help improve the resolution of satellite images, making it easier to monitor and analyze subtle changes in large areas; in security monitoring, super-resolution technology can enhance the clarity of video images and more clearly identify facial features or license plate numbers.

[0003] Traditional image super-resolution methods mainly rely on interpolation algorithms, reconstruction methods based on sparse representation, etc. These methods often perform poorly in complex texture areas and have difficulty capturing high-frequency detail information. With the rapid development of deep learning, image super-resolution methods based on convolutional neural networks (CNNs) have received widespread attention. This type of method can adaptively learn the mapping relationship from low resolution to high resolution through large-scale data training, and has shown significant advantages in image detail restoration and high-frequency information reconstruction.

[0004] In recent years, image super-resolution methods based on Transformer structures have gradually become a research hotspot. Compared with traditional convolutional networks, Transformer has stronger global modeling capabilities and can better capture the long-range dependencies of images, thereby improving the quality of reconstruction. Especially in high-frequency texture recovery, detail preservation, and processing of complex structures, the Transformer framework has demonstrated its unique advantages. In the field of SISR, a noteworthy work based on Transformer is SwinIR, which uses Swin Transformer for image super-resolution. Unlike earlier works such as ViT, SwinIR overcomes problems such as the loss of high-frequency details and has achieved significant improvements in image super-resolution tasks. Subsequently, a series of works involving Transformer-based designs, such as Swin-FIR and ELAN, further improved SwinIR and achieved excellent SISR results.

[0005] Transformer-based SISR methods all use a window-based self-attention mechanism, where the computational complexity of the self-attention mechanism is quadratically related to the window size. Recent work has shown that the performance of the window-based attention mechanism is related to the number of key pixels contained in the window. Therefore, a direct way to improve the quality of super-resolution is to increase the size of the local window, but this will result in a high computational cost. To address this problem, some methods have tried to use carefully designed attention patterns to allow the window to include more pixels in the attention process. Although these manually designed attention patterns are effective, some important long-distance features may still be omitted during data processing, while less important features may be retained, which has high processor computation requirements and quadratically increases in computational complexity with the window size. In addition, the manually designed attention patterns perform poorly when processing important long-distance features, limiting the effect of image reconstruction. Summary of the invention

[0006] In order to overcome the problem that the manually designed attention mode in the prior art performs poorly when processing important features at a long distance, thus limiting the effect of image reconstruction, the present invention proposes an image super-resolution reconstruction method and system, which can fully utilize the key information at different positions in the image during the image reconstruction process, avoid the omission of important features due to a fixed window size, and effectively improve the effect of image reconstruction.

[0007] To achieve the purpose of the present invention, the present invention adopts the following technical solutions: A method for super-resolution image reconstruction, the method comprising the following steps: Acquiring image data, and preprocessing the image data; Constructing a super-resolution network model, and dividing the preprocessed image data into a training set and a test set, using the training set to perform image super-resolution reconstruction training on the super-resolution network model, and using the test set to test the trained super-resolution network model, and obtaining a trained super-resolution network model after the test passes; The trained super-resolution network model is used to perform super-resolution reconstruction on the image data to be super-resolved, and the reconstructed super-resolution image is output.

[0008] In the above technical scheme, by processing the acquired image data, the practicability and reliability of the image data can be effectively improved; after the constructed super-resolution network model is trained with the image data in the training set and tested with the test set, a super-resolution network model that effectively grasps the changes in the image data can be obtained, so that the image data to be super-resolution reconstructed can be reconstructed through the super-resolution network model, so as to make full use of the key information at different positions in the image during the image reconstruction process, avoid the omission of important features due to the fixed window size, and effectively improve the image reconstruction effect.

[0009] Furthermore, the process of preprocessing the image data includes: Extract the HR image from the image data, and downsample the HR image using a bilinear downsampling method to extract the corresponding LR image; The extracted HR images and LR images are combined into an LR-HR image dataset, and the image data in the LR-HR image dataset is normalized; The HR image represents a high-resolution image, and the LR image represents a low-resolution image.

[0010] In the above technical scheme, the bilinear downsampling method is used to downsample the HR image, which can improve the accuracy and speed of extraction in the process of extracting LR images; the image data in the LR-HR image data set is normalized, which can effectively improve the practicality and reliability of the image data.

[0011] Furthermore, the process of training the super-resolution network model for image super-resolution reconstruction using the training set includes: The constructed super-resolution network model includes a shallow feature extraction module, N deformable window attention modules, N convolutional feedforward networks and an upsampling reconstruction module; The shallow features in the LR image are extracted using the shallow feature extraction module; Each deformable window attention module is used to perform deep feature extraction on the extracted shallow feature map; Use each convolutional feed-forward network to adjust the weights of the extracted deep features; The extracted shallow features are fused with the deep features after the final weight adjustment, and the fused features are reconstructed using the upsampling reconstruction module, and the reconstructed image is output; The loss function is set to iteratively optimize the super-resolution network model during the training process, and the performance of the trained super-resolution network model is tested using the test set. When the loss function converges, the trained super-resolution network model is obtained.

[0012] In the above technical solution, the present invention proposes a deformable window attention block (DWAB) based on the design concept of deformable convolution and deformable attention. The module automatically locates the important information area in the image by dynamically adjusting the window position, and expands the local window attention mechanism to multi-window non-local attention, thereby effectively capturing relevant feature information at a long distance while maintaining computational efficiency.

[0013] Furthermore, the process of extracting the shallow feature map in the LR image using the shallow feature extraction module is as follows: A 3×3 convolution layer is set in the shallow feature extraction module, and the 3×3 convolution layer is used to extract shallow features of the input LR image. The expression is: ; in, represents the resolution of the LR image, represents a 3×3 convolutional layer, Represents shallow features.

[0014] Furthermore, the process of performing deep feature extraction on the extracted shallow feature map using each deformable window attention module includes: The deformable window attention module includes several groups of deformable receptive field depth extraction units; The shallow features extracted Input several groups of deformable receptive field depth extraction units to perform deep feature extraction, the expression is: ; in, Denotes a deformable receptive field depth extraction unit. The superscript n denotes the nth group of deformable receptive field depth extraction units.

[0015] Furthermore, the process of using each convolutional feedforward network to adjust the weights of the extracted deep features includes: Normalize and deeply convolve the deep features extracted by the deformable window attention module; The convolutional feedforward network introduces a channel attention mechanism, which adjusts the weights of each channel adaptively to adjust the weights of the deep features after normalization and deep convolution processing.

[0016] Furthermore, the expression for normalizing and deep convolution processing the deep features extracted by the deformable window attention module is: ; The weights of the deep features after normalization and deep convolution are adjusted, and the expression is: ; ; in, represents the normalization operation, represents the depthwise convolution operation, Representation characteristics The global average pooling value in the c channel, represents a linear projection, represents the final output feature of the feedforward network, and s represents the adjusted weight.

[0017] Furthermore, the process of reconstructing the image using the upsampling reconstruction module on the fused features includes: The global residual connection method is used to extract shallow features With deep features to integrate; The PixelShuffle layer is used as a decoder to reconstruct a high-resolution image of the fused features. The expression is: ; in, represents the upsampling reconstruction module, Represents the reconstructed high-resolution image.

[0018] Furthermore, the loss function L is set 1 The iterative training of the super-resolution network model during training is optimized, and the expression is: ; Where N represents the number of iterations. represents the high-resolution images in the test set, Represents the high-resolution image obtained after reconstruction.

[0019] In the above technical scheme, the shallow feature extraction module, the deformable window attention module and the upsampling reconstruction module in the super-resolution network model are iteratively optimized and trained, and a 3×3 convolution layer, several groups of deformable receptive field depth extraction units and a loss function are set in the process of iterative optimization training. In the process of iterative training of the super-resolution network model, the accuracy of the super-resolution network model in grasping the changes in image data can be further improved, so as to avoid the omission of important features due to the fixed window size, and effectively improve the image reconstruction effect; a convolutional feedforward network branch is connected after the deformable window attention module to more effectively encode and restore detail information. This branch helps to enhance the model's ability to capture high-frequency textures, making the model more accurate in processing complex details. A channel attention mechanism is introduced in the convolutional feedforward network branch. By adaptively adjusting the weights of each channel, the information exchange between feature channels is further strengthened, which not only improves the model's ability to capture important features, but also enhances the overall feature expression effect.

[0020] An image super-resolution reconstruction system, the system comprising: A data acquisition module, used for acquiring image data; A data processing module, used for preprocessing the image data; Model building module, used to build super-resolution network model; The data processing module is also used to divide the preprocessed image data into a training set and a test set, use the training set to perform image super-resolution reconstruction training on the super-resolution network model, use the test set to test the trained super-resolution network model, and obtain the trained super-resolution network model after the test passes; The super-resolution image reconstruction module is used to perform super-resolution reconstruction on the image data to be super-resolved using the trained super-resolution network model, and output the reconstructed super-resolution image.

[0021] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of an image super-resolution reconstruction method when executing the program.

[0022] A computer-readable storage medium stores a computer program, which implements the steps of an image super-resolution reconstruction method when executed by a processor.

[0023] Compared with the prior art, the present invention has the following beneficial effects: The present invention proposes an image super-resolution reconstruction method and system, which can effectively improve the practicability and reliability of image data by processing the acquired image data; after the constructed super-resolution network model is trained with image data in a training set and tested with a test set, a super-resolution network model that effectively grasps the changes in image data can be obtained, so that the image data to be super-resolution reconstructed can be reconstructed through the super-resolution network model, so that key information at different positions in the image can be fully utilized during the image reconstruction process, and the situation of missing important features due to a fixed window size is avoided, thereby effectively improving the effect of image reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flowchart of a method for super-resolution reconstruction of an image provided by an embodiment of the present application; Figure 2 A data flow diagram of a super-resolution network model provided in an embodiment of the present application; Figure 3 A schematic diagram of the principle of a deformable window attention module provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of an image super-resolution reconstruction system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Preferred embodiments of the present invention are provided in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0027] Embodiment 1: This embodiment provides an image super-resolution reconstruction method. Figure 1 , the method comprises the following steps: Step S1: acquiring image data and preprocessing the image data; Step S2: construct a super-resolution network model, and divide the preprocessed image data into a training set and a test set, use the training set to perform image super-resolution reconstruction training on the super-resolution network model, and use the test set to test the trained super-resolution network model, and obtain a trained super-resolution network model after the test passes; Step S3: Use the trained super-resolution network model to perform super-resolution reconstruction on the image data to be super-resolved, and output the reconstructed super-resolution image.

[0028] In step S1, the process of preprocessing the image data includes: Extract the HR image from the image data, and downsample the HR image using a bilinear downsampling method to extract the corresponding LR image; The extracted HR images and LR images are combined into an LR-HR image dataset, and the image data in the LR-HR image dataset is normalized; The HR image represents a high-resolution image, and the LR image represents a low-resolution image.

[0029] Specifically, HR images and LR images are paired to construct network training image pairs, and all LR-HR images are cropped and data augmented. For all training LR-HR image pair data, the original images with the same content are captured at the same position. More specifically, a 48×48 window is captured on the LR image, and then the window on the HR image is enlarged by the same multiple of the data set scaling factor at the same position of the HR image for capture. All LR-HR image pair data are converted into tensor data structures and normalized. The original image pairs are randomly rotated or rotated. The original image pairs are randomly horizontally flipped.

[0030] It can be understood that downsampling HR images using a bilinear downsampling method can improve the accuracy and speed of extraction in the process of extracting LR images; normalizing the image data in the LR-HR image dataset can effectively improve the practicality and reliability of the image data.

[0031] In this embodiment, by processing the acquired image data, the practicability and reliability of the image data can be effectively improved; after the constructed super-resolution network model is trained with the image data in the training set and tested with the test set, a super-resolution network model that effectively grasps the changes in the image data can be obtained, so that the image data to be super-resolution reconstructed can be reconstructed through the super-resolution network model, so as to make full use of the key information at different positions in the image during the image reconstruction process, avoid the omission of important features due to the fixed window size, and effectively improve the image reconstruction effect.

[0032] Embodiment 2: This embodiment further describes steps S2 to S3 in the first embodiment, as follows: In step S2, see Figure 2 The process of using the training set to train the super-resolution network model for image super-resolution reconstruction includes: The constructed super-resolution network model includes a shallow feature extraction module, N deformable window attention modules, N convolutional feedforward networks and an upsampling reconstruction module; S21: Set N rounds of training times and use the shallow feature extraction module to extract shallow features in the LR image; S22: Use each deformable window attention module to perform deep feature extraction on the extracted shallow feature map; S23: Use each convolutional feed-forward network to adjust the weights of the extracted deep features; S24: Fusing the extracted shallow features with the deep features after the final weight adjustment, reconstructing the image of the fused features using an upsampling reconstruction module, and outputting the reconstructed image; S25: Setting a loss function to iteratively optimize the super-resolution network model during the training process, and using a test set to perform a performance test on the trained super-resolution network model. When the loss function converges, a trained super-resolution network model is obtained.

[0033] As a preferred embodiment, in step S21, the process of extracting the shallow feature map in the LR image using the shallow feature extraction module is as follows: A 3×3 convolution layer is set in the shallow feature extraction module, and the 3×3 convolution layer is used to extract shallow features of the input LR image. The expression is: ; in, represents the resolution of the LR image, represents a 3×3 convolutional layer, Represents shallow features.

[0034] In step S22, participate Figure 2 and Figure 3 ,The process of using each deformable window attention module to extract deep features from the extracted shallow feature map includes: The deformable window attention module includes several groups of deformable receptive field depth extraction units; The shallow features extracted Input several groups of deformable receptive field depth extraction units to perform deep feature extraction, the expression is: ; in, Denotes a deformable receptive field depth extraction unit. The superscript n denotes the nth group of deformable receptive field depth extraction units.

[0035] Specifically, the deformable receptive field depth extraction unit samples and aggregates non-local information by expanding the receptive field using a deformable window. Figure 3 , in the input feature map The position of each window divided on is regarded as a reference point The feature reference point positions inside the window are linearly spaced 2D coordinates ranging from arrive The receptive field of a window of size 4×4 is defined as : ; The nth token in the window is defined as: ; See also Figure 3 ,In order to enable the module to capture non-local key information, a lightweight sub-network is used, which is represented as , which identifies key feature locations based on the input feature map. This subnetwork generates offsets for the window through a simple convolutional layer . Subsequently, after applying the offset, the window is moved to the location of the global key feature. The token is redefined as: ; in is a bilinear interpolation function. Next, the channel dimension of the features is grouped, and different offsets are applied to each group of features, resulting in a regular receptive field The receptive field of a single local window is deformed and expanded to a multi-window deformable receptive field.

[0036] After all non-local information is collected through the deformable window, the collected information is passed to the attention module for attention calculation. First, the query, key and value for attention are generated. Apply linear projection to generate the query, then reconstruct the features by the deformable window receptive field Apply linear projection to generate keys and values. The expressions are: ; Finally, the attention calculation is performed using the converted key and value matrices and the query matrix, expressed as: ; In step S23, see Figure 3, the deformable receptive field depth extraction unit is connected to a convolutional feedforward network branch after the attention module to more effectively encode and restore detail information. This branch helps to enhance the model's ability to capture high-frequency textures, making the model more accurate in processing complex details. Input features First, the output features are obtained through the deformable window attention module It can be expressed as:

[0037] in, Represents a deformable window attention operation.

[0038] See also Figure 3 , the extracted features are passed through a normalization layer and a 5×5 deep convolution to further extract high-frequency texture information and improve the perception of local details. The expression is: ; in, represents the normalization operation, Represents a deep convolution operation. The deep convolution layer performs convolution operations on each channel of the input feature separately, which significantly reduces the computational overhead. Due to the high computational efficiency of deep convolution, larger convolution kernels (such as 5×5) can be used to capture larger scale detail information, thereby effectively extracting complex features in the image. The channel attention mechanism is introduced in the convolution feedforward network branch, which further strengthens the information exchange between feature channels by adaptively adjusting the weights of each channel. This not only improves the model's ability to capture important features, but also enhances the overall feature expression effect. For input features , this process can be expressed as: ; ; ; in, Representation characteristics Global average pooling value in the c channel. Represents a linear projection. Represents the final output feature of the feedforward network. By enhancing the synergy between features, the performance of the super-resolution network model can be further improved, and s represents the adjusted weight.

[0039] It can be understood that the present invention proposes a deformable window attention block (DWAB) based on the design concept of deformable convolution and deformable attention. The module automatically locates the important information area in the image by dynamically adjusting the window position, and expands the local window attention mechanism to multi-window non-local attention, thereby effectively capturing relevant feature information at a long distance while maintaining computational efficiency.

[0040] In step S24, the process of reconstructing the image of the fused features using the upsampling reconstruction module includes: The global residual connection method is used to extract shallow features With deep features to integrate; The PixelShuffle layer is used as a decoder to reconstruct a high-resolution image of the fused features. The expression is: ; in, represents the upsampling reconstruction module, Represents the reconstructed high-resolution image.

[0041] In step S25, the loss function L is set 1 The iterative training of the super-resolution network model during training is optimized, and the expression is: ; Where N represents the number of iterations. represents the high-resolution images in the test set, Represents the high-resolution image obtained after reconstruction.

[0042] In this embodiment, the shallow feature extraction module, the deformable window attention module and the upsampling reconstruction module in the super-resolution network model are iteratively optimized and trained, and a 3×3 convolution layer, several groups of deformable receptive field depth extraction units and a loss function are set in the process of iterative optimization training. In the process of iterative training of the super-resolution network model, the accuracy of the super-resolution network model in grasping the changes in image data can be further improved, so as to avoid the omission of important features due to the fixed window size, and effectively improve the image reconstruction effect; a convolutional feedforward network branch is connected after the deformable window attention module to more effectively encode and restore detail information. The branch helps to enhance the model's ability to capture high-frequency textures, making the model more accurate in processing complex details. A channel attention mechanism is introduced in the convolutional feedforward network branch. By adaptively adjusting the weights of each channel, the information exchange between feature channels is further strengthened, which not only improves the model's ability to capture important features, but also enhances the overall feature expression effect.

[0043] In order to overcome the limitations of existing Transformer-based single image super-resolution (SISR) methods in terms of computational complexity and long-distance feature capture, this paper proposes a new framework with a deformable adaptive window receptive field, referred to as DWSR (Deformable Window Super-Resolution). By dynamically adjusting the window position, the important information area in the image is automatically located, and the local window attention mechanism is expanded to multi-window non-local attention, thereby effectively capturing relevant feature information at a distance while maintaining computational efficiency. Compared with existing methods, DWAB can make full use of key information at different positions in the image during the reconstruction process, avoiding the omission of important features due to fixed window size.

[0044] In addition, to further enhance the network's ability to capture high-frequency details, the present invention introduces deep convolution and channel attention mechanisms into the original feedforward network (FFN) to enhance the model's ability to restore high-frequency textures. The feedforward structure combined with deep convolution strengthens the processing of local details while retaining global features, thereby significantly improving the overall quality of image super-resolution.

[0045] Embodiment three: This embodiment provides an image super-resolution reconstruction system. Figure 4 , the system comprising: A data acquisition module, used for acquiring image data; A data processing module, used for preprocessing the image data; Model building module, used to build super-resolution network model; The data processing module is also used to divide the preprocessed image data into a training set and a test set, use the training set to perform image super-resolution reconstruction training on the super-resolution network model, use the test set to test the trained super-resolution network model, and obtain the trained super-resolution network model after the test passes; The super-resolution image reconstruction module is used to perform super-resolution reconstruction on the image data to be super-resolved using the trained super-resolution network model, and output the reconstructed super-resolution image.

Claims

1. A method for super-resolution image reconstruction, characterized in that: The method comprises the following steps: Acquiring image data, and preprocessing the image data; Constructing a super-resolution network model, and dividing the preprocessed image data into a training set and a test set, using the training set to perform image super-resolution reconstruction training on the super-resolution network model, and using the test set to test the trained super-resolution network model, and obtaining a trained super-resolution network model after the test passes; The trained super-resolution network model is used to perform super-resolution reconstruction on the image data to be super-resolved, and the reconstructed super-resolution image is output.

2. The image super-resolution reconstruction method according to claim 1, characterized in that: The process of preprocessing the image data includes: Extract the HR image from the image data, and downsample the HR image using a bilinear downsampling method to extract the corresponding LR image; The extracted HR images and LR images are combined into an LR-HR image dataset, and the image data in the LR-HR image dataset is normalized; The HR image represents a high-resolution image, and the LR image represents a low-resolution image.

3. The image super-resolution reconstruction method according to claim 2, characterized in that: The process of using the training set to train the super-resolution network model for image super-resolution reconstruction includes: The constructed super-resolution network model includes a shallow feature extraction module, N deformable window attention modules, N convolutional feedforward networks and an upsampling reconstruction module; The shallow features in the LR image are extracted using the shallow feature extraction module; Each deformable window attention module is used to perform deep feature extraction on the extracted shallow feature map; Use each convolutional feed-forward network to adjust the weights of the extracted deep features; The extracted shallow features are fused with the deep features after the final weight adjustment, and the fused features are reconstructed using the upsampling reconstruction module, and the reconstructed image is output; The loss function is set to iteratively optimize the super-resolution network model during the training process, and the performance of the trained super-resolution network model is tested using the test set. When the loss function converges, the trained super-resolution network model is obtained.

4. The image super-resolution reconstruction method according to claim 3, characterized in that: The process of extracting shallow feature maps from LR images using the shallow feature extraction module is as follows: A 3×3 convolution layer is set in the shallow feature extraction module, and the 3×3 convolution layer is used to extract shallow features of the input LR image. The expression is: ; in, represents the resolution of the LR image, represents a 3×3 convolutional layer, Represents shallow features.

5. The image super-resolution reconstruction method according to claim 4, characterized in that: The process of using each deformable window attention module to extract deep features from the extracted shallow feature map includes: The deformable window attention module includes several groups of deformable receptive field depth extraction units; The shallow features extracted Input several groups of deformable receptive field depth extraction units to perform deep feature extraction, the expression is: ; in, Denotes a deformable receptive field depth extraction unit. The superscript n denotes the nth group of deformable receptive field depth extraction units.

6. The image super-resolution reconstruction method according to claim 5, characterized in that: The process of weighting the extracted deep features using each convolutional feedforward network includes: Normalize and deeply convolve the deep features extracted by the deformable window attention module; The convolutional feedforward network introduces a channel attention mechanism, which adjusts the weights of each channel adaptively to adjust the weights of the deep features after normalization and deep convolution processing.

7. The image super-resolution reconstruction method according to claim 6, characterized in that: The expression for normalizing and deep convolution of the deep features extracted by the deformable window attention module is: ; The weights of the deep features after normalization and deep convolution are adjusted, and the expression is: ; ; in, represents the normalization operation, represents the depthwise convolution operation, Representation characteristics The global average pooling value in the c channel, represents a linear projection, represents the final output feature of the feedforward network, and s represents the adjusted weight.

8. The image super-resolution reconstruction method according to claim 7, characterized in that: The process of reconstructing the image using the upsampling reconstruction module includes: The global residual connection method is used to extract shallow features With deep features to integrate; The PixelShuffle layer is used as a decoder to reconstruct a high-resolution image of the fused features. The expression is: ; in, represents the upsampling reconstruction module, Represents the reconstructed high-resolution image.

9. The image super-resolution reconstruction method according to claim 3, characterized in that: The loss function L1 is set to optimize the iterative training of the super-resolution network model during the training process, and the expression is: ; Where N represents the number of iterations. represents the high-resolution images in the test set, Represents the high-resolution image obtained after reconstruction.

10. An image super-resolution reconstruction system, characterized in that: The system comprises: A data acquisition module, used for acquiring image data; A data processing module, used for preprocessing the image data; Model building module, used to build super-resolution network model; The data processing module is also used to divide the preprocessed image data into a training set and a test set, use the training set to perform image super-resolution reconstruction training on the super-resolution network model, use the test set to test the trained super-resolution network model, and obtain the trained super-resolution network model after the test passes; The super-resolution image reconstruction module is used to perform super-resolution reconstruction on the image data to be super-resolved using the trained super-resolution network model, and output the reconstructed super-resolution image.

Citation Information

Patent Citations

  • Image super-resolution method based on hybrid Transform super-resolution network

    CN117333365A

  • Lightweight image super-resolution reconstruction method and system, storage medium and product

    CN118552408A

  • Image super-resolution reconstruction method and system based on convolution and Transform hybrid architecture

    CN118780984A