Image super-resolution reconstruction method and device and storage medium

Abstract

The invention provides an image super-resolution reconstruction method and device and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: receiving a low-resolution image; shallow layer feature extraction: performing a convolution operation on the low-resolution image to obtain a shallow layer feature map; deep feature extraction: forming RHSG units based on a plurality of Swin Transformer layers, an overlapping and crossing attention module and a double-domain convolution module, performing deep feature extraction on the shallow feature map through a plurality of RHSG units in sequence, and performing convolution operation once after passing through continuous RHSG units to obtain a deep feature map; high-quality image reconstruction: obtaining a deep feature map; utilizing a global residual path to obtain a shallow feature map; fusing the shallow feature map and the deep feature map; and performing image reconstruction based on the sub-pixel convolutional neural network to obtain a super-resolution image. According to the method, the spatial domain and the frequency domain are considered at the same time, global information is utilized, more pixels are activated in the image reconstruction process, and the performance of an image reconstruction task is optimized.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image super-resolution reconstruction method, device and storage medium. Background Art

[0002] Single image super-resolution is a classic problem in the field of computer vision and image processing, and its goal is to reconstruct a high-resolution image from a low-resolution image. With the rapid development of deep learning technology, many methods based on CNN (Convolutional Neural Network) have been widely used to design various image super-resolution models. In the process of implementing the present invention, the applicant found that: in the prior art, most of the image reconstruction performance is improved by residual learning, dense connection, or attention mechanism. Although these methods show significant performance improvement compared with traditional methods, the core operation of CNN is insensitive to content. Therefore, applying the same convolution kernel in different image regions may not be optimal. In addition, in order to balance network performance and computational cost, the existing methods perform self-attention operations in independent windows, which limits the use of large receptive fields in early layers to obtain sufficient global information, resulting in poor expression of the reconstructed image at the detail level.

[0003] Therefore, how to improve the performance of image super-resolution reconstruction tasks in actual work has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art or related technology, and discloses an image super-resolution reconstruction method, device and storage medium, which improve the performance of image super-resolution reconstruction tasks in actual work.

[0005] The first aspect of the present invention discloses an image super-resolution reconstruction method, comprising: receiving a low-resolution image; shallow feature extraction: performing a convolution operation on the low-resolution image to obtain a shallow feature map; deep feature extraction: forming an RHSG unit based on multiple Swin Transformer layers, an overlapping cross attention module and a dual-domain convolution module, performing deep feature extraction on the shallow feature map through multiple RHSG units in sequence, and performing a convolution operation after passing through consecutive RHSG units to obtain a deep feature map; wherein the overlapping cross attention module is formed by replacing the window-based multi-head self-attention algorithm in the Swin Transformer layer with the overlapping cross attention algorithm, and the dual-domain convolution module includes parallel spatial branches and frequency branches, the spatial branch is used to capture local features in the spatial domain, the frequency branch is used to obtain global features from the frequency domain, and the outputs of the two branches are connected in series and convolution operation is performed to obtain the output of the dual-domain convolution module; image reconstruction: obtaining a deep feature map; using a global residual path to obtain a shallow feature map; fusing the shallow feature map with the deep feature map; image reconstruction based on a sub-pixel convolutional neural network to obtain a super-resolution image.

[0006] In this technical solution, spatial domain and frequency domain information are combined to make full use of their complementary advantages. The spatial domain better captures the local features of the image, while the frequency domain helps to analyze the overall structure of the image. Thanks to these designs, the present application can utilize more overall information in the image reconstruction process, thereby achieving significant performance improvements. Among them, the RHSG unit is a residual hybrid Swin group, consisting of a series of Swin Transformer layers (STLs), an OCAB (overlapping cross attention module) and a DDCB (dual domain convolution module). Among them, OCAB is used to realize the transmission of pixel information between adjacent windows to improve performance, and DDCB consists of two branches: the frequency branch maps spatial features to the frequency domain, and uses ResBlocks to promote global high-frequency learning of the image. Because of the characteristics of the fast Fourier transform, this branch can obtain the full image receptive field from the early layers. The spatial branch uses ordinary convolution in the spatial domain and focuses more on local details.

[0007] According to the image super-resolution reconstruction method disclosed in the present invention, preferably, the calculation process of the overlapping cross attention module includes: for a dimension of The input features X, where X Q is decomposed into non-overlapping windows of size M×M, X K and S V It is also expanded to There are overlapping windows of size M0×M0, and the relationship between M and M0 is as follows:

[0008]

[0009] Among them, the constant Used to specify the scale of the overlapping area.

[0010] In this technical solution, the overlapping cross attention module (OCAB) is used to enhance the information interaction between adjacent windows and make full use of the information in each window; the design structure adopted by OCAB is similar to the standard Swin Transformer block, but in OCAB, overlapping cross attention is used instead of the window-based multi-head self-attention (W-MSA) part in the standard Swin Transformer block. In OCA, the projected features are divided into different window sizes. Therefore, OCAB can calculate key / value from a wider spatial range, enhance the interaction of information between windows, and thus improve the expressiveness of the model. The size of the overlapping area affects the symmetry of the window cross attention. By adjusting the overlap ratio, the size of the overlapping area can be controlled. In addition, it is very important to ensure that the product of the window size and the overlap ratio is an even number. Specifically, when the overlap ratio is 0.5, the window size is preferably divisible by 4; when the window size is 15, the overlap ratio can be set to 2 / 3, which helps to maintain the symmetry of the cross attention, thereby improving the stability and performance of the model. In addition, multi-head self-attention is applied to the embedding of the above query, key, and value, using the attention mechanism in the Transformer and adopting relative position encoding The specific formula is as follows:

[0011]

[0012] in, is a query matrix, It is a key and value matrix, d represents the query / key dimension, and B represents the relative position encoding.

[0013] According to the image super-resolution reconstruction method disclosed in the present invention, preferably, the calculation process of the spatial branch of the dual-domain convolution module includes: inputting the feature map X into the dual-domain convolution module; using convolution layers and residual connections to enhance the expression ability of features, and the expression is:

[0014] X spatial =H CLC (X)+X

[0015] Among them, X spatial represents the output features of the spatial branch, H CLC It indicates that a 1×1 convolution layer is used at the beginning and end, and a LeakyReLU activation function is used in the middle.

[0016] According to the image super-resolution reconstruction method disclosed in the present invention, preferably, the calculation process of the frequency branch of the dual-domain convolution module includes: inputting the feature map X into the dual-domain convolution module; and obtaining a finer feature map X0 through a convolution layer, which is expressed as:

[0017] X0=H CL (X)

[0018] Among them, H CL represents a 3×3 convolutional layer with a LeakyReLU activation function;

[0019] The fast Fourier transform (FFT) is used to convert the spatial features into frequency features in order to capture long-range contextual information in the frequency domain, and then the inverse Fourier transform (IFT) is performed to convert the frequency features back to spatial features; finally, a convolution operation is performed to halve the channel dimension so that it can be concatenated with the spatial features. The expression is as follows:

[0020]

[0021] Among them, X frequency represents the output characteristics of the frequency branch, H CLC It means that a 1×1 convolution layer is used at the beginning and end, and a LeakyReLU activation function is used in the middle. and represent the fast Fourier transform and inverse Fourier transform operations respectively, H conv Represents a 1×1 convolutional layer.

[0022] According to the image super-resolution reconstruction method disclosed in the present invention, preferably, in the step of shallow feature extraction, the convolution kernel size of the convolution operation is 3×3.

[0023] In this technical solution, when performing shallow feature extraction, a given input low-resolution (LR) image H, W and C in are the height, width and number of input channels of the low-resolution image respectively. A simple convolutional layer is used to extract shallow features. Where C is the number of channels of the intermediate feature, expressed as:

[0024] F0=H SF (I LR )

[0025] Among them, H SFRepresents a 3×3 convolution operation. Using only one convolution layer not only preserves the low-frequency information of the image itself, but also provides a simple way to map the input image space to a higher-dimensional feature space.

[0026] According to the image super-resolution reconstruction method disclosed in the present invention, preferably, when a convolution operation is performed after continuous RHSG units, the convolution kernel size is 3×3.

[0027] In this technical solution, the deep feature extraction stage includes a series of consecutive RHSG units (residual hybrid Swin groups) and a 3×3 convolution operation, expressed as:

[0028]

[0029] F DF =H conv (F RHSGs )

[0030] in, represents the kth RHSG unit, K represents the number of RHSGs in deep feature extraction, and H conv Denotes the last convolutional layer. Using convolution operations before entering the image reconstruction stage can introduce the inductive bias of CNN, which is very effective in Transformer-based networks. It enables the advantages of CNN and Transformer architectures to be effectively integrated, thereby improving the overall network performance.

[0031] According to the image super-resolution reconstruction method disclosed in the present invention, preferably, it also includes: optimizing the model by combining the FFT-based frequency loss function and the L1 loss function, and the loss function is defined as follows:

[0032] L(θ)=L1(θ)+βL FFT (θ)

[0033] Among them, θ represents the model parameters, L1() and L FFT () represent L1 loss function and FFT loss function respectively, β is a weight parameter set by experience, L FFT The expression is as follows:

[0034]

[0035] Among them, I SR Represents a low-resolution image, L HR represents a high-resolution image, stands for Fast Fourier Transform.

[0036] The second aspect of the present invention discloses an image super-resolution reconstruction device, comprising: a memory for storing program instructions; a processor for calling the program instructions stored in the memory to implement an image super-resolution reconstruction method as described in any of the above technical solutions.

[0037] A third aspect of the present invention discloses a computer-readable storage medium storing a program code, wherein the program code is used to implement an image super-resolution reconstruction method as described in any of the above technical solutions.

[0038] The beneficial effects of the present invention include at least: the introduced dual-domain convolution block (DDCB) module captures finer global features to improve the quality of reconstructing high-resolution results. It combines spatial domain and frequency domain information at the same time, comprehensively utilizes their complementary advantages, captures local and global image features, and integrates low-frequency and high-frequency information. Overlapping cross attention blocks (OCABs) are used to promote pixel transfer between adjacent windows to enhance the global modeling ability of the model and improve network performance while maintaining linear computational complexity. In the model training stage, a fast Fourier transform (FFT) loss is additionally introduced to give full play to the model performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of the network model structure of an image super-resolution reconstruction method according to an embodiment of the present invention is shown.

[0040] Figure 2 A schematic diagram of the network structure of a Swin Transformer layer according to an embodiment of the present invention is shown.

[0041] Figure 3 A schematic diagram of the network structure of an overlapping cross-attention module according to an embodiment of the present invention is shown.

[0042] Figure 4 A schematic diagram of the network structure of a dual-domain convolution module according to an embodiment of the present invention is shown.

[0043] Figure 5 A schematic block diagram of an image super-resolution reconstruction device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0044] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0046] like Figure 1 As shown, according to an embodiment of the present invention, the image super-resolution reconstruction method disclosed in the present invention includes: receiving a low-resolution image; shallow feature extraction: performing a convolution operation on the low-resolution image to obtain a shallow feature map; deep feature extraction: forming an RHSG unit based on multiple Swin Transformer layers, an overlapping cross attention module and a dual-domain convolution module, performing deep feature extraction on the shallow feature map through multiple RHSG units in sequence, and performing a convolution operation after passing through consecutive RHSG units to obtain a deep feature map; wherein the overlapping cross attention module is formed by replacing the window-based multi-head self-attention algorithm in the Swin Transformer layer with the overlapping cross attention algorithm, and the dual-domain convolution module includes parallel spatial branches and frequency branches, the spatial branch is used to capture local features in the spatial domain, and the frequency branch is used to obtain global features from the frequency domain, and the outputs of the two branches are connected in series and convolution operation is performed to obtain the output of the dual-domain convolution module; image reconstruction: obtaining a deep feature map; using a global residual path to obtain a shallow feature map; fusing the shallow feature map with the deep feature map; image reconstruction based on a sub-pixel convolutional neural network to obtain a super-resolution image.

[0047] In this embodiment, the present application provides a Transformer-based space-frequency dual-domain information fusion image super-resolution reconstruction algorithm (SFFT). The overall network architecture of the algorithm is as follows: Figure 1 As shown in the figure, it includes three main modules: shallow feature extraction module, deep feature extraction module and image reconstruction module. The calculation process mainly includes: the low-resolution image enters the continuous RHSG unit after a convolution layer to extract deep features, and performs another convolution operation after the continuous RHSG operation to obtain deep features. At the same time, the output (low-frequency information) of the shallow feature extraction module is directly passed to the image reconstruction module through a global residual path, so that the image reconstruction module can fuse shallow features and deep features for image reconstruction.

[0048] According to the above embodiment, preferably, in the shallow feature extraction module, the specific calculation process includes:

[0049] Given an input LR (low resolution) image L LR , Among them, H, W and C in are the height, width and number of input channels of the LR image respectively;

[0050] First, a simple convolutional layer is used to extract the shallow feature F0. Where C is the number of channels of the intermediate feature, expressed as:

[0051] F0=H SF (I LR )

[0052] Among them, H SF Represents a 3×3 convolution operation. Using only one convolution layer not only preserves the low-frequency information of the image itself, but also provides a simple way to map the input image space to a higher-dimensional feature space.

[0053] According to the above embodiment, preferably, in the deep feature extraction module, the deep features are extracted The steps specifically include:

[0054] The deep feature extraction stage consists of a series of consecutive residual hybrid Swin groups (RHSGs) and a 3×3 convolution operation, expressed as:

[0055]

[0056] F DP =H conv (F RHSGs )

[0057] in, represents the kth RHSG unit, K represents the number of RHSGs in deep feature extraction, and H conv represents the last convolutional layer.

[0058] Using convolution operations before entering the image reconstruction module can introduce the inductive bias of CNN, which is very effective in Transformer-based networks. It enables the advantages of CNN and Transformer architectures to be effectively integrated, thereby improving the overall network performance.

[0059] According to the above embodiment, preferably, the calculation process of the image reconstruction module specifically includes:

[0060] A global residual path is used to directly pass low-frequency information to the image reconstruction module, thereby fusing shallow and deep features. Specifically, the shallow feature F0 mainly contains low-frequency components, while the deep feature F DF Then we focus on restoring the lost high-frequency components. For the implementation of the image reconstruction module, a sub-pixel convolution layer is used to upsample the features to high resolution, expressed as:

[0061] I SR =H up (F DF+F0)=H SFFT (I LR )

[0062] Among them, H up is the image reconstruction module, H SFFT represents the SFFT network proposed in this application, I SR It is the final reconstructed image. The image reconstruction module performs image reconstruction based on the PixelShuffle algorithm. PixelShuffle is an upsampling method based on deep learning, which is mainly used in the field of image processing. It upsamples low-resolution images to high-resolution images by learning the mapping relationship between low-resolution images and high-resolution images.

[0063] like Figure 2 and Figure 3 As shown, according to the above embodiment, preferably, the design structure adopted by the overlapping cross attention module (OCAB) is similar to the standard Swin Transformer block, but in OCAB, the overlapping cross attention (OCA) is used to replace the window-based multi-head self-attention (W-MSA) part in the standard Swin Transformer block. In OCA, the projected features are divided into different window sizes. Therefore, OCAB can calculate key / value from a wider spatial range, enhance the interaction of information between windows, and thus improve the expression ability of the model. For the input features Unlike the standard Swin Transformer block, which calculates query, key, and value within the same window feature, the OCA of this application calculates key and value from a larger range. The specific calculation process includes:

[0064] For the dimension The input features X, X Q is decomposed into non-overlapping windows of size M×M, and X K and X V It is also expanded to There are overlapping windows of size M0×M0. The relationship between M and M0 is as follows:

[0065]

[0066] Among them, the constant Is used to specify the ratio of the overlapping area. The size of the overlapping area affects the symmetry of the window cross attention. By adjusting the overlap ratio, the size of the overlapping area can be controlled. In addition, it is very important to ensure that the product of the window size and the overlap ratio is an even number. Specifically, when the overlap ratio is 0.5, the window size is preferably divisible by 4; when the window size is 15, the overlap ratio can be set to 2 / 3. This helps maintain the symmetry of the cross attention, thereby improving the stability and performance of the model. In addition, multi-head self-attention is applied to the above query, key, and value embeddings, utilizing the attention mechanism in Transformer and adopting relative position encoding The specific formula is as follows:

[0067]

[0068] in, is a query matrix, is the key and value matrix. d represents the query / key dimension and B represents the relative position encoding.

[0069] like Figure 4 As shown in the figure, according to the above embodiment, the dual-domain convolution module can enhance the representation ability of the Transformer from the perspective of image frequency, realize full-image receptive field modeling, and improve super-resolution performance. The dual-domain convolution module (DDCB) includes two main parallel branches: a spatial branch and a frequency branch. The spatial branch aims to capture local features in the spatial domain, while the frequency branch focuses on obtaining global features from the frequency domain. Specifically, the feature map X derived from OCAB is used as the input of the DDCB and is simultaneously input to the two branches X. frequency and X spatial In the formula, the following is the formula:

[0070] X spatial =H spatial (X)

[0071] X frequency =H frequency (X)

[0072] Among them, H spatial and H frequency They represent the spatial branch and frequency branch of the DDCB module respectively, and X spatial and X frequency They represent the outputs of the corresponding branches respectively.

[0073] In the frequency branch, the input X first passes through a convolutional layer to obtain a more refined X0, expressed as:

[0074] X0=H CL (X)

[0075] Among them, H CL Represents a 3×3 convolutional layer with a LeakyReLU activation function.

[0076] Next, FFT is used to convert traditional spatial features into frequency features, which helps capture long-range contextual information in the frequency domain. Subsequently, the inverse Fourier transform (IFT) is used to convert the frequency features back into spatial features. Finally, a convolution operation reduces the channel dimension by half for concatenation with the spatial features. The formal representation is as follows:

[0077]

[0078] Among them, H CLC It indicates that a 1×1 convolution layer is used at the beginning and end, and a LeakyReLU activation function is used in the middle. and Represents FFT and inverse FFT operations, H conv Represents a 1×1 convolutional layer.

[0079] In the spatial branch, convolutional layers and residual connections are used to enhance the expressiveness of features, thereby improving the performance of the model. It can be expressed as the following process:

[0080] X spatial =H CLC (X)+X

[0081] The outputs of the two branches are concatenated and convolved to get the final result, as shown below:

[0082] X DDCB =H conv (H concatenate (X spatial ,X frequency ))

[0083] Among them, H conv and H concatenate denote the convolution and concatenation operations respectively, and represent the overall output of DDCB.

[0084] According to the above embodiment, preferably, the performance of the model depends not only on the design of the network structure, but also on the choice of the loss function. For the method (SFFT) proposed in the above embodiment of the present application, a combination of a frequency loss function based on FFT and an L1 loss function is used for model optimization. This allows the frequency information and spatial structure information to be fully considered in the recovery results, thereby achieving performance improvement in super-resolution tasks. The loss function used in this application is defined as follows:

[0085] L(θ)=L1(θ)+βL FFT (θ)

[0086] Among them, θ represents the parameters of SFFT, L1() and L FFT () represent the L1 loss function and the FFT loss function respectively. β is a weight parameter set empirically and is set to 0.075 during training.

[0087] L FFT The expression is as follows:

[0088]

[0089] Among them, I HR represents the HR image, Represents FFT.

[0090] like Figure 5 As shown, according to another embodiment of the present invention, an image super-resolution reconstruction device 500 is also disclosed, including: a memory 501, used to store program instructions; a processor 502, used to call the program instructions stored in the memory to implement the image super-resolution reconstruction method as described in the above embodiment.

[0091] According to yet another embodiment of the present invention, a computer-readable storage medium is disclosed. The computer-readable storage medium stores program codes, and the program codes are used to implement the image super-resolution reconstruction method as described in the above embodiment.

[0092] According to the above embodiments, the present application discloses an image super-resolution reconstruction method, which is a Transformer-based space-frequency dual-domain information fusion image super-resolution reconstruction algorithm (SFFT), which combines spatial domain and frequency domain information at the same time, comprehensively utilizes their complementary advantages, captures local and global image features, and integrates low-frequency and high-frequency information. An overlapping cross-attention module (OCAB) is used to promote pixel transmission between adjacent windows and improve network performance. In the network training stage, a fast Fourier transform (FFT) loss is additionally introduced to give full play to the performance of the proposed module. A large number of qualitative and quantitative evaluations of benchmark data sets show that the algorithm proposed in this application is superior to existing image super-resolution reconstruction methods in terms of accuracy, and does not generate a larger amount of parameters and calculations.

[0093] All or part of the steps in the various methods of the above embodiments can be completed by controlling the relevant hardware through a program, and the program can be stored in a readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other readable medium that can be used to carry or store data.

[0094] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for super-resolution image reconstruction, characterized in that: include: Receive low-resolution images; Shallow feature extraction: performing a convolution operation on the low-resolution image to obtain a shallow feature map; Deep feature extraction: Based on multiple Swin Transformer layers, an overlapping cross attention module and a dual-domain convolution module, an RHSG unit is formed, and the shallow feature map is subjected to deep feature extraction through multiple RHSG units in sequence, and a convolution operation is performed after passing through consecutive RHSG units to obtain a deep feature map; wherein, the overlapping cross attention module is formed by replacing the window-based multi-head self-attention algorithm in the Swin Transformer layer with the overlapping cross attention algorithm, and the dual-domain convolution module includes parallel spatial branches and frequency branches, the spatial branch is used to capture local features in the spatial domain, and the frequency branch is used to obtain global features from the frequency domain, and the outputs of the two branches are connected in series and subjected to a convolution operation to obtain the output of the dual-domain convolution module; Image reconstruction: obtaining the deep feature map; obtaining the shallow feature map using a global residual path; fusing the shallow feature map with the deep feature map; performing image reconstruction based on a sub-pixel convolutional neural network to obtain a super-resolution image.

2. The image super-resolution reconstruction method according to claim 1, characterized in that: The calculation process of the overlapping cross attention module includes: For the dimension The input features X, where X Q is decomposed into non-overlapping windows of size M×M, X K and X V It is also expanded to There are overlapping windows of size M0×M0, and the relationship between M and M0 is as follows: Among them, the constant Used to specify the scale of the overlapping area.

3. The image super-resolution reconstruction method according to claim 1, characterized in that: The calculation process of the spatial branch of the dual-domain convolution module includes: The feature map X is input into the dual-domain convolution module; Convolutional layers and residual connections are used to enhance the expressiveness of features. The expression is: X spatial =H CLC (X)+X Among them, X spatial represents the output features of the spatial branch, H CLC It indicates that a 1×1 convolution layer is used at the beginning and end, and a LeakyReLU activation function is used in the middle.

4. The image super-resolution reconstruction method according to claim 1, characterized in that: The calculation process of the frequency branch of the dual-domain convolution module includes: The feature map X is input into the dual-domain convolution module; Through a convolution layer, a more refined feature map X0 is obtained, which is expressed as: X0=H CL (X) Among them, H CL represents a 3×3 convolutional layer with a LeakyReLU activation function; The spatial features are converted into frequency features using fast Fourier transform to capture long-range contextual information in the frequency domain, and then the frequency features are converted back to spatial features using inverse Fourier transform. Finally, a convolution operation is performed to halve the channel dimension so that it can be concatenated with the spatial features. The expression is as follows: Among them, X frequency represents the output characteristics of the frequency branch, H CLC It means that a 1×1 convolution layer is used at the beginning and end, and a LeakyReLU activation function is used in the middle. and represent the fast Fourier transform and inverse Fourier transform operations respectively, H conv Represents a 1×1 convolutional layer.

5. The image super-resolution reconstruction method according to claim 1, characterized in that: In the shallow feature extraction step, the convolution kernel size of the convolution operation is 3×3.

6. The image super-resolution reconstruction method according to claim 1, characterized in that: When a convolution operation is performed after consecutive RHSG units, the convolution kernel size is 3×3.

7. The image super-resolution reconstruction method according to any one of claims 1 to 6, characterized in that: Also includes: The model is optimized by combining the FFT-based frequency loss function and the L1 loss function. The loss function is defined as follows: L(θ)=L1(θ)+βL FFT (i) Among them, θ represents the model parameters, L1( ) and L FFT ( ) represent L1 loss function and FFT loss function respectively, β is a weight parameter set by experience, L FFT The expression is as follows: Among them, I SR Represents a low-resolution image, I HR represents a high-resolution image, stands for Fast Fourier Transform.

8. An image super-resolution reconstruction device, characterized in that: include: A memory for storing program instructions; A processor, configured to call the program instructions stored in the memory to implement the image super-resolution reconstruction method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program code, and the program code is used to implement the image super-resolution reconstruction method according to any one of claims 1 to 7.

Images