Frequency domain adaptive super-resolution reconstruction method

Through the frequency domain adaptive super-resolution reconstruction method, the frequency pyramid attention network and fast Fourier transform loss function are used, combined with sub-pixel upsampling technology, the problems of excessive complexity and low reconstruction efficiency in the existing technology are solved, and efficient and stable image reconstruction effect is achieved.

CN119809936AActive Publication Date: 2025-04-11XIAN UNIV OF SCI & TECH

Patent Information

Application Number
CN202510002506.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-01-02
Publication Date
2025-04-11
Estimated Expiration
2045-01-02

AI Technical Summary

Technical Problem

In the existing super-resolution reconstruction technology, the high complexity of the model leads to low reconstruction efficiency, unstable training process and difficult to converge.

Method used

The frequency domain adaptive super-resolution reconstruction method is adopted to extract multi-scale frequency domain features and weight processing through frequency pyramid attention network and fast Fourier transform loss function, and combine sub-pixel upsampling technology to achieve efficient image reconstruction.

Benefits of technology

It significantly improves the efficiency and quality of image reconstruction, enhances the ability to capture image details and high-frequency information, and improves the stability and convergence speed of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119809936A_ABST
    Figure CN119809936A_ABST
Patent Text Reader

Abstract

The invention discloses a frequency domain adaptive super-resolution reconstruction method. The method comprises the following steps: acquiring an original image; and processing an original image through the trained frequency domain self-adaptive super-resolution reconstruction model to obtain a reconstructed picture. According to the frequency domain self-adaptive super-resolution reconstruction method, a multi-scale frequency domain feature extraction module can extract multi-scale features from different levels of feature maps by combining a feature pyramid structure and Fourier transform, and the multi-scale feature extraction is beneficial for a model to capture different levels of information of an image and enhance the detail recognition capability; the frequency pyramid attention network decomposes the feature map into different frequency components, calculates the importance of each frequency component, and weights the components through attention weights, so that the model can pay more attention to high-frequency information which has great influence on image reconstruction quality.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of image enhancement and relates to a frequency domain adaptive super-resolution reconstruction method. Background Art

[0002] At present, super-resolution reconstruction methods can be roughly divided into traditional methods and deep learning methods based on their development history. Traditional super-resolution reconstruction methods can be further divided into interpolation-based methods, reconstruction-based methods, and learning-based methods. Interpolation-based super-resolution methods mainly use basis functions or interpolation kernels to approximately restore lost feature information to achieve image reconstruction. Typical interpolation methods include nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation. However, these methods are relatively simple and usually cause problems such as jagged artifacts, missing details, and blurring in the reconstructed image, and cannot effectively restore the high-frequency information of the image. Reconstruction-based super-resolution methods first simulate the image degradation process to generate a low-quality degraded image, and then introduce the prior constraints of the high-resolution image through the regularization method, and optimize the solution to generate the corresponding high-resolution image. Typical reconstruction methods include iterative back-projection method, maximum a posteriori probability method, and convex set projection method. However, these methods usually have high computational overhead, and it is difficult to select appropriate constraints or make full use of the prior information of the image, which limits their effectiveness and adaptability in practical applications. With the development of deep learning, models such as convolutional neural networks, graph neural networks, and generative adversarial networks obtain feature information from input images, and finally the reconstruction module generates the corresponding high-resolution image. Although the performance and effect of different models in the image reconstruction process are gradually improving, they face the challenges of excessive memory consumption and computational complexity.

[0003] The emergence of generative adversarial networks can generate high-quality and realistic samples. Therefore, generative adversarial networks can be used for high-resolution image generation and can effectively capture the details and structural information in the data. SRGAN (Super-Resolution Generative Adversarial Network) introduced generative adversarial networks into the field of super-resolution reconstruction. SRGAN learns the process of restoring high-resolution images from low-resolution images by adversarially training the generator and the discriminator. After SRGAN was proposed, it attracted widespread attention in the field of super-resolution. However, its shortcomings cannot be ignored: the training process is unstable, difficult to converge, and prone to mode collapse; the restoration of details may not be perfect, and the generated images may have artifacts or unnatural textures; high computing resource consumption, slow training and generation speed. In order to further improve the performance of SRGAN, researchers have innovated and improved in many aspects such as network architecture, loss function, training strategy, multi-scale feature fusion, introduction of auxiliary models, specific application optimization and inference acceleration. Through these contributions, SRGAN has made significant progress in the field of image super-resolution. The generated images are not only more natural and detailed in visual effects, but also more stable and efficient in training process. Existing super-resolution reconstruction technology usually processes images in the spatial domain. The higher the image resolution, the greater the complexity of the algorithm, and more computing resources and running time are required to complete the reconstruction task. Now more emphasis is placed on restoration effects and model complexity is ignored, resulting in an unstable training process and difficulty in convergence.

[0004] In summary, the existing technology has the problem of over-emphasizing reconstruction quality, which leads to high model complexity and low reconstruction efficiency. Summary of the invention

[0005] The purpose of the present invention is to provide a frequency domain adaptive super-resolution reconstruction method, which solves the problem in the prior art that too much emphasis is placed on reconstruction quality, resulting in excessive model complexity and low reconstruction efficiency.

[0006] The technical solution adopted by the present invention is a frequency domain adaptive super-resolution reconstruction method, comprising the following steps: S1, obtain the original image; S2. The original image is processed by the trained frequency domain adaptive super-resolution reconstruction model to obtain a reconstructed image.

[0007] The present invention is also characterized in that: The frequency domain adaptive super-resolution reconstruction model includes a shallow feature extraction module, a multi-scale frequency domain feature extraction module, and a feature fusion module; The multi-scale frequency domain feature extraction module includes several convolutional layers, feature enhancement modules and frequency pyramid attention network; The feature enhancement module includes several convolutional layers and activation units connected in sequence, and the activation unit is a Gaussian error linear unit activation function.

[0008] S2 includes the following steps: S2.1, shallow feature extraction module extracts shallow features of original image through nonlinear mapping; S2.2, several stacked multi-scale frequency domain feature extraction modules process the shallow features of the original image to obtain the deep features of the original image; S2.3, the feature fusion module fuses the deep features of the original image with the shallow features of the original image and rearranges them through sub-pixel convolution to obtain a reconstructed image.

[0009] The process of multi-scale frequency domain feature extraction module processing shallow features of original image is as follows: A1. The shallow features of the original image are processed through several convolutional layers to obtain the distilled features of the original image; A2, the feature enhancement module processes the distilled features of the original image to obtain a multi-scale feature map of the original image; A3. Process the multi-scale feature map of the original image through the frequency pyramid attention network to obtain the deep features of the original image.

[0010] A3 includes the following steps: A3.1. Convert the original image multi-scale feature map from the spatial domain to the frequency domain through Fourier transform; A3.2. The frequency domain features of the real and imaginary parts of the original image multi-scale feature map in the frequency domain are extracted through the convolution layer, and then enhanced and merged in turn according to the attention weight coefficient to obtain the enhanced detail features of the original image; A3.3, transform the original image enhancement detail features from the frequency domain to the spatial domain through inverse Fourier transform; A3.4. The original image enhanced detail features are fused with the original image multi-scale feature map to obtain the original image deep features.

[0011] The attention weight coefficient is calculated by the fast Fourier transform loss function; The calculation formula of the fast Fourier transform loss function is as follows:

[0012] in, represents the fast Fourier transform loss value, represents the loss weight coefficient, represents the real part of the predicted image in the frequency domain, represents the real part of the target image in the frequency domain, represents the imaginary part of the predicted image in the frequency domain, Represents the imaginary part of the target image in the frequency domain.

[0013] The trained frequency domain adaptive super-resolution reconstruction model is obtained through the following steps: B1. Obtain training sets and test sets, and set model training indicators; B2. Use the training set to train the frequency domain adaptive super-resolution reconstruction model; B3. Input the test set into the trained frequency domain adaptive super-resolution reconstruction model to obtain the test result. If the test result meets the model training index, the training is completed and the trained frequency domain adaptive super-resolution reconstruction model is obtained. If the test result does not meet the model training index, repeat steps B2-B3.

[0014] Model training metrics include peak signal-to-noise ratio and structural similarity index.

[0015] The reconstructed image has a higher resolution than the original image.

[0016] The beneficial effects of the present invention are as follows: the multi-scale frequency domain feature extraction module in the present invention can extract multi-scale features from feature maps of different levels by combining a feature pyramid structure with Fourier transform, and this multi-scale feature extraction helps the model capture information of different levels of the image and enhance the ability to recognize details; the frequency pyramid attention network decomposes the feature map into different frequency components, calculates the importance of each frequency component, and weights these components by attention weights, so that the model can pay more attention to high-frequency information that has a greater impact on the quality of image reconstruction; multiple stacked multi-scale frequency domain feature extraction modules are used to achieve deep information capture and feature enhancement, and the model can deeply capture Acquiring deep information in the network is conducive to improving the details and quality of the reconstructed image, so that the model can better understand and reconstruct the complex structure in the image, especially when processing high-resolution images, it can significantly improve the reconstruction effect; combining the fast Fourier transform loss function and sub-pixel upsampling operation, it can effectively reconstruct high-resolution images. The fast Fourier transform loss function reduces the gap between the generated image and the real image in the frequency domain, while sub-pixel upsampling improves the resolution of the reconstructed image while maintaining the details and quality of the image. This method of combining frequency domain analysis and spatial domain upsampling not only improves the efficiency of image reconstruction, but also optimizes the visual quality of the reconstructed image. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a schematic diagram of the network structure of a frequency domain adaptive super-resolution reconstruction model in the frequency domain adaptive super-resolution reconstruction method in an embodiment; Figure 2 is a schematic diagram of the network structure of a multi-scale frequency domain feature extraction module in a frequency domain adaptive super-resolution reconstruction model in an embodiment; Figure 3 is a schematic diagram of the network structure of a feature enhancement module in a frequency domain adaptive super-resolution reconstruction model in an embodiment; Figure 4 Schematic diagram of the network structure of the frequency pyramid attention network in the frequency domain adaptive super-resolution reconstruction model in the embodiment. DETAILED DESCRIPTION

[0018] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] The frequency domain adaptive super-resolution reconstruction method comprises the following steps: S1, obtain the original image; S2, processing the original image through the trained frequency domain adaptive super-resolution reconstruction model to obtain a reconstructed image; S2.1, shallow feature extraction module extracts shallow features of original image through nonlinear mapping; S2.2, several stacked multi-scale frequency domain feature extraction modules process the shallow features of the original image to obtain the deep features of the original image; The process of multi-scale frequency domain feature extraction module processing shallow features of original image is as follows: A1. The shallow features of the original image are processed through several convolutional layers to obtain the distilled features of the original image; A2, the feature enhancement module processes the distilled features of the original image to obtain a multi-scale feature map of the original image; A3. Process the multi-scale feature map of the original image through the frequency pyramid attention network to obtain the deep features of the original image; A3.1. Convert the original image multi-scale feature map from the spatial domain to the frequency domain through Fourier transform; A3.2. The frequency domain features of the real and imaginary parts of the original image multi-scale feature map in the frequency domain are extracted through the convolution layer, and then enhanced and merged in turn according to the attention weight coefficient to obtain the enhanced detail features of the original image; The attention weight coefficient is calculated by the fast Fourier transform loss function; the calculation formula of the fast Fourier transform loss function is as follows:

[0020] in, represents the fast Fourier transform loss value, represents the loss weight coefficient, represents the real part of the predicted image in the frequency domain, represents the real part of the target image in the frequency domain, represents the imaginary part of the predicted image in the frequency domain, Represents the imaginary part of the target image in the frequency domain; A3.3, transform the original image enhancement detail features from the frequency domain to the spatial domain through inverse Fourier transform; A3.4. The original image enhanced detail features are fused with the original image multi-scale feature map to obtain the original image deep features; S2.3, the feature fusion module fuses the deep features of the original image with the shallow features of the original image and rearranges them through sub-pixel convolution to obtain a reconstructed image; The resolution of the reconstructed image is higher than the original image; The trained frequency domain adaptive super-resolution reconstruction model is obtained through the following steps: B1. Obtain training sets and test sets, and set model training indicators; Model training metrics include peak signal-to-noise ratio and structural similarity index; B2. Use the training set to train the frequency domain adaptive super-resolution reconstruction model; B3, input the test set into the trained frequency domain adaptive super-resolution reconstruction model to obtain the test result. If the test result meets the model training index, the training is completed and the trained frequency domain adaptive super-resolution reconstruction model is obtained. If the test result does not meet the model training index, repeat steps B2-B3. The frequency domain adaptive super-resolution reconstruction model includes a shallow feature extraction module, a multi-scale frequency domain feature extraction module, and a feature fusion module; The multi-scale frequency domain feature extraction module includes several convolutional layers, feature enhancement modules and frequency pyramid attention network; The feature enhancement module includes several convolutional layers and activation units connected in sequence, and the activation unit is a Gaussian error linear unit activation function.

[0021] In the present invention, several multi-scale frequency domain feature extraction modules can be used to extract deeper and more abstract feature information from the original image; the multi-scale frequency domain feature extraction modules can enhance the representation ability of the feature map, thereby helping the model to learn the ability to restore clear images from various transformations; the multi-scale frequency domain feature extraction module uses adaptive grouped convolution and multi-level attention mechanisms to enhance feature extraction and fusion; in the multi-scale frequency domain feature extraction module, the frequency pyramid attention network is used to enhance the extraction and fusion of feature information of the original image; the residual connection is processed by the feature enhancement module to retain the details and texture information of the image; the frequency pyramid attention network weights the features in the frequency domain so that the network can pay more attention to important feature parts; the frequency pyramid attention network enhances the detail recovery capability in the image super-resolution task by combining Fourier transform and convolution operations.

[0022] In the frequency pyramid attention network of the present invention, the input image is converted from the spatial domain to the frequency domain to capture the high-frequency detail information of the image, and then the real and imaginary parts in the frequency domain are processed separately through three layers of convolution, the frequency domain features are extracted and the details are enhanced, and then the processed real and imaginary parts are merged, and the frequency domain signal is restored to the spatial domain through inverse Fourier transform, and finally the input image is fused with the processed image, thereby improving the image quality, especially in terms of details and textures, and enhancing the clarity and detail recovery effect of the image. The frequency pyramid attention network of the present invention can improve the quality of image reconstruction through the advantages of frequency domain enhancement, multi-scale feature processing, and adaptive feature selection. At the same time, compared with the traditional spatial domain convolution method, it has obvious advantages in computational efficiency, detail recovery and robustness, especially in processing high-frequency details and reducing noise.

[0023] The frequency pyramid attention network processes images in the frequency domain and uses convolution to enhance high-frequency information such as texture and edges in the frequency domain. The fast Fourier transform loss function can ensure that image details are enhanced during training by optimizing the difference between the real and imaginary parts in the frequency domain. The combination of the two can significantly improve image detail recovery, image quality and convergence speed.

[0024] The present invention calculates the difference between the predicted image and the target image in the frequency domain through the fast Fourier transform loss function to obtain the attention weight coefficient: the image is converted from the spatial domain to the frequency domain through the Fourier transform, and then the absolute error between the real part and the imaginary part is calculated respectively to measure the similarity between the predicted image and the real image in the frequency domain. Finally, the volume error of the real part and the imaginary part is summed and multiplied by the loss coefficient to obtain the fast Fourier transform loss function value as the attention weight coefficient. Since the difference in the frequency domain can often better capture the details, texture and high-frequency components of the image, the fast Fourier transform loss function helps to improve the detail restoration effect of the image.

[0025] The training set in the present invention can use the high-quality DIV2k dataset, each image has extremely rich texture information, which is very suitable as training data for super-resolution reconstruction. The test set uses the Set5 and Set14 test sets widely used in super-resolution reconstruction. The Set5 test set consists of human and animal images, and the Set14 test set consists of human, animal, plant, and posters. The original image during model training, i.e., the low-resolution image, is obtained by downsampling the original clear image by interpolation.

[0026] Peak Signal-to-Noise Ratio (PSNR) and Structure Similarity Index (SSIM) are two commonly used methods for evaluating the quality of image super-resolution reconstruction technology. The higher the PSNR and SSIM values, the closer the reconstructed image is to the original image, and the better the reconstruction effect. The present invention combines the feature pyramid structure with Fourier transform to design a multi-scale frequency domain feature extraction module (Multi-Scale Frequency-Aware Block, MFAB). This module uses the frequency pyramid attention network (FPAN) to decompose the feature map into different frequency components, calculate the importance of each frequency component, and weight the different frequency components by the attention weight coefficient. The weighted frequency components are recombined to obtain the adjusted feature map, and multiple stacked MFAB modules are used to capture deeper information in the network. The high-resolution image is reconstructed by combining the frequency pyramid attention network with the fast Fourier transform loss function and sub-pixel upsampling operation.

[0027] Example 1 This embodiment proposes a frequency domain adaptive super-resolution reconstruction method, including the following steps: S1, obtain the original image; S2. The original image is processed by the trained frequency domain adaptive super-resolution reconstruction model to obtain a reconstructed image.

[0028] Example 2 This embodiment proposes a frequency domain adaptive super-resolution reconstruction method, including the following steps: S1, obtain the original image; S2, processing the original image through the trained frequency domain adaptive super-resolution reconstruction model to obtain a reconstructed image; like Figure 1 As shown in , the frequency domain adaptive super-resolution reconstruction model includes a shallow feature extraction module, a multi-scale frequency domain feature extraction module, and a feature fusion module; Figure 2 As shown in , the multi-scale frequency domain feature extraction module includes convolutional layers such as c1_d, c2_d, c3_d, feature enhancement module and frequency pyramid attention network; Figure 3 As shown, the feature enhancement module includes a Conv_3 convolution layer, a Conv_x convolution layer and an activation unit connected in sequence, and the activation unit is a Gaussian error linear unit activation function.

[0029] Example 3 This embodiment proposes a frequency domain adaptive super-resolution reconstruction method, including the following steps: S1, obtain the original image; S2, processing the original image through the trained frequency domain adaptive super-resolution reconstruction model to obtain a reconstructed image; S2.1, shallow feature extraction module extracts shallow features of original image through nonlinear mapping; S2.2, seven stacked multi-scale frequency domain feature extraction modules process the shallow features of the original image to obtain the deep features of the original image; S2.3, the feature fusion module fuses the deep features of the original image with the shallow features of the original image and rearranges them through sub-pixel convolution to obtain a reconstructed image.

[0030] Example 4 This embodiment proposes a frequency domain adaptive super-resolution reconstruction method, including the following steps: S1, obtain the original image; S2, processing the original image through the trained frequency domain adaptive super-resolution reconstruction model to obtain a reconstructed image; S2.1, shallow feature extraction module extracts shallow features of original image through nonlinear mapping; S2.2, seven stacked multi-scale frequency domain feature extraction modules process the shallow features of the original image to obtain the deep features of the original image; The process of multi-scale frequency domain feature extraction module processing shallow features of original image is as follows: A1. Process the shallow features of the original image through the convolution layer to obtain the distilled features of the original image; A2, the feature enhancement module processes the distilled features of the original image to obtain a multi-scale feature map of the original image; A3, the original image multi-scale feature map is passed through Figure 4 The frequency pyramid attention network shown is used to obtain the deep features of the original image; S2.3, the feature fusion module fuses the deep features of the original image with the shallow features of the original image and rearranges them through sub-pixel convolution to obtain a reconstructed image.

[0031] Example 5 This embodiment proposes a frequency domain adaptive super-resolution reconstruction method, including the following steps: S1, obtain the original image; S2, processing the original image through the trained frequency domain adaptive super-resolution reconstruction model to obtain a reconstructed image; S2.1, shallow feature extraction module extracts shallow features of original image through nonlinear mapping; S2.2, seven stacked multi-scale frequency domain feature extraction modules process the shallow features of the original image to obtain the deep features of the original image; The process of multi-scale frequency domain feature extraction module processing shallow features of original image is as follows: A1. Process the shallow features of the original image through the convolution layer to obtain the distilled features of the original image; A2, the feature enhancement module processes the distilled features of the original image to obtain a multi-scale feature map of the original image; A3, the original image multi-scale feature map is passed through Figure 4 The frequency pyramid attention network shown is used to obtain the deep features of the original image; A3.1. Convert the original image multi-scale feature map from the spatial domain to the frequency domain through Fourier transform; A3.2. The frequency domain features of the real and imaginary parts of the original image multi-scale feature map in the frequency domain are extracted through the convolution layer, and then enhanced and merged in turn according to the attention weight coefficient to obtain the enhanced detail features of the original image; The attention weight coefficient is calculated by the fast Fourier transform loss function; the calculation formula of the fast Fourier transform loss function is as follows:

[0032] in, represents the fast Fourier transform loss value, represents the loss weight coefficient, represents the real part of the predicted image in the frequency domain, represents the real part of the target image in the frequency domain, represents the imaginary part of the predicted image in the frequency domain, Represents the imaginary part of the target image in the frequency domain; A3.3, transform the original image enhancement detail features from the frequency domain to the spatial domain through inverse Fourier transform; A3.4. The original image enhanced detail features are fused with the original image multi-scale feature map to obtain the original image deep features; S2.3, the feature fusion module fuses the deep features of the original image with the shallow features of the original image and rearranges them through sub-pixel convolution to obtain a reconstructed image.

[0033] Example 6 This embodiment proposes a frequency domain adaptive super-resolution reconstruction method, including the following steps: S1, obtain the original image; S2, processing the original image through the trained frequency domain adaptive super-resolution reconstruction model to obtain a reconstructed image; The resolution of the reconstructed image is higher than the original image; The trained frequency domain adaptive super-resolution reconstruction model is obtained through the following steps: B1. Obtain training sets and test sets, and set model training indicators; Model training metrics include peak signal-to-noise ratio and structural similarity index; B2. Use the training set to train the frequency domain adaptive super-resolution reconstruction model; B3. Input the test set into the trained frequency domain adaptive super-resolution reconstruction model to obtain the test result. If the test result meets the model training index, the training is completed and the trained frequency domain adaptive super-resolution reconstruction model is obtained. If the test result does not meet the model training index, repeat steps B2-B3.

[0034] In order to better verify the effectiveness and accuracy of the proposed algorithm, the FDASRN (frequency domain adaptive super-resolution reconstruction method) of the present invention is compared with other super-resolution reconstruction methods such as Bicubic (bicubic interpolation), SRCNN (Super-Resolution Convolutional Neural Network, super-resolution convolutional neural network, ESPCN (Efficient Sub-Pixel Convolutional Neural Network, efficient sub-pixel convolutional neural network), LapSRN (Laplacian Pyramid Super-Resolution Network, Laplacian pyramid super-resolution network). All methods use PSNR and SSIM objective standards to quantitatively judge the reconstruction results of each network model, and the optimal value is displayed in bold.

[0035] Table 1 PSNR (dB) values ​​of different algorithms on Set5 and Set14 datasets

[0036] Table 2 SSIM values ​​of different algorithms on Set5 and Set14 datasets

[0037] As shown in Table 1 and Table 2, when the scaling factors are 2, 3, and 4, respectively, the evaluation values ​​of the present invention and the comparison method on the test data set show that the PSNR value and SSIM value of the proposed algorithm are better than those of the comparison algorithm. This proves that the proposed network structure has a better performance improvement than the comparison algorithm and can obtain better reconstruction results.

Claims

1. A frequency domain adaptive super-resolution reconstruction method, characterized in that: The following steps are involved: S1, obtain the original image; S2. The original image is processed by the trained frequency domain adaptive super-resolution reconstruction model to obtain a reconstructed image.

2. The frequency domain adaptive super-resolution reconstruction method according to claim 1, characterized in that: The frequency domain adaptive super-resolution reconstruction model includes a shallow feature extraction module, a multi-scale frequency domain feature extraction module, and a feature fusion module; The multi-scale frequency domain feature extraction module includes several convolutional layers, feature enhancement modules and frequency pyramid attention network; The feature enhancement module includes a plurality of convolutional layers and activation units connected in sequence, and the activation unit is a Gaussian error linear unit activation function.

3. The frequency domain adaptive super-resolution reconstruction method according to claim 1, characterized in that: The S2 comprises the following steps: S2.1, shallow feature extraction module extracts shallow features of original image through nonlinear mapping; S2.2, several stacked multi-scale frequency domain feature extraction modules process the shallow features of the original image to obtain the deep features of the original image; S2.3, the feature fusion module fuses the deep features of the original image with the shallow features of the original image and rearranges them through sub-pixel convolution to obtain a reconstructed image.

4. The frequency domain adaptive super-resolution reconstruction method according to claim 3, characterized in that: The process of the multi-scale frequency domain feature extraction module processing the shallow features of the original image is as follows: A1. The shallow features of the original image are processed through several convolutional layers to obtain the distilled features of the original image; A2, the feature enhancement module processes the distilled features of the original image to obtain a multi-scale feature map of the original image; A3. Process the multi-scale feature map of the original image through the frequency pyramid attention network to obtain the deep features of the original image.

5. The frequency domain adaptive super-resolution reconstruction method according to claim 4, characterized in that: The A3 comprises the following steps: A3.

1. Convert the original image multi-scale feature map from the spatial domain to the frequency domain through Fourier transform; A3.

2. The frequency domain features of the real and imaginary parts of the original image multi-scale feature map in the frequency domain are extracted through the convolution layer, and then enhanced and merged in turn according to the attention weight coefficient to obtain the enhanced detail features of the original image; A3.3, transform the original image enhancement detail features from the frequency domain to the spatial domain through inverse Fourier transform; A3.

4. The original image enhanced detail features are fused with the original image multi-scale feature map to obtain the original image deep features.

6. The frequency domain adaptive super-resolution reconstruction method according to claim 5, characterized in that: The attention weight coefficient is calculated by the fast Fourier transform loss function; The calculation formula of the fast Fourier transform loss function is as follows: in, represents the fast Fourier transform loss value, represents the loss weight coefficient, represents the real part of the predicted image in the frequency domain, represents the real part of the target image in the frequency domain, represents the imaginary part of the predicted image in the frequency domain, Represents the imaginary part of the target image in the frequency domain.

7. The frequency domain adaptive super-resolution reconstruction method according to claim 1, characterized in that: The trained frequency domain adaptive super-resolution reconstruction model is obtained by the following steps: B1. Obtain training sets and test sets, and set model training indicators; B2. Use the training set to train the frequency domain adaptive super-resolution reconstruction model; B3. Input the test set into the trained frequency domain adaptive super-resolution reconstruction model to obtain the test result. If the test result meets the model training index, the training is completed and the trained frequency domain adaptive super-resolution reconstruction model is obtained. If the test result does not meet the model training index, repeat steps B2-B3.

8. The frequency domain adaptive super-resolution reconstruction method according to claim 7, characterized in that: The model training indicators include peak signal-to-noise ratio and structural similarity index.

9. The frequency domain adaptive super-resolution reconstruction method according to any one of claims 1 to 8, characterized in that: The resolution of the reconstructed image is higher than the original image.

Citation Information

Patent Citations

  • Large-scale MIMO channel state information feedback method based on pseudo complex value input

    CN114157331A

  • Image super-resolution reconstruction method and system based on frequency domain-texture feature fusion

    CN116128722A

  • Fourier lamination microscopic imaging reconstruction method, device and equipment

    CN116579924A

  • Mine image super-resolution reconstruction system and method based on overall attention

    CN117173024A

  • Target re-identification method and system based on target perception high-frequency enhanced Transform network

    CN118587741A

Cited By

  • Image rain removal method based on multi-scale state space model

    CN120031750A

  • Image reconstruction method and system and storage medium

    CN120147137A

  • Super-resolution image enhancement system and method based on variational mode decomposition algorithm

    CN120598777A

  • Frequency aggregation self-attention-based arbitrary-size image enhancement method

    CN120997066A