A frequency domain difference image reconstruction method and system based on SwinIR model

Through the frequency domain differential image reconstruction method based on the SwinIR model, the problems of artifacts and noise in image super-resolution reconstruction are solved by using hierarchical feature extraction and frequency domain differential strategies, and efficient image reconstruction and model generalization are achieved, which is suitable for image denoising and super-resolution reconstruction.

CN120198295BActive Publication Date: 2025-08-26WUHAN UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510674585.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-05-23
Publication Date
2025-08-26
Estimated Expiration
2045-05-23

AI Technical Summary

Technical Problem

With limited data and computing resources, existing image super-resolution reconstruction technology is difficult to effectively reduce artifacts and noise, and the model generalization capabilities are insufficient, which cannot meet the practical application needs.

Method used

The frequency domain differential image reconstruction method based on the SwinIR model is adopted, and the window multi-head self-attention mechanism and frequency domain differential technology of the SwinIR model are used to suppress artifacts and noise, and improve reconstruction quality and model adaptability through hierarchical feature extraction and frequency domain differential strategy.

Benefits of technology

Effectively suppress artifacts and noise in the image reconstruction process, improve image reconstruction quality, reduce computational redundancy, and enhance the model's adaptability in different data scenarios. It is suitable for image denoising, super-resolution reconstruction and image restoration related to frequency domain analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120198295B_ABST
    Figure CN120198295B_ABST
Patent Text Reader

Abstract

The present invention discloses a frequency-domain differential image reconstruction method and system based on the SwinIR model, belonging to the field of image processing technology. The method comprises preprocessing a low-resolution image; performing a Fourier transform on the preprocessed low-resolution image to convert the image from the spatial domain to the frequency domain to obtain an image spectrum representation; inputting the image spectrum representation into a differential mode generator to obtain a differential mode signal image spectrum; inputting the differential mode signal image spectrum into a pre-trained SwinIR model to output a high-resolution reconstructed differential mode image spectrum; performing differential output on the high-resolution reconstructed differential mode image spectrum to obtain a high-resolution reconstructed image spectrum; and performing an inverse Fourier transform on the high-resolution reconstructed image spectrum to output a high-resolution reconstructed image. The present invention effectively suppresses artifacts and noise during image reconstruction, improving reconstruction quality, by utilizing the hierarchical feature extraction and frequency-domain differential strategy of the SwinIR model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a frequency domain differential image reconstruction method and system based on a SwinIR model, and belongs to the technical field of image processing. Background Art

[0002] With the rapid development of artificial intelligence and computer vision technologies, image super-resolution reconstruction has become a key research area in image processing. Super-resolution technology aims to address the problem of insufficient image resolution, particularly in fields such as medical imaging, satellite remote sensing, and surveillance, where high-resolution images are crucial for detailed analysis and decision-making. Traditional methods, such as interpolation algorithms, are simple and fast, but they can easily lose detail. Signal processing-based super-resolution technology has made some progress, but its effectiveness in single-image super-resolution reconstruction is limited, making it difficult to meet practical application requirements. The introduction of deep learning has enabled breakthroughs in super-resolution technology, particularly the convolutional neural network (CNN)-based super-resolution reconstruction method proposed by the SRCNN model in 2014, as well as subsequent methods such as VDSR, ESPCN, and SRGAN, which have significantly improved image reconstruction performance.

[0003] While deep learning can adaptively extract features and process large amounts of data, it also faces challenges. For example, the generated images may contain artifacts, unnatural textures, and noise. Furthermore, the model's heavy reliance on training data and computational complexity limit its practical applications.

[0004] Researchers at home and abroad have proposed some solutions. For example, GAN combined with perceptual loss can generate more realistic details, but it is easy to introduce false artifacts in high-frequency areas, resulting in unnatural texture structures; regularization methods can reduce noise, but will lead to the loss of image details, making the image too smooth. Multi-scale technology and attention mechanisms can better capture image details, but the high computing resource requirements limit their application in real-time processing and resource-constrained environments. Adaptive attention mechanisms have also been applied to super-resolution reconstruction to enhance image details and global consistency and effectively suppress irrelevant noise. By adaptively selecting important areas in the image, the attention mechanism can more intelligently allocate computing resources and improve reconstruction effects. However, the increased network complexity and computational burden make this method challenging in practical applications.

[0005] Therefore, how to reduce artifacts and noise and improve the generalization ability of the model when data and computing resources are limited is an important research topic in current image super-resolution reconstruction. Summary of the Invention

[0006] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a frequency domain differential image reconstruction method and system based on the SwinIR model. Through the hierarchical feature extraction and frequency domain differential strategy of the SwinIR model, the method can effectively suppress artifacts and noise in the image reconstruction process and improve the reconstruction quality. At the same time, the method makes full use of frequency domain information, reduces computational redundancy, and enhances the adaptability and generalization ability of the model in different data scenarios.

[0007] In order to achieve the above objectives / solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0008] A first aspect: A frequency domain difference image reconstruction method based on the SwinIR model, the method comprising:

[0009] Preprocess low-resolution images;

[0010] Perform Fourier transform on the preprocessed low-resolution image to convert the image from the spatial domain to the frequency domain to obtain the image spectrum representation;

[0011] Input the image spectrum representation into the differential mode generator to obtain the differential mode signal image spectrum;

[0012] The differential mode signal image spectrum is input into the pre-trained SwinIR model, and the high-resolution reconstructed differential mode image spectrum is output;

[0013] Perform differential output on the high-resolution reconstructed differential mode image spectrum to obtain a high-resolution reconstructed image spectrum;

[0014] The high-resolution reconstructed image spectrum is subjected to inverse Fourier transform to output a high-resolution reconstructed image.

[0015] Optionally, the SwinIR model includes a shallow feature extraction layer, a deep feature extraction layer and a high-quality image spectrum reconstruction layer, the shallow feature extraction layer includes a convolutional network layer and a LeakyReLU activation layer, wherein the number of input channels of the convolutional network layer is the number of channels of the low-resolution image spectrum, and the number of output channels is the dimension of the shallow feature tensor, and the deep feature extraction layer includes 6 RSTB modules and 1 3×3 convolutional network layer; the high-quality image spectrum reconstruction layer includes 1 jump connection module, 1 upsampling module and 1 convolutional network layer, and the jump connection module directly adds the shallow feature tensor to the deep feature tensor; the number of input channels of the convolutional network layer is the dimension of the shallow feature tensor, and the number of output channels is the number of channels of the high-resolution image spectrum.

[0016] Optionally, the deep feature extraction layer processing process is as follows:

[0017] The residual block module processes the input signal and extracts features through a windowed multi-head self-attention mechanism. The windowed multi-head self-attention mechanism divides the extracted features into fixed 7×7 windows and independently calculates self-attention within each window to capture detailed features.

[0018] The captured detail features enter the offset window multi-head self-attention mechanism, and the position of the window is offset by 3×3 in the adjacent layer; the input features and the features processed by the attention mechanism are added through a multi-layer perceptron module, and the window multi-head self-attention mechanism and the offset window multi-head self-attention mechanism are alternately cycled, and the features are integrated through a 3×3 convolutional network layer to obtain a set of deep features.

[0019] Optionally, inputting the image spectrum representation into a differential mode generator to obtain the differential mode signal image spectrum includes:

[0020] The differential mode generator performs differential mode on the low-resolution image spectrum after Fourier transformation to obtain two low-resolution image spectra, one of which is the original low-resolution image spectrum and the other is the inverse low-resolution image spectrum for input into the SwinIR model. The inverse low-resolution image refers to the original image with all pixels converted according to the The relationship is mapped one to one.

[0021] Optionally, the preprocessing of the low-resolution image includes:

[0022] Perform image normalization preprocessing on the input low-resolution image.

[0023] Optionally, the image normalization preprocessing includes: mapping the range of pixel values ​​to 0 to 1.

[0024] Optionally, performing differential output on the high-resolution reconstructed differential mode image spectrum to obtain the high-resolution reconstructed image spectrum includes:

[0025] The high-resolution reconstructed differential mode image spectrum is differentially output to remove common mode noise, wherein the differential output represents the subtraction of the high-resolution reconstructed differential mode image spectrum from the inverse low-resolution image spectrum of the differential mode output.

[0026] Optionally, the training process of the SwinIR model includes:

[0027] Preprocessing the DIV2K dataset containing high- and low-resolution images includes normalizing the input images to map pixel values ​​to a range of 0 to 1, and dividing the processed dataset into a training set and a test set.

[0028] Perform Fourier transform on the dataset image to obtain low-resolution image spectrum and high-resolution image spectrum;

[0029] Inputting the low-resolution image spectrum into the differential mode generator to obtain the differential mode signal image spectrum;

[0030] The differential signal image spectrum and the corresponding high-resolution image spectrum are input into the SwinIR model network for training;

[0031] The back-propagation algorithm continuously optimizes the model parameters to obtain the trained SwinIR model.

[0032] Optionally, the training further includes:

[0033] By comparing the difference between the high-resolution image spectrum output by the SwinIR model and the target high-resolution image spectrum, the loss value is calculated using the pixel-level L1 loss function;

[0034] The SwinIR model uses the calculated loss value to pass the error gradient back to each parameter of the SwinIR model layer by layer through the back propagation algorithm;

[0035] Use the optimizer to update each parameter according to the learning rate so that the generated high-resolution spectrum approaches the true target spectrum. Repeat until the loss curve stabilizes and the indicators on the validation set reach the optimal level, indicating that the model has converged and training is complete.

[0036] A second aspect: A frequency domain difference image reconstruction system based on the SwinIR model, the system comprising:

[0037] Image processing module, used for preprocessing low-resolution images;

[0038] The image conversion module is used to perform Fourier transform on the pre-processed low-resolution image, converting the image from the spatial domain to the frequency domain to obtain the image spectrum representation;

[0039] A generating module, configured to input the image spectrum representation into a differential mode generator to obtain a differential mode signal image spectrum;

[0040] The differential mode image spectrum reconstruction module is used to input the differential mode signal image spectrum into the pre-trained SwinIR model and output a high-resolution reconstructed differential mode image spectrum;

[0041] A differential processing module is used to perform differential output on the high-resolution reconstructed differential mode image spectrum to obtain a high-resolution reconstructed image spectrum;

[0042] The high-resolution reconstruction module is used to perform inverse Fourier transform on the high-resolution reconstructed image spectrum and output a high-resolution reconstructed image.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The present invention is used to extract and optimize frequency domain features. It improves the expressiveness of frequency domain features through a deep learning network, effectively solves the problems of noise interference and artifacts, and ultimately achieves efficient reconstruction of low-resolution images to high-resolution images.

[0045] The present invention effectively suppresses artifacts and noise in the image reconstruction process and improves the reconstruction quality through the hierarchical feature extraction and frequency domain difference strategy of the SwinIR model. It is suitable for image denoising, super-resolution reconstruction, and image restoration related to frequency domain analysis, and has important application value, especially in fields with high demand for high-quality image reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0047] Figure 2 This is a diagram of the SwinIR model structure of an embodiment of the present invention;

[0048] Figure 3 This is a SwinIR model training flowchart of an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below in conjunction with specific implementation methods.

[0050] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0051] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0052] Example 1, as Figure 1 The invention discloses a frequency domain differential image reconstruction method based on the SwinIR model, comprising:

[0053] Step 1: Input a low-resolution image. The low-resolution image to be processed is used as input.

[0054] Step 2: Image preprocessing, perform normalization preprocessing operations on the input image.

[0055] Step 3: Perform Fourier transform on the preprocessed image to convert the image from the spatial domain to the frequency domain to obtain the image spectrum representation for subsequent processing.

[0056] Step 4: Input the differential mode generator to obtain the differential mode signal (image) spectrum, and use the differential mode generator to extract the differential mode signal (image) spectrum from the input spectrum.

[0057] Step 5: Input the differential mode signal (image) spectrum into the SwinIR model. The differential mode signal (image) spectrum is used as input and sent into the SwinIR model to obtain a high-resolution reconstructed differential mode image spectrum.

[0058] In the specific implementation process of this embodiment, for step 2, the input low-resolution image is subjected to image normalization preprocessing, and the range of pixel values ​​is mapped to 0 to 1 to ensure that the image data has a uniform scale and range in the subsequent processing process, thereby improving the stability and efficiency of the algorithm and eliminating interference caused by differences in brightness, contrast, etc. between different images.

[0059] In the specific implementation process of this embodiment, for step 4, the differential mode generator first performs differential mode on the low-resolution image spectrum after Fourier transformation to obtain two low-resolution image spectra, one of which is the original low-resolution image spectrum and the other is the inverse low-resolution image spectrum for input into the SwinIR model. The inverse low-resolution image refers to the original image with all pixels converted according to The relationship is mapped one to one.

[0060] Please see Figure 2In step 5, during the specific implementation process of this embodiment, the SwinIR model includes a shallow feature extraction layer, a deep feature extraction layer, and a high-quality image spectrum reconstruction layer. The shallow feature extraction layer includes a convolutional network layer and a LeakyReLU activation layer, wherein the number of input channels of the convolutional network layer is the number of channels of the low-resolution image spectrum, and the number of output channels is the dimension of the shallow feature tensor. The deep feature extraction layer includes 6 RSTB modules and 1 3×3 convolutional network layer. The high-quality image spectrum reconstruction layer includes 1 skip connection module, 1 upsampling module, and 1 convolutional network layer. The skip connection module directly adds the shallow feature tensor to the deep feature tensor. The number of input channels of the convolutional network layer is the dimension of the shallow feature tensor, and the number of output channels is the number of channels of the high-resolution image spectrum.

[0061] To further elaborate on step 5, the deep feature extraction layer processing process is as follows:

[0062] The residual block module processes the input signal and extracts features through a windowed multi-head self-attention mechanism. The windowed multi-head self-attention mechanism divides the extracted features into fixed 7×7 windows and independently calculates self-attention within each window to capture detailed features.

[0063] The captured detail features enter the offset window multi-head self-attention mechanism, and the position of the window is offset by 3×3 in the adjacent layer; the input features and the features processed by the attention mechanism are added through a multi-layer perceptron module, and the window multi-head self-attention mechanism and the offset window multi-head self-attention mechanism are alternately cycled, and the features are integrated through a 3×3 convolutional network layer to obtain a set of deep features.

[0064] Please see Figure 3 In one embodiment, the SwinIR model is a trained SwinIR model; the training process includes the following sub-steps:

[0065] Step S5.1, using the DIV2K dataset containing high- and low-resolution images;

[0066] In one embodiment, the DIV2K dataset contains a variety of image scenes, including natural scenery, buildings, and objects. This diversity enables trained and tested models to better generalize to different real-world application scenarios, rather than simply performing well on a single scene. DIV2K is also a standard dataset in the field of image super-resolution, widely used in academic research and industrial applications. Experimental results using the DIV2K dataset facilitate fair comparisons with other research works, thereby measuring the superiority and performance of the proposed methods.

[0067] Step S5.2, preprocessing the data set;

[0068] In one embodiment, the preprocessing includes normalizing the input image by mapping the pixel values ​​to a range of 0 to 1, and then dividing the processed data set into a training set and a test set;

[0069] Step S5.3, performing Fourier transform on the dataset image to obtain a low-resolution image spectrum and a high-resolution image spectrum;

[0070] Step S5.4, inputting the low-resolution image spectrum into the differential mode generator to obtain the differential mode signal (image) spectrum;

[0071] In one embodiment, the differential mode generator first performs differential mode on the low-resolution image spectrum after Fourier transformation to obtain two low-resolution image spectra, one of which is the original low-resolution image spectrum and the other is the inverse low-resolution image spectrum for input into the SwinIR model. The inverse low-resolution image refers to the original image with all pixels converted according to The relationship is mapped one to one.

[0072] Step S5.5, inputting the differential mode signal (image) spectrum and the corresponding high-resolution image spectrum into the SwinIR model network for training;

[0073] Step S5.6, the back propagation algorithm continuously optimizes the model parameters;

[0074] In one embodiment, the training process uses the SwinIR model for training, and the loss value is calculated using the pixel-level L1 loss function by comparing the difference between the high-resolution image spectrum output by the model and the target high-resolution image spectrum. The model uses the calculated loss value to pass the error gradient back to the various parameters of the model layer by layer through the back propagation algorithm, including the convolution layer weights of the shallow feature extraction module, the weights and biases of the attention mechanism in the RSTB in the deep feature extraction module, etc. The optimizer is used to update these parameters according to the learning rate, so that the generated high-resolution spectrum gradually approaches the true target spectrum. This process is repeated until the loss curve tends to stabilize and the indicators on the validation set reach the optimal level, indicating that the model has converged and the training is complete.

[0075] The SwinIR model has the same structure as the SwinIR model described in step 4.

[0076] Step 6: Reconstruct the differential mode image spectrum with high resolution for differential output, remove common mode noise, and obtain a high-resolution image spectrum.

[0077] In one embodiment, the differential output represents a subtraction of a high-resolution reconstructed differential mode image spectrum from an inverse low-resolution image spectrum of the differential mode output to obtain a high-resolution reconstructed image spectrum.

[0078] Step 7: Inverse Fourier transform to output high-resolution image.

[0079] In one embodiment, the inverse Fourier transform outputs a high-resolution image by performing an inverse Fourier transform on the spectrum of the high-resolution reconstructed image to obtain a final high-resolution reconstructed image.

[0080] Embodiment 2 discloses a frequency domain differential image reconstruction system based on the SwinIR model, the system comprising:

[0081] Image processing module, used for preprocessing low-resolution images;

[0082] The image conversion module is used to perform Fourier transform on the pre-processed low-resolution image, converting the image from the spatial domain to the frequency domain to obtain the image spectrum representation;

[0083] A generating module, configured to input the image spectrum representation into a differential mode generator to obtain a differential mode signal image spectrum;

[0084] The differential mode image spectrum reconstruction module is used to input the differential mode signal image spectrum into the pre-trained SwinIR model and output a high-resolution reconstructed differential mode image spectrum;

[0085] A differential processing module is used to perform differential output on the high-resolution reconstructed differential mode image spectrum to obtain a high-resolution reconstructed image spectrum;

[0086] The high-resolution reconstruction module is used to perform inverse Fourier transform on the high-resolution reconstructed image spectrum and output a high-resolution reconstructed image.

[0087] The present invention is used to extract and optimize frequency domain features. It improves the expressiveness of frequency domain features through a deep learning network, effectively solves the problems of noise interference and artifacts, and ultimately achieves efficient reconstruction of low-resolution images to high-resolution images.

[0088] The present invention effectively suppresses artifacts and noise in the image reconstruction process and improves the reconstruction quality through the hierarchical feature extraction and frequency domain difference strategy of the SwinIR model. It is suitable for image denoising, super-resolution reconstruction, and image restoration related to frequency domain analysis, and has important application value, especially in fields with high demand for high-quality image reconstruction.

[0089] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A frequency domain difference image reconstruction method based on the SwinIR model, characterized in that: The method comprises: Preprocess low-resolution images; Perform Fourier transform on the preprocessed low-resolution image to convert the image from the spatial domain to the frequency domain to obtain the image spectrum representation; Input the image spectrum representation into the differential mode generator to obtain the differential mode signal image spectrum; The differential mode signal image spectrum is input into the pre-trained SwinIR model, and the high-resolution reconstructed differential mode image spectrum is output; Perform differential output on the high-resolution reconstructed differential mode image spectrum to obtain a high-resolution reconstructed image spectrum; Perform inverse Fourier transform on the high-resolution reconstructed image spectrum to output a high-resolution reconstructed image; The SwinIR model includes a shallow feature extraction layer, a deep feature extraction layer, and a high-quality image spectrum reconstruction layer. The shallow feature extraction layer includes a convolutional network layer and a LeakyReLU activation layer, wherein the number of input channels of the convolutional network layer is the number of channels of the low-resolution image spectrum, and the number of output channels is the dimension of the shallow feature tensor. The deep feature extraction layer includes 6 RSTB modules and 1 3×3 convolutional network layer; the high-quality image spectrum reconstruction layer includes 1 skip connection module, 1 upsampling module, and 1 convolutional network layer. The skip connection module directly adds the shallow feature tensor to the deep feature tensor; the number of input channels of the convolutional network layer is the dimension of the shallow feature tensor, and the number of output channels is the number of channels of the high-resolution image spectrum; the processing process of the deep feature extraction layer is as follows: The residual block module processes the input signal and extracts features through a windowed multi-head self-attention mechanism. The windowed multi-head self-attention mechanism divides the extracted features into fixed 7×7 windows and independently calculates self-attention within each window to capture detailed features. The captured detail features enter the offset window multi-head self-attention mechanism, and the position of the window is offset by 3×3 in the adjacent layer; the input features and the features processed by the attention mechanism are added through a multi-layer perceptron module, and the window multi-head self-attention mechanism and the offset window multi-head self-attention mechanism are alternately cycled, and the features are integrated through a 3×3 convolutional network layer to obtain a set of deep features.

2. The frequency domain difference image reconstruction method based on the SwinIR model according to claim 1, characterized in that: The step of inputting the image spectrum representation into the differential mode generator to obtain the differential mode signal image spectrum includes: The differential mode generator performs differential mode on the low-resolution image spectrum after Fourier transformation to obtain two low-resolution image spectra, one of which is the original low-resolution image spectrum and the other is the inverse low-resolution image spectrum for input into the SwinIR model. The inverse low-resolution image refers to the original image with all pixels converted according to the The relationship is mapped one to one.

3. The frequency domain difference image reconstruction method based on the SwinIR model according to claim 1, characterized in that: The preprocessing of the low-resolution image includes: Perform image normalization preprocessing on the input low-resolution image.

4. The frequency domain difference image reconstruction method based on the SwinIR model according to claim 3, characterized in that: Image normalization preprocessing includes mapping the range of pixel values ​​to 0 to 1.

5. The frequency domain difference image reconstruction method based on the SwinIR model according to claim 1, characterized in that: The step of performing differential output on the high-resolution reconstructed differential mode image spectrum to obtain the high-resolution reconstructed image spectrum includes: The high-resolution reconstructed differential mode image spectrum is differentially output to remove common mode noise, wherein the differential output represents the subtraction of the high-resolution reconstructed differential mode image spectrum from the inverse low-resolution image spectrum of the differential mode output.

6. The frequency domain difference image reconstruction method based on the SwinIR model according to claim 1, characterized in that: The training process of the SwinIR model includes: Preprocessing the DIV2K dataset containing high- and low-resolution images includes normalizing the input images to map pixel values ​​to a range of 0 to 1, and dividing the processed dataset into a training set and a test set. Perform Fourier transform on the dataset image to obtain low-resolution image spectrum and high-resolution image spectrum; Inputting the low-resolution image spectrum into the differential mode generator to obtain the differential mode signal image spectrum; The differential signal image spectrum and the corresponding high-resolution image spectrum are input into the SwinIR model network for training; The back-propagation algorithm continuously optimizes the model parameters to obtain the trained SwinIR model.

7. The frequency domain difference image reconstruction method based on the SwinIR model according to claim 6, characterized in that: The training also includes: By comparing the difference between the high-resolution image spectrum output by the SwinIR model and the target high-resolution image spectrum, the loss value is calculated using the pixel-level L1 loss function; The SwinIR model uses the calculated loss value to pass the error gradient back to each parameter of the SwinIR model layer by layer through the back propagation algorithm; Use the optimizer to update each parameter according to the learning rate so that the generated high-resolution spectrum approaches the true target spectrum. Repeat until the loss curve stabilizes and the indicators on the validation set reach the optimal level, indicating that the model has converged and training is complete.

8. A frequency domain difference image reconstruction system based on the SwinIR model according to any one of claims 1 to 7, characterized in that: The system comprises: Image processing module, used for preprocessing low-resolution images; The image conversion module is used to perform Fourier transform on the pre-processed low-resolution image, converting the image from the spatial domain to the frequency domain to obtain the image spectrum representation; A generating module, configured to input the image spectrum representation into a differential mode generator to obtain a differential mode signal image spectrum; The differential mode image spectrum reconstruction module is used to input the differential mode signal image spectrum into the pre-trained SwinIR model and output a high-resolution reconstructed differential mode image spectrum; A differential processing module is used to perform differential output on the high-resolution reconstructed differential mode image spectrum to obtain a high-resolution reconstructed image spectrum; The high-resolution reconstruction module is used to perform inverse Fourier transform on the high-resolution reconstructed image spectrum and output a high-resolution reconstructed image.

Citation Information

Patent Citations

  • Image reconstruction method and system based on differential output

    CN117455774A

  • Image super-resolution reconstruction method and device and storage medium

    CN119228649A