Image restoration method combining fast Fourier convolution and diffusion model
By combining Fast Fourier Convolution and Diffusion Models in an image inpainting method, the problems of network instability and slow iteration process in generative deep learning are solved, achieving high-quality and efficient image inpainting results.
Patent Information
- Application Number
- CN202511530612.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-24
- Publication Date
- 2026-02-27
AI Technical Summary
Existing generative deep learning methods suffer from network instability, low quality of generated images, and slow inference speed in image inpainting, especially in diffusion-based image inpainting, where the Markov chain iteration process affects algorithm efficiency.
By combining Fast Fourier Convolution and Diffusion models, a Fast Fourier Convolution network is introduced into the reverse diffusion process to perform Fourier transform and convolution operations, processing the frequency and temporal features of the image. A hybrid loss function is used to evaluate the model performance.
It improves the quality and efficiency of image restoration, reduces algorithm complexity, maintains the accuracy of image restoration or repair, and enhances the execution efficiency of image restoration processing.
Smart Images
Figure CN121582104A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of image processing, and particularly relates to an image restoration method combining fast Fourier convolution and diffusion model. BACKGROUND
[0002] Low-resolution image restoration technology plays an important role in the field of computer vision. Since generative deep learning methods perform well in generating image visual perception and image fidelity, generative deep learning has become the main research direction of image restoration. According to the network structure of generative deep learning, it can be divided into generative adversarial method, standardized flow method and diffusion method. The generative adversarial method solves the limitations of convolutional neural network in high-fidelity detail recovery of images through the adversarial game between the generator and the discriminator; the core is to extract high-frequency features by using residual structure, and to optimize the visual quality of the generated image by combining adversarial loss and content loss, so as to realize high-fidelity image reconstruction from low-resolution to high-resolution image; but the stability of the generative adversarial method is affected by the generator and the discriminator, and if one of them has unstable network structure, the performance of the whole network will be affected. The standardized flow method converts a simple distribution into a target distribution through a series of reversible transformations; the reversibility ensures that the generation process is reversible, so that the likelihood value can be calculated through the reverse flow; but the standardized flow method has low image quality in terms of image details and texture richness. The diffusion method realizes image super-resolution through the iterative process of forward diffusion and reverse denoising in the diffusion model; the core is to input the low-resolution image as a condition to guide the model to gradually recover the high-resolution image consistent with the content of the low-resolution image from random noise, while maintaining the diversity and detail authenticity of the generated results; but this method is affected by the Markov chain iteration process, which slows down the inference speed and affects the efficiency of the algorithm. SUMMARY
[0003] The present application provides an image restoration method combining fast Fourier convolution and diffusion model, which introduces fast Fourier convolution network for Fourier transform and convolution operation of the image in the reverse diffusion process of the diffusion model, to improve the quality of image restoration.
[0004] Technical scheme: To achieve the above object, the present application provides an image restoration method combining fast Fourier convolution and diffusion model, comprising the following steps:
[0005] S1, acquiring an image data set and dividing it into a training set and a test set, and inputting the training set and the test set into a diffusion model for training and testing, respectively;
[0006] S2, in the forward diffusion process in the diffusion model, Gaussian noise is gradually added to the input initial image until the input initial image becomes a low-resolution image with Gaussian noise;
[0007] S3, in the reverse diffusion process in the diffusion model, the low-resolution image with Gaussian noise is input into the neural network, a fast Fourier convolution network is used as the neural network for image prediction, and the low-resolution image with Gaussian noise is gradually removed to obtain the final image data after repair;
[0008] The fast Fourier convolution network performs feature processing on the low-resolution image with Gaussian noise in the frequency domain and the time domain to obtain a first feature image and a second feature image; the first feature image and the second feature image obtained are spliced and fused to obtain an image after fast Fourier convolution network denoising.
[0009] Further, in the forward diffusion process in the diffusion model, Gaussian noise is gradually added to the input initial image until the input initial image becomes a low-resolution image with Gaussian noise; in the forward diffusion process, x0~q(x0) is set as the probability distribution of the initial image after T-step addition of Gaussian noise to obtain a low-resolution image x T with Gaussian noise, in the forward diffusion process, x t is only related to x t-1 at the previous time, and the calculation process of gradually adding Gaussian noise is as follows:
[0010]
[0011]
[0012] ,
[0013] In the formula, βt represents the weight of the Gaussian noise added at time t, the range value of β is a linear transformation of 10 -4 → 2 × 10 -2 , and ε represents a standard Gaussian noise, ε∈N(0, I).
[0014] Further, in the reverse diffusion process in the diffusion model, a fast Fourier convolution network is used as a neural network for image prediction, and the low-resolution image with Gaussian noise is gradually removed to obtain the final image data after repair; in the reverse diffusion process, the low-resolution image with Gaussian noise x T is gradually restored to the initial image data, and then the final image data after repair is obtained; in the denoising diffusion probability model, a neural network p θ (x t-1 |xt To fit the reverse diffusion process, the calculation process is as follows:
[0015]
[0016]
[0017]
[0018] In the formula, p θ (x t-1 |x t This represents the denoising process during reverse diffusion, where the input x is denoised. t Restore to x t-1 The probability distribution, μ θ (x t Let Σ(t) represent the expected mean at time t. θ (x t Let ,t) represent the expected variance at time t; βt-1 represents the weight of the Gaussian noise added at time t-1, and the range of β is 10. -4 → 2 × 10 -2 The linear transformation of ; ε represents standard Gaussian noise, ε∈N(0,I); ε θ (x t Let ,t) represent the mapping relationship of the neural network at time t, and σ t This is expressed as the noise oscillation amplitude.
[0019] Furthermore, the Fast Fourier Convolutional Network includes a dual-branch network constructed from a convolutional network and a Fast Fourier Network; in the convolutional network, a small kernel convolutional network is used to perform convolution processing on low-resolution images with Gaussian noise, and image features are processed in the temporal domain to obtain a convolutional feature image; in the Fast Fourier Network, the low-resolution image with Gaussian noise is sequentially subjected to Fast Fourier Transform, high-pass or low-pass filter filtering, and Fast Inverse Fourier Transform to obtain a first initial feature image and a second initial feature image; the first initial feature image and the second initial feature image are obtained by fusing them with the convolutional feature image respectively.
[0020] Furthermore, the Fast Fourier Transform (FFT) network also includes two branch networks. One branch network converts the low-resolution image with Gaussian noise into the frequency domain using FFT, filters it using a high-pass filter, and then converts it into the time domain using FIRST to obtain a first initial feature image. The first initial feature image is then fused with the convolutional feature image obtained through a small kernel convolutional network to obtain the first feature image F. h The calculation process is as follows:
[0021]
[0022] In the formula, I is a low-resolution image with Gaussian noise, FFT is a fast Fourier transform, HPF is a filter of a high-pass filter, iFFT is a fast inverse Fourier transform, and Conv is a convolution operation in a small kernel convolution network.
[0023] Further, in the fast Fourier network, another branch network converts the low-resolution image with Gaussian noise into a frequency domain through a fast Fourier transform, filters through a low-pass filter, and converts into a time domain through a fast inverse Fourier transform to obtain a second initial feature image; the second initial feature image is fused with a convolution feature image obtained through the small kernel convolution network to obtain a second feature image F l ; the calculation process is as follows:
[0024]
[0025] In the formula, I is a low-resolution image with Gaussian noise, FFT is a fast Fourier transform, LPF is a filter of a low-pass filter, iFFT is a fast inverse Fourier transform, and Conv is a convolution operation in a small kernel convolution network.
[0026] Further, the first feature image and the second feature image are spliced and fused through a feature fusion module to obtain denoised image data, and the calculation process is as follows:
[0027]
[0028] In the formula, I predict is denoised image data, F h is the first feature image, and F l is the second feature image.
[0029] Further, in the fast Fourier convolution network, to evaluate the effect of the fast Fourier convolution network, a hybrid loss function is used to evaluate the performance of the model in the time domain and the frequency domain; the calculation formula of the hybrid loss function is as follows:
[0030]
[0031] In the formula, I0 is an initial image, and E represents a mathematical expectation.
[0032] Beneficial effects: the image restoration method combining the fast Fourier convolution and the diffusion model, in the inverse diffusion process of the diffusion model, the fast Fourier convolution network is introduced to perform Fourier transform and convolution operation on the image, the quality of the image restoration is improved; the combination of the fast Fourier transform and the convolution network is adopted, the complexity of the algorithm is reduced while the accuracy of the image restoration or repair is maintained, the execution efficiency of the image restoration processing is effectively improved. BRIEF DESCRIPTION OF DRAWINGS
[0033] Figure 1 is a flow chart of the image restoration method combining the fast Fourier convolution and the diffusion model;
[0034] Figure 2 is a diffusion model demonstration diagram;
[0035] Figure 3 is a convolution network structure diagram combining the Fourier transform;
[0036] Figure 4 is a qualitative comparison diagram of various algorithms and the present application on the DIV2K data set. DETAILED DESCRIPTION
[0037] The present application will be further described below in combination with the drawings.
[0038] As shown in Figure 1 , an image restoration method combining fast Fourier convolution and diffusion model, comprising the following steps:
[0039] S1, acquiring an image data set and dividing it into a training set and a test set, and inputting the training set and the test set into a diffusion model for training and testing;
[0040] S2, in the forward diffusion process in the diffusion model, gradually adding Gaussian noise to the input initial image until the input initial image becomes a low-resolution image with Gaussian noise; the forward diffusion process simulates the entropy increase process of physical diffusion, and provides a learning goal for the inverse diffusion process;
[0041] S3, in the inverse diffusion process in the diffusion model, inputting the low-resolution image with Gaussian noise into a neural network, using a fast Fourier convolution network as the neural network to predict the image, and gradually removing the Gaussian noise from the low-resolution image with Gaussian noise to obtain the final image data after restoration; the inverse diffusion process realizes data generation by predicting noise distribution, and the essence is approximate sampling of data distribution, which has the advantage of increasing learning samples and better training model effect;
[0042] The fast Fourier convolution network performs feature processing on the low-resolution image with Gaussian noise in the frequency domain and the time domain to obtain a first feature image and a second feature image; the first feature image and the second feature image are spliced and fused to obtain an image denoised by the fast Fourier convolution network; the fast Fourier convolution network is used to perform a denoising operation on the low-resolution image with Gaussian noise, that is, one-step denoising operation, and the denoising operation of the fast Fourier convolution network is gradually performed in the reverse diffusion process, and finally the repaired image data is obtained.
[0043] As shown in Figure 2 , in the forward diffusion process in the diffusion model, Gaussian noise is gradually added to the input initial image until the input initial image becomes a low-resolution image with Gaussian noise; in the forward diffusion process, x0~q(x0) is set as the probability distribution of the low-resolution image x T with Gaussian noise obtained by adding Gaussian noise to the initial image for T steps; since the noise adding process in the forward diffusion process is affected by the Markov chain, x t in the forward diffusion process is only related to x t-1 at the previous time, and the calculation process of gradually adding Gaussian noise is as follows:
[0044]
[0045]
[0046] ,
[0047] In the formula, βt represents the weight of the Gaussian noise added at t time, the range value of β is a linear transformation of 10 -4 → 2 × 10 -2 , ε represents a standard Gaussian noise, ε∈N(0, I), where I is a covariance matrix, which is generally a unit matrix; x0 represents the input initial image, xt represents the low-resolution image with Gaussian noise at t time; T can represent the number of noise adding, or the forward diffusion process is performed for T rounds, q((x 1:T )|x0) represents the probability distribution of the low-resolution image x T with Gaussian noise after T times of adding Gaussian noise.
[0048] In the reverse diffusion process in the diffusion model, a fast Fourier convolution network is used as a neural network to predict an image, and the low-resolution image with Gaussian noise is gradually removed to obtain the repaired final image data; in the reverse diffusion process, the low-resolution image x TBy gradually restoring the initial image data, the final repaired image data can be obtained; the probability distribution of the reverse diffusion process can then be obtained from the low-resolution image x with Gaussian noise. T The original effect image is gradually restored, therefore a neural network p is used in the denoising diffusion probability model. θ (x t-1 |x t To fit the reverse diffusion process, the calculation process is as follows:
[0049]
[0050]
[0051] ,
[0052] In the formula, p θ (x t-1 |x t This represents the denoising process during reverse diffusion, where the input x is denoised. t Restore to x t-1 The probability distribution, μ θ (x t ,t) is represented as x t The expected mean at time t, Σ θ (x t Let x be the expression for t). t The expected variance at time t; βt-1 represents the weight of the Gaussian noise added at time t-1, and the range of β is 10. -4 → 2 × 10 -2 The linear transformation of; ε represents standard Gaussian noise, ε∈N(0,I), where I is the covariance matrix, generally the identity matrix; ε θ (x t Let ,t) represent the mapping relationship of the neural network at time t, and σ t The noise oscillation amplitude is represented by x0, where x0 represents the initial input image, and N is a normal distribution. The neural network is a Fast Fourier Convolutional Network, p θ (x t-1 |x t ε can be understood as the probability representation of a Fast Fourier Convolutional Network. θ (x t ,t) can be understood as a symbolic representation of a Fast Fourier Convolutional Network.
[0053] In the reverse diffusion process, with ε θIn the neural network of (xt, t), a fast Fourier convolution network is used to replace a traditional convolution; the traditional convolution is difficult to achieve optimal effects of both the convolution kernel size and the receptive field, while the fast Fourier transform is directed to the entire frequency domain information, and the fast Fourier convolution network has higher execution efficiency than the traditional convolution, so the fast Fourier convolution network is introduced into the reverse diffusion process.
[0054] The small-kernel-based convolution operation can efficiently process features, but limits the receptive field of the convolution, which is not conducive to the processing of global features, but increasing the convolution kernel to expand the receptive field will affect the efficiency of the convolution operation, so it is difficult to achieve optimal effects of both the size of the convolution kernel and the receptive field. However, the Fourier transform converts the time domain into the frequency domain for processing, and is directed to the entire frequency domain, and the algorithm complexity of the fast Fourier transform is lower than that of the convolution; the complexity of the direct convolution is O(n2), and the fast Fourier transform can be calculated in O(n log n) time; therefore, the small-kernel convolution network and the fast Fourier transform are combined to process features of an image; the combination of the small-kernel convolution network and the fast Fourier transform enables the diffusion model to improve the inference speed and enhance the accuracy of image restoration or repair in the image restoration process.
[0055] As shown in Figure 3 , the fast Fourier convolution network includes a double-branch network constructed by a convolution network and a fast Fourier network; in the convolution network, a small-kernel convolution network is used to convolve a low-resolution image with Gaussian noise to process features of the image in the time domain and obtain a convolution feature image; in the fast Fourier network, the low-resolution image with Gaussian noise is sequentially subjected to fast Fourier transform, high-pass or low-pass filter filtering, and fast inverse Fourier transform to obtain a first initial feature image and a second initial feature image; the first initial feature image and the second initial feature image are fused with the convolution feature image respectively to obtain a first feature image and a second feature image. The small-kernel convolution network is a 3x3 convolution kernel convolution network; the fast Fourier network is divided into two branch networks of high-pass filtering and low-pass filtering, which mainly separate the processing of high-frequency signals and low-frequency signals to improve the processing efficiency and the image repair accuracy.
[0056] The fast Fourier network also includes two branch networks, one of which converts a low-resolution image with Gaussian noise into a frequency domain through fast Fourier transform, filters the image through a high-pass filter to reduce noise interference, and converts the image into a time domain through fast inverse Fourier transform to obtain a first initial feature image; the first initial feature image is fused with a convolution feature image obtained through a small-kernel convolution network to obtain a first feature image F h ; the calculation process is as follows:
[0057]
[0058] In the formula, I is a low-resolution image with Gaussian noise, FFT is a fast Fourier transform, HPF is a filter of a high-pass filter, iFFT is a fast inverse Fourier transform, and Conv is a convolution operation in a small kernel convolution network.
[0059] In the fast Fourier network, another branch network converts the low-resolution image with Gaussian noise into a frequency domain through a fast Fourier transform, filters through a low-pass filter to reduce the interference of noise, and converts into a time domain through a fast inverse Fourier transform to obtain a second initial feature image; the second initial feature image is fused with a convolution feature image obtained through the small kernel convolution network to obtain a second feature image F l ; the calculation process is as follows:
[0060]
[0061] In the formula, I is a low-resolution image with Gaussian noise, FFT is a fast Fourier transform, HPF is a filter of a high-pass filter, iFFT is a fast inverse Fourier transform, and Conv is a convolution operation in a small kernel convolution network.
[0062] The first feature image and the second feature image are spliced and fused through the feature fusion module to obtain denoised image data, and the calculation process is as follows:
[0063]
[0064] In the formula, I predict is denoised image data, F h is the first feature image, and F l is the second feature image. For example, a low-resolution image x t with high-speed noise is subjected to a denoising operation of the fast Fourier convolution network to obtain denoised image data I predict , which is the low-resolution image x t-1 with high-speed noise, and then the low-resolution image x t-1 with high-speed noise is subjected to a denoising operation of the fast Fourier convolution network again to obtain denoised image data I predict , which is the low-resolution image x t-2 with high-speed noise. In this way, step-by-step denoising operations are performed in the reverse diffusion process to obtain the final image data after repair.
[0065] In the fast Fourier convolution network, in order to evaluate the effect of the fast Fourier convolution network, a hybrid loss function is used to evaluate the performance of the model in the time domain and the frequency domain; the calculation formula of the hybrid loss function is as follows:
[0066]
[0067] where I0 is the initial image, is the loss of the fast Fourier convolution network and the real image, is the loss of the convolution neural network and the real image, I FFC represents the predicted image processed by the fast Fourier convolution network FFC, and is also understood as the final repaired image data; I conv represents the image predicted by the convolution network, and E represents the mathematical expectation.
[0068] Embodiment
[0069] The image repairing method combining the fast Fourier convolution and diffusion model of the present application uses the DIV2K and Urban100 data sets as the image data set. The evaluation criteria are used to measure the performance of the image repairing model, and three evaluation indexes are selected: the peak signal-to-noise ratio PSNR, which is a value used to measure the ratio of the maximum possible power between two images and the noise power, the higher the value, the better the performance of the model; the structural similarity SSIM, which is a value used to consider the similarity between the generated image and the original image by considering three dimensions of brightness, contrast and structural change, the higher the value, the better the model; and the Fréchet Inception Distance FID, which is a value used to measure the difference between the generated image and the real image distribution, the lower the value, the closer the generated image is to the real image distribution.
[0070] As shown in Figure 4 , the comparative effect diagrams of different algorithms and the method of the present application on the same DIV2K data set are shown, i.e., the processing results of the Bicubic algorithm, the SR3 algorithm, the SRDiff algorithm, the ResDiff algorithm and the image repairing method combining the fast Fourier convolution and diffusion model of the present application on the same image data set are compared; Figure 4 , Input is the initial input image, and Reference is the reference standard image. The closer to the reference standard image, the more excellent the image algorithm processing is. Through qualitative comparison and analysis, the method of the present application is more superior than other algorithms in terms of detail feature fidelity and clarity, so the superiority of the method of the present application can be proved, and the final image obtained is clearer and more accurate.
[0071] To further verify the performance of the model, quantitative comparison is made on the DIV2K and Urban100 data sets; the image processing results of the SR3 algorithm, the SRDiff algorithm and the ResDiff algorithm are compared with the image processing results of the present application in terms of the PSNR, SSIM and FID evaluation indexes, and the evaluation index comparison table of various algorithms and the present application on the DIV2K and Urban100 data sets is obtained, as shown in Table 1 below:
[0072] Table 1 Comparison of evaluation indexes of various algorithms and the present application on DIV2K and Urban100 data sets
[0073]
[0074] In the table, Ground Truth is the golden standard, that is, the best index value of PSNR, SSIM and FID evaluation indexes in the ideal state; it can be seen from the table that whether in processing DIV2K data set or processing Urban100 data set, the image processing result of the present application shows better performance in PSNR, SSIM and FID evaluation indexes compared with other algorithms such as SR3 algorithm, SRDiff algorithm and ResDiff algorithm.
[0075] The above is only a description of the preferred embodiments of the present application, and those skilled in the art can make some modifications and optimizations according to the above disclosure without departing from the above basic principles, and these improvements and optimizations should be regarded as the protection scope of the present application.
Claims
1. An image inpainting method combining fast Fourier convolution and diffusion models, characterized in that: Includes the following steps: S1. Obtain the image dataset and divide it into training and test sets. Input the training and test sets into the diffusion model for training and testing, respectively. S2. In the forward diffusion process of the diffusion model, Gaussian noise is gradually added to the initial input image until the initial input image becomes a low-resolution image with Gaussian noise. S3. In the reverse diffusion process of the diffusion model, the low-resolution image with Gaussian noise is input into the neural network. The fast Fourier convolutional network is used as the neural network to predict the image. The Gaussian noise is gradually removed from the low-resolution image with Gaussian noise to obtain the final image data after restoration. The Fast Fourier Convolutional Network performs frequency and time domain feature processing on low-resolution images with Gaussian noise to obtain a first feature image and a second feature image; the first feature image and the second feature image are then concatenated and fused to obtain an image denoised by the Fast Fourier Convolutional Network.
2. The image inpainting method combining Fast Fourier Convolution and Diffusion Model according to claim 1, characterized in that: In the forward diffusion process in the diffusion model, Gaussian noise is added to the input initial image step by step until the input initial image becomes a low-resolution image with Gaussian noise; in the forward diffusion process, x0~q(x0) is set as the probability distribution of the low-resolution image x with Gaussian noise obtained after the initial image is added with Gaussian noise for T steps T In the forward diffusion process, x t is only related to x t-1 at the previous time, and the calculation process of adding Gaussian noise step by step is as follows: ， In the formula, βt represents a weight of Gaussian noise added at time t, and the range value of β is 10 -4 → 2 × 10 -2 linear transformation, and ε represents a standard Gaussian noise, ε ∈ N(0, I).
3. The image inpainting method combining Fast Fourier Convolution and Diffusion Model according to claim 2, characterized in that: In the reverse diffusion process in the diffusion model, a fast Fourier convolution network is used as a neural network to predict the image, and the low-resolution image with Gaussian noise is gradually removed from the Gaussian noise to obtain the final image data after repair; in the reverse diffusion process, the low-resolution image with Gaussian noise x T is gradually restored to the initial image data to obtain the final image data after repair; in the denoising diffusion probability model, a neural network p θ (x t-1 | x t ) is used to fit the reverse diffusion process, and the calculation process is as follows: In the formula, p θ (x t-1 |x t This represents the denoising process during reverse diffusion, where the input x is denoised. t Restore to x t-1 The probability distribution, μ θ (x t Let Σ(t) represent the expected mean at time t. θ (x t Let ,t) represent the expected variance at time t; βt-1 represents the weight of the Gaussian noise added at time t-1, and the range of β is 10. -4 → 2 × 10 -2 The linear transformation of; ε represents standard Gaussian noise, ε∈N(0,I); ε θ (x t Let ,t) represent the mapping relationship of the neural network at time t, and σ t This is expressed as the noise oscillation amplitude.
4. The image inpainting method combining Fast Fourier Convolution and Diffusion Model according to claim 1, characterized in that: The Fast Fourier Convolutional Network comprises a dual-branch network constructed from a convolutional network and a Fast Fourier Network. In the convolutional network, a small-kernel convolutional network is used to convolve low-resolution images with Gaussian noise, processing image features in the temporal domain to obtain a convolutional feature image. In the Fast Fourier Network, the low-resolution image with Gaussian noise is sequentially subjected to Fast Fourier Transform, high-pass or low-pass filtering, and Fast Inverse Fourier Transform to obtain a first initial feature image and a second initial feature image. The first initial feature image and the second initial feature image are then fused with the convolutional feature image to obtain a first feature image and a second feature map.
5. The image inpainting method combining fast Fourier convolution and diffusion models according to claim 4, characterized in that: The fast Fourier network also includes two branch networks, one branch network converts the low-resolution image with Gaussian noise into the frequency domain through fast Fourier transform, filters through a high-pass filter, and converts into the time domain through inverse fast Fourier transform to obtain a first initial feature image; the first initial feature image is fused with a convolution feature image obtained through a small kernel convolution network to obtain a first feature image F h The calculation process is as follows: In the formula, I represents a low-resolution image with Gaussian noise, FFT is Fast Fourier Transform, HPF is the filtering of a high-pass filter, iFFT is the Inverse Fast Fourier Transform, and Conv is the convolution operation in a small kernel convolutional network.
6. The image inpainting method combining fast Fourier convolution and diffusion models according to claim 4, characterized in that: In the Fast Fourier Transform network, another branch network converts the low-resolution image with Gaussian noise into the frequency domain through Fast Fourier Transform, filters it through a low-pass filter, and then converts it into the time domain through Fast Inverse Fourier Transform to obtain the second initial feature image. The second initial feature image is fused with the convolution feature image obtained through the small kernel convolution network to obtain a second feature image F l The calculation process is as follows: In the formula, I represents a low-resolution image with Gaussian noise, FFT is the Fast Fourier Transform, LPF is the filtering of a low-pass filter, iFFT is the Inverse Fast Fourier Transform, and Conv is the convolution operation in a small kernel convolutional network.
7. The image inpainting method combining Fast Fourier Convolution and Diffusion Model according to claim 1, characterized in that: The first feature image and the second feature image are concatenated and fused by the feature fusion module to obtain the denoised image data. The calculation process is as follows: In the formula, I predict is the denoised image data, F h is the first feature image, F l is the second feature image.
8. The image inpainting method combining fast Fourier convolution and diffusion models according to claim 1, characterized in that: In Fast Fourier Convolutional Networks (FFTs), a hybrid loss function is used to evaluate the model's performance in both the time and frequency domains. The formula for calculating the hybrid loss function is shown below: In the formula, I0 is the initial image, and E represents the mathematical expectation.