Image deblurring model and method based on multi-scale feature fusion

Abstract

The invention relates to an image deblurring model and method based on multi-scale feature fusion. The rough deblurring processing and the refined deblurring processing of a blurred image under multi-aspect UNet network optimization are achieved. The method comprises the following steps: optimizing an MIMO-UNet model to obtain an MIMO-FWNet model, inputting an original size blurred image into the MIMO-FWNet model, and sequentially carrying out rough deblurring processing and refined deblurring processing on the original size blurred image through two U-shaped networks; a deblurring loss function is adopted to train the MIMO-FWNet model, a trained MIMO-FWNet model is obtained, and the deblurring loss function is composed of a spatial domain loss function and a frequency domain loss function; and inputting a to-be-processed image into the trained MIMO-FWNet model, and outputting the to-be-processed image to obtain a deblurred image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image deblurring model and method based on multi-scale feature fusion. BACKGROUND

[0002] Traditional image deblurring is mainly based on variational model, sparse representation and blur kernel estimation, etc. It often has the shortcomings of high complexity and difficulty in wide application. In recent years, deep learning technology has developed rapidly, bringing new methods to solve the problem of image deblurring. Compared with traditional methods, deep neural networks can automatically learn multi-level features of images, without the need for manual feature extractor design. The model can directly learn the mapping relationship from the blurred and clear image pairs, so as to deal with various different blur states. MIMO-UNet, as a representative of multi-scale input and multi-scale output deep learning deblurring model, achieves good results by fusing rich image features. However, MIMO-UNet is based on a single UNet network, and there is still room for improvement in structure. Image downsampling will lose pixel details, thus reducing the deblurring effect. In addition, directly fusing features of different sizes will cause information confusion, as they contain asymmetric information. And the convolutional network is limited by the size of the convolution kernel, making it difficult to capture the overall information of the image, thus the effect is not good in the case of large blur.

[0003] After searching the existing technical literature, it is found that the patent with the application number 202211009494.0 and the patent name A deep learning-based image deblurring method and system is related to the present application. The patent includes obtaining a target image, a first scale image and a second scale image; the first scale image and the second scale image are images obtained by reducing the target image by a first scale and a second scale respectively; inputting the target image, the first scale image and the second scale image into a trained image deblurring model to obtain a deblurred image corresponding to the target image; wherein the image deblurring model uses an improved MIMO-Unet model, which is obtained by replacing the convolution block in the MIMO-Unet model with a depth separable convolution block. However, this patent is based on a single UNet network, which is difficult to achieve better deblurring effect from coarse to fine, and the optimization of the UNet network does not consider problems such as insufficient comprehensive image feature extraction and loss of part of the features by sampling operation. SUMMARY

[0004] Therefore, it is necessary to provide an image deblurring model and method based on multi-scale feature fusion to achieve rough deblurring and refined deblurring processing of blurred images under multi-UNet network optimization.

[0005] To solve the above technical problems, the present application provides the following technical solutions: In one aspect, the present application provides an image deblurring model, comprising: a first U-shaped network and a second U-shaped network; The first U-shaped network and the second U-shaped network are based on UNet network modification; The first U-shaped network and the second U-shaped network have the same architecture, and the first U-shaped network is connected in series with the second U-shaped network; The input of the first U-shaped network is an original size blurred image, a half size blurred image, and a quarter size blurred image, and the output is an original size deblurred image, a half size deblurred image, and a quarter size deblurred image; The input of the second U-shaped network is an original size deblurred image, a half size deblurred image, and a quarter size deblurred image, and the output is an original size refined deblurred image, a half size refined deblurred image, and a quarter size refined deblurred image.

[0006] Preferably, the first U-shaped network comprises an encoder layer, a feature fusion layer, and a decoder layer; the blurred image sequentially passes through the encoder layer, the feature fusion layer, and the decoder layer to obtain a deblurred image; The encoder layer comprises a shallow convolution module, a frequency domain and spatial domain convolution module, a feature attention module, and a residual module; The feature fusion layer comprises an attention fusion module; The decoder layer comprises a frequency domain and spatial domain convolution module, a residual module, and an inverse convolution module.

[0007] Preferably, the frequency domain and spatial domain convolution module comprises: The input image is respectively spliced after passing through the frequency domain convolution module and the spatial domain convolution module, and then output after passing through the double-flow attention calculation module.

[0008] In another aspect, the present application provides an image deblurring method based on multi-scale feature fusion, applied to the image deblurring model, comprising: S1: optimizing the MIMO-UNet model to obtain a MIMO-FWNet model, inputting an original size blurred image into the MIMO-FWNet model, and sequentially performing rough deblurring processing and refined deblurring processing through two U-shaped networks; S2: training the MIMO-FWNet model using a deblurring loss function to obtain a trained MIMO-FWNet model, wherein the deblurring loss function is composed of a spatial domain loss function and a frequency domain loss function; S3: inputting a to-be-processed image into the trained MIMO-FWNet model to output a deblurred image.

[0009] Preferably, before sequentially passing through the two U-shaped networks, the original size blur image further comprises: The original size blur image is subjected to a downsampling operation to obtain a half size blur image and a quarter size blur image.

[0010] Preferably, the rough deblurring processing comprises: The original size blur image is sequentially subjected to a first frequency domain and spatial domain convolution module and a first residual module to obtain first residual features; The first residual features are subjected to a second frequency domain and spatial domain convolution module to obtain first convolution features, the half size blur image is subjected to a first shallow layer convolution module to obtain second convolution features, the first convolution features and the second convolution features are jointly input into a first feature attention module, and then subjected to a second residual module to obtain second residual features; The second residual features are subjected to a third frequency domain and spatial domain convolution module to obtain third convolution features, the quarter size blur image is subjected to a second shallow layer convolution module to obtain fourth convolution features, the third convolution features and the fourth convolution features are jointly input into a second feature attention module, and then subjected to a third residual module to obtain third residual features; The second residual features and the third residual features are subjected to an upsampling operation and are jointly input into a first attention fusion module with the first residual features to obtain first fusion features; The first residual features are subjected to a downsampling operation, the third residual features are subjected to an upsampling operation, and the first residual features and the third residual features are jointly input into a second attention fusion module with the second residual features to obtain second fusion features; The first residual features and the second residual features are subjected to a downsampling operation and are jointly input into a third attention fusion module with the third residual features to obtain third fusion features; The third fusion features are sequentially subjected to a fourth residual module and a first deconvolution module, and then subjected to a first splicing operation with the second fusion features to obtain first splicing features; The first splicing features are sequentially subjected to a fourth frequency domain and spatial domain convolution module, a fifth residual module, and a second deconvolution module, and then subjected to a second splicing operation with the first fusion features to obtain second splicing features; The second splicing features are sequentially subjected to a fifth frequency domain and spatial domain convolution module, a sixth residual module, and a sixth frequency domain and spatial domain convolution module to obtain original size deblurring features; The first splicing features are sequentially subjected to a fourth frequency domain and spatial domain convolution module, a fifth residual module, and a seventh frequency domain and spatial domain convolution module to obtain half size deblurring features; The third fusion features are sequentially subjected to a fourth residual module and an eighth frequency domain and spatial domain convolution module to obtain quarter size deblurring features; The original size deblurred feature and the original size blurred image are added together in the first addition operation to obtain the original size deblurred image. The half-size deblurred feature and the half-size blurred image are added together in the second addition operation to obtain the half-size deblurred image. The quarter-size deblurred feature and the quarter-size blurred image are added together in the third addition operation to obtain the quarter-size deblurred image.

[0011] Preferably, the downsampling operation and the upsampling operation adopt the space-to-channel downsampling method and the channel-to-space upsampling method, respectively.

[0012] Preferably, before step S2, the method further includes: The original sample set was obtained by collecting data from publicly available databases; Clear images are obtained from public sources by web crawling. A blurring algorithm is applied to the clear images to obtain corresponding blurred images. A synthetic sample set is obtained based on the clear images and their corresponding blurred images. The blur kernel in the image blurring algorithm is randomly set. The original sample set and the synthetic sample set are merged to obtain a sample set, which includes image pairs consisting of blurred sample images and their corresponding clear sample images; The clear images in the sample set are downsampled to obtain half-size clear images and quarter-size clear images. The clear images, half-size clear images and quarter-size clear images constitute the clear image labels. The sample set is divided into a training set, a validation set, and a test set according to a preset ratio; Set the hyperparameters for training the MIMO-FWNet model.

[0013] Preferably, step S2 includes: The training set, validation set, and test set are input into the MIMO-FWNet model with the hyperparameters set to obtain sample deblurred images. The sample deblurred images include: original size sample deblurred images, half-size sample deblurred images, quarter-size sample deblurred images, original size thinned sample deblurred images, half-size thinned sample deblurred images, and quarter-size thinned sample deblurred images. The deblurred images of the samples are subjected to supervised learning using a deblurring loss function until the loss converges, resulting in a trained MIMO-FWNet model.

[0014] Preferably, the deblurring loss function is calculated as follows:

[0015] Where L1 represents the L1 loss function, SSIM represents the structural loss function, the L1 loss function and the structural loss function together constitute the spatial domain loss function, and FFT represents the Fourier transform, i.e., the frequency domain loss function. , These represent the clear image label and the deblurred image of the sample, respectively.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention provides an image deblurring model and method based on multi-scale feature fusion. It employs two stacked U-shaped networks to achieve a multi-scale input-output deblurring strategy from coarse to fine. The U-shaped networks are based on the UNet network of the MIMO-UNet model and have been optimized in various aspects. This improves the extraction and fusion of effective features and preserves richer features during upsampling and downsampling. For the optimized MIMO-UNet model, i.e., the MIMO-FWNet model, multi-scale deblurring loss supervision is also performed in the frequency and spatial domains to further guide the learning of network parameters. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of an image deblurring model in one embodiment; Figure 2 This is a schematic diagram of an image deblurring method based on multi-scale feature fusion in one embodiment. Figure 3 This is a schematic diagram of the MIMO-FWNet model architecture, an image deblurring method based on multi-scale feature fusion, in one embodiment. Figure 4 This is a schematic diagram of the frequency domain spatial domain convolution module architecture of an image deblurring method based on multi-scale feature fusion in one embodiment. Figure 5 This is a schematic diagram of the attention fusion module architecture of an image deblurring method based on multi-scale feature fusion in one embodiment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] Example 1 like Figure 1 As shown, this embodiment proposes an image deblurring model, including: a first U-shaped network and a second U-shaped network; The first U-shaped network and the second U-shaped network are derived from the UNet network; The first U-shaped network and the second U-shaped network have the same architecture, and the first U-shaped network is connected in series with the second U-shaped network; The inputs of the first U-shaped network are the original size blurred image, the half size blurred image, and the quarter size blurred image, and the outputs are the original size deblurred image, the half size deblurred image, and the quarter size deblurred image; The inputs to the second U-shaped network are the original size deblurred image, the half-size deblurred image, and the quarter-size deblurred image, and the outputs are the original size thinned deblurred image, the half-size thinned deblurred image, and the quarter-size thinned deblurred image.

[0020] The first U-shaped network includes: an encoder layer, a feature fusion layer, and a decoder layer; the blurred image passes through the encoder layer, the feature fusion layer, and the decoder layer in sequence to obtain a deblurred image; The encoder layer includes: a shallow convolution module, a frequency-domain spatial-domain convolution module, a feature attention module, and a residual module; The feature fusion layer includes: an attention fusion module; The decoder layer includes: a frequency domain spatial domain convolution module, a residual module, and a deconvolution module.

[0021] The frequency-domain spatial-domain convolution module includes: The input image is concatenated after passing through the frequency domain convolution module and the spatial domain convolution module, and then output after passing through the two-stream attention calculation module.

[0022] The specific implementation of this step is as follows, such as Figure 4 As shown, frequency and spatial domain features are extracted through a frequency-space domain convolution module (FRConv), resulting in complementary features and a more comprehensive set of image features. First, a Fourier convolution module (frequency domain convolution module) extracts the frequency domain features of the image. Then, it is fused with the spatial domain features extracted by the spatial domain convolution module using channel attention. Specifically, the input feature channel is divided into two parts. Branch one undergoes a Fourier transform before being input to the convolution module, followed by an inverse Fourier transform. Branch two directly extracts features through the convolution module. The feature maps output from the two branches are concatenated and then subjected to two-stream attention calculation to further improve the extraction of effective features.

[0023] Example 2 like Figure 2 As shown, this embodiment proposes an image deblurring method based on multi-scale feature fusion, the method comprising: S1: Optimize the MIMO-UNet model to obtain the MIMO-FWNet model. Input the original size blurred image into the MIMO-FWNet model, and pass it through two U-shaped networks in sequence for coarse deblurring and fine deblurring. The specific implementation of this step is as follows, such as Figure 3 As shown, the MIMO-FWNet model employs a stacked approach of two multi-input multi-output (MIMO) U-shaped networks to implement a coarse-to-fine multi-size input / output deblurring strategy. This coarse-to-fine strategy manifests in two ways: firstly, through stacking, the first U-shaped network obtains a preliminary deblurred image and passes it to the next U-shaped network; secondly, within a single U-shaped network, at the decoder end, the smaller deblurred feature map serves as one of the larger inputs, resulting in richer deblurred features. The original size blurred image is downsampled to obtain three sizes of blurred images: original, half-size, and quarter-size. These three sizes of blurred images are first input into the first U-shaped network for feature learning, as shown... Figure 5 As shown, the attention fusion module adaptively weights and fuses the encoded features of the three sizes, and through attention calculation, obtains more effective encoded features. After feature fusion, deblurred features of the three sizes are obtained. These deblurred features are added to the input blurred image to obtain deblurred images of the three sizes. The three sizes of deblurred images are then input into a second U-shaped network for further learning, resulting in three sizes of refined deblurred images. The original size deblurred image serves as the final result. The three sizes of deblurred features are used as auxiliary features for the original size deblurred image.

[0024] S2: The MIMO-FWNet model is trained using a deblurring loss function to obtain a trained MIMO-FWNet model. The deblurring loss function consists of a spatial domain loss function and a frequency domain loss function. The specific implementation of this step is as follows: through multi-size deblurring loss supervision, the learning of network parameters is further effectively guided. The model has a total of six deblurred image outputs (three sizes of deblurred images and three sizes of thinned deblurred images), all of which are supervised by the deblurring loss function to learn parameters, thereby achieving a deblurring effect from coarse to fine.

[0025] S3: Input the image to be processed into the trained MIMO-FWNet model and output the deblurred image.

[0026] Example 3 This embodiment further explains the image deblurring method based on multi-scale feature fusion proposed in Embodiment 2.

[0027] Before the original size blurred image passes through two U-shaped networks, it also includes: The original size blurred image is downsampled to obtain a half-size blurred image and a quarter-size blurred image.

[0028] The coarse deblurring process includes: The original size blurred image is sequentially passed through the first frequency domain spatial domain convolution module and the first residual module to obtain the first residual feature; The first residual feature is passed through the second frequency domain spatial domain convolution module to obtain the first convolution feature. The half-size blurred image is passed through the first shallow convolution module to obtain the second convolution feature. The first convolution feature and the second convolution feature are input together into the first feature attention module, and then passed through the second residual module to obtain the second residual feature. The second residual feature is passed through the third frequency domain spatial domain convolution module to obtain the third convolution feature. The quarter-size blurred image is passed through the second shallow convolution module to obtain the fourth convolution feature. The third and fourth convolution features are input together into the second feature attention module, and then passed through the third residual module to obtain the third residual feature. After upsampling, the second and third residual features, together with the first residual feature, are input into the first attention fusion module to obtain the first fused feature; After the first residual feature is downsampled and the third residual feature is upsampled, they are input together with the second residual feature into the second attention fusion module to obtain the second fused feature. The first residual feature, the second residual feature, and the third residual feature are downsampled and then input together with the third residual feature into the third attention fusion module to obtain the third fused feature. The third fusion feature is passed through the fourth residual module and the first deconvolution module in sequence, and then combined with the second fusion feature through the first concatenation operation to obtain the first concatenated feature; The first spliced ​​feature is sequentially processed through the fourth frequency domain spatial domain convolution module, the fifth residual module, and the second deconvolution module, and then combined with the first fused feature through the second splicing operation to obtain the second spliced ​​feature; The second concatenated feature is passed sequentially through the fifth frequency domain spatial domain convolution module, the sixth residual module, and the sixth frequency domain spatial domain convolution module to obtain the original size deblurred feature; The first concatenated feature is passed sequentially through the fourth frequency domain spatial domain convolution module, the fifth residual module, and the seventh frequency domain spatial domain convolution module to obtain a half-size deblurred feature. The third fusion feature is passed sequentially through the fourth residual module and the eighth frequency-space domain convolution module to obtain a quarter-size deblurred feature; The original size deblurred feature and the original size blurred image are added together in the first addition operation to obtain the original size deblurred image. The half-size deblurred feature and the half-size blurred image are added together in the second addition operation to obtain the half-size deblurred image. The quarter-size deblurred feature and the quarter-size blurred image are added together in the third addition operation to obtain the quarter-size deblurred image.

[0029] The downsampling and upsampling operations respectively employ a space-to-channel downsampling method and a channel-to-space upsampling method.

[0030] The specific implementation of this step is as follows: Since direct downsampling will lose some pixels or features, which is not conducive to image deblurring, the U-shaped network modifies the direct downsampling and upsampling of the UNet network's input and feature maps to perform downsampling and upsampling operations by converting space to channel (Space2Depth) and channel to space (Depth2Space), which can preserve richer features. For example, when a half-size blurred image is obtained by downsampling, the original size blurred image is (320, 320, 64). By converting space to channel downsampling, a half-size blurred image of size (160, 160, 256) is obtained. The features discarded during downsampling are stored by channel expansion. The channel to space upsampling in the feature upsampling process is exactly the inverse operation of space to channel downsampling.

[0031] Before step S2, the following are also included: The original sample set was obtained by collecting data from publicly available databases; Clear images are obtained from public sources by web crawling. A blurring algorithm is applied to the clear images to obtain corresponding blurred images. A synthetic sample set is obtained based on the clear images and their corresponding blurred images. The blur kernel in the image blurring algorithm is randomly set. The original sample set and the synthetic sample set are merged to obtain a sample set, which includes image pairs consisting of blurred sample images and their corresponding clear sample images; The clear images in the sample set are downsampled to obtain half-size clear images and quarter-size clear images. The clear images, half-size clear images and quarter-size clear images constitute the clear image labels. The sample set is divided into a training set, a validation set, and a test set according to a preset ratio; Set the hyperparameters for training the MIMO-FWNet model.

[0032] The specific implementation of this step is as follows: First, for common scenarios such as transportation and daily life, an original sample set is obtained by collecting data from public databases. The original sample set includes original blurred images and their corresponding original clear images. Since it is often difficult to obtain a large number of paired blurred-clear image samples, the sample data is expanded by using a blurring algorithm. Commonly available image data (containing only clear images) is obtained through methods such as web crawling. The corresponding blurred images are obtained through the blurring algorithm, and then a synthetic sample set is obtained. During the process, the blur kernel in the image blurring algorithm is randomly set so that the synthetic sample set can achieve blur diversity by controlling the blur angle range and blur direction range, etc. The original sample set and the synthetic sample set are merged to obtain the final required sample set.

[0033] The sample set includes blurred sample images and clear sample images. The clear sample images in the sample set are downsampled to obtain half-size clear sample images and quarter-size clear sample images. The clear sample images, half-size clear sample images, and quarter-size clear sample images constitute the clear sample image labels. The clear sample image labels are used as the parameter input of the deblurring loss function when training the model later. Set up the environment required for training the MIMO-FWNet model. Divide the sample set into training, validation, and test sets according to a preset ratio, with the training, validation, and test sets accounting for 60%, 20%, and 20% of the data, respectively. Stratified sampling is performed during data partitioning, meaning the allocation of image data for different scene types follows the above ratio. Set the hyperparameters for network model training, with some hyperparameters set as follows: maximum number of training epochs: 3000 epochs, batch size: 8, training image cropping: 256*256, and validation interval: 100 epochs.

[0034] Step S2 includes: The training set, validation set, and test set are input into the MIMO-FWNet model with the hyperparameters set to obtain sample deblurred images. The sample deblurred images include: original size sample deblurred images, half-size sample deblurred images, quarter-size sample deblurred images, original size thinned sample deblurred images, half-size thinned sample deblurred images, and quarter-size thinned sample deblurred images. The deblurred images of the samples are subjected to supervised learning using a deblurring loss function until the loss converges, resulting in a trained MIMO-FWNet model.

[0035] The expression for calculating the deblurring loss function is as follows:

[0036] Where L1 represents the L1 loss function, SSIM represents the structural loss function, the L1 loss function and the structural loss function together constitute the spatial domain loss function, and FFT represents the Fourier transform, i.e., the frequency domain loss function. , These represent the clear image label and the deblurred image of the sample, respectively.

Claims

1. An image deblurring model, characterized in that, include: First U-shaped network and second U-shaped network; The first U-shaped network and the second U-shaped network are derived from the UNet network; The first U-shaped network and the second U-shaped network have the same architecture, and the first U-shaped network is connected in series with the second U-shaped network; The inputs of the first U-shaped network are the original size blurred image, the half size blurred image, and the quarter size blurred image, and the outputs are the original size deblurred image, the half size deblurred image, and the quarter size deblurred image; The inputs to the second U-shaped network are the original size deblurred image, the half-size deblurred image, and the quarter-size deblurred image, and the outputs are the original size thinned deblurred image, the half-size thinned deblurred image, and the quarter-size thinned deblurred image.

2. The image deblurring model according to claim 1, characterized in that, The first U-shaped network includes: an encoder layer, a feature fusion layer, and a decoder layer; the blurred image passes through the encoder layer, the feature fusion layer, and the decoder layer in sequence to obtain a deblurred image; The encoder layer includes: a shallow convolution module, a frequency-domain spatial-domain convolution module, a feature attention module, and a residual module; The feature fusion layer includes: an attention fusion module; The decoder layer includes: a frequency domain spatial domain convolution module, a residual module, and a deconvolution module.

3. The image deblurring model according to claim 2, characterized in that, The frequency-domain spatial-domain convolution module includes: The input image is concatenated after passing through the frequency domain convolution module and the spatial domain convolution module, and then output after passing through the two-stream attention calculation module.

4. An image deblurring method based on multi-scale feature fusion, characterized in that, Applied to the image deblurring model as described in claim 1, comprising: S1: Optimize the MIMO-UNet model to obtain the MIMO-FWNet model. Input the original size blurred image into the MIMO-FWNet model, and pass it through two U-shaped networks in sequence for coarse deblurring and fine deblurring. S2: The MIMO-FWNet model is trained using a deblurring loss function to obtain a trained MIMO-FWNet model. The deblurring loss function consists of a spatial domain loss function and a frequency domain loss function. S3: Input the image to be processed into the trained MIMO-FWNet model and output the deblurred image.

5. The image deblurring method based on multi-scale feature fusion according to claim 4, characterized in that, Before the original size blurred image passes through two U-shaped networks, it also includes: The original size blurred image is downsampled to obtain a half-size blurred image and a quarter-size blurred image.

6. The image deblurring method based on multi-scale feature fusion according to claim 5, characterized in that, The coarse deblurring process includes: The original size blurred image is sequentially passed through the first frequency domain spatial domain convolution module and the first residual module to obtain the first residual feature; The first residual feature is passed through the second frequency domain spatial domain convolution module to obtain the first convolution feature. The half-size blurred image is passed through the first shallow convolution module to obtain the second convolution feature. The first convolution feature and the second convolution feature are input together into the first feature attention module, and then passed through the second residual module to obtain the second residual feature. The second residual feature is passed through the third frequency domain spatial domain convolution module to obtain the third convolution feature. The quarter-size blurred image is passed through the second shallow convolution module to obtain the fourth convolution feature. The third and fourth convolution features are input together into the second feature attention module, and then passed through the third residual module to obtain the third residual feature. After upsampling, the second and third residual features, together with the first residual feature, are input into the first attention fusion module to obtain the first fused feature; After the first residual feature is downsampled and the third residual feature is upsampled, they are input together with the second residual feature into the second attention fusion module to obtain the second fused feature. The first residual feature, the second residual feature, and the third residual feature are downsampled and then input together with the third residual feature into the third attention fusion module to obtain the third fused feature. The third fusion feature is passed through the fourth residual module and the first deconvolution module in sequence, and then combined with the second fusion feature through the first concatenation operation to obtain the first concatenated feature; The first spliced ​​feature is sequentially processed through the fourth frequency domain spatial domain convolution module, the fifth residual module, and the second deconvolution module, and then combined with the first fused feature through a second splicing operation to obtain the second spliced ​​feature; The second concatenated feature is passed sequentially through the fifth frequency domain spatial domain convolution module, the sixth residual module, and the sixth frequency domain spatial domain convolution module to obtain the original size deblurred feature; The first concatenated feature is passed sequentially through the fourth frequency domain spatial domain convolution module, the fifth residual module, and the seventh frequency domain spatial domain convolution module to obtain a half-size deblurred feature. The third fusion feature is passed through the fourth residual module and the eighth frequency-space domain convolution module to obtain a quarter-size deblurred feature. The original size deblurred feature and the original size blurred image are added together in the first addition operation to obtain the original size deblurred image. The half-size deblurred feature and the half-size blurred image are added together in the second addition operation to obtain the half-size deblurred image. The quarter-size deblurred feature and the quarter-size blurred image are added together in the third addition operation to obtain the quarter-size deblurred image.

7. The image deblurring method based on multi-scale feature fusion according to claim 6, characterized in that, The downsampling and upsampling operations respectively employ a space-to-channel downsampling method and a channel-to-space upsampling method.

8. The image deblurring method based on multi-scale feature fusion according to claim 4, characterized in that, Before step S2, the following are also included: The original sample set was obtained by collecting data from publicly available databases; Clear images are obtained from public sources by web crawling. A blurring algorithm is applied to the clear images to obtain corresponding blurred images. A synthetic sample set is obtained based on the clear images and their corresponding blurred images. The blur kernel in the image blurring algorithm is randomly set. The original sample set and the synthetic sample set are merged to obtain a sample set, which includes image pairs consisting of blurred sample images and their corresponding clear sample images; The clear images in the sample set are downsampled to obtain half-size clear images and quarter-size clear images. The clear images, half-size clear images and quarter-size clear images constitute the clear image labels. The sample set is divided into a training set, a validation set, and a test set according to a preset ratio; Set the hyperparameters for training the MIMO-FWNet model.

9. The image deblurring method based on multi-scale feature fusion according to claim 8, characterized in that, Step S2 includes: The training set, validation set, and test set are input into the MIMO-FWNet model with the hyperparameters set to obtain sample deblurred images. The sample deblurred images include: original size sample deblurred images, half-size sample deblurred images, quarter-size sample deblurred images, original size thinned sample deblurred images, half-size thinned sample deblurred images, and quarter-size thinned sample deblurred images. The deblurred images of the samples are subjected to supervised learning using a deblurring loss function until the loss converges, resulting in a trained MIMO-FWNet model.

10. The image deblurring method based on multi-scale feature fusion according to claim 9, characterized in that, The expression for calculating the deblurring loss function is as follows: Where L1 represents the L1 loss function, SSIM represents the structural loss function, the L1 loss function and the structural loss function together constitute the spatial domain loss function, and FFT represents the Fourier transform, i.e., the frequency domain loss function. , These represent the clear image label and the deblurred image of the sample, respectively.

Images