Spatial frequency hybrid restoration method for JPEG (Joint Photographic Experts Group) compressed blurred image restoration

Abstract

The invention relates to the technical field of image restoration, in particular to a spatial frequency hybrid restoration method for JPEG (Joint Photographic Experts Group) compressed blurred image restoration, which is characterized in that a spatial frequency hybrid restoration network is constructed for compressed blurred image restoration and comprises three coding blocks and three decoding blocks, each of the coding block and the decoding block is composed of a plurality of spatial frequency mixing blocks; the spatial frequency mixing block comprises a spatial frequency double-branch mixing structure and an ISS block; the SFHRN provided by the invention can realize a more efficient restoration effect on the compressed blurred image with lower model parameter quantity and calculation complexity.

Description

Technical Field

[0001] The present invention relates to the technical field of image restoration, and particularly relates to a spatial frequency hybrid restoration method for JPEG compressed blurred image restoration. Background Art

[0002] Currently, there are problems of JPEG compression artifacts and motion blur in the field of image processing. Specifically, JPEG compression first divides an image into 8×8 blocks, then performs discrete cosine transform on each block to obtain DCT coefficients, and then performs quantization and rounding operations on the coefficients on each block. And JPEG compression will cause information loss and compression artifacts, which will not only lead to poor visual quality, but also lead to a decline in the performance of subsequent computer vision tasks. And image blur is caused by motion during the imaging process (such as camera shake, object motion). The existence of blur significantly reduces the level of details in the image, making key information such as edges and textures unidentifiable.

[0003] Therefore, how to restore a clear image has received more and more attention. People have proposed many deep learning-based methods to eliminate JPEG compression artifacts and motion blur from aspects such as compression artifact removal, deblurring, and image restoration. In terms of compression artifact removal, someone proposed a deep dual-domain neural network containing two branches, with the two branches in the low-frequency domain and the spatial domain respectively, and the output characteristics of the two branches are fused in the spatial domain; someone proposed two encoders and decoders, which adopted dilated convolutional layers for the low-frequency domain and the spatial domain respectively, and designed corresponding loss functions for the two different domains respectively; and someone extracted shallow features through dense diffusion convolutional layers and further corrected the features of the corresponding channels according to the quantization tables of the luminance channel and the color channel. In terms of deblurring, some scholars proposed a cross-channel transformer to capture the global attention map and a region-based blur-aware attention module for capturing different scales of blur patterns respectively; subsequently, someone proposed intra-strip and inter-strip self-attention modules in the vertical and horizontal directions on this basis to capture different blur patterns. In terms of image restoration, someone designed a double U-shaped network structure based on cross-scale feature fusion and feature supervision, and proposed a residual structure for instance normalization of half-channel features; and someone used a residual block with dilated convolution and channel attention to restore the channel information related to the image, and also introduced a JPEG autoencoder loss function to use the difference between the compressed image and the uncompressed image to restore image details.

[0004] Although there have been various research methods in the field of compressed blurred image restoration and significant progress has been made, existing methods either only utilize local and global feature information in the spatial domain or only mine frequency features along the width and height dimensions. This cannot fully utilize the features and information buried in compressed blurred images, hindering the restoration of compressed blurred images. Therefore, the present invention comprehensively considers two image spaces, namely the spatial domain and the frequency domain, as well as two dimensions, namely channels and width-height, to extract and screen local and global features of compressed blurred images in all aspects, improve the performance of the model in this visual task, and restore clearer images. Summary of the Invention

[0005] To solve the above problems, the present invention provides a spatial-frequency hybrid restoration method for JPEG compressed blurred image restoration, wherein a spatial-frequency hybrid restoration network is constructed for compressed blurred image restoration. The spatial-frequency hybrid restoration network includes three encoding blocks and three decoding blocks, and both the encoding blocks and the decoding blocks are composed of multiple spatial-frequency hybrid blocks;

[0006] The specific process of compressed blurred image restoration through the spatial-frequency hybrid restoration network includes the following steps:

[0007] S1. The compressed blurred image is passed through a 3×3 convolutional layer to obtain shallow features, and the shallow features are passed through the first encoding block to obtain the first encoded feature;

[0008] S2. The first encoded feature is downsampled and then passed through the second encoding block to obtain the second encoded feature, and the second encoded feature is downsampled and then passed through the third encoding block to obtain the third encoded feature;

[0009] S3. The third encoded feature is passed through the first decoding block to obtain the first decoded feature, and the first decoded feature is upsampled to obtain the first upsampled feature;

[0010] S4. The first upsampled feature is fused with the second encoded feature and then input into the second decoding block to obtain the second decoded feature, and the second decoded feature is upsampled to obtain the second upsampled feature;

[0011] S5. The second upsampled feature is fused with the first encoded feature and then input into the third decoding block to obtain the third decoded feature, and the third decoded feature is passed through a 3×3 convolutional layer to obtain convolutional features;

[0012] S6. The compressed blurred image and the convolutional features are residually connected to obtain the restored image.

[0013] Furthermore, the spatial frequency mixing block includes a first LN layer, a first convolutional layer, a depth convolutional layer, an SG layer, a spatial frequency dual-branch mixing structure, a second LN layer, a second convolutional layer, and an ISS block connected in sequence; among them, there is a residual connection between the input of the first LN layer and the output of the spatial frequency dual-branch mixing structure, and there is a residual connection between the input of the second LN layer and the output of the ISS block.

[0014] Furthermore, the spatial frequency dual-branch mixing structure includes a patch-level channel attention branch and a pixel-level global attention branch; the processing process of the spatial frequency dual-branch mixing structure includes

[0015] The input features pass through the patch-level channel attention branch and the pixel-level global attention branch respectively to obtain the structural features and channel attention;

[0016] Multiply the structural features by the input features to obtain the PCAB output, and multiply the channel attention by the input features to obtain the PGAB output;

[0017] Connect the PCAB output and the PGAB output and then pass through a 1×1 convolution to obtain the output of the spatial frequency dual-branch mixing structure.

[0018] Furthermore, the patch-level channel attention branch includes a sliding window operation layer, a first depth convolutional layer, a second depth convolutional layer, an activation function layer, and a third depth convolutional layer cascaded in sequence; the sliding window size in the sliding window operation layer is 8×8, and the sliding step is 4; the convolutional kernel size of the first depth convolutional layer is 8×8, and the step is 8; the convolutional kernel sizes of the second depth convolutional layer and the third depth convolutional layer are both 1×1.

[0019] Furthermore, the processing process of the pixel-level global attention branch includes:

[0020] Perform a fast Fourier transform on each pixel point in the input features along the channel dimension to obtain the image frequency characteristics;

[0021] Perform global average pooling on the image frequency characteristics along the spatial dimension to obtain the pooled features;

[0022] Perform an inverse fast Fourier transform on the pooled features along the channel dimension to obtain the channel attention.

[0023] Furthermore, the processing process of the ISS block includes:

[0024] Divide the input features into 8×8 non-overlapping image patches;

[0025] For each non-overlapping image patch, perform a fast Fourier transform (FFT) in the spatial domain to obtain the transform result. Multiply the transform result by a learnable matrix and then perform an inverse fast Fourier transform (IFFT) to obtain the important spatial information in the frequency domain. Pass the important spatial information in the frequency domain through a depth convolutional layer and then perform an element-wise multiplication operation on the features to obtain the processed features. Pass the processed features through a 1×1 convolutional layer and multiply them by a learnable vector to obtain the screened features.

[0026] Output the screened features of all non-overlapping image patches.

[0027] Furthermore, the downsampling operation uses a convolutional layer with a kernel size of 2×2 and a stride of 2; the upsampling operation uses a convolutional layer with a kernel size of 1×1.

[0028] Advantages of the present invention

[0029] The present invention further explores more information features from both the spatial domain and the frequency domain, and proposes a Spatial Frequency Hybrid Restoration Network (SFHRN). It fully utilizes the spatial domain and frequency domain information features in the spatial dimension and channel dimension, deeply integrates the spatial local context information and the global frequency characteristics, screens the information frequency components and spatial features, and clearly and effectively restores the compressed blurred images.

[0030] Compared with the existing technologies, the SFHRN proposed by the present invention can achieve a more efficient restoration effect for compressed blurred images with a lower number of model parameters and computational complexity. Description of the drawings

[0031] Figure 1 It is a schematic diagram of the SFHRN network of the present invention;

[0032] Figure 2 It is a schematic diagram of the SFDBHS of the present invention;

[0033] Figure 3 It is a schematic diagram of the ISS of the present invention;

[0034] Figure 4 It is a visual comparison of the existing method and the SFHRN for restoring blurred compressed images in the embodiments of the present invention. Detailed implementation manners

[0035] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0036] The present invention provides a spatial frequency hybrid restoration method for JPEG compressed blurred image restoration, including constructing a spatial frequency hybrid restoration network for compressed blurred image restoration. The spatial frequency hybrid restoration network includes a downsampling path and an upsampling path. The downsampling path includes three encoding blocks, and the upsampling path includes three decoding blocks. Both the encoding blocks and the decoding blocks are composed of multiple spatial frequency hybrid blocks (SFHBs). To help retain the original feature information, the present invention adds shortcut connections between the same layers of the encoding blocks and the decoding blocks (except for the top layer). The number of SFHBs included in the three encoding blocks gradually increases with downsampling, and the number of SFHBs included in the three decoding blocks gradually decreases with upsampling. As Figure 1 shown, in the embodiment of the present invention, the number of SFHBs in the three encoding blocks is 4, 8, and 12 in sequence, and the number of SFHBs in the three decoding blocks is 12, 8, and 4 in sequence.

[0037] As Figure 1 shown, the specific process of compressed blurred image restoration through the spatial frequency hybrid restoration network includes the following steps:

[0038] S1. The compressed blurred image I D ∈R 3×H×W obtains the shallow feature F S ∈R C×H×W through a 3×3 convolutional layer, and the shallow feature is passed through the first encoding block to obtain the first encoded feature; H and W are the height and width of the compressed blurred image respectively, and C represents the number of channels.

[0039] S2. The first encoded feature is downsampled and then passed through the second encoding block to obtain the second encoded feature, and the second encoded feature is downsampled and then passed through the third encoding block to obtain the third encoded feature;

[0040] S3. The third encoded feature passes through the first decoding block to obtain the first decoded feature, and the first decoded feature is upsampled to obtain the first upsampled feature;

[0041] S4. The first upsampled feature is fused with the second encoded feature and then input into the second decoding block to obtain the second decoded feature, and the second decoded feature is upsampled to obtain the second upsampled feature;

[0042] S5. The second upsampled feature is fused with the first encoded feature and then input into the third decoding block to obtain the third decoded feature, and the third decoded feature passes through a 3×3 convolutional layer to obtain the convolutional feature;

[0043] S6. The compressed blurred image and the convolutional feature are residually connected to obtain the restored image.

[0044] Specifically, the spatial frequency mixing block includes a first LN layer, a first convolutional layer, a depth convolutional layer, an SG layer, a spatial frequency dual-branch mixing structure, a second LN layer, a second convolutional layer, and an ISS block connected in sequence; among them, there is a residual connection between the input of the first LN layer and the output of the spatial frequency dual-branch mixing structure, and there is a residual connection between the input of the second LN layer and the output of the ISS block.

[0045] The processing process of the spatial frequency mixing block can be expressed as

[0046]

[0047] Among them, represents the input feature of the l-th spatial frequency mixing block; LN 1 represents the first LN layer, that is, the first normalization layer; Conv 1 represents the first convolutional layer, DwConv 3×3 represents a depth convolutional layer with a convolution kernel of 3×3, SG represents the element-wise multiplication operation of features, represents the output of the SG layer in the l-th spatial frequency mixing block, SFHDBS() represents the spatial frequency dual-branch mixing structure; represents the fusion result of the first residual connection in the l-th spatial frequency mixing block, and is also the input of the second LN layer; LN 2 represents the second LN layer, that is, the second normalization layer; Conv 2 represents the second convolutional layer, represents the output of the second convolutional layer in the l-th spatial frequency mixing block, ISS() represents the ISS block, represents the fusion result of the second residual connection in the l-th spatial frequency mixing block, and is also the output of the l-th spatial frequency mixing block. Among them, the convolution kernels of the first convolutional layer and the second convolutional layer are both 1×1.

[0048] Specifically, in the embodiments of the present invention, all convolutional layers involved in the downsampling operation between the three encoding blocks have a convolution kernel size of 2×2 and a stride of 2; all convolutional layers in the upsampling operation between the three decoding blocks use a 1×1 convolution kernel.

[0049] Specifically, as Figure 2 shown, the spatial frequency dual-branch mixing structure includes a patch-level channel attention branch (PCAB) and a pixel-level global attention branch (PGAB); the processing process of the spatial frequency dual-branch mixing structure includes

[0050] The input feature passes through the patch-level channel attention branch and the pixel-level global attention branch respectively to obtain the structural feature and the channel attention degree;

[0051] Element-wise multiply the structural features with the input features to obtain the PCAB output, and element-wise multiply the channel attention with the input features to obtain the PGAB output;

[0052] Connect the PCAB output and the PGAB output and then perform a 1×1 convolution to obtain the output of the spatial frequency dual-branch hybrid structure.

[0053] Specifically, the patch-level channel attention branch includes a sliding window operation layer, a first depth convolution layer, a second depth convolution layer, an activation function layer, and a third depth convolution layer connected in series; the sliding window size in the sliding window operation layer is 8×8, and the sliding stride is 4; the convolution kernel size of the first depth convolution layer is 8×8, and the stride is 8; the convolution kernel sizes of the second depth convolution layer and the third depth convolution layer are both 1×1.

[0054] Specifically, in the patch-level channel attention branch, first divide the input features into multiple 8×8 sub-blocks, and slide 4 pixels horizontally and vertically respectively at one time through the sliding window, and then use a depth convolution with a convolution kernel size of 8×8 and a stride of 8 to clearly obtain the structural features of multiple non-overlapping 8×8 sub-blocks along the channel dimension in the spatial domain; convert the structural features into structural features with the same shape as the input features through the broadcast mechanism, element-wise multiply the input features with the converted structural features, and integrate all the multiplication results together to obtain the PCAB output. The whole process can be expressed as

[0055]

[0056] Among them, WS() represents the sliding window operation, DwConv 8×8 represents the first depth convolution layer, DwConv 2 represents the second depth convolution layer, DwConv 3 represents the third depth convolution layer, σ represents the LeakyReLU activation function, M pcab represents the structural information of each non-overlapping 8×8 image block in the feature image.

[0057] Specifically, the processing process of the pixel-level global attention branch includes:

[0058] Perform a fast Fourier transform on each pixel point in the input features along the channel dimension to obtain the frequency characteristics of each pixel point, and obtain the image frequency characteristics;

[0059] Perform global average pooling on the image frequency characteristics along the spatial dimension to obtain the pooled features;

[0060] Perform an inverse fast Fourier transform on the pooled features along the channel dimension to summarize the frequency characteristics of all pixel points, and obtain the channel attention.

[0061] Specifically, asFigure 3 As shown in the figure, the present invention proposes an information screening strategy (ISS), in which a learnable matrix and a learnable vector are defined, which are used to screen important spatial information in the frequency domain and important channels in the spatial domain respectively. Specifically, since JPEG compression performs discrete cosine transform on each 8×8 non-overlapping image patch, the processing process of the ISS block includes:

[0062] Dividing the input features into 8×8 non-overlapping image patches;

[0063] For each non-overlapping image patch, performing a fast Fourier transform in the spatial domain to obtain a transform result, multiplying the transform result by the learnable matrix M fs and then performing an inverse fast Fourier transform to obtain important spatial information in the frequency domain; passing the important spatial information in the frequency domain through a deep convolutional layer and then performing an element-wise multiplication operation on the features to obtain processed features, passing the processed features through a 1×1 convolutional layer and multiplying by the learnable vector V sc to obtain screening features;

[0064] Fusing the screening features of all non-overlapping image patches to obtain the output of the ISS block.

[0065] In one embodiment, in order to prove the superiority of the method proposed by the present invention, the method proposed by the present invention is compared with existing highly representative image restoration methods.

[0066] For fair comparison, the present invention first uniformly compresses the test datasets of GoPro and HIDE with quality factors (QF) of 10, 20, 30, and 40 respectively to obtain synthetic compressed blurred images. The present invention for the first time establishes a degradation dataset with both blur and compression artifacts, named GoPro-Test-Compressed and HIDE-Compressed respectively. Then, SFHRN is trained on the GoPro-Train-Compressed dataset, and the performance of the model is evaluated on the GoPro-Test-Compressed, HIDE-Compressed, Realblur-J dataset, and REDS dataset.

[0067] The model of the present invention is trained by the Adam optimizer (β 1 =0.9, β 2 =0.9). The initial learning rate is set to 1×10 -3 , and then it is reduced to 1×10 -7 through the cosine annealing strategy. The training patch size is 256×256, and the batch size is 16. The training images are enhanced by random rotation and flipping. We use the TLC strategy to avoid artifacts from patches and improve performance. The loss function of SFHRN is:

[0068] L total = L content + λL fft

[0069] where L content is the L1 loss between the restored image I R and the clear image I GT in the spatial domain. L fft is the L1 loss between the restored image I R and the clear image I GT in the frequency domain, and λ = 0.1. We implemented model training and testing on the NVIDIA RTX A6000 GPU and the PyTorch platform.

[0070] The present invention uses the peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), number of model parameters (Params), and floating-point operation count (FLOPs) as evaluation metrics to evaluate the performance of SFHRN.

[0071] Table 1 Comparison methods

[0072]

[0073] Table 2 Comparison results

[0074]

[0075] As shown in Table 2, the present invention quantitatively compares the performance of recent existing image restoration and deblurring methods in terms of average PSNR, average SSIM, number of parameters, and FLOPs on GoPro-Test-Compressed, REDS, ReaBlur-J, and HIDE-Compressed. It can be seen from Table 2 that when the model is trained on GoPro-Compressed, in most cases, SFHRN can achieve the best performance with less computational cost on all test datasets. Specifically, the average PSNR of SFHRN on GoPro-Test-Compressed, REDS, and HIDE-Compress is 0.08 dB, 0.02 dB, and 0.02 dB higher than that of FFTformer (the method ranked second on GoPro-Test-Compressed, REDS, and HIDE-Compress), respectively, while reducing the number of parameters and FLOP by nearly 13.2% and 26.9%. In addition, the average PSNR of SFHRN on RealBlur-J is 0.04 dB higher than that of Restormer (the method ranked second on RealBlur-J), while reducing the number of parameters by nearly 44.8% and FLOP by 31.9%. At the same time, the average PSNR of SFHRN on GoPro-Compress, REDS, and HIDE-Compress is 1.12 dB, 0.71 dB, and 1.5 dB higher than that of MIMO-UNet, respectively.

[0076] Figure 4 shows the visual comparison between SFHRN and recent existing image restoration and deblurring methods on GoPro-Test-Compressed, REDS, RealBlur-J, and HIDE-Compressed. As Figure 4As shown, compared with other methods, the images restored by SFHRN contain clearer and more accurate edges and details. For example, in Img_GOPR0854_11_00000004, the model of the present invention can accurately restore the numbers and Korean characters in the license plate. Some methods cannot restore the Korean characters in the license plate, and the restoration results of other methods have obvious motion blur. In "Img_1fromGOPR0973", the model of the present invention not only restores clear text, but also effectively avoids the adhesion of the letter "O" and quotation marks, while other methods cannot restore the letter "O" and the word "Marc" remains blurred. In "Img_002_00000009", the model of the present invention can relatively accurately restore the word "Innisfree", while the restoration results of other methods have obvious blur and artifacts respectively. In Img_scene056_2, the model of the present invention can accurately restore the names and prices of the dishes in the menu, while some methods cannot restore the content of the menu and other methods cannot eliminate motion blur. In short, our method can restore more accurate structures and clearer textures (see Figure 4 and the enlarged area).

[0077] In the present invention, unless otherwise clearly specified and defined, terms such as "installation", "setting", "connection", "fixation", "rotation", etc. shall be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal communication of two components or the interaction relationship between two components. Unless otherwise clearly defined, for those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0078] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A spatial frequency hybrid restoration method for restoring JPEG compressed blurred images, characterized in that: A spatial frequency hybrid restoration network is constructed for compressed blurred image restoration, wherein the spatial frequency hybrid restoration network includes three encoding blocks and three decoding blocks, and each of the encoding block and the decoding block is composed of a plurality of spatial frequency hybrid blocks; The specific process of restoring a compressed blurred image through a spatial-frequency hybrid restoration network includes the following steps: S1. The compressed blurred image is passed through a 3×3 convolutional layer to obtain shallow features, and the shallow features are passed through a first coding block to obtain first coded features; S2. downsampling the first coding feature and obtaining a second coding feature through a second coding block, and downsampling the second coding feature and obtaining a third coding feature through a third coding block; S3. Passing the third encoded feature through the first decoding block to obtain a first decoded feature, and upsampling the first decoded feature to obtain a first upsampled feature; S4. The first up-sampled feature and the second encoded feature are fused and input into the second decoding block to obtain a second decoded feature, and the second decoded feature is up-sampled to obtain a second up-sampled feature; S5. The second up-sampled feature is fused with the first encoded feature and input into the third decoding block to obtain a third decoded feature, and the third decoded feature is passed through a 3×3 convolution layer to obtain a convolution feature; S6. Connect the compressed blurred image with the convolution feature residual to obtain the restored image.

2. The spatial frequency hybrid restoration method for restoring a JPEG compressed blurred image according to claim 1, characterized in that: The spatial-frequency mixing block includes a first LN layer, a first convolutional layer, a depth convolutional layer, an SG layer, a spatial-frequency dual-branch mixing structure, a second LN layer, a second convolutional layer and an ISS block connected in sequence; wherein, there is a residual connection between the input of the first LN layer and the output of the spatial-frequency dual-branch mixing structure, and there is a residual connection between the input of the second LN layer and the output of the ISS block.

3. The spatial frequency hybrid restoration method for restoring a JPEG compressed blurred image according to claim 2, characterized in that: The spatial-frequency dual-branch hybrid structure includes a patch-level channel attention branch and a pixel-level global attention branch; The processing of the spatial frequency dual-branch hybrid structure includes The input features are passed through the patch-level channel attention branch and the pixel-level global attention branch to obtain structural features and channel attention; Multiply the structural features by the input features to get PCAB output, and multiply the channel attention by the input features to get PGAB output; The PCAB output is connected to the PGAB output and then subjected to a 1×1 convolution to obtain the output of the spatial-frequency dual-branch hybrid structure.

4. The spatial frequency hybrid restoration method for restoring a JPEG compressed blurred image according to claim 3, characterized in that: The patch-level channel attention branch includes a cascaded sliding window operation layer, a first depth convolution layer, a second depth convolution layer, an activation function layer, and a third depth convolution layer; the sliding window size in the sliding window operation layer is 8×8, and the sliding step is 4; the convolution kernel size of the first depth convolution layer is 8×8, and the step is 8; the convolution kernel size of the second and third depth convolution layers are both 1×1.

5. The spatial frequency hybrid restoration method for restoring JPEG compressed blurred images according to claim 3, characterized in that: The processing of the pixel-level global attention branch includes: Perform fast Fourier transform on each pixel in the input feature along the channel dimension to obtain the image frequency characteristics; Perform global average pooling on the image frequency characteristics along the spatial dimension to obtain pooled features; The pooled features are inversely fast Fourier transformed along the channel dimension to obtain the channel attention.

6. The spatial frequency hybrid restoration method for restoring a JPEG compressed blurred image according to claim 2, characterized in that: The processing of the ISS block includes: Divide the input features into 8×8 non-overlapping image patches; For each non-overlapping image patch, a fast Fourier transform is performed in the spatial domain to obtain the transformation result, and the transformation result is multiplied by the learnable matrix and then an inverse fast Fourier transform is performed to obtain the important spatial information in the frequency domain; the important spatial information in the frequency domain passes through a deep convolution layer and then performs an element-by-element multiplication operation to obtain the processing feature, and the processed feature passes through a 1×1 convolution layer and is multiplied with the learnable vector to obtain the screening feature.

7. The spatial frequency hybrid restoration method for restoring a JPEG compressed blurred image according to claim 1, characterized in that: The downsampling operation uses a convolutional layer with a kernel size of 2×2 and a stride of 2; the upsampling operation uses a convolutional layer with a kernel size of 1×1.

Images