Image restoration method and system

Abstract

The invention relates to the field of image processing, in particular to an image restoration method and system, and the method comprises the steps: inputting a degraded image into an image restoration network; the method comprises the following steps: respectively extracting channel information, space information and frequency domain information of a degraded image from three dimensions of an image channel, space and frequency through an image restoration network, restoring the degraded image by utilizing the extracted channel information, space information and frequency domain information, and outputting a restored image. While degeneration is removed, texture details in multiple complex degeneration scenes can be recovered, and the calculation complexity of the model can be reduced.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to an image restoration method and system. Background Technology

[0002] In recent years, data-driven deep neural networks have made rapid progress in the field of image restoration, and many ingenious functional modules based on convolutional neural networks (CNNs) and Transformers have been designed to obtain high-quality output predictions, such as:

[0003] 1. Channel attention blocks and pixel attention blocks were designed using convolution to adaptively learn feature information at different levels. To capture long-range dependencies, many researchers have customized Transformer-based models for image restoration tasks. 2. To improve the computational efficiency of the Transformer model, it is proposed to apply self-attention in the channel dimension.

[0004] The methods described above primarily operate in the spatial and channel domains. While they improve the visual quality of image inpainting to some extent, degradation in real-world scenes exhibits complex distribution characteristics, and edge information plays a crucial role in restoring the texture details of a sharp image. However, the aforementioned Transformer operations are inherently insensitive to local texture details, and layer-by-layer convolution operations often ignore edge points, making it difficult to guarantee inpainting quality. Image inpainting not only needs to remove degradation but also needs to restore fine details and textures.

[0005] Low-frequency components of an image retain more structural information, such as color and brightness, while high-frequency components represent finer details, such as edges and textures. Therefore, frequency information is crucial for restoring the structure and texture of an image. Recently, some studies have explored image inpainting in the frequency domain. However, the lack of frequency separation hinders targeted processing of features across different frequency bands, increasing the complexity of feature learning and reducing the interpretability of the method. Although high and low-frequency information have been processed separately, interactive complementary enhancement between these frequency components has not been achieved. Given that different types of degradation depend on high and low-frequency information to varying degrees—dehazing tasks prioritize restoring low-frequency information, while denoising tasks focus more on enhancing high-frequency details—it is necessary to adjust the relative importance of high and low-frequency features through interactive modulation to adapt to various complex degradation scenarios.

[0006] Meanwhile, it was also observed that many methods operate in the frequency domain at deeper layers of the U-net network. However, since the feature map size is already quite small, the frequency information is almost lost. Excessive batch processing increases computational complexity and also affects model performance. Summary of the Invention

[0007] The purpose of this invention is to overcome the problems of insufficient image restoration quality, lack of sensitivity to local texture details, and severe loss of frequency information in the existing technology, and to provide an image restoration method and system.

[0008] In a first aspect, the present invention provides an image restoration method, comprising: Input the degraded image into the image restoration network; The image restoration network repairs the degraded image from three dimensions: image channel, space, and frequency, and outputs the restored image. The image restoration network learns hierarchical representations through an encoder-decoder architecture; During the encoding phase, the original features from the input are supplemented by skip connections, which include average pooling, pixel-wise convolution, and depthwise convolution. During the decoding stage, encoder features are connected to the decoder via skip connections for auxiliary reconstruction; The encoder-decoder architecture consists of three-domain blocks in the first two layers and two-domain blocks in the last two layers. The three-domain blocks are used to extract channel information, spatial information, and frequency domain information and capture texture and details. The two-domain blocks are used to extract semantic information and recover the low-frequency information of the image.

[0009] Based on the divide-and-conquer approach, this method comprehensively extracts spatial, channel, and frequency domain information from the input image from three dimensions: channel, space, and frequency. While removing degradation, it can restore texture details in complex degradation scenes and reduce the computational complexity of the model itself.

[0010] Preferably, the process of extracting channel information includes: The query, key, and value projections of the input image are generated by 1×1 pointwise convolution and 3×3 depthwise convolution. The query and key are reshaped, and an attention map is generated by the dot product of the query and key as channel information.

[0011] Preferably, the process of extracting spatial information includes: The input image is fed into the upper branch, and the upper branch features are extracted by sequentially passing it through 1×1 convolution and 3×3 depth convolution. The input image is input into the lower branch, and the features of the lower branch are extracted sequentially through 1×1 convolution, 3×3 depth convolution and GELU activation function; The spatial information is obtained by multiplying the upper and lower branches element by element and then restoring the number of channels through a 1×1 convolution.

[0012] Preferably, the process of extracting frequency domain information includes: Frequency separation: The input image and intermediate features are aligned, and two-dimensional wavelet coefficients are calculated using forward discrete wavelet transform. The obtained two-dimensional wavelet coefficients are zero-padded, and a discrete inverse wavelet transform is performed to separate low-frequency and high-frequency information; the intermediate features are features containing channel information and spatial information. Frequency mining: Based on low-frequency and high-frequency information, a transposed channel attention mechanism is used to obtain low-frequency mining features and high-frequency mining features from the intermediate features; Interactive integration: Low-frequency mining features and high-frequency mining features are interactively refined to obtain low-frequency refined features and high-frequency refined features; 1×1 convolution is used to aggregate the low-frequency refined features and high-frequency refined features to generate global frequency features, and the transposed channel attention mechanism is used to fuse the global frequency features into the intermediate features to obtain frequency domain information.

[0013] Preferably, the process of refining low-frequency mining features includes: Global max pooling and global average pooling are performed on the high-frequency mining features respectively to generate two single-channel spatial feature maps, which are then connected along the channel dimension. The number of channels is adjusted and features are refined by using 7×7 convolution, and a spatial attention map is generated by using the sigmoid function. The spatial attention map is multiplied by the low-frequency mining features to generate low-frequency refined features.

[0014] Preferably, the process of refining high-frequency mining features includes: Adaptive average pooling and adaptive max pooling are applied to the low-frequency mining features along the spatial dimension, respectively. The features after adaptive average pooling are sequentially subjected to two 1×1 convolutions to obtain the upper branch features; The features after adaptive max pooling are sequentially subjected to two 1×1 convolutions to obtain the next branch features; After adding the features from the upper branch to the features from the lower branch, a channel attention map is generated using the sigmoid function; The channel attention map is multiplied by the high-frequency mining features to generate high-frequency refined features.

[0015] Preferably, L1 loss and SSIM loss are used as pixel fidelity losses to optimize the image restoration network.

[0016] In a second aspect, the present invention provides an image restoration system for performing the method described in the first aspect, comprising: The image restoration unit stores the image restoration network, including a channel attention module, a spatial awareness module, and a frequency learning module; The channel attention module is used to extract channel information of the input image from the channel dimension; The spatial perception module is used to extract spatial information from the input image from a spatial dimension; The frequency learning module is used to extract high-frequency and low-frequency information from the input image and integrate it into frequency domain information.

[0017] Preferably, the encoder-decoder architecture of the image restoration network is as follows: the first two layers of the encoder and the decoder are three-domain blocks, and the last two layers are two-domain blocks; The three-domain block includes a transposed channel attention module, a spatial awareness module, and a frequency learning module, which are used to extract features and capture texture and details; The dual-domain block includes a transposed channel attention and a spatial awareness module, used to extract semantic information.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention provides an image restoration method and system based on the three-domain divide-and-conquer approach. It comprehensively extracts spatial, channel, and frequency domain information of the input image from three dimensions: channel, space, and frequency. From a divide-and-conquer perspective, in the shallow layers, i.e., layers with relatively high feature map resolution, rich features are extracted and complex textures and details are captured; in the deeper layers, i.e., layers with reduced feature map resolution, the focus is on abstract semantic information. This invention can restore texture details in complex degraded scenes while removing degradation, and it can also reduce the computational complexity of the model itself, enhancing the network's ability to represent various features. Attached Figure Description

[0019] Figure 1 This is a flowchart of an image restoration method in one or more embodiments of the present invention; Figure 2 This is a diagram illustrating the image restoration network and its internal modules in one or more embodiments of the present invention. Among them, (a) is the architecture diagram of the image restoration network; (b) is the architecture diagram of the frequency learning module; (c) is the architecture diagram of the spatial perception module; (d) is the architecture diagram of the transposed channel attention module; (e) is the architecture diagram of the spatial attention unit; and (f) is the architecture diagram of the channel attention unit. Figure 3 This is a qualitative comparison result diagram of the experiment on [Setting 1] provided in one embodiment of the present invention; Figure 4 This is a graph showing the qualitative comparison results of an experiment provided in one embodiment of the present invention on the [Set 2] Rain1400 dataset; Figure 5 This is a graph showing the qualitative comparison results of an experiment provided in one embodiment of the present invention on the [Setting 2] RESIDE dataset; Figure 6This is a graph showing the qualitative comparison results of an experiment provided in one embodiment of the present invention on the [Set 2] Snow100K-L dataset; Figure 7 This is a qualitative comparison result diagram of the experiment on [Setting 3] provided in one embodiment of the present invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to specific embodiments. However, this should not be construed as limiting the scope of the present invention to the following embodiments; all technologies implemented based on the content of the present invention fall within the scope of the present invention.

[0021] Example 1 Please refer to Figure 1 This embodiment provides an image restoration method, including: Input the degraded image into the image restoration network; The image restoration network repairs the degraded image from three dimensions: image channel, space, and frequency, and outputs the restored image. The image restoration network learns hierarchical representations through an encoder-decoder architecture; During the encoding phase, the original features from the input are supplemented by skip connections, which include average pooling, pixel-wise convolution, and depthwise convolution. During the decoding stage, encoder features are connected to the decoder via skip connections for auxiliary reconstruction; The encoder-decoder architecture consists of three-domain blocks in the first two layers and two-domain blocks in the last two layers. The three-domain blocks are used to extract channel information, spatial information, and frequency domain information and capture texture and details. The two-domain blocks are used to extract semantic information and recover the low-frequency information of the image.

[0022] In embodiments of the present invention, "degraded image" refers to image data whose visual quality and information fidelity decrease due to interference from various factors during acquisition, encoding, transmission or storage; "low-frequency information of the image" refers to the background component that characterizes the overall brightness distribution and smooth color transition of the image, and the structural component that reflects the main outline and slow changing trend of the image.

[0023] This embodiment is based on the three-domain divide-and-conquer approach, comprehensively extracting spatial, channel, and frequency domain information from the input image from three dimensions: channel, space, and frequency. From a divide-and-conquer perspective, in the shallow layers, i.e., layers with relatively high feature map resolution, rich features are extracted and complex textures and details are captured; in the deeper layers, i.e., layers with reduced feature map resolution, the focus is on abstract semantic information. This invention can restore texture details in complex degraded scenes while removing degradation, and can also reduce the computational complexity of the model itself, enhancing the network's ability to represent various features.

[0024] Example 2 Please refer to Figure 2 This embodiment is a further optimization of embodiment 1.

[0025] As one possible implementation, the image restoration network learns hierarchical representations through an encoder-decoder architecture; During the encoding phase, the original features from the input are supplemented by skip connections, which include average pooling, pixel-wise convolution, and depthwise convolution. During the decoding stage, encoder features are connected to the decoder via skip connections for assisted reconstruction.

[0026] Specifically, the image restoration network consists of a four-layer encoder and a four-layer decoder. The first two layers of the encoder and decoder consist of three-domain blocks, in which the feature maps are processed sequentially by TCA (Transposed Channel Attention), SPM (Spatial Perception Module), and FLM (Frequency Learning Module) modules in the three domains respectively. In the last two layers of the encoder and decoder, considering that the frequency information in the feature maps is almost completely lost at this point, a two-domain block is used for processing. The two-domain block consists of TCA and SPM, in which the feature maps are processed in both the channel domain and the spatial domain.

[0027] Given a degraded image of shape H×W×3, where 3 represents the number of channels and H×W represents the spatial size, a shallow feature of size H×W×C is extracted using a 3×3 convolution. This shallow feature is then generated through a four-scale encoder (two three-domain blocks and two two-domain blocks). Downsampling is performed using PixelUnshuffle, gradually reducing the resolution while expanding the number of channels. At each stage of the encoding process, our model is equipped with a coarse skip connection to supplement the original features from the input. This skip connection consists of a series of operations, including average pooling, pixel-wise convolution, and depthwise convolution. The resulting deepest features are then fed into the four-scale decoder, where PixelShuffle restores the features to their original size, and FLM restores details and textures. Simultaneously, the encoder features are connected to the decoder via skip connections to aid reconstruction, and the number of channels is halved using a 1×1 convolution.

[0028] As one possible implementation, this embodiment utilizes a transposed channel attention module to extract channel information; The transposed channel attention module processes the input image as follows: The query, key, and value projections of the input image are generated by 1×1 pointwise convolution and 3×3 depthwise convolution. The query and key are reshaped, and an attention map is generated by the dot product of the query and key as channel information.

[0029] In traditional self-attention, the computational complexity of the query (Q) and key (K) dot product increases quadratically with the resolution of the input space, limiting its application on high-resolution maps and making it unsuitable for hardware-restricted scenarios. Here, we apply TCA (Transmission-Channel Alignment) to perform self-attention in the channel dimension rather than the spatial dimension, thereby achieving global channel interaction of intermediate feature maps, such as... Figure 2 As shown in (d).

[0030] Given input features First, projections of the query (Q), key (K), and value (V) are generated through 1×1 pointwise convolutions and 3×3 depthwise convolutions. Then, Q and K are reshaped so that their dot product produces a C×C attention map. Similar to traditional multi-head attention, we divide the number of channels into 'heads' for parallel attention learning. The specific processing steps are given below: (1) , , (2) in, This represents a 3×3 depthwise convolution. This represents a 1×1 pointwise convolution. Q , K and V These are query, key, and value projection, respectively. α It is a scaling factor used to control Q and K The size of the dot product between them.

[0031] As one possible implementation, this embodiment utilizes a spatial sensing module to extract spatial information; The spatial perception module's processing of the input image includes: The input image is fed into the upper branch, and the upper branch features are extracted by sequentially passing it through 1×1 convolution and 3×3 depth convolution. The input image is input into the lower branch, and the features of the lower branch are extracted sequentially through 1×1 convolution, 3×3 depth convolution and GELU activation function; The spatial information is obtained by multiplying the upper and lower branches element by element and then restoring the number of channels through a 1×1 convolution.

[0032] SPM is a two-branch structure, such as Figure 2As shown in (c), the upper branch expands the number of channels through a 1×1 convolution, and then uses a 3×3 depthwise convolution to learn local image structure information. The difference is that the lower branch adds a GELU nonlinear function after the convolution. Finally, the upper and lower branches are multiplied element-wise and then a 1×1 convolution is used to restore the number of channels. Given input... The expression for SPM is: (3) (4) in, It is the GELU activation function.

[0033] As one possible implementation, this embodiment utilizes a frequency learning module to extract frequency domain information.

[0034] The frequency learning module processes the input image as follows: Frequency separation: The input image and intermediate features are aligned, and two-dimensional wavelet coefficients are calculated using forward discrete wavelet transform. The obtained two-dimensional wavelet coefficients are then zero-filled and subjected to discrete inverse wavelet transform to separate low-frequency and high-frequency information. The intermediate features are features that contain channel information and spatial information. Frequency mining: Based on low-frequency and high-frequency information, a transposed channel attention mechanism is used to obtain low-frequency mining features and high-frequency mining features from the intermediate features; Interactive integration: Low-frequency mining features and high-frequency mining features are interactively refined to obtain low-frequency refined features and high-frequency refined features; 1×1 convolution is used to aggregate the low-frequency refined features and high-frequency refined features to generate global frequency features, and the transposed channel attention mechanism is used to fuse the global frequency features into the intermediate features to obtain frequency domain information.

[0035] In the description of the embodiments of this application, "low frequency" refers to the background component that characterizes the overall brightness distribution and smooth color transition of the image, and the structural component that reflects the main outline and slow change trend of the image; "high frequency" refers to the edge and texture details that characterize drastic changes in pixel values, and the fine component caused by sensor noise or random disturbance.

[0036] Although TCA and SPM processes remove some degradation in the degraded image, convolution and Transformer operations struggle to restore texture details, resulting in inconsistent image restoration quality. To mitigate this issue, we designed a frequency learning module. Figure 2 As shown in (b), the degraded image X and intermediate features Simultaneous input, processed through discrete wavelet transform and its inverse transform. X Perform spectral decoupling, from FWe learn different frequency representations. Our proposed FLM mainly consists of three parts: frequency separation, mining, and interactive integration.

[0037] For the frequency separation section X First, it is processed by a convolutional group to combine its spatial dimensions and channel dimensions. F Alignment is then performed, followed by computation of two-dimensional wavelet coefficients using the forward discrete wavelet transform (DWT): (5) in, , , and They represent The approximation coefficients, horizontal coefficients, vertical coefficients, and diagonal coefficients are used. Each coefficient is processed as follows: (6) (7) in, This represents a depthwise convolution with a kernel size of 3×3. To process low-frequency and high-frequency information separately, we employ zero-padding and discrete inverse wavelet transform to obtain the denoised low-frequency components (in...). Figure 2 (represented in the upper branch of (b)) and high-frequency components containing texture details (in Figure 2 (represented in the lower branch of (b)). The specific processing steps are as follows: (8) (9) Among them, low frequency ( ) and high frequency ( ( ) respectively refer to the degraded image X The low-frequency and high-frequency information separated from it.

[0038] Next, we employ the transposed channel attention mechanism in and Guided by this, we extract different feature parts from the intermediate feature map F. Here we will extract low-frequency ( and high frequency ( ) Components as queries in the attention mechanism ( Q ), F As key (K) and value ( V The overall attention calculation using the transposed channel attention mechanism is as follows: (10)

[0039]

[0040] (11) in Indicates high frequency and low frequency.

[0041] Subsequently, we used two lightweight spatial attention units (SAU) and channel attention units (CAU) to mine low-frequency features. and high-frequency mining features Interactive integration, such as Figure 2 As shown in (e) and (f), due to It contains rich texture details, so SAU is used to enrich it. Similarly, The global information existing in the system is supplemented by CAU. middle.

[0042] The process of refining low-frequency mining features includes: Global max pooling and global average pooling are performed on the high-frequency mining features respectively to generate two single-channel spatial feature maps, which are then connected along the channel dimension. The number of channels is adjusted and features are refined by using 7×7 convolution, and a spatial attention map is generated by using the sigmoid function. The spatial attention map is multiplied by the low-frequency mining features to generate low-frequency refined features.

[0043] For example, such as Figure 2 As shown in (e), in SAU, given , Two single-channel spatial feature maps, each with dimension 1, are generated in parallel using two different channel pooling techniques. H×W×1 Then, the features are connected along the channel dimension, and the number of channels is adjusted and the features are refined through a 7×7 convolution. Finally, a spatial attention map with dimensions H×W×1 is formed through a sigmoid function. The SAU processing is represented as follows: (12) (13) in, It is the sigmoid function. It is a convolution operation with a kernel size of 7 and half the number of channels.

[0044] The process of refining high-frequency mining features includes: Adaptive average pooling and adaptive max pooling are applied to the low-frequency mining features along the spatial dimension, respectively. The features after adaptive average pooling are sequentially subjected to two 1×1 convolutions to obtain the upper branch features; The features after adaptive max pooling are sequentially subjected to two 1×1 convolutions to obtain the next branch features; After adding the features from the upper branch to the features from the lower branch, a channel attention map is generated using the sigmoid function; The channel attention map is multiplied by the high-frequency mining features to generate high-frequency refined features.

[0045] For example, such as Figure 2 As shown in (f), CAU is a two-branch structure. In the upper branch, given The process involves adaptive average pooling along the spatial dimension to obtain a 1×1×C vector, followed by two 1×1 convolutions. The only difference in the lower branch is that adaptive max pooling is applied to the header. The results of the two branches are then summed, and finally, the sigmoid function is applied to form a 1×1×C attention map. The CAU process can be represented as follows: (14) (15) Where γ is the ReLU activation function, and AAP and AMP are adaptive average pooling and adaptive max pooling, respectively.

[0046] In the final part of the frequency learning module, we use 1×1 convolutional aggregation. and produce Then, a transposed channel attention mechanism is used to fuse it into the intermediate features. F In the middle, search here Q Vectors are intermediate features F The mapping, and the key K Sum V yes The mapping.

[0047] This invention uses a frequency learning module as the frequency domain processing part. By modulating the high-frequency and low-frequency content of the input features separately, it strengthens key frequency bands, suppresses useless information, and realizes the detailed restoration of complex degradation scenes.

[0048] As one possible implementation, L1 loss (mean absolute error) and SSIM loss (structural similarity loss) are used as pixel fidelity losses to optimize the image restoration network, as shown in the following formula: (16) (17) The total loss is: (18) Preferably, The value is 0.25.

[0049] To verify the effectiveness of our proposed method, we conducted our experiments on three dataset settings, as shown in Table 1. Previous severe weather removal methods have only tested their model performance on synthetic datasets, such as [Setting 1] and [Setting 2]. In addition, we also used a real-world benchmark dataset with mixed degradation ([Setting 3]).

[0050] For [Setting 1], to ensure a fair comparison, the training set contains 9000 image pairs from Snow100K-L, 1069 image pairs from RainDrop, and 8250 image pairs from Outdoor-Rain. For evaluation, we use the Test1 dataset, the RainDrop test dataset, and the Snow100K-L test set. [Setup 2] is a combination of three datasets: Rain1400, RESIDE-OTS, and Snow100K-L. Rain1400 contains 12,600 training images and 1,400 test images, RESIDE-OTS contains 72,135 training images and 500 test images, and Snow100K-L is the same as in [Setup 1]. In [Settings 3], it contains two real-world rain and fog hybrid degradation datasets: SPA+ and GT-RAIN, which are used for testing purposes only. The SPA+ dataset contains 1000 test images, while the GT-RAIN dataset contains 300 test images.

[0051] Table 1. Training Data Description

[0052] In severe weather images, the low frequencies that need to be removed are fog, and the high frequencies that need to be removed are rain, raindrops, and snow.

[0053] Training details: Batch size set to 24, image patch size 128×128. Each patch is randomly horizontally flipped with a probability of 0.5 for data augmentation. A linear warm-up cosine annealing scheduler is used, with a total training duration of 160 epochs and a maximum of 180 epochs. The learning rate is gradually increased from 0 to 2e in the first 15 epochs. -4 For the remaining epochs, the learning rate is gradually reduced using a cosine annealing strategy.

[0054] Experimental results are as follows Figures 3-7 As shown, Figure 3The qualitative comparison on the test set in [Setup 1] is presented, showing that other algorithms suffer from distortion problems in image restoration, such as distorted text on roads and roof lights being mistaken for snow and removed. In contrast, our method consistently recovers more texture details and provides a more realistic restored image.

[0055] Figure 4 , Figure 5 and Figure 6 The qualitative comparison results shown in [Setting 2] demonstrate that the method provided by this invention is more thorough in removing degradations such as rain streaks, fog, and snow. Figure 4 In Figure 5, the texture details of the grass and houses were restored more perfectly. These results demonstrate that the method provided by this invention can effectively remove degradation, preserve texture details, and generate more natural images, improving both global and local contrast. In contrast, other methods either failed to remove degradation or produced incorrect texture patterns.

[0056] exist Figure 7 Compared with other methods, the method provided by this invention is more thorough in removing real-world rain streaks, while restoring tree details more faithfully.

[0057] The qualitative comparison results above demonstrate the superiority of our method over other methods in image restoration tasks.

[0058] Example 3 This embodiment provides an image restoration system for performing the method described in Embodiment 1 or Embodiment 2, including: Image restoration network, including channel attention module, spatial awareness module and frequency learning module; The channel attention module is used to extract channel information of the input image from the channel dimension; The spatial perception module is used to extract spatial information from the input image from a spatial dimension; The frequency learning module is used to extract high-frequency and low-frequency information from the input image and integrate it into frequency domain information.

[0059] Specifically, the encoder-decoder architecture of the image restoration network is as follows: the first two layers of the encoder and the decoder are three-domain blocks, and the last two layers are two-domain blocks; The three-domain block includes a transposed channel attention module, a spatial awareness module, and a frequency learning module, which are used to extract features and capture texture and details; The dual-domain block includes a transposed channel attention and a spatial awareness module, used to extract semantic information.

[0060] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An image restoration method, characterized in that, include: Input the degraded image into the image restoration network; The image restoration network repairs the degraded image from three dimensions: image channel, space, and frequency, and outputs the restored image. The image restoration network learns hierarchical representations through an encoder-decoder architecture; During the encoding phase, the original features from the input are supplemented by skip connections, which include average pooling, pixel-wise convolution, and depthwise convolution. During the decoding stage, encoder features are connected to the decoder via skip connections for auxiliary reconstruction; The encoder-decoder architecture consists of three-domain blocks in the first two layers and two-domain blocks in the last two layers. The three-domain blocks are used to extract channel information, spatial information, and frequency domain information and capture texture and details. The two-domain blocks are used to extract semantic information and recover the low-frequency information of the image.

2. The image restoration method according to claim 1, characterized in that, The process of extracting channel information includes: The query, key, and value projections of the input image are generated by 1×1 pointwise convolution and 3×3 depthwise convolution. The query and key are reshaped, and an attention map is generated by the dot product of the query and key as channel information.

3. The image restoration method according to claim 1, characterized in that, The process of extracting spatial information includes: The input image is fed into the upper branch, and the upper branch features are extracted by sequentially passing it through 1×1 convolution and 3×3 depth convolution. The input image is input into the lower branch, and the features of the lower branch are extracted sequentially through 1×1 convolution, 3×3 depth convolution and GELU activation function; The spatial information is obtained by multiplying the upper and lower branches element by element and then restoring the number of channels through a 1×1 convolution.

4. The image restoration method according to claim 1, characterized in that, The process of extracting frequency domain information includes: Frequency separation: The input image and intermediate features are aligned, and two-dimensional wavelet coefficients are calculated using forward discrete wavelet transform. The obtained two-dimensional wavelet coefficients are zero-padded, and a discrete inverse wavelet transform is performed to separate low-frequency and high-frequency information; the intermediate features are features containing channel information and spatial information. Frequency mining: Based on low-frequency and high-frequency information, a transposed channel attention mechanism is used to obtain low-frequency mining features and high-frequency mining features from the intermediate features; Interactive integration: Low-frequency mining features and high-frequency mining features are interactively refined to obtain low-frequency refined features and high-frequency refined features; 1×1 convolution is used to aggregate the low-frequency refined features and high-frequency refined features to generate global frequency features, and the transposed channel attention mechanism is used to fuse the global frequency features into the intermediate features to obtain frequency domain information.

5. The image restoration method according to claim 4, characterized in that, The process of refining low-frequency mining features includes: Global max pooling and global average pooling are performed on the high-frequency mining features respectively to generate two single-channel spatial feature maps, which are then connected along the channel dimension. The number of channels is adjusted and features are refined by using 7×7 convolution, and a spatial attention map is generated by using the sigmoid function. The spatial attention map is multiplied by the low-frequency mining features to generate low-frequency refined features.

6. The image restoration method according to claim 4, characterized in that, The process of refining high-frequency mining features includes: Adaptive average pooling and adaptive max pooling are applied to the low-frequency mining features along the spatial dimension, respectively. The features after adaptive average pooling are sequentially subjected to two 1×1 convolutions to obtain the upper branch features; The features after adaptive max pooling are sequentially subjected to two 1×1 convolutions to obtain the next branch features; After adding the features from the upper branch to the features from the lower branch, a channel attention map is generated using the sigmoid function; The channel attention map is multiplied by the high-frequency mining features to generate high-frequency refined features.

7. The image restoration method according to claim 1, characterized in that, The image restoration network is optimized using L1 loss and SSIM loss as pixel fidelity losses.

8. An image restoration system, characterized in that, For performing the method according to any one of claims 1-7, comprising: The image restoration unit stores the image restoration network, including a channel attention module, a spatial awareness module, and a frequency learning module; The channel attention module is used to extract channel information of the input image from the channel dimension; The spatial perception module is used to extract spatial information from the input image from a spatial dimension; The frequency learning module is used to extract high-frequency and low-frequency information from the input image and integrate it into frequency domain information.

9. The image restoration system according to claim 8, characterized in that, The encoder-decoder architecture of the image restoration network is as follows: the first two layers of the encoder and the decoder are three-domain blocks, and the last two layers are two-domain blocks; The three-domain block includes a transposed channel attention module, a spatial awareness module, and a frequency learning module, which are used to extract features and capture texture and details; The dual-domain block includes a transposed channel attention and a spatial awareness module, used to extract semantic information.

Images