An image rain removal method and system based on direction perception frequency domain filtering

Abstract

This invention discloses an image deraining method and system based on direction-aware frequency domain filtering. The method involves preprocessing rain and no-rain images with data augmentation; using a direction-aware frequency domain filter combined with a ResNet50 network to extract frequency domain fusion feature maps from the augmented data of the rain image; inputting the frequency domain fusion feature map into a neural compression-based feature optimizer to obtain a noise-robust fusion feature map; upsampling the noise-robust fusion feature map to obtain the final feature map as the generated derained image; constructing a loss function to train and form a deraining model; and having the user input a rainy image to be processed into the deraining model to obtain the generated derained image. This invention combines the distribution characteristics of rainwater in images and differentiates and processes different frequency components of the image, effectively removing rainwater interference while preserving image structure and detail information. The introduction of a neural compression-based feature optimization module avoids interference from noise that may exist in the features.

Description

Technical Field

[0001] This invention relates to the field of image processing, and specifically to an image deraining method and system based on direction-aware frequency domain filtering. Background Technology

[0002] In the field of computer vision, image deraining refers to designing various feasible algorithms based on existing hardware to remove rainwater and its effects, such as blurring and occlusion, from images, thereby restoring a clean and clear image free of rain. As a low-level vision task, image deraining is usually used as a preprocessing step to provide performance support for other high-level vision tasks, such as image segmentation and object detection.

[0003] The challenges of image deraining lie in the complexity and diversity of rainy weather and the scarcity of high-quality rain datasets. The complexity and diversity of rainy weather present two major challenges to image deraining: First, most existing deraining methods only process single rain types, such as rain streaks, raindrops, or rain fog. These methods experience a sharp decline in performance when dealing with unprocessed rain types. Furthermore, in real-world rainy weather, multiple rain types often overlap. The second challenge is that the unpredictability and wide-ranging nature of rainfall make it virtually impossible to collect real-world rainy datasets with clean, rain-free labels. This has led to existing methods typically relying on artificially synthesized rain datasets.

[0004] Existing image deraining methods can be broadly categorized into two types: model-based methods and data-based methods. Model-based methods typically first utilize various prior knowledge to complete and refine the model of rain image composition, and then design effective optimization algorithms to continuously improve the model so that it approximates the composition of the real rain image. Data-based methods extract deep features from images by learning from a large amount of rainy data, and then achieve image deraining by removing rain features from the image and retaining clean background features.

[0005] One existing technique is the single-image deraining method based on direction-aware feature-level frequency decomposition, as described in the paper "Direction-aware Feature-level Frequency Decomposition for Single Image Deraining". This method first uses a convolution operation to extract shallow features from the image. Then, it uses a novel direction-aware cross-mean filter to explicitly extract high-frequency and low-frequency information from the features. Next, it combines an interactive adapter to implicitly enhance the decomposition of high-frequency and low-frequency information through interactive connections. Finally, it processes the extracted high-frequency and low-frequency information separately using a convolution operation and sums the results to obtain the final derained image. The drawbacks of this method are that it primarily considers frequency domain feature extraction, with less attention paid to spatial domain features. This results in spatially inconsistent and less refined deraining results. Furthermore, this invention directly uses the extracted high and low-frequency information to simply reconstruct the derained image, without considering the possibility of noise interference within the information, leading to insufficient deraining performance.

[0006] The second existing technology is a video restoration method based on neural compression feature learning, as described in the paper "Neural Compression-Based Feature Learning for Video Restoration". This method first uses a pre-trained optical flow estimation network to estimate motion vectors, then uses a lightweight encoder-decoder network to optimize the obtained motion vectors, generating aligned temporal features using the optimized motion vectors. Next, an attention module and a neural compression module are used to enhance and denoise the temporal features. Finally, a restoration module generates the final clear video. The drawback of this method is that it focuses on processing temporal features in the video while neglecting the distribution characteristics of noise in individual video frames, lacking processing of single frames in the spatial and frequency domains, resulting in mediocre performance on a single frame in the final video restoration.

[0007] The third existing technology is the patent "Method and System for Removing Rain Stripes from Rainy Images Based on Image Filtering and CNN (CN111815526A)". This invention first expands the training rainy image dataset using image dataset augmentation methods, then obtains the high-frequency components of the rainy image through image filtering, and then uses the high-frequency components of the image as input to the rain removal network, optimizing the network using the SSIM loss function. Finally, the rain-removed image is input into an HG discriminant network, and the discrimination results are used to further optimize the rain removal network. The disadvantages of this technology are that it discards the low-frequency components of the image, resulting in the loss of image structural information and a decrease in rain removal performance. Furthermore, the invention only uses SSIM as the network loss function, which makes it impossible to guarantee the smoothness and continuity of the image. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of existing methods and propose an image deraining method and system based on direction-aware frequency domain filtering. The main problem addressed by this invention is how to achieve high-quality derained image restoration by performing parallel processing of high-frequency and low-frequency features in a dual-branch manner, thereby optimizing the noise reduction of the high-low frequency fusion features and restoring the optimized features.

[0009] To address the above problems, this invention proposes an image deraining method based on direction-aware frequency domain filtering, the method comprising:

[0010] For each pair of rain maps and corresponding no-rain maps in the synthetic rainwater dataset, data augmentation preprocessing is performed to obtain the augmented dataset;

[0011] The frequency domain fusion feature map of the rain map in the augmented dataset is extracted by combining a direction-aware frequency domain filter with a ResNet50 network.

[0012] The frequency domain fusion feature map is input into a neural compression-based feature optimizer to obtain a noise-robust fusion feature map.

[0013] The noise-robust fusion feature map is upsampled until it is gradually restored to the original image size. The resulting final feature map is used as the generated derained image.

[0014] A loss function is constructed to calculate the difference between the generated derained image and the corresponding rainless image in the augmented dataset. The difference is used as the global loss to train and form a derained model.

[0015] The user inputs the rainy map to be processed into the rain removal model, and the generated rain removal map is obtained.

[0016] Preferably, the step of performing data augmentation preprocessing on each pair of rain maps and corresponding no-rain maps in the synthetic rainwater dataset to obtain an augmented dataset specifically involves:

[0017] For each pair of rain-no-rain images in the synthetic rain dataset, the same preprocessing operations of random flipping, random rotation, and fixed-size cropping are performed sequentially to obtain a single image size of 512×512, and the augmented dataset is output.

[0018] Preferably, the extraction of frequency domain fusion feature maps from the rain maps in the augmented dataset using a direction-aware frequency domain filter combined with a ResNet50 network specifically involves:

[0019] The rain map in the augmented dataset is decomposed into high-frequency and low-frequency components using a direction-aware frequency domain filter.

[0020] The high-frequency part is input into the high-frequency feature processing branch, and the low-frequency part is input into the low-frequency feature processing branch. The two branches use the Block 1 module in the ResNet50 network to extract and output the initial high-frequency features and the initial low-frequency features, respectively.

[0021] The low-frequency feature processing branch first uses a direction-aware frequency domain filter to decompose the initial low-frequency feature into a high-frequency part LH1 and a low-frequency part LL1. Then, LL1 is used as the input for the next stage of the low-frequency branch, and LH1 is passed to the high-frequency feature processing branch. The operation of the high-frequency feature processing branch corresponds to that of the low-frequency feature processing branch. First, a direction-aware frequency domain filter is used to decompose the initial high-frequency feature into a high-frequency part HH1 and a low-frequency part HL1. Then, HH1 is used as the input for the next stage of the high-frequency branch, and HL1 is passed to the low-frequency feature processing branch, outputting LL1, LH1, HL1, and HH1.

[0022] LL1 is concatenated with HL1 and LH1 with HH1 respectively to obtain the next stage inputs for the low-frequency processing branch and the high-frequency processing branch. This operation is repeated 3 times to obtain the outputs of the third stage as LL3, LH3, HL3, and HH3 respectively. LL3 is concatenated with HL3 and LH3 with HH3 respectively to obtain the final extracted low-frequency and high-frequency features of the low-frequency feature processing branch and the high-frequency processing branch. The frequency domain fusion feature map obtained by summing the low-frequency and high-frequency features is output as the result.

[0023] Preferably, the step of inputting the frequency domain fusion feature map into a neural compression-based feature optimizer to obtain a noise-robust fusion feature map specifically involves:

[0024] The frequency domain fused feature map is input into a feature encoder, which alternately uses two convolutional layers with a kernel size of 3×3 and a stride of 2, and a GDN module that can reduce feature dependencies to generate an implicit feature representation e.

[0025] The frequency domain fusion feature map is input into the prior module, which alternately uses three convolutional layers with a kernel size of 3×3 and a stride of 2, and two LeakyReLU activation layers to generate the data distribution [μ,σ,q], where μ represents the mean, σ represents the variance, and q represents the stride output.

[0026] The implicit representation e and the data distribution [μ,σ,q] are input into the quantization module to obtain the noise robust feature quantization result; the specific working principle of the quantization module is shown in the following equation:

[0027]

[0028] in This represents the quantization result of the noise robust feature, and "·" indicates the integer approximation operation;

[0029] The noise robust feature quantization result is input into the feature decoder, which has a structure corresponding to the feature encoder. It alternately uses two deconvolutional layers with a kernel size of 3×3 and a stride of 2, as well as an inverse GDN module. The feature decoder restores the noise robust feature quantization result to the size of the frequency domain fusion feature map to obtain the noise robust fusion feature map.

[0030] Preferably, the upsampling of the noise-robust fused feature map is performed until the noise-robust fused feature map is gradually restored to the original image size, and the resulting final feature map is used as the generated derained image. Specifically:

[0031] The noise-robust fusion feature map is progressively upsampled using four sequentially connected upsampling layers until it is gradually restored to the original image size. Each upsampling layer includes a deconvolution operation and a regularization operation. The sizes of the noise-robust fusion feature map in the four sequentially connected upsampling layers and the final output are 32×32, 64×64, 128×128, 256×256 and 512×512, respectively. The final feature map has the same size as the rain map in the augmented dataset, and this final feature map is the generated de-rain image.

[0032] Preferably, the loss function is constructed to calculate the difference between the generated derained image and the corresponding rainless image in the augmented dataset, and the difference is used as the global loss to train and form a deraining model, specifically:

[0033] The difference between the generated derained image and the corresponding rainless image in the augmented dataset is calculated using the L1 loss shown in the following formula. This difference is then used as the global loss to guide the training of the model until the model converges:

[0034]

[0035] Where H and W represent the length and width of the image, respectively, x represents the input derained image, f(x) represents the rainless image output by the network, y represents the clean label corresponding to the input derained image, and i and j represent the subscripts of each pixel in the image.

[0036] Accordingly, the present invention also provides an image deraining system based on direction-aware frequency domain filtering, comprising:

[0037] The data preprocessing unit is used to perform data augmentation preprocessing on each pair of rain maps and corresponding no-rain maps in the synthetic rainwater dataset to obtain the augmented dataset.

[0038] The frequency domain fusion feature map extraction unit is used to extract the frequency domain fusion feature map of the rain map in the augmented dataset by using a direction-aware frequency domain filter in combination with the ResNet50 network.

[0039] The noise-robust fusion feature map extraction unit is used to input the frequency domain fusion feature map into a neural compression-based feature optimizer to obtain a noise-robust fusion feature map.

[0040] The upsampling image generation unit is used to upsample the noise-robust fusion feature map until the noise-robust fusion feature map is gradually restored to the original image size, and the resulting final feature map is used as the generated deraining image.

[0041] The model training unit is used to construct a loss function to calculate the difference between the generated derained image and the corresponding rainless image in the augmented dataset, and use the difference as a global loss to train and form a derained model.

[0042] The image deraining unit is used to input a rainy image to be processed into the deraining model by the user, and to obtain a generated derained image.

[0043] Implementing this invention has the following beneficial effects:

[0044] The rain removal method proposed in this invention combines the distribution characteristics of rainwater in images, namely that rain streaks and raindrops are usually distributed in the high-frequency part of the image. By distinguishing and processing different frequency parts of the image, it can effectively remove rainwater interference while preserving the image's structure and detail information. The invention introduces a feature optimization module based on neural compression, avoiding interference from noise that may exist in the features, making the optimized features more noise-robust. The invention effectively extracts and integrates features, improving efficiency and making the obtained rain removal results more consistent with real-life situations. Attached Figure Description

[0045] Figure 1 This is a flowchart of an image deraining method based on direction-aware frequency domain filtering according to an embodiment of the present invention;

[0046] Figure 2 This is a flowchart of the extraction of frequency domain fusion feature maps according to an embodiment of the present invention;

[0047] Figure 3 This is a flowchart of generating a noise-robust fusion feature map according to an embodiment of the present invention;

[0048] Figure 4 This is a flowchart of generating a derained image according to an embodiment of the present invention;

[0049] Figure 5This is a structural diagram of an image deraining system based on direction-aware frequency domain filtering according to an embodiment of the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Figure 1 This is a flowchart illustrating the overall process of an image deraining method based on direction-aware frequency domain filtering according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0052] S1, perform data augmentation preprocessing on each pair of rain maps and corresponding no-rain maps in the synthetic rainwater dataset to obtain the augmented dataset;

[0053] S2, using a direction-aware frequency domain filter in conjunction with a ResNet50 network to extract frequency domain fusion feature maps from the rain maps in the augmented dataset;

[0054] S3, input the frequency domain fusion feature map into the neural compression-based feature optimizer to obtain a noise-robust fusion feature map;

[0055] S4, Upsample the noise-robust fusion feature map until the noise-robust fusion feature map is gradually restored to the original image size, and the resulting final feature map is used as the generated rain-removed image.

[0056] S5, construct a loss function to calculate the difference between the generated derained image and the corresponding rainless image in the augmented dataset, use the difference as the global loss to train and form a derained model;

[0057] S6, the user inputs the rainy map to be processed into the rain removal model to obtain the generated rain removal map.

[0058] Step S1 is as follows:

[0059] For each pair of rain-no-rain images in the synthetic rain dataset, the same preprocessing operations of random flipping, random rotation, and fixed-size cropping are performed sequentially to obtain a single image size of 512×512, and the augmented dataset is output.

[0060] Step S2, as follows Figure 2As shown in the figure, Blocki (i = 0, 1, 2, 3, 4) represents the five multi-stage feature extraction modules in the ResNet50 network. The feature map size is reduced to the size of the previous stage during the processing of the last four stages. The feature map sizes in each stage are 512×512, 256×256, 128×128, 64×64, and 32×32, respectively. Frequency domain filter representing direction sensing. This indicates a splicing operation. This indicates that the matrix is ​​added element by element.

[0061] Specifically as follows:

[0062] S2-1, using a direction-aware frequency domain filter to decompose the rain map in the augmented dataset into high-frequency and low-frequency components;

[0063] S2-2, the high-frequency part is input into the high-frequency feature processing branch, and the low-frequency part is input into the low-frequency feature processing branch. The two branches use the Block1 module in the ResNet50 network to extract and output the initial high-frequency features and the initial low-frequency features, respectively.

[0064] In S2-3, the low-frequency feature processing branch first uses a direction-aware frequency domain filter to decompose the initial low-frequency features of S2-2 into a high-frequency part LH1 and a low-frequency part LL1. Then, LL1 is used as the input for the next stage of the low-frequency branch, and LH1 is passed to the high-frequency feature processing branch. The operation of the high-frequency feature processing branch corresponds to that of the low-frequency feature processing branch. First, a direction-aware frequency domain filter is used to decompose the initial high-frequency features of S2-2 into a high-frequency part HH1 and a low-frequency part HL1. Then, HH1 is used as the input for the next stage of the high-frequency branch, and HL1 is passed to the low-frequency feature processing branch, outputting LL1, LH1, HL1, and HH1.

[0065] S2-4: Concatenate LL1 and HL1, LH1 and HH1 obtained in S2-3 respectively to obtain the next stage inputs for the low-frequency processing branch and the high-frequency processing branch. Repeat the above operation 3 times to obtain the outputs of the third stage as LL3, LH3, HL3 and HH3 respectively. Concatenate LL3 and HL3, LH3 and HH3 respectively to obtain the final extracted low-frequency features and high-frequency features of the low-frequency feature processing branch and the high-frequency processing branch. The frequency domain fusion feature map obtained by summing the low-frequency features and the high-frequency features is output as the result.

[0066] Step S3, as follows Figure 3 As shown, the details are as follows:

[0067] S3-1, input the frequency domain fused feature map from S2 into the feature encoder, which alternately uses two convolutional layers with a kernel size of 3×3 and a stride of 2, and a GDN module that can reduce feature dependencies to generate implicit feature representation e;

[0068] S3-2, input the frequency domain fusion feature map in S2 into the prior module. This prior module alternately uses three convolutional layers with a kernel size of 3×3 and a stride of 2, as well as two LeakyReLU activation layers to generate the data distribution [μ,σ,q], where μ represents the mean, σ represents the variance, and q represents the stride output.

[0069] S3-3, the implicit representation e of S3-1 and S3-2 and the data distribution [μ,σ,q] are input into the quantization module to obtain the noise robust feature quantization result; the specific working principle of this quantization module is shown in the following equation:

[0070]

[0071] in This represents the quantization result of the noise robust feature, and "·" indicates the integer approximation operation;

[0072] S3-4: The noise robust feature quantization result of S3-3 is input into the feature decoder. The structure of the feature decoder corresponds to that of the feature encoder. It alternately uses two deconvolution layers with a kernel size of 3×3 and a stride of 2, as well as an inverse GDN module. The feature decoder restores the noise robust feature quantization result to the size of the frequency domain fusion feature map in S2 to obtain the noise robust fusion feature map.

[0073] Step S4, as follows Figure 4 As shown, the details are as follows:

[0074] The noise-robust fusion feature map of the input S3 is progressively upsampled using four serially connected upsampling layers until it is gradually restored to the original image size. Each upsampling layer includes a deconvolution operation and a regularization operation. The noise-robust fusion feature map is 32×32, 64×64, 128×128, 256×256 and 512×512 in the four serially connected upsampling layers and the final output, respectively. The final feature map has the same size as the rain map in the augmented dataset of S1. This final feature map is the generated de-rain image.

[0075] Step S5 is as follows:

[0076] The difference between the generated derained image and the corresponding rainless image in the augmented dataset is calculated using the L1 loss shown in the following formula. This difference is then used as the global loss to guide the training of the model until the model converges:

[0077]

[0078] Where H and W represent the length and width of the image, respectively, x represents the input derained image, f(x) represents the rainless image output by the network, y represents the clean label corresponding to the input derained image, and i and j represent the subscripts of each pixel in the image.

[0079] Accordingly, the present invention also provides an image deraining system based on direction-aware frequency domain filtering, such as... Figure 5 As shown, it includes:

[0080] Data preprocessing unit 1 is used to perform data augmentation preprocessing on each pair of rain maps and corresponding no-rain maps in the synthetic rainwater dataset to obtain an augmented dataset;

[0081] Frequency domain fusion feature map extraction unit 2 is used to extract frequency domain fusion feature maps from the rain map in the augmented dataset by using a direction-aware frequency domain filter in conjunction with a ResNet50 network.

[0082] The noise-robust fusion feature map extraction unit 3 is used to input the frequency domain fusion feature map into a neural compression-based feature optimizer to obtain a noise-robust fusion feature map.

[0083] The upsampling image generation unit 4 is used to upsample the noise-robust fusion feature map until the noise-robust fusion feature map is gradually restored to the original image size, and the resulting final feature map is used as the generated deraining image.

[0084] Model training unit 5 is used to construct a loss function to calculate the difference between the generated derained image and the corresponding rainless image in the augmented dataset, and use the difference as a global loss to train and form a derained model;

[0085] Image deraining unit 6 is used for the user to input a rainy image to be processed into the deraining model to obtain a generated derained image.

[0086] Therefore, the rain removal method proposed in this invention combines the distribution characteristics of rainwater in images, namely that rain streaks and raindrops are usually distributed in the high-frequency part of the image. By distinguishing and processing different frequency parts of the image, it can effectively remove rainwater interference while preserving the image's structure and detail information. The invention introduces a feature optimization module based on neural compression, avoiding interference from noise that may exist in the features, making the optimized features more noise-robust. The invention effectively extracts and integrates features, improving efficiency and making the obtained rain removal results more consistent with real-life situations.

[0087] The above provides a detailed description of an image rain removal method and system based on direction-aware frequency domain filtering provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for image deraining based on directional perceptual frequency domain filtering, characterized in that, The method includes: For each pair of rain maps and corresponding no-rain maps in the synthetic rainwater dataset, data augmentation preprocessing is performed to obtain the augmented dataset; The frequency domain fusion feature map of the rain map in the augmented dataset is extracted by combining a direction-aware frequency domain filter with a ResNet50 network. The frequency domain fusion feature map is input into a neural compression-based feature optimizer to obtain a noise-robust fusion feature map. The noise-robust fusion feature map is upsampled until it is gradually restored to the original image size. The resulting final feature map is used as the generated derained image. A loss function is constructed to calculate the difference between the generated derained image and the corresponding rainless image in the augmented dataset. The difference is used as the global loss to train and form a derained model. The user inputs the rainy map to be processed into the rain removal model, and the generated rain removal map is obtained; Specifically, the extraction of frequency domain fusion feature maps from rain images in the augmented dataset using a direction-aware frequency domain filter combined with a ResNet50 network is as follows: The rain map in the augmented dataset is decomposed into high-frequency and low-frequency components using a direction-aware frequency domain filter. The high-frequency part is input into the high-frequency feature processing branch, and the low-frequency part is input into the low-frequency feature processing branch. The two branches use the Block 1 module in the ResNet50 network to extract and output the initial high-frequency features and the initial low-frequency features, respectively. The low-frequency feature processing branch first uses a direction-aware frequency domain filter to decompose the initial low-frequency feature into a high-frequency part LH1 and a low-frequency part LL1. Then, LL1 is used as the input for the next stage of the low-frequency branch, and LH1 is passed to the high-frequency feature processing branch. The operation of the high-frequency feature processing branch corresponds to that of the low-frequency feature processing branch. First, a direction-aware frequency domain filter is used to decompose the initial high-frequency feature into a high-frequency part HH1 and a low-frequency part HL1. Then, HH1 is used as the input for the next stage of the high-frequency branch, and HL1 is passed to the low-frequency feature processing branch, outputting LL1, LH1, HL1, and HH1. LL1 is concatenated with HL1 and LH1 with HH1 respectively to obtain the next stage inputs for the low-frequency processing branch and the high-frequency processing branch. This operation is repeated 3 times to obtain the outputs of the third stage as LL3, LH3, HL3, and HH3 respectively. LL3 is concatenated with HL3 and LH3 with HH3 respectively to obtain the final extracted low-frequency and high-frequency features of the low-frequency feature processing branch and the high-frequency processing branch. The frequency domain fusion feature map obtained by summing the low-frequency and high-frequency features is output as the result. 2.The image rain removal method based on direction-aware frequency domain filtering of claim 1, wherein, The process of performing data augmentation preprocessing on each pair of rain maps and corresponding no-rain maps in the synthetic rainwater dataset to obtain the augmented dataset is as follows: For each pair of rain-no-rain images in the synthetic rain dataset, the same preprocessing operations of random flipping, random rotation, and fixed-size cropping are performed sequentially to obtain a single image size of 512×512, and the augmented dataset is output.

3. The image deraining method based on direction-aware frequency domain filtering as described in claim 1, characterized in that, The step of inputting the frequency domain fused feature map into a neural compression-based feature optimizer to obtain a noise-robust fused feature map is as follows: The frequency domain fused feature map is input into a feature encoder, which alternately uses two convolutional layers with a kernel size of 3×3 and a stride of 2, and a GDN module that can reduce feature dependencies to generate an implicit feature representation e. The frequency domain fused feature map is input into a priori module, which alternately uses three convolutional layers with a kernel size of 3×3 and a stride of 2, and two LeakyReLU activation layers to generate the data distribution. , This represents the mean. Represents variance. Indicates the step size output; The implicit feature representation e and the data distribution The input is fed into the quantization module to obtain the quantization result of the noise robust feature; the specific working principle of this quantization module is shown in the following formula: ， in This represents the quantization result of the noise robustness feature. Represents integer approximation operations; The noise robust feature quantization result is input into the feature decoder, which has a structure corresponding to the feature encoder. It alternately uses two deconvolutional layers with a kernel size of 3×3 and a stride of 2, as well as an inverse GDN module. The feature decoder restores the noise robust feature quantization result to the size of the frequency domain fusion feature map to obtain the noise robust fusion feature map.

4. The image deraining method based on direction-aware frequency domain filtering as described in claim 1, characterized in that, The noise-robust fused feature map is upsampled until it is gradually restored to the original image size. The resulting final feature map is used as the generated derained image. Specifically: The noise-robust fusion feature map is progressively upsampled using four sequentially connected upsampling layers until it is gradually restored to the original image size. Each upsampling layer includes a deconvolution operation and a regularization operation. The sizes of the noise-robust fusion feature map in the four sequentially connected upsampling layers and the final output are 32×32, 64×64, 128×128, 256×256 and 512×512, respectively. The final feature map has the same size as the rain map in the augmented dataset, and this final feature map is the generated de-rain image.

5. The image deraining method based on direction-aware frequency domain filtering as described in claim 1, characterized in that, The loss function is constructed to calculate the difference between the generated derained image and the corresponding rainless image in the augmented dataset. This difference is used as the global loss to train and form a deraining model. Specifically: The difference between the generated derained image and the corresponding rainless image in the augmented dataset is calculated using the L1 loss shown in the following formula. This difference is then used as the global loss to guide the training of the model until the model converges: ， in and These represent the length and width of the image, respectively. This indicates that the input image is a derained image. This represents the rainless image output by the network. This represents the clean label corresponding to the input derained image. , This indicates the index of each pixel in the image.

6. An image deraining system based on direction-aware frequency domain filtering, characterized in that, The system includes: The data preprocessing unit is used to perform data augmentation preprocessing on each pair of rain maps and corresponding no-rain maps in the synthetic rainwater dataset to obtain the augmented dataset. The frequency domain fusion feature map extraction unit is used to extract the frequency domain fusion feature map of the rain map in the augmented dataset by using a direction-aware frequency domain filter in combination with the ResNet50 network. The noise-robust fusion feature map extraction unit is used to input the frequency domain fusion feature map into a neural compression-based feature optimizer to obtain a noise-robust fusion feature map. The upsampling image generation unit is used to upsample the noise-robust fusion feature map until the noise-robust fusion feature map is gradually restored to the original image size, and the resulting final feature map is used as the generated deraining image. The model training unit is used to construct a loss function to calculate the difference between the generated derained image and the corresponding rainless image in the augmented dataset, and use the difference as a global loss to train and form a derained model. The image deraining unit is used to input a rainy image to be processed into the deraining model by the user, and to obtain a generated derained image; Specifically, in the frequency domain fusion feature map extraction unit, a direction-aware frequency domain filter combined with a ResNet50 network is used to extract the frequency domain fusion feature map from the rain map in the augmented dataset, specifically as follows: The rain map in the augmented dataset is decomposed into high-frequency and low-frequency components using a direction-aware frequency domain filter. The high-frequency part is input into the high-frequency feature processing branch, and the low-frequency part is input into the low-frequency feature processing branch. The two branches use the Block 1 module in the ResNet50 network to extract and output the initial high-frequency features and the initial low-frequency features, respectively. The low-frequency feature processing branch first uses a direction-aware frequency domain filter to decompose the initial low-frequency feature into a high-frequency part LH1 and a low-frequency part LL1. Then, LL1 is used as the input for the next stage of the low-frequency branch, and LH1 is passed to the high-frequency feature processing branch. The operation of the high-frequency feature processing branch corresponds to that of the low-frequency feature processing branch. First, a direction-aware frequency domain filter is used to decompose the initial high-frequency feature into a high-frequency part HH1 and a low-frequency part HL1. Then, HH1 is used as the input for the next stage of the high-frequency branch, and HL1 is passed to the low-frequency feature processing branch, outputting LL1, LH1, HL1, and HH1. LL1 is concatenated with HL1 and LH1 with HH1 respectively to obtain the next stage inputs for the low-frequency processing branch and the high-frequency processing branch. This operation is repeated 3 times to obtain the outputs of the third stage as LL3, LH3, HL3, and HH3 respectively. LL3 is concatenated with HL3 and LH3 with HH3 respectively to obtain the final extracted low-frequency and high-frequency features of the low-frequency feature processing branch and the high-frequency processing branch. The frequency domain fusion feature map obtained by summing the low-frequency and high-frequency features is output as the result.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images