Rain and fog removing method for image in water area scene

Abstract

The invention provides a rain and fog removing method for an image in a water area scene. The method comprises the following steps: collecting clear images in various water area scenes; constructing a rain and fog scattering model, outputting a rain and fog image, inputting the clear image into the rain and fog scattering model, and constructing a clear-rain and fog image data set; constructing an image progressive rain and fog removal network, randomly dividing the image data set into a training set and a test set, training and testing the image progressive rain and fog removal network by using the training set and the test set, and introducing a loss function for evaluation to obtain a trained image progressive rain and fog removal network; and inputting the rain and fog image of the to-be-detected water area into the trained image progressive rain and fog removal network to obtain a clear rain and fog recovery image of the to-be-detected water area. According to the method, structural similarity loss and frequency domain loss are introduced into the loss function, water surface texture detail features are effectively reserved, the color gamut correction module is added into the image progressive rain and fog removal network, accurate restoration of the original color of the water surface is achieved, and the rain and fog removal quality of the image is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and particularly relates to a method for removing rain and fog from images in a water area scene. Background Art

[0002] Traditional methods for removing rain and fog from images mainly include the dark channel prior method, frequency domain filtering method, and partial differential equation method. However, although the dark channel prior method based on physical prior can handle uniform light rain or light fog scenes, it is prone to misjudgment of transmittance when dealing with water surfaces or similar bright areas with strong reflection characteristics, and this method has a high computational complexity and slow processing speed; the frequency domain filtering method is particularly sensitive to the non-uniform characteristics of the spatial distribution of rain and fog. Especially in complex weather conditions such as thick fog or heavy rain, the frequency domain aliasing effect will cause artifacts in the processed water area monitoring images; while the partial differential equation method over-smooths high-frequency components during gradient domain processing, resulting in a large loss of water surface texture.

[0003] Chinese Patent CN118887109A discloses a method for removing rain and fog from images. By obtaining the image to be de-rained and de-fogged, and inputting the image to be de-rained and de-fogged into a preset network model for eliminating the influence of rain and fog, a target rain and fog-free image is obtained, which further improves the applicability and accuracy of UAV power inspection under various weather conditions. However, the model structure is relatively complex; Chinese Patent CN118587125A discloses a method for restoring rain and fog scene images based on staged learning. By performing image restoration in stages, context features are learned using an encoder-decoder structure in each stage, and combined with a resolution network branch to retain local information, and then local adjustment is re-weighted through an instant supervision attention mechanism to achieve staged restoration of rain and fog scene images. However, the images restored by the deep learning-based algorithm are prone to color distortion. Summary of the Invention

[0004] To solve the problems of loss of image detail information and color distortion of the water surface during the restoration of water area images in harsh environments, the present invention provides a method for removing rain and fog from images in a water area scene.

[0005] Based on the above purpose, the technical solution adopted by the present invention is as follows:

[0006] A method for removing rain and fog from images in a water area scene, comprising the following steps:

[0007] S1. Collect clear images without rain and fog interference under different lighting conditions and shooting angles in various water area scenes;

[0008] S2. Construct a rain and fog scattering model, the output of which is a rain and fog image. Input the clear image into the rain and fog scattering model, and generate a rain image, a fog image, and a rain and fog image according to a certain ratio to construct a clear-rain and fog image dataset;

[0009] The rain and fog scattering model includes an image visibility module, a rain layer intensity module, and a fog layer intensity module. The image visibility module is used to reflect the visibility change of the water area scene. The rain layer intensity module is used to randomly generate rain layer maps with different rainfall intensities. The fog layer intensity module is used to randomly generate fog layer maps with different fog concentrations.

[0010] S3. Construct an image progressive rain and fog removal network, whose output is a clear rain and fog restored image. Randomly divide the clear-rain and fog image dataset into a training set and a test set according to a ratio, and use the training set to train the image progressive rain and fog removal network to obtain a trained image progressive rain and fog removal network.

[0011] The image progressive rain and fog removal network includes preliminary feature extraction processing, feature fusion processing, and detail enhancement processing. The preliminary feature extraction processing is used to extract multi-scale features and context information of the rain and fog image, including low-level detail information and high-level semantic information. The feature fusion processing is used to fuse feature information at different levels to enhance the feature representation. The detail enhancement processing is used to restore the image detail information and the original color.

[0012] S4. Input the test set into the trained image progressive rain and fog removal network to generate test results, and introduce a loss function to evaluate the test results. If it meets the evaluation criteria, obtain a trained image progressive rain and fog removal network and enter S5. If it does not meet the evaluation criteria, return to S3.

[0013] S5. Input the rain and fog image of the water area to be measured into the trained image progressive rain and fog removal network to obtain a clear rain and fog restored image of the water area to be measured.

[0014] Preferably, step S2 further includes: the ratio of the rain map, the fog map, and the rain and fog map is 4:4:2.

[0015] The expression of the rain and fog scattering model is:

[0016] ;

[0017] In the formula, x represents the image pixel, Q(x) represents the rain and fog image, V(x) represents the image visibility module, R(x) represents the rain layer intensity module, A(x) represents the fog layer intensity module, and A0 represents the atmospheric light intensity.

[0018] The expression of the image visibility module V(x) is:

[0019] ;

[0020] In the formula, I(x) represents the clear image.

[0021] The expression of the rain layer intensity module R(x) is as follows:

[0022] ;

[0023] In the formula, R p (x) represents the spatial distribution intensity of rain streaks, and T r (x) represents the intensity of rain streaks;

[0024] The expression of the fog layer intensity module A(x) is as follows:

[0025] ;

[0026] In the formula, σ represents the fog layer intensity attenuation coefficient, and d(x) represents the scene depth.

[0027] Preferably, the calculation formula of the rain streak intensity T r (x) is as follows:

[0028] ;

[0029] In the formula, μ represents the rain layer intensity attenuation coefficient, and d1 represents the maximum scene depth.

[0030] Preferably, step S3 further includes: randomly dividing the training set and the test set according to a ratio of 8:2;

[0031] The expression of the image progressive rain and fog removal network is as follows:

[0032] ;

[0033] In the formula, X out represents the clear rain and fog restored image, F IFE represents the preliminary feature extraction process, F FF represents the feature fusion process, F DE represents the detail enhancement process, and X in represents the original rain and fog image.

[0034] Preferably, the preliminary feature extraction process F IFE includes a channel attention module, a feature extraction network, and a feature filtering module, and its processing process is as follows:

[0035] ;

[0036] In the formula, X1 represents the preliminary feature extraction reconstructed image, Conv represents the convolution process, F AB represents the channel attention module, F UNet represents the feature extraction network, F FFT represents the feature filtering module;

[0037] The feature fusion process F FF The processing procedure is as follows:

[0038] ;

[0039] Wherein, X2 represents the reconstructed image of the feature fusion process, and concat represents the concatenation in the channel dimension;

[0040] The detail enhancement process F DE includes a detail retention module and a color gamut correction module, and its processing procedure is as follows:

[0041] ;

[0042] Wherein, F RSNet represents the detail retention module, and F CR represents the color gamut correction module.

[0043] Preferably, the processing procedure of the channel attention module F AB is as follows:

[0044] ;

[0045] Wherein, O AB represents the shallow feature extraction image, ReLU represents the activation function, and F CA represents the processing of the channel attention layer;

[0046] The processing procedure of the feature extraction network F UNet is as follows:

[0047] ;

[0048] Wherein, O UNet represents the feature representation image, F encode represents the feature encoder, F decode represents the feature decoder, and F SC represents the skip connection;

[0049] The processing procedure of the feature filtering module F FFT is as follows:

[0050] ;

[0051] Wherein, O FFT represents the attention map, sigmoid represents the activation function, and C s represents the restored image of the preliminary feature extraction;

[0052] The detail retention module F RSNet includes n of the channel attention modules, and its processing procedure is as follows:

[0053] ;

[0054] Wherein, C RSNet and O RSNet respectively represent the input image and the output image of the detail retention module, F AB1 , F AB2 …F ABn respectively represent the 1st to the nth channel attention modules;

[0055] The processing process of the color gamut correction module F CR is as follows:

[0056] ;

[0057] Wherein, and respectively represent the image transformation pixel and the original image pixel, and F represents the mapping diagonal matrix, and its expression is as follows:

[0058] ;

[0059] Wherein, α, β, and γ respectively represent the weight coefficients of the R, G, and B color channels.

[0060] Preferably, the expression of the preliminary feature extraction and restoration image C s is:

[0061] ;

[0062] Wherein, O UNet represents the feature representation image, and X in represents the original rainy and foggy image;

[0063] The expression of the input image C RSNet of the detail retention module is:

[0064] ;

[0065] Wherein, X1 represents the preliminary feature extraction and reconstruction image, and X2 represents the feature fusion processing and reconstruction image.

[0066] Preferably, step S4 further includes: The calculation formula of the loss function is:

[0067] ;

[0068] Wherein, λ and δ represent weight coefficients, and L Total represents the loss function, and L Derain represents the de-raining loss function, and L DehazeDenote the de-raining and de-hazing loss function;

[0069] The evaluation criteria include the loss function L Total If ≤ 0.1, a trained image progressive de-raining and de-hazing network is obtained, and proceed to S5. If the evaluation criteria are not met, return to S3.

[0070] Preferably, the de-raining loss function L Derain has the following expression:

[0071] ;

[0072] In the formula, represents the predicted value of the k-th pixel by the image progressive de-raining and de-hazing network, and y k represents the true value of the k-th pixel, K represents the total number of pixels, and I pred represents the clear rain and fog restored image, and I gt represents the clear image;

[0073] The de-hazing loss function L Dehaze has the following expression:

[0074] ;

[0075] In the formula, FT represents the fast Fourier transform, represents the L1 norm;

[0076] The sum of the weight coefficients λ and δ is 1. The initial values of the weight coefficients λ and δ are set to 0.7 and 0.3 respectively. If the evaluation criteria are not met, update the weight coefficients λ and δ according to the loss function L Total and return to step S3 for retraining.

[0077] The advantages of the present invention compared with existing methods are as follows:

[0078] (1) The present invention fuses the image visibility module, the rain layer intensity module, and the fog layer intensity module to construct a unified rain and fog scattering model. Inputting a clear water image into the rain and fog scattering model can generate corresponding rain maps, fog maps, and rain and fog maps, overcoming the limitation that a single model in traditional methods can only simulate one weather type, and the model has a wider adaptability;

[0079] (2) For the problem that the image is too smooth and there is detail loss after processing by existing image de-raining and de-hazing algorithms, the present invention realizes the enhancement of water area edge features by introducing structural similarity loss, retains the water surface texture features by using frequency domain loss, and strengthens the water flow pattern features, ensuring the accuracy of subsequent water area segmentation and water surface flow velocity measurement;

[0080] (3) Since the rain and fog removal algorithm based on deep learning is too sensitive to high-frequency detail information and insufficient in restoring low-frequency information, the restored image after processing has a certain degree of color distortion and dullness. In the present invention, a color gamut correction module is incorporated into the image progressive rain and fog removal network, and a mapping relationship between the image transformation pixels and the original pixels is established in the form of convolutional residuals to calibrate the brightness and color of the image after rain and fog removal, effectively solving the problem of color distortion in water area images and achieving accurate restoration of the original color of the water surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0082] Figure 1 It is a flowchart of an image rain and fog removal method in a water area scene according to an embodiment of the present invention;

[0083] Figure 2 It is a schematic diagram of the architecture of an image progressive rain and fog removal network according to an embodiment of the present invention;

[0084] Figure 3 It is a schematic diagram of the architecture of a feature filtering module according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0085] To make the objectives, technical solutions, and advantages of the present invention more obvious and understandable, the following will clearly and completely describe the technical solutions in the present invention in conjunction with the drawings and specific embodiments. It should be noted that the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0086] Please refer to Figure 1 As shown, an image rain and fog removal method in a water area scene provided by the present invention includes the following steps:

[0087] S1. Collect clear images without rain and fog interference under different lighting conditions and shooting angles in a variety of water area scenes.

[0088] Step S1 in the embodiment of the present invention specifically includes:

[0089] Using a camera or camera acquisition device, collect water surface images under different lighting conditions and different shooting angles in a variety of water area scenes, and the water surface images have no rain and fog interference and the water surface texture is clearly visible.

[0090] S2. Construct a rain and fog scattering model, the output of which is a rain and fog image. Input the clear image into the rain and fog scattering model, and generate a rain image, a fog image, and a rain and fog image according to a certain ratio to construct a clear-rain and fog image dataset.

[0091] The step S2 in the embodiment of the present invention specifically includes:

[0092] To simulate rain and fog images under real weather conditions, a rain and fog scattering model is constructed. The rain and fog scattering model includes an image visibility module, a rain layer intensity module, and a fog layer intensity module. The expression of the rain and fog scattering model is:

[0093] ;

[0094] In the formula, x represents an image pixel, Q(x) represents the rain and fog image, V(x) represents the image visibility module, R(x) represents the rain layer intensity module, A(x) represents the fog layer intensity module, A0 represents the atmospheric light intensity, representing the scattered light intensity of the atmosphere in the water area scene, and its value is a constant.

[0095] The image visibility module V(x) is used to reflect the visibility change of the water area scene, and its expression is:

[0096] ;

[0097] In the formula, I(x) represents the clear image, reflecting the image brightness distribution of the water area scene without any interference. The image visibility module V(x) describes that as the rain layer intensity module R(x) and the fog layer intensity module A(x) increase, the superposition effect of rain and fog gradually obscures the water area scene, and the field of view visibility gradually decreases.

[0098] The value range of the rain layer intensity module R(x) is in [0,1], and it is used to randomly generate rain layer images with different rainfall intensities. Its expression is:

[0099] ;

[0100] In the formula, R p (x) represents the rain pattern spatial distribution intensity, reflecting the relative intensity of the rain pattern at different pixel positions. Its value is [0,1], 0 represents no rain pattern at this pixel, and 1 represents the maximum intensity of the rain pattern at this pixel. T r (x) represents the rain pattern intensity, and its calculation formula is as follows:

[0101] ;

[0102] Wherein, μ represents the attenuation coefficient of the rain layer intensity, and its value is uniformly distributed within the range of [0, 0.1]. d(x) represents the scene depth, which represents the actual physical distance from the rain pattern to the acquisition device. d1 represents the maximum scene depth. When the scene depth d(x) exceeds the maximum scene depth d1, the intensity T r (x) gradually decreases to 0.

[0103] The fog layer intensity module A(x) is generated based on the standard atmospheric scattering optical model and is used to randomly generate fog layer maps with different fog concentrations. Its expression is:

[0104] ;

[0105] Wherein, σ represents the attenuation coefficient of the fog layer intensity, and its value is uniformly distributed within the range of [0, 0.02], representing the degree of scattering and absorption of light by the atmosphere. A larger σ value means that the fog in the fog layer is denser, thereby enhancing the shielding effect of the fog on the water area scene.

[0106] Specifically, input the clear image into the rain and fog scattering model. Using the rain layer intensity model R(x) and the fog layer intensity model A(x), randomly generate rain layer maps with different rainfall intensities and fog layer maps with different fog concentrations respectively. Use the rain and fog scattering model to combine the clear image, the rain pattern image and the fog layer image, and output the corresponding rain and fog image. To avoid the singularity of the rain and fog image, the rain and fog image includes a rain image, a fog image and a rain and fog image, and the ratio of the rain image, the fog image and the rain and fog image is 4:4:2. Construct a clear-rain and fog image dataset from the clear image and its corresponding rain and fog image.

[0107] S3. Construct an image progressive rain and fog removal network, and its output is a clear rain and fog restored image. Randomly divide the clear-rain and fog image dataset into a training set and a test set according to a ratio, and use the training set to train the image progressive rain and fog removal network to obtain a trained image progressive rain and fog removal network.

[0108] Step S3 in the embodiment of the present invention specifically includes:

[0109] Please refer to Figure 2 As shown, construct an image progressive rain and fog removal network. The image progressive rain and fog removal network includes preliminary feature extraction processing, feature fusion processing and detail enhancement processing. The expression of the image progressive rain and fog removal network is:

[0110] ;

[0111] Wherein, X out represents the clear rain and fog restored image, F IFEDenote the preliminary feature extraction process as F FF Denote the feature fusion process as F DE Denote the detail enhancement process as X in Denote the original rainy and foggy image

[0112] The image progressive deraining and defogging network can effectively balance the spatial details and high-level context information in the image restoration process. By gradually extracting the high-level context information and preserving the spatial details, it ensures that the output of each stage can improve the overall image restoration quality

[0113] The preliminary feature extraction process F IFE Includes a channel attention module, a feature extraction network, and a feature filtering module. The specific processing process is as follows: Use the channel attention module to extract the shallow features of the original rainy and foggy image X in To obtain a shallow feature extraction image, transfer the shallow feature extraction image to the feature extraction network, use the encoder-decoder structure in the feature extraction network to capture the multi-scale context information of the shallow feature extraction image, generate a more semantically meaningful feature representation image, and use the feature filtering module to perform per-pixel weighting on the feature representation image to optimize the output of the preliminary feature extraction reconstruction image. The processing process of the preliminary feature extraction process F IFE The processing process is as follows

[0114] ;

[0115] In the formula, X1 represents the preliminary feature extraction reconstruction image, Conv represents the convolution process, F AB Represents the channel attention module, F UNet Represents the feature extraction network, F FFT Represents the feature filtering module

[0116] The channel attention module F AB Includes convolution operations and channel attention layer processing. The processing process is as follows

[0117] ;

[0118] In the formula, O AB Represents the shallow feature extraction image, ReLU represents the activation function, F CA Represents the channel attention layer processing, which is used to enhance the feature expression in the image restoration process

[0119] The feature extraction network F UNetIt includes an encoding-decoding network that extracts multi-scale features through a feature encoder, then gradually restores spatial information using a feature decoder, and combines skip connections to retain the spatial details, achieving high-resolution and detail-rich image restoration. The processing process is as follows:

[0120] ;

[0121] In the formula, O UNet represents the feature representation image, F encode represents the feature encoder, which is used for multi-scale feature extraction and gradually downsamples for context information capture, F decode represents the feature decoder, which is used to gradually upsample the output result of the feature encoder and combine the skip connections to generate a high-resolution restored image, F SC represents the skip connection, which is used to transfer detail information between the feature encoder and the decoder to help restore the spatial resolution.

[0122] Please refer to Figure 3 as shown. The feature filtering module F FFT is placed between two adjacent processing stages of the image progressive rain and fog removal network, and is used to perform feature enhancement representation on the output features of the previous stage, significantly improving the performance. Specifically, it includes providing stage-by-stage real supervision information to gradually optimize the image restoration effect; generating an attention map through local supervision prediction to filter out features with low information content in the current stage, and only allowing useful features to be transferred to the next stage. The processing process is as follows:

[0123] ;

[0124] In the formula, O FFT represents the attention map, sigmoid represents the activation function, C s represents the preliminary feature extraction and restored image, which is used to provide a supervision signal, and its expression is:

[0125] ;

[0126] In the formula, O UNet represents the feature representation image, X in represents the original rainy and foggy image.

[0127] The feature fusion process F FF includes fusing the shallow features of the original rainy and foggy image X in and the preliminary feature extraction and reconstruction image X1 to form a more abundant feature image. Specifically, the original rainy and foggy image X inExtract the shallow features again through the channel attention module, and cascade it with the preliminary feature extraction and reconstruction image X1 and then transfer it to the feature extraction module. The cascading method enables the feature fusion stage to simultaneously utilize the detail information of the original rain and fog image X in and the initial feature extraction F IFE to output context features, and then through the feature filtering module F FFT supervise again on the preliminary feature extraction and reconstruction image X1 to guide the adaptive weighting of features and generate a feature fusion processing and reconstruction image. The processing process of the feature fusion processing F FF is as follows:

[0128] ;

[0129] In the formula, X2 represents the feature fusion processing and reconstruction image, and concat represents concatenation in the channel dimension.

[0130] The detail enhancement processing F DE includes a detail retention module and a color gamut correction module. Specifically, combine the shallow features of the original rain and fog image X in with the feature fusion processing and reconstruction image X2 to form new input features, and transfer the new input features to the detail retention module. The detail retention module processes directly at the resolution of the original rain and fog image X in to retain the image detail information and ensure that the output has fine texture and accurate spatial details. Add the output of the detail retention module to the original rain and fog image X in to enrich the context information and retain the image details, and perform image brightness and color calibration through the color gamut correction module to output the clear rain and fog recovery image X out , and the processing process of the detail enhancement processing F DE is as follows:

[0131] ;

[0132] In the formula, F RSNet represents the detail retention module, and F CR represents the color gamut correction module.

[0133] The detail retention module F RSNet includes n channel attention modules to further enhance the processing effect, and its processing process is as follows:

[0134] ;

[0135] In the formula, C RSNet and O RSNetrespectively represent the input and output images of the detail retention module, F AB1 , F AB2 …F ABn respectively represent the first to the nth channel attention modules, and the C RSNet The expression of is:

[0136] ;

[0137] In the formula, X1 represents the preliminary feature extraction and reconstruction image, and X2 represents the feature fusion processing and reconstruction image.

[0138] The color gamut correction module F CR is used to perform brightness and color calibration on the color distortion problem of the restored image, and its processing process is:

[0139] ;

[0140] In the formula, and respectively represent the image transformation pixel and the original image pixel, F represents the mapping diagonal matrix, which is a generalized 3×3 matrix and the diagonal elements are greater than 0. The expression of the mapping diagonal matrix F is as follows:

[0141] ;

[0142] In the formula, α, β, and γ respectively represent the weight coefficients of the R, G, and B color channels. When obtaining a brighter image, α, β, and γ are all greater than 1. The processing process of the color gamut correction module can be rewritten as:

[0143] ;

[0144] The color gamut correction module F CR can be implemented by a residual block. Since the convolutional kernel is initialized between 0 and 1, reducing the diagonal elements makes the learning process converge faster, and the residual block can enhance the accuracy of the color gamut correction module.

[0145] Randomly divide the clear-rain-fog image dataset into a training set and a test set according to a ratio of 8:2, and use the training set to train the image progressive de-raining and de-fogging network to obtain a trained image progressive de-raining and de-fogging network.

[0146] S4. Input the test set into the trained image progressive de-raining and de-fogging network to generate test results, introduce a loss function to evaluate the test results. If it meets the evaluation criteria, obtain a trained image progressive de-raining and de-fogging network and enter S5. If it does not meet the evaluation criteria, return to S3.

[0147] Step S4 in the embodiments of the present invention specifically includes:

[0148] Inputting the test set into the trained image progressive rain and fog removal network to generate a test result, where the test result is the clear rain and fog restored image corresponding to the test set, and introducing a loss function to evaluate the clear rain and fog restored image. The calculation formula of the loss function is:

[0149] ;

[0150] In the formula, λ and δ represent weight coefficients, and their sum is 1. L Total represents the loss function, and L Derain represents the rain removal loss function, and L Dehaze represents the fog removal loss function.

[0151] The rain removal loss function L Derain adopts the mean absolute error L1 loss to measure the per-pixel reconstruction accuracy, and uses the SSIM loss to calculate the structural similarity. Its expression is:

[0152] ;

[0153] In the formula, represents the predicted value of the k-th pixel by the image progressive rain and fog removal network, y k represents the true value of the k-th pixel, K represents the total number of pixels, I pred represents the clear rain and fog restored image, and I gt represents the clear image.

[0154] The fog removal loss function L Dehaze adopts the frequency domain reconstruction loss function. Its expression is:

[0155] ;

[0156] In the formula, FT represents the fast Fourier transform, represents the L1 norm.

[0157] The evaluation criteria include that the loss function L Total ≤0.1. The initial values of the weight coefficients λ and δ are respectively set to 0.7 and 0.3. If the evaluation criteria are met, a trained image progressive rain and fog removal network is obtained, and it enters S5. If the evaluation criteria are not met, the weight coefficients λ and δ are updated according to the loss function L Total and return to step S3 for retraining.

[0158] S5. Input the rain and fog image of the water area to be measured into the trained image progressive de-raining and de-fogging network to obtain a clear rain and fog restored image of the water area to be measured.

[0159] Step S5 in the embodiment of the present invention specifically includes: collecting the rain and fog image of the water area to be measured under real weather conditions, inputting the rain and fog image of the water area to be measured into the trained image progressive de-raining and de-fogging network, obtaining a clear rain and fog restored image of the water area to be measured, and effectively removing the rain and fog interference in the rain and fog image of the water area to be measured.

[0160] In the embodiment of the present invention, clear images in different water area scenes are collected to construct a rain and fog scattering model. The clear images are input into the rain and fog scattering model, and corresponding rain and fog images are output. A clear-rain and fog data set is constructed by using the clear images and the rain and fog images. The clear-rain and fog data set is randomly divided into a training set and a test set. An image progressive de-raining and de-fogging network is constructed. The training set and the test set are used to train and test the image progressive de-raining and de-fogging network. A trained image progressive de-raining and de-fogging network is obtained by introducing a loss function. The rain and fog image of the water area to be measured is input into the trained image progressive de-raining and de-fogging network to obtain a clear rain and fog restored image of the water area to be measured. The method of the present invention realizes the real simulation of complex weather conditions by using the constructed rain and fog scattering model, integrates the image de-raining and image de-fogging tasks, effectively retains the water area edge information and water surface texture details by introducing the structural similarity loss and the frequency domain loss, and accurately restores the original color of the water surface by combining the color gamut correction module.

[0161] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the scope of the present invention.

Claims

1. An image de-raining and de-fogging method in a water area scene, characterized in that, It includes the following steps: S1. Collect clear images under different lighting conditions, shooting angles in various water area scenes without rain and fog interference; S2. Construct a rain and fog scattering model, the output of which is a rain and fog image. Input the clear image into the rain and fog scattering model, generate a rain image, a fog image and a rain and fog image according to a certain ratio, and construct a clear-rain and fog image dataset; The rain and fog scattering model includes an image visibility module, a rain layer intensity module and a fog layer intensity module. The image visibility module is used to reflect the visibility change of the water area scene. The rain layer intensity module is used to randomly generate rain layer images with different rainfall intensities. The fog layer intensity module is used to randomly generate fog layer images with different fog concentrations; S3. Construct an image progressive rain and fog removal network, the output of which is a clear rain and fog restored image. Randomly divide the clear-rain and fog image dataset into a training set and a test set according to a ratio. Use the training set to train the image progressive rain and fog removal network to obtain a trained image progressive rain and fog removal network; The image progressive rain and fog removal network includes preliminary feature extraction processing, feature fusion processing and detail enhancement processing. The preliminary feature extraction processing is used to extract multi-scale features and context information of the rain and fog image, including low-level detail information and high-level semantic information. The feature fusion processing is used to fuse feature information at different levels to enhance feature representation. The detail enhancement processing is used to restore image detail information and original color; S4. Input the test set into the trained image progressive rain and fog removal network to generate a test result. Introduce a loss function to evaluate the test result. If it meets the evaluation criteria, obtain a trained image progressive rain and fog removal network and enter S5. If it does not meet the evaluation criteria, return to S3; S5. Input the rain and fog image of the water area to be measured into the trained image progressive rain and fog removal network to obtain a clear rain and fog restored image of the water area to be measured.

2. The method for removing rain and fog from images in a water area scene according to claim 1, characterized in that, Step S2 further includes: the ratio of the rain image, the fog image and the rain and fog image is 4:4:2; The expression of the rain and fog scattering model is: ； In the formula, x represents an image pixel, Q(x) represents the rain and fog image, V(x) represents the image visibility module, R(x) represents the rain layer intensity module, A(x) represents the fog layer intensity module, and A0 represents the atmospheric light intensity; The expression of the image visibility module V(x) is: ； In the formula, I(x) represents the clear image; The expression of the rain layer intensity module R(x) is: ； where R p (x) represents the spatial distribution intensity of rain streaks, and T r (x) represents the intensity of rain streaks; The expression of the fog layer intensity module A(x) is: ； In the formula, σ represents the fog layer intensity attenuation coefficient, and d(x) represents the scene depth.

3. A method for removing rain and fog from images in a water area scene according to claim 2, characterized in that, The rain pattern intensity T r (x) is calculated by the following formula: ； In the formula, μ represents the rain layer intensity attenuation coefficient, and d1 represents the maximum scene depth.

4. A method for removing rain and fog from images in a water area scene according to claim 1, characterized in that, Step S3 further includes: the training set and the test set are randomly divided according to a ratio of 8:2; The expression of the image progressive rain and fog removal network is: ； Wherein, X out represents the clear rain and fog restored image, F IFE represents the preliminary feature extraction process, F FF represents the feature fusion process, F DE represents the detail enhancement process, X in represents the original rain and fog image.

5. A method for removing rain and fog from images in a water area scene according to claim 4, characterized in that, The preliminary feature extraction process F IFE includes a channel attention module, a feature extraction network, and a feature filtering module, and its processing process is as follows: ； Wherein, X1 represents the preliminary feature extraction and reconstruction image, Conv represents the convolutional process, F AB represents the channel attention module, F UNet represents the feature extraction network, F FFT represents the feature filtering module; The feature fusion process F FF The processing procedure is as follows: ； In the formula, X2 represents the feature fusion processing reconstructed image, and concat represents the channel dimension splicing; The detailed enhancement process F DE includes a detail retention module and a color gamut correction module, and its processing process is as follows: ； In the formula, F RSNet represents the detail retention module, and F CR represents the color gamut correction module.

6. The method for removing rain and fog from an image in a water area scene according to claim 5, wherein, The channel attention module F AB has the following processing procedure: ； Where, O AB represents the shallow feature extraction image, ReLU represents the activation function, F CA represents the processing of the channel attention layer; The feature extraction network F UNet The processing procedure is as follows: ； where, O UNet represents a feature representation image, F encode represents a feature encoder, F decode represents a feature decoder, F SC represents a skip connection; The feature filtering module F FFT has the following processing procedure: ； where, O FFT represents the attention map, sigmoid represents the activation function, and C s represents the preliminary feature extraction restored image; The detail retention module F RSNet includes n of the channel attention modules, and its processing process is as follows: ； Wherein, C RSNet and O RSNet respectively represent the input image and the output image of the detail retention module, F AB1 , F AB2 …F ABn respectively represent the 1st to the nth channel attention modules; The gamut correction module F CR The processing procedure is as follows: ； In the formula, and respectively represent the image-transformed pixels and the original image pixels. F represents the mapping diagonal matrix, and its expression is as follows: ； In the formula, α, β, and γ respectively represent the weight coefficients of the R, G, and B three color channels.

7. A method for removing rain and fog from images in a water area scene according to claim 6, characterized in that, The preliminary feature extraction and restored image C s is expressed as: ； where, O UNet represents the feature representation image, and X in represents the original rain and fog image; The input image C of the detailed retention module RSNet has the following expression: ； Wherein, X1 represents the preliminary feature extraction and reconstruction image, and X2 represents the feature fusion processing and reconstruction image.

8. A method for removing rain and fog from images in a water area scene according to claim 1, characterized in that, Step S4 further includes: The calculation formula of the loss function is: ； where λ and δ represent weight coefficients, and L Total represents the loss function, and L Derain represents the rain removal loss function, and L Dehaze represents the haze removal loss function; The evaluation criteria include the loss function L Total If ≤ 0.1, a trained image progressive de-raining and de-hazing network is obtained, and step S5 is entered; otherwise, if the evaluation criteria are not met, return to step S3.

9. A method for removing rain and fog from images in a water area scene according to claim 8, characterized in that, The rain removal loss function L Derain has the following expression: ； In the formula, represents the predicted value of the k-th pixel by the image progressive rain and fog removal network, y k represents the true value of the k-th pixel, K represents the total number of pixels, I pred represents the clear rain and fog restored image, I gt represents the clear image; The defogging loss function L Dehaze has the following expression: ； Wherein, FT represents the fast Fourier transform, represents the L1 norm; The sum of the weight coefficients λ and δ is 1. The initial values of the weight coefficients λ and δ are set to 0.7 and 0.3 respectively. If the evaluation criteria are not met, then according to the loss function L Total update the weight coefficients λ and δ, and return to step S3 for retraining.

Images