Method and device for removing rain and fog from image, equipment and medium

Abstract

The invention relates to an image rain and fog removing method, device and equipment and a medium, and belongs to the field of computer vision, and the method comprises the steps: carrying out the filtering processing of a preset conduction function through guiding filtering; performing defogging processing on the input original image according to the filtered preset conduction function to obtain a defogged image; inputting the defogged image into an attention circulation network for processing to obtain an attention map; and carrying out rain removal processing on the defogged image through a context automatic encoder in combination with the attention map to obtain a target image. According to the scheme, rain fog and raindrops in the image shot in the cloudy and rainy weather are removed, the obtained clear image is transmitted to other downstream visual tasks, the calculation efficiency of an existing defogging and rain removing scheme is improved, resources occupied by the model are reduced, and the rain removing performance of the model is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method, device, equipment and medium for removing rain and fog from an image. Background Art

[0002] The rapid development of intelligent driving technology is inseparable from the application of computer vision technology. Currently, computer vision is widely used in target detection and tracking, lane line detection, traffic sign recognition, etc. in road traffic.

[0003] The existing end-to-end progressive recurrent network PreNet rain removal or defogging algorithm model only removes single rain lines and raindrops, and does not consider the impact of rain and fog generated on rainy days; and the overall end-to-end model has low computational efficiency and occupies a large amount of computing resources, which is not suitable for intelligent driving scenarios. Summary of the invention

[0004] In view of this, it is necessary to provide a method, device, equipment and medium for removing rain and fog from an image to solve the problems that the existing defogging or deraining solutions have low computational efficiency, occupy large computing resources, and are not suitable for intelligent driving scenarios.

[0005] In order to solve the problem that the existing defogging or deraining solutions have low computational efficiency, occupy large computing resources, and are not suitable for intelligent driving scenarios, the present invention provides a method for removing rain and fog from an image, comprising: Filtering the preset conduction function through guided filtering; Defogging the input original image according to the preset conduction function after filtering to obtain a defogging image; inputting the defogging image into the attention recurrent network for processing to obtain an attention map; The dehazed image is derained by combining the contextual autoencoder with the attention map to obtain a target image.

[0006] In a possible implementation, the defogging process on the input original image specifically includes: Obtain the first imaging equation of the atmospheric scattering model describing the foggy environment; Obtain the second equation of the RGB dark channel of the natural image in a non-sky environment; Solving the conduction function by using the first equation of the atmospheric scattering model imaging and the second equation of the RGB dark channel; The conduction function is filtered by guided filtering, and the original image is defogged based on the filtered conduction function.

[0007] In a possible implementation, the rain removal process specifically includes: Determine an attention area in the original image by using the attention recurrent network, and generate an attention map; The original image is derained by combining the contextual autoencoder with the attention map.

[0008] In one possible implementation, improving the attentiveGAN model specifically includes: The convLSTM in the attention recurrent network is improved to convGRU to obtain the improved attentiveGAN model.

[0009] In a possible implementation, the improved attentiveGAN model includes: a generator and a discriminator; wherein the generator includes: an attention recurrent network and a contextual autoencoder.

[0010] In a possible implementation, the improved attentiveGAN model further includes: extracting features using a convolutional neural network through the discriminator and using the attention map for guidance to judge the authenticity of the target image. In a possible implementation, the improved attentiveGAN model further includes: The image processing network VGG16 in the contextual autoencoder is improved to an EfficientNetV2-m network to obtain the improved attentiveGAN model.

[0011] The present invention also provides an image defogging device, comprising: a defogging module and a rain removal module; The defogging module is used to filter the preset conduction function through guided filtering. And defogging the input original image according to the preset conduction function after filtering to obtain a defogging image; The deraining module is used to input the defogging image into the attention loop network for processing to obtain an attention map. The dehazed image is derained by combining the contextual autoencoder with the attention map to obtain a target image.

[0012] The present invention also provides an electronic device, comprising a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in a method for removing rain and fog from an image in any of the above solutions.

[0013] The present invention also provides a computer-readable storage medium for storing a computer-readable program or instruction, wherein the program or instruction, when executed by a processor, can implement the steps in a method for removing rain and fog from an image in any of the above-mentioned solutions.

[0014] The beneficial effects of the present invention are as follows: the scheme of the present invention is mainly used in the image preprocessing stage of the on-board camera of an intelligent driving vehicle to remove rain fog and raindrops from images taken in rainy weather, obtain clear images for transmission to other downstream visual tasks, improve the computational efficiency of existing defogging and deraining schemes, reduce the resources occupied by the model, and improve the model deraining performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of a flow chart of an embodiment of a method for removing rain and fog from an image provided by the present invention; Figure 2 A comparison diagram of an image before and after defogging provided by an embodiment of the present invention; Figure 3 A comparison diagram of an image before and after rain removal provided by an embodiment of the present invention; Figure 4 for Figure 1 A schematic flow chart of an embodiment of step S101; Figure 5 for Figure 1 A schematic flow chart of an embodiment of step S102; Figure 6 A system structure framework diagram of an embodiment of a device for removing rain and fog from an image provided by the present invention; Figure 7 The present invention is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention but not to limit the scope of the present invention.

[0017] like Figure 1 As shown, a method for removing rain and fog from an image provided by an embodiment of the present invention includes: It should be noted that the image deraining and image defogging involved in the present invention belong to the underlying visual technology in the field of computer vision. The fog and raindrops in the image are removed by the method proposed in the present invention to obtain a clear image for downstream target detection, image classification, etc. The existing algorithm model only removes single rain lines and raindrops, and does not consider the impact of rain and fog generated simultaneously on rainy days; the existing model MAXIM, which can complete image processing tasks such as denoising, deblurring, deraining, defogging, and enhancement, also only targets a single task scenario, and the model has a large number of parameters and occupies a large amount of computing resources. Although the existing joint processing method can remove raindrops and fog in the image end-to-end, this method needs to train a CNN-based defogging model and a GAN-based deraining model separately, and then splice and fuse the two models. The model training is difficult, and the overall end-to-end model has low computational efficiency and occupies a large amount of computing resources, which is not suitable for intelligent driving scenarios.

[0018] S101, filtering a preset conduction function by guided filtering; S102, performing defogging processing on the input original image according to the preset filtered conduction function to obtain a defogging image; It should be noted that, in a certain embodiment, the method of the dark channel defogging algorithm based on guided filtering includes: The atmospheric scattering model imaging equation constructed to describe the fog environment is: , (1) Among them, I is the fog image, J is the fog-free image, A is the atmospheric light, t is the transfer function, and x represents the pixel information.

[0019] In the RGB three channels of non-sky natural images, the brightness of at least one channel is very low and close to 0. According to the above rules, the mathematical expression of the dark channel is: (2) Among them, Jc represents each channel of the RGB image, Ω(x) represents the window centered on pixel x, and y represents the pixel information in this window. According to the dark channel prior theory, Jdark tends to 0.

[0020] Normalize equation 1 and divide both sides by the atmospheric light A to obtain the following expression: , (3) If A is a known quantity, t(x) is a constant, the superscript c represents each channel of the RGB image, and x represents the pixel information, it is defined as t~(x) to minimize equation 3: , (4) According to the dark channel prior theory, we get: , (5) For the obtained transfer function, use a filter to refine it. In the original method, the soft matting method is used, which is inefficient. Therefore, guided filtering is used to filter the transfer function: , (6) where r is the filtering radius, I is the foggy image, and ε is the smoothness parameter, and guidedfilter represents the guided filtering function.

[0021] Based on the above reasoning, the fog-free image is restored. To prevent the denominator from being 0, a threshold t0 is set. When t < t0, take t = t0.

[0022] , (7) where I is the foggy image and A is a known quantity.

[0023] It should be noted that in a certain embodiment, the atmospheric light A is a known quantity. In this example, the top 0.1% pixels with the highest brightness are extracted from the dark channel map according to the brightness, and then the value with the highest brightness at the corresponding position in the original foggy image is taken as the A value. In Equation 7, to prevent the denominator of the formula from being 0, a threshold t0 is set. When t < t0, let t = t0. In this example, the value of t0 is 0.1. In the dark channel algorithm based on guided filtering, the minimum filtering radius of the dark channel and the mean filtering radius in the guided filtering are two relatively important parameters. In this example, the minimum filtering radius of the dark channel is 7, and the mean filtering radius in the guided filtering is 81. According to the above description, this example uses the dark channel defogging algorithm based on guided filtering to process and obtain the defogged picture, and the comparison before and after processing is as Figure 2 shown.

[0024] S103. Input the defogged image into an attention recurrent network for processing to obtain an attention map; S104, the defogging image is derained by combining the contextual autoencoder with the attention map to obtain a target image. Based on the attentiveGAN model, the operating efficiency of the attention map generation is improved, the network complexity is reduced, and the original performance of the attention loop network is guaranteed. The perceptual loss of the autoencoder is optimized, and the difference between the generated image and the true value image is extracted using EfficientNetV2-m with stronger image feature extraction capability. The present invention can better remove the rain fog and raindrops in the rainy image to obtain a clear image. It should be noted that attentiveGAN is based on the generative adversarial network (GAN) architecture and introduces an attention mechanism. The model mainly includes two parts: a generator and a discriminator. The generator is used to generate a corresponding rain-free image from a rainy image. It consists of two sub-networks: an attentive-recurrent network and a contextual autoencoder. The attention loop network determines the attention area in the input image and generates an attention map. These areas are mainly raindrop areas and their surrounding structures. The contextual autoencoder is used to generate rain-free images, and its input is the input image and the attention map output by the attention recurrent network. The discriminator is used to judge the authenticity of the generated image, using a convolutional neural network to extract features and using the attention map for guidance. It should be noted that the authenticity judgment may include: during the training process, a real image (groundtruth), i.e., a rain-free image, is used, the input image is a rainy image, and the generator processes the input image to obtain a rain-free image. As the training progresses, the rain-free images generated by the generator become more and more realistic, and the discriminator's ability to distinguish between true and false is also improving. Finally, a balance is reached, that is, the generator can generate sufficiently realistic images that the discriminator cannot accurately distinguish between true and false. Among them, the discriminator compares the difference between the image generated by the generator and the real image (groundtruth). Here, the generated image is the rain-free image, and the real image refers to the rain-free image. In order to guide the discriminator to focus on the area indicated by the attention map, a loss function of the discriminator is the loss between the features extracted from the inner layer of the discriminator and the final attention map.The attentiveGAN model mentioned in the present invention adopts an existing publicly available model framework as its model architecture, see Rui Qian, Robby T Tan, Wenhan Yang, Jiajun Su, and Jiaying Liu. Attentive generative adversarial network for raindrop removal from a single image. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, pp. 2482–2491, 2018.

[0025] In one embodiment, the process of improving the attentiveGAN model specifically includes: using 5 layers of ResNet and convLSTM and a convolutional layer each time the attention recurrent network generates an attention map. In order to improve the computational efficiency of the attention map and reduce the complexity of the attention recurrent network, the present invention improves convLSTM to convGRU, which merges the forget gate and output gate of convLSTM into an update gate, cancels the output gate, and directly uses the reset gate and update gate to control the information flow. After the improvement, there are only two gates, which makes the recurrent attention network structure more concise, effectively reduces the number of parameters, reduces the risk of overfitting, and improves the training speed, while also maintaining almost the same performance as convLSTM.

[0026] The improvement process may also include: there are 16 Conv+Relu layers in the contextual autoencoder, and two loss functions are used in the autoencoder: multi-scale loss and perceptual loss. By extracting features from different decoder layers to form outputs of different sizes, the MSE loss is calculated between the true value image to obtain multi-scale loss, and context information is obtained from different scales. The perceptual loss is obtained by calculating the global difference between the features of the autoencoder output image and the features of the corresponding true value image. The network used to extract image features here is VGG16 pre-trained on the ImageNet dataset. The present invention improves VGG16 to EfficientNetV2-m. As a more advanced backbone network, EfficientNetV2 has stronger computational efficiency and feature extraction capabilities than VGG16, and can better capture the difference between the generated image and the original true value. The improved perceptual loss function is as follows: (8) Among them, Lp represents perceptual loss, O represents generated image, T represents true image, and EFF represents EfficientNetV2-m.

[0027] In one embodiment, for the improved attentiveGAN algorithm, the following parameters are used to train the improved attentiveGAN. Epochs=1000, learning_rate=0.0002, batch_size=8. The defogging image processed by the dark channel defogging algorithm based on guided filtering described in S1 is input into the improved attentiveGAN to obtain the final generated derained image, as shown in FIG. Figure 3 By using the improved attentiveGAN image defogging and deraining algorithm provided by the present invention, rain fog and raindrops in the image can be removed in stages to obtain a clear image.

[0028] The present invention first uses a dark channel algorithm based on guided filtering to remove rain fog in the original input image, and then uses an improved attentiveGAN to remove raindrops in the image. The present invention makes two improvements to attentiveGAN. Improvement 1 improves the convLSTM in the attention recurrent network to convCRU, which improves the operating efficiency of attention map generation, reduces network complexity, and ensures the original performance of the attention recurrent network. Improvement 2 optimizes the perceptual loss of the autoencoder and uses EfficientNetV2-m with stronger image feature extraction capabilities to extract the difference between the generated image and the true value image. The present invention can better remove rain fog and raindrops in rainy images to obtain clear images.

[0029] Preferably, in any of the above embodiments, Figure 4 As shown, the defogging process specifically includes: S401, obtaining the first imaging equation of the atmospheric scattering model describing the fog environment; It should be noted that, in a certain embodiment, constructing the first equation of atmospheric scattering model imaging specifically includes: The atmospheric scattering model imaging equation constructed to describe the fog environment is: , (1) Among them, I is the fog image, J is the fog-free image, A is the atmospheric light, t is the transfer function, and x represents the pixel information.

[0030] S402, obtaining a second equation of the RGB dark channel of a natural image in a non-sky environment; It should be noted that, in a certain embodiment, constructing the second RGB dark channel equation includes: In the RGB three channels of a non-sky natural image, the brightness of at least one channel is very low and approaches 0. According to the above rule, the mathematical expression of the dark channel is: , (2) where Jc represents each channel of the RGB image, Ω(x) represents the window centered on pixel x, and according to the dark channel prior theory, Jdark tends to 0.

[0031] S403. Solve the transmission function through the first equation of the atmospheric scattering model imaging and the second equation of the RGB dark channel; S404. Filter the transmission function using guided filtering, and perform defogging processing on the original image based on the filtered transmission function.

[0032] In a certain embodiment, S403 and S404 may include: Normalize Equation (1), divide both sides by the atmospheric light A to obtain the following expression: , (3) If A is a known quantity and t(x) is a constant, define it as t~(x) and perform a minimization operation on Equation (3): , (4) According to the dark channel prior theory, it is obtained that: , (5) For the obtained transmission function, use a filter to refine it. In the original method, the method of soft matting is used, which is inefficient. Therefore, guided filtering is used to filter the transmission function: , (6) where r is the filtering radius, is the foggy image, is the smoothness parameter, and guidedfilter represents the guided filtering function.

[0033] Based on the above reasoning, to restore the fog-free image, to prevent the denominator from being 0, set a threshold t0. When t < t0, take t = t0.

[0034] , (7) where I is the foggy image and A is a known quantity.

[0035] It should be noted that in a certain embodiment, the atmospheric light A is a known quantity. In this example, the top 0.1% pixels with the highest brightness will be extracted from the dark channel map according to the brightness, and then the value with the highest brightness at the corresponding position in the original foggy image will be used as the value of A. In Equation 7, to prevent the denominator of the formula from being 0, a threshold t0 is set. When t < t0, let t = t0. In this example, the value of t0 is 0.1. In the dark channel algorithm based on guided filtering, the minimum filtering radius of the dark channel and the mean filtering radius in guided filtering are two relatively important parameters. In this example, the minimum filtering radius of the dark channel is 7, and the mean filtering radius in guided filtering is 81.

[0036] Preferably, in a certain embodiment above, the improved Attentive GAN model includes: a generator and a discriminator; wherein, the generator includes: an attention recurrent network and a context autoencoder.

[0037] It should be noted that Attentive GAN is based on the generative adversarial network (GAN) architecture and introduces an attention mechanism. The model mainly includes two parts: a generator and a discriminator. The generator is used to generate a corresponding rain-free image from a rainy image, and it consists of two sub-networks: an attention recurrent network and a context autoencoder. The attention recurrent network determines the attention regions in the input image and generates an attention map. These regions are mainly the raindrop regions and the surrounding structures. The context autoencoder is used to generate a rain-free image, and its input is the input image and the attention map output by the attention recurrent network. The discriminator is used to judge the authenticity of the generated image, extracts features using a convolutional neural network, and is guided by the attention map.

[0038] Preferably, in a certain embodiment above, as Figure 5 shown, the rain removal process specifically includes: S501. Determine the attention regions in the original image through the attention recurrent network and generate an attention map; S502. Perform rain removal processing on the original image through the context autoencoder in combination with the attention map.

[0039] Preferably, in a certain embodiment above, it further includes: using the discriminator to extract features using a convolutional neural network and being guided by the attention map to judge the authenticity of the target image.

[0040] Preferably, in a certain embodiment above, the improved AttentiveGAN model specifically includes: The convLSTM in the attention recurrent network is improved to convGRU to obtain the improved attentiveGAN model. It should be noted that the attention recurrent network uses 5 layers of ResNet and convLSTM and a convolution layer each time it generates an attention map. In order to improve the computational efficiency of the attention map and reduce the complexity of the attention recurrent network, the present invention improves convLSTM to convGRU, which merges the forget gate and output gate of convLSTM into an update gate, cancels the output gate, and directly uses the reset gate and update gate to control the information flow. After the improvement, there are only two gates, which makes the structure of the recurrent attention network more concise, effectively reduces the number of parameters, reduces the risk of overfitting, and improves the training speed, while also maintaining almost the same performance as convLSTM.

[0041] Preferably, in one of the above embodiments, improving the attentiveGAN model further includes: The image processing network VGG16 in the contextual autoencoder is improved to an EfficientNetV2-m network to obtain the improved attentiveGAN model.

[0042] It should be noted that, in a certain embodiment, the improved attentiveGAN model may include: Improvement 1 uses convGRU to replace convLSTM, reduces the parameters of the attention loop network, and improves the computational efficiency. The main content of the improvement is to remove the calculation of the forget gate and the output gate, and replace them with the update gate and the reset gate. When calculating the candidate hidden state, the reset gate is first multiplied with the hidden state of the previous moment, and then concatenated with the current input tensor, and then the convolution operation is performed. The update of the hidden state is directly based on the update gate and the candidate hidden state, as well as the hidden state of the previous moment. The cell state is removed from the return value, and only the hidden state is retained. The update gate con_z of the improved convGRU has a convolution kernel of 3*3, a number of channels of 32C, and a step size of 1. The reset gate conv_r of the improved convGRU has a convolution kernel of 3*3, a number of channels of 32C, and a step size of 1. Improvement 2 uses the optimized perceptual loss of the autoencoder. The loss function of the autoencoder consists of multi-scale loss and perceptual loss. The multi-scale loss obtains contextual information from different scales, and the perceptual loss measures the difference between the image generated by the generator and the true image. In order to improve the performance of feature extraction of generated images and true images, EfficientNetV2-m is used to replace the VGG16 network in the original structure for feature extraction, thereby optimizing the perceptual loss. The structure of EfficientNetV2-m uses a publicly available parameter model, see Tan, Mingxing, and QuocLe. "Efficientnetv2: Smaller models and faster training." International conference on machine learning .PMLR , 2021. Use EfficientNetV2-m to extract features of generated images and true images, and calculate perceptual loss.

[0043] In one embodiment, for the improved attentiveGAN algorithm, the following parameters are used to train the improved attentiveGAN. Epochs=1000, learning_rate=0.0002, batch_size=8. The defogging image processed by the dark channel defogging algorithm based on guided filtering described in S1 is input into the improved attentiveGAN to obtain the final generated derained image, as shown in FIG. Figure 3 By using the improved attentiveGAN image defogging and deraining algorithm provided by the present invention, rain fog and raindrops in the image can be removed in stages to obtain a clear image.

[0044] The solution of the present invention uses a dark channel defogging algorithm based on guided filtering to remove rain and fog in images. This method uses a priori theory instead of a deep neural network, has higher computational efficiency, and occupies less resources.

[0045] We use attentiveGAN as the backbone network and make improvements to optimize the generation efficiency of the attention map and the perceptual loss in the network model. This improves the computational efficiency of the model, reduces the complexity of the network, and improves the model's deraining performance. Compared with the existing end-to-end model, it has higher computational efficiency and occupies less resources.

[0046] In one embodiment, if Figure 6 As shown, an image defogging device 600 includes: a defogging module 601 and a deraining module 602; The defogging module 601 is used to filter the preset conduction function through guided filtering. And defogging the input original image according to the preset conduction function after filtering to obtain a defogging image; The deraining module 602 is used to input the dehazed image into the attention recurrent network for processing, obtain an attention map, and perform deraining processing on the dehazed image in combination with the attention map through a contextual autoencoder to obtain a target image.

[0047] The image demisting device provided in the above embodiment can implement the technical solution described in the above image demisting method embodiment. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above image demisting method embodiment, which will not be repeated here.

[0048] like Figure 7 As shown, the present invention also provides an electronic device 700. The electronic device 700 includes a processor 701, a memory 702 and a display 703. Figure 7 Only some components of the electronic device 700 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0049] In some embodiments, the memory 702 may be an internal storage unit of the electronic device 700, such as a hard disk or memory of the electronic device 700. In other embodiments, the memory 702 may also be an external storage device of the electronic device 700, such as a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), etc. equipped on the electronic device 700.

[0050] Furthermore, the memory 702 may include both an internal storage unit of the electronic device 700 and an external storage device. The memory 702 is used to store application software installed in the electronic device 700 and various data.

[0051] In some embodiments, the processor 701 may be a central processing unit (CPU), a microprocessor or other data processing chip, which is used to run the program code or process data stored in the memory 702, such as an image deraining method in the present invention.

[0052] In some embodiments, the display 703 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 703 is used to display information on the electronic device 700 and to display a visual user interface. The components 701-703 of the electronic device 700 communicate with each other via a system bus.

[0053] In some embodiments of the present invention, when the processor 701 executes the image deraining and fogging program in the memory 702, the following steps may be implemented: Filtering the preset conduction function through guided filtering; Defogging the input original image according to the preset conduction function after filtering to obtain a defogging image; Inputting the dehazed image into an attention recurrent network for processing to obtain an attention map; The dehazed image is derained by combining the contextual autoencoder with the attention map to obtain a target image.

[0054] It should be understood that: when the processor 701 executes the image rain and fog removal program in the memory 702, in addition to the above functions, other functions may also be implemented. For details, please refer to the description of the corresponding method embodiment above.

[0055] Furthermore, the embodiment of the present invention does not specifically limit the type of the electronic device 700 mentioned, and the electronic device 700 may be a portable electronic device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer, etc. Exemplary embodiments of portable electronic devices include but are not limited to portable electronic devices equipped with IOS, Android, Microsoft or other operating systems. The above-mentioned portable electronic devices may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 700 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0056] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, a method for removing rain and fog from an image provided by the above methods is implemented, and the method comprises: Filtering the preset conduction function through guided filtering; Defogging the input original image according to the preset conduction function after filtering to obtain a defogging image; Inputting the dehazed image into an attention recurrent network for processing to obtain an attention map; The dehazed image is derained by combining the contextual autoencoder with the attention map to obtain a target image.

[0057] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0058] The above is a detailed introduction to the image deraining and fogging method, system, device and medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for removing rain and fog from an image, characterized in that: include: Filtering the preset conduction function through guided filtering; Defogging the input original image according to the preset conduction function after filtering to obtain a defogging image; Inputting the dehazed image into an attention recurrent network for processing to obtain an attention map; The dehazed image is derained by combining the contextual autoencoder with the attention map to obtain a target image.

2. The method for removing rain and fog from an image according to claim 1, characterized in that: The defogging process specifically includes: Obtain the first imaging equation of the atmospheric scattering model describing the foggy environment; Obtain the second equation of the RGB dark channel of the natural image in a non-sky environment; Solving the conduction function by using the first equation of the atmospheric scattering model imaging and the second equation of the RGB dark channel; The conduction function is filtered by guided filtering, and the original image is defogged based on the filtered conduction function.

3. The method for removing rain and fog from an image according to claim 1 or 2, characterized in that: The rain removal process specifically includes: Determine an attention area in the original image by using the attention recurrent network, and generate an attention map; The original image is derained by combining the contextual autoencoder with the attention map.

4. The method for removing rain and fog from an image according to claim 1, characterized in that: Also includes: Improve the convLSTM in the attention recurrent network to convGRU to obtain an improved attentiveGAN model; The performing rain removal processing on the defogging image specifically includes: performing rain removal processing on the defogging image based on an improved attentiveGAN model.

5. The method for removing rain and fog from an image according to claim 4, characterized in that: The improved attentiveGAN model includes: a generator and a discriminator; wherein the generator includes: an attention recurrent network and a contextual autoencoder.

6. The method for removing rain and fog from an image according to claim 5, characterized in that: Also includes: The discriminator uses a convolutional neural network to extract features and uses the attention map for guidance to judge the authenticity of the target image.

7. The method for removing rain and fog from an image according to claim 4, characterized in that: Also includes: Improve the image processing network VGG16 in the contextual autoencoder to an EfficientNetV2-m network to obtain the improved attentiveGAN model; The performing rain removal processing on the defogging image specifically includes: performing rain removal processing on the defogging image based on an improved attentiveGAN model.

8. An image defogging device, characterized in that: include: Defogging module and deraining module; The defogging module is used to filter the preset conduction function through guided filtering, and to defog the input original image according to the filtered preset conduction function to obtain a defogging image; The deraining module is used to input the dehazed image into the attention recurrent network for processing, obtain an attention map, and perform deraining processing on the dehazed image in combination with the attention map through a contextual autoencoder to obtain a target image.

9. An electronic device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the method for removing rain and fog from an image as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps of the image deraining and fogging method described in any one of claims 1 to 7.

Images