A weakly supervised image deraining system and method integrating dual attention mechanism

Abstract

This invention discloses a weakly supervised image deraining system and method that incorporates a dual attention mechanism. The system consists of a camera unit, an image processing unit, and a display unit; the method includes a weakly supervised image deraining process within the image processing unit. This invention not only reduces the cost of collecting paired sample datasets and improves the applicability of network models, but also produces derained images with clearer background details and better color preservation. The system is simple and intuitive, the method offers superior performance, and its application is universal.

Description

Technical Field

[0001] The present invention relates to an image deraining system and method, and in particular to a weakly supervised image deraining system and method integrating a dual attention mechanism. Background Art

[0002] In recent years, with the continuous development of science and technology, image processing technology has become increasingly widely used, particularly in the field of computer vision systems, creating numerous practical benefits. However, in rainy weather, the image quality obtained by computer vision systems is generally poor. Raindrops or rain lines often cause severe image degradation, reducing the application value of the image information. Image deraining tasks involve the field of image processing technology, aiming to recover a derained image from a given rain-affected image. This is of great significance for computer vision systems in completing subsequent high-level visual tasks, such as autonomous driving, surveillance, and aerial photography.

[0003] Deep learning methods have made recent progress in image deraining. Most existing methods use strongly supervised learning, requiring paired rainy and rain-free images as input for training. This leads to a high reliance on paired sample datasets. Furthermore, existing methods often perform poorly in image deraining, with issues such as blurred background details and color variations after raindrop removal. Summary of the Invention

[0004] To address the problems of the prior art, the present invention aims to overcome the shortcomings of existing technologies and provide a weakly supervised image deraining system and method that incorporates a dual attention mechanism. By designing a weakly supervised image deraining network based on a recurrent generative network architecture, the training sample set does not need to be paired during training, allowing for direct generation of derained images. Simultaneously, dual channel and spatial attention mechanisms are introduced into the two generators of the weakly supervised image deraining network, resulting in clearer background details in the derained image and color information that more closely resembles the derained image to be restored. This invention not only reduces the cost of collecting paired sample datasets and improves the applicability of the network model, but also produces derained images with clearer background details and better preserved color information. The system is simple and intuitive, and the method performs excellently, making it universally applicable in the field of computer vision systems.

[0005] In order to achieve the above invention purpose, the present invention adopts the following technical solutions:

[0006] A weakly supervised image deraining system incorporating a dual-attention mechanism consists of a camera unit, an image processing unit, and a display unit. The camera unit is connected to the image processing unit via a wired or wireless connection; the image processing unit is also connected to the display unit via a wired or wireless connection. This weakly supervised image deraining system incorporating a dual-attention mechanism not only reduces the cost of collecting paired sample datasets and improves the applicability of network models, but also produces derained images with clearer background details and better color preservation.

[0007] Preferably, the weakly supervised image deraining system of the present invention, which integrates a dual attention mechanism, comprises an image processing unit comprising a data receiving module, a sample set construction module, a sample set preprocessing module, an image deraining network construction module, an image deraining model training module, an image deraining processing module, and a data sending module. The data receiving module is connected to the sample set construction module in a wired manner, the sample set construction module is connected to the sample set preprocessing module in a wired manner, the sample set preprocessing module is connected to the image deraining model training module and the image deraining processing module in a wired manner, the image deraining network construction module is connected to the image deraining model training module in a wired manner, the image deraining model training module is connected to the image deraining processing module in a wired manner, and the image deraining processing module is connected to the data sending module in a wired manner. The weakly supervised image deraining system of the present invention, which integrates a dual attention mechanism, is simple and intuitive and can achieve end-to-end restoration of rainy images.

[0008] A weakly supervised image deraining method integrating a dual attention mechanism is operated using a weakly supervised image deraining system integrating a dual attention mechanism of the present invention. The operation steps include a weakly supervised image deraining process of an image processing unit; the weakly supervised image deraining process of the image processing unit:

[0009] S1, using the data receiving module to receive image data from the camera unit and send it to the sample set construction module to construct a training sample set and a test sample set to obtain the rain image to be restored;

[0010] S2, the sample set preprocessing module performs preprocessing operations on the training sample set, the test sample set, and the rainy image to be restored;

[0011] S3, the image deraining network construction module is based on the recurrent generative adversarial network, introduces the channel and spatial dual attention mechanism, and constructs a weakly supervised image deraining network that integrates the dual attention mechanism;

[0012] S4, the image deraining model training module uses the preprocessed training sample set to train the above network. The input is unpaired rainy images and rain-free images. The network is constrained by using a hybrid loss function and the test sample set is used for inference testing to obtain a weakly supervised image deraining model that integrates a dual attention mechanism.

[0013] S5, the image deraining processing module directly inputs the rain image to be restored into the above model to generate a derained image;

[0014] S6, the data sending module sends the finally generated derained image to the display unit to achieve image deraining.

[0015] Preferably, step S3 includes an image deraining network construction process; the image deraining network construction process:

[0016] S31, based on the basic architecture of the cyclic generative adversarial network, designed and constructed a weakly supervised image deraining network, using the autoencoder architecture to implement two generators and the Markov discriminator to implement two discriminators;

[0017] S32, introduces the channel attention mechanism into the encoders of the two generators, designs and constructs the channel attention block and channel attention residual block for feature extraction;

[0018] S33, introduces the spatial attention mechanism into the decoders of the two generators, designs and constructs spatial attention blocks and multi-branch attention generation subnetworks to highlight rainy areas and background areas.

[0019] Preferably, the step S32 includes a channel attention residual block construction process; the channel attention residual block construction process:

[0020] S321, build a channel attention block: use a 2D discrete cosine transform, two fully connected layers, and a Sigmoid function to obtain channel attention; use multiplication to obtain output features after multiplying with the input features;

[0021] S322, combine the channel attention block and the residual block to construct a channel attention residual block.

[0022] Preferably, the step S33 includes a multi-branch attention generation sub-network construction process; the multi-branch attention generation sub-network construction process:

[0023] S331, build a spatial attention block: use a pooling layer, a 3×3 convolution layer, and a Sigmoid function to obtain spatial attention; use multiplication to obtain output features after multiplying with the input features;

[0024] S332, build branch 1: use a spatial attention module and a 1×1 convolution layer to generate multi-channel rain features;

[0025] S333, construct branch 2: use a 1×1 convolution layer to generate single-channel background features;

[0026] S334, construct branch three: use a 7×7 convolution layer and a tanh activation function layer to generate image content features;

[0027] S335, concatenate the rain feature and background feature, and use the softmax function to generate rain attention and background attention;

[0028] S336, using multiplication, multiplies the rain-containing attention and the image content features to generate a rain-containing attention map; multiplies the background features and the input rainy image to generate a background attention map;

[0029] S337, using addition, the rain-containing attention map is added to the background attention map to generate a rain-free image.

[0030] Compared with the prior art, the present invention has the following obvious outstanding substantial features and significant advantages:

[0031] 1. The system of the present invention comprises a camera unit, an image processing unit, and a display unit. The method includes a weakly supervised image deraining process within the image processing unit. This invention not only reduces the cost of collecting paired sample datasets and improves the applicability of network models, but also produces derained images with clearer background details and better color preservation.

[0032] 2. The system of the present invention is simple and intuitive, the method has superior performance, and its application is universal. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the system structure of an embodiment of the present invention.

[0034] Figure 2 yes Figure 1 Block diagram of the image processing unit of the example.

[0035] Figure 3 yes Figure 2 The workflow diagram of the example implementation of the image processing unit for weakly supervised image deraining.

[0036] Figure 4 yes Figure 3 The example implementation workflow diagram for building an image deraining network.

[0037] Figure 5 yes Figure 4 Workflow diagram of the example implementation of channel attention residual block construction.

[0038] Figure 6 yes Figure 4 Example implementation of the multi-branch attention generation sub-network construction workflow diagram. DETAILED DESCRIPTION

[0039] The above solution is further described below with reference to specific implementation examples. The preferred embodiments of the present invention are described in detail as follows:

[0040] Example 1:

[0041] In this embodiment, see Figure 1 The weakly supervised image deraining system integrating the dual attention mechanism consists of a camera unit 1, an image processing unit 2 and a display unit 3, and is characterized in that the camera unit 1 is connected to the image processing unit 2 in a wired or wireless manner; the image processing unit 2 is connected to the display unit 3 in a wired or wireless manner.

[0042] The weakly supervised image deraining system in this embodiment that integrates the dual attention mechanism can not only reduce the collection cost of paired sample datasets and improve the applicability of network models, but also obtain derained images with clearer background details and better color information preservation.

[0043] Example 2:

[0044] This embodiment is basically the same as the first embodiment, with the following special features:

[0045] In this embodiment, Figure 2 The structure of the image processing unit 2 is composed of a data receiving module 4, a sample set construction module 5, a sample set preprocessing module 6, an image deraining network construction module 7, an image deraining model training module 8, an image deraining processing module 9, and a data sending module 10. The data receiving module 4 is connected to the sample set construction module 5 in a wired manner, the sample set construction module 5 is connected to the sample set preprocessing module 6 in a wired manner, the sample set preprocessing module 6 is connected to the image deraining model training module 8 and the image deraining processing module 9 in a wired manner, the image deraining network construction module 7 is connected to the image deraining model training module 8 in a wired manner, the image deraining model training module 8 is connected to the image deraining processing module 9 in a wired manner, and the image deraining processing module 9 is connected to the data sending module 10 in a wired manner.

[0046] This embodiment incorporates a weakly supervised image deraining system with a dual attention mechanism. By designing a weakly supervised image deraining network based on a recurrent generative network architecture, the network can directly generate derained images without requiring paired training samples. This embodiment not only reduces the cost of collecting paired sample datasets and improves the applicability of network models, but also produces derained images with clearer background details and better color preservation. The system is simple and intuitive, and its method performs well, demonstrating its universal applicability in computer vision systems.

[0047] Example 3:

[0048] This embodiment is basically the same as the above embodiment, with the following special features:

[0049] In this embodiment, a weakly supervised image deraining method integrating a dual attention mechanism is operated using the above system, and the operation steps include a weakly supervised image deraining process of an image processing unit.

[0050] join Figure 3 , the image processing unit weakly supervises the image deraining process:

[0051] S1, using the data receiving module 4 to receive the image data from the camera unit 1, and sending it to the sample set construction module 5, to construct a training sample set and a test sample set, and obtain the rain image to be restored;

[0052] S2, the sample set preprocessing module 6 performs preprocessing operations on the training sample set, the test sample set, and the rain image to be restored;

[0053] S3, image deraining network construction module 7 is based on the cyclic generative adversarial network, introduces the channel and spatial dual attention mechanism, and constructs a weakly supervised image deraining network that integrates the dual attention mechanism;

[0054] S4, the image deraining model training module 8 uses the preprocessed training sample set to train the above network, inputs unpaired rainy images and rain-free images, constrains the network using a hybrid loss function, and performs inference testing using a test sample set to obtain a weakly supervised image deraining model that integrates a dual attention mechanism;

[0055] S5, the image deraining processing module 9 directly inputs the rain image to be restored into the above model to generate a derained image;

[0056] S6 , the data sending module 10 sends the finally generated derained image to the display unit 3 , thus achieving image deraining.

[0057] This embodiment introduces a dual channel and spatial attention mechanism into the two generators of a weakly supervised image deraining network, resulting in clearer background details in the derained image and a color information that is closer to the original derained image. This embodiment not only reduces the cost of collecting paired sample datasets and improves the applicability of the network model, but also produces derained images with clearer background details and better preserved color information.

[0058] Example 4:

[0059] This embodiment is basically the same as the above embodiment, with the following special features:

[0060] In this embodiment, step S3 includes an image deraining network construction process.

[0061] join Figure 4 , the image deraining network construction process:

[0062] S31, based on the basic architecture of the cyclic generative adversarial network, designed and constructed a weakly supervised image deraining network, using the autoencoder architecture to implement two generators and the Markov discriminator to implement two discriminators;

[0063] S32, introduces the channel attention mechanism into the encoders of the two generators, designs and constructs the channel attention block and channel attention residual block for feature extraction;

[0064] S33, introduces the spatial attention mechanism into the decoders of the two generators, designs and constructs spatial attention blocks and multi-branch attention generation subnetworks to highlight rainy areas and background areas.

[0065] In this embodiment, step S32 includes a channel attention residual block construction process.

[0066] See also Figure 5 , the channel attention residual block construction process:

[0067] S321, build a channel attention block: use a 2D discrete cosine transform, two fully connected layers, and a Sigmoid function to obtain channel attention; use multiplication to obtain output features after multiplying with the input features;

[0068] S322, combine the channel attention block and the residual block to construct a channel attention residual block.

[0069] In this embodiment, see Figure 6 , the multi-branch attention generation sub-network construction process:

[0070] S331, build a spatial attention block: use a pooling layer, a 3×3 convolution layer, and a Sigmoid function to obtain spatial attention; use multiplication to obtain output features after multiplying with the input features;

[0071] S332, build branch 1: use a spatial attention module and a 1×1 convolution layer to generate multi-channel rain features;

[0072] S333, construct branch 2: use a 1×1 convolution layer to generate single-channel background features;

[0073] S334, construct branch three: use a 7×7 convolution layer and a tanh activation function layer to generate image content features;

[0074] S335, concatenate the rain feature and background feature, and use the softmax function to generate rain attention and background attention;

[0075] S336, using multiplication, multiplies the rain-containing attention and the image content features to generate a rain-containing attention map; multiplies the background features and the input rainy image to generate a background attention map;

[0076] S337, using addition, the rain-containing attention map is added to the background attention map to generate a rain-free image.

[0077] In step S4, the network is constrained using a hybrid loss function, which is specifically:

[0078] L=L GAN +λ cycle ×L cycle +λ idt ×L idt (1)

[0079] Where, L GAN To generate the adversarial loss, L cycle is the cycle consistency loss, L idt is the unit mapping loss, λ cycle ,λ idt Is to control L cycle ,L idt Parameters of relative relationship. It should be noted that λ cycle and λ idt The value of can be set using the settings in the cyclic generative adversarial network (λ cycle =10,λ idt =0.5), and can also be adjusted according to actual training conditions.

[0080] Use L GAN Used to judge real images and fake images:

[0081]

[0082] Where E represents the expectation, x represents the input real rainy image, y represents the input real rainless image, and p data (x) and p data (y) are the sample distributions of x and y respectively;

[0083] Use L cycle To ensure cycle consistency:

[0084]

[0085] Where ||.||1 is the L1 function, which calculates the absolute value distance between two samples.

[0086] Use L idt Used to reduce the change of the overall color-independent domain information of the image controlled by pixel points:

[0087]

[0088] The above-described embodiment of the present invention uses a camera unit to capture rain-affected images to be restored, uses an image processing unit to restore the rain-affected images, and outputs the processing results via a display unit. This embodiment of the present invention not only reduces the cost of collecting paired sample datasets and improves the applicability of network models, but also produces rain-removed images with clearer background details and better color preservation. The system of the present invention is simple and intuitive, the method offers superior performance, and has universal applicability.

[0089] The above describes the embodiments of the present invention in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Various changes can be made according to the purpose of the invention. Any changes, modifications, substitutions, combinations or simplifications made according to the spirit and principles of the technical solution of the present invention should be equivalent replacement methods. As long as they comply with the purpose of the invention and do not deviate from the technical principles and inventive concepts of the present invention, they belong to the scope of protection of the present invention.

Claims

1. A weakly supervised image deraining system integrating a dual attention mechanism, comprising a camera unit (1), an image processing unit (2) and a display unit (3), characterized in that: The camera unit (1) is connected to the image processing unit (2) in a wired or wireless manner; the image processing unit (2) is connected to the display unit (3) in a wired or wireless manner; The weakly supervised image deraining system integrating the dual attention mechanism is used for operation, including a weakly supervised image deraining process of an image processing unit; the weakly supervised image deraining process of the image processing unit: S1, using the data receiving module (4) to receive the image data transmitted by the camera unit (1), and sending it to the sample set construction module (5), constructing a training sample set and a test sample set, and obtaining a rain image to be restored; S2, the sample set preprocessing module (6) performs preprocessing operations on the training sample set, the test sample set, and the rain image to be restored; S3, image deraining network construction module (7) is based on the cyclic generative adversarial network, introduces the channel and spatial dual attention mechanism, and constructs a weakly supervised image deraining network that integrates the dual attention mechanism; S4, image deraining model training module (8) uses the preprocessed training sample set to train the above network, with unpaired rainy images and rain-free images as input, uses the hybrid loss function to constrain the network, and uses the test sample set for inference testing to obtain a weakly supervised image deraining model that integrates the dual attention mechanism; S5, the image deraining processing module (9) directly inputs the rainy image to be restored into the above model to generate a derained image; S6, the data sending module (10) sends the finally generated derained image to the display unit (3), thereby achieving image deraining; The step S3 includes the image deraining network construction process; the image deraining network construction process: S31, based on the basic architecture of the cyclic generative adversarial network, designed and constructed a weakly supervised image deraining network, using the autoencoder architecture to implement two generators and the Markov discriminator to implement two discriminators; S32, introduces the channel attention mechanism into the encoders of the two generators, designs and constructs the channel attention block and channel attention residual block for feature extraction; S33, introduces the spatial attention mechanism in the decoders of the two generators, designs and constructs a spatial attention block and a multi-branch attention generation sub-network to highlight the rainy areas and background areas; The step S33 includes a multi-branch attention generation sub-network construction process; the multi-branch attention generation sub-network construction process: S331, build a spatial attention block: use a pooling layer, a 3×3 convolution layer, and a Sigmoid function to obtain spatial attention; use multiplication to obtain output features after multiplying with the input features; S332, build branch 1: use a spatial attention module and a 1×1 convolution layer to generate multi-channel rain features; S333, construct branch 2: use a 1×1 convolution layer to generate single-channel background features; S334, construct branch three: use a 7×7 convolution layer and a tanh activation function layer to generate image content features; S335, concatenate the rain feature and background feature, and use the softmax function to generate rain attention and background attention; S336, using multiplication, multiplies the rain-containing attention and the image content features to generate a rain-containing attention map; multiplies the background features and the input rainy image to generate a background attention map; S337, using addition, the rain-containing attention map is added to the background attention map to generate a rain-free image.

2. The weakly supervised image deraining system integrating dual attention mechanism according to claim 1 is characterized in that: The image processing unit (2) is composed of a data receiving module (4), a sample set construction module (5), a sample set preprocessing module (6), an image deraining network construction module (7), an image deraining model training module (8), an image deraining processing module (9) and a data sending module (10). The data receiving module (4) is connected to the sample set construction module (5) in a wired manner, the sample set construction module (5) is connected to the sample set preprocessing module (6) in a wired manner, the sample set preprocessing module (6) is connected to the image deraining model training module (8) and the image deraining processing module (9) in a wired manner, the image deraining network construction module (7) is connected to the image deraining model training module (8) in a wired manner, the image deraining model training module (8) is connected to the image deraining processing module (9) in a wired manner, and the image deraining processing module (9) is connected to the data sending module (10) in a wired manner.

3. A weakly supervised image deraining method integrating a dual attention mechanism, which is operated using the weakly supervised image deraining system integrating a dual attention mechanism according to claim 1, characterized in that: The operation steps include a weakly supervised image deraining process of an image processing unit; the weakly supervised image deraining process of the image processing unit: S1, using the data receiving module (4) to receive the image data transmitted by the camera unit (1), and sending it to the sample set construction module (5), constructing a training sample set and a test sample set, and obtaining a rain image to be restored; S2, the sample set preprocessing module (6) performs preprocessing operations on the training sample set, the test sample set, and the rain image to be restored; S3, image deraining network construction module (7) is based on the cyclic generative adversarial network, introduces the channel and spatial dual attention mechanism, and constructs a weakly supervised image deraining network that integrates the dual attention mechanism; S4, image deraining model training module (8) uses the preprocessed training sample set to train the above network, with unpaired rainy images and rain-free images as input, uses the hybrid loss function to constrain the network, and uses the test sample set for inference testing to obtain a weakly supervised image deraining model that integrates the dual attention mechanism; S5, the image deraining processing module (9) directly inputs the rainy image to be restored into the above model to generate a derained image; S6, the data sending module (10) sends the finally generated derained image to the display unit (3), thereby achieving image deraining.

4. The weakly supervised image deraining method integrating dual attention mechanism according to claim 3 is characterized in that: The step S3 includes the image deraining network construction process; the image deraining network construction process: S31, based on the basic architecture of the cyclic generative adversarial network, designed and constructed a weakly supervised image deraining network, using the autoencoder architecture to implement two generators and the Markov discriminator to implement two discriminators; S32, introduces the channel attention mechanism into the encoders of the two generators, designs and constructs the channel attention block and channel attention residual block for feature extraction; S33, introduces the spatial attention mechanism into the decoders of the two generators, designs and constructs spatial attention blocks and multi-branch attention generation subnetworks to highlight rainy areas and background areas.

5. The weakly supervised image deraining method integrating dual attention mechanism according to claim 4 is characterized in that: The step S32 includes a channel attention residual block construction process; the channel attention residual block construction process: S321, build a channel attention block: use a 2D discrete cosine transform, two fully connected layers, and a Sigmoid function to obtain channel attention; use multiplication to obtain output features after multiplying with the input features; S322, combine the channel attention block and the residual block to construct a channel attention residual block.

6. The weakly supervised image deraining method integrating dual attention mechanism according to claim 4 is characterized in that: The step S33 includes a multi-branch attention generation sub-network construction process; the multi-branch attention generation sub-network construction process: S331, build a spatial attention block: use a pooling layer, a 3×3 convolution layer, and a Sigmoid function to obtain spatial attention; use multiplication to obtain output features after multiplying with the input features; S332, build branch 1: use a spatial attention module and a 1×1 convolution layer to generate multi-channel rain features; S333, construct branch 2: use a 1×1 convolution layer to generate single-channel background features; S334, construct branch three: use a 7×7 convolution layer and a tanh activation function layer to generate image content features; S335, concatenate the rain feature and background feature, and use the softmax function to generate rain attention and background attention; S336, using multiplication, multiplies the rain-containing attention and the image content features to generate a rain-containing attention map; multiplies the background features and the input rainy image to generate a background attention map; S337, using addition, the rain-containing attention map is added to the background attention map to generate a rain-free image.

Images