Image deraining method, system, electronic device and storage medium

Abstract

The present invention relates to the technical field of image processing, and in particular to an image deraining method, system, electronic device and storage medium. The image deraining method comprises: obtaining a rain streak feature map, a background feature map and a residual channel map of an initial rainy day image; fusing the rain streak feature map, the background feature map and the initial rainy day image to obtain a preliminary derained image, and fusing the preliminary derained image with the residual channel map to obtain a target derained image. The present invention improves the clarity restoration effect of rainy day images in real scenes, so that the result after deraining is closer to a real rain-free image, thereby better meeting the image requirements for clarity and accuracy.

Description

Background Art

[0002] When surveillance systems are installed outdoors, they face the challenge of changing weather conditions, especially the impact of rainy days. On rainy days, raindrops block the quality of images captured by surveillance cameras, reducing the effectiveness of surveillance systems. In recent years, deep learning has been widely used in image deraining tasks and has achieved impressive performance. However, deep learning-based deraining methods have poor generalization ability and limited clarity restoration effects on rainy day images in real scenes, making it difficult to meet the requirements of surveillance systems for clarity and accuracy.

[0003] Therefore, in order to improve the availability and performance of surveillance systems on rainy days, a technology that can effectively restore the quality of rainy day images is urgently needed. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides an image deraining method, system, electronic device and storage medium.

[0005] In a first aspect, the present invention provides an image deraining method, and the technical solution of the method is as follows:

[0006] Obtain rain streak feature map, background feature map and residual channel map of the initial rainy day image;

[0007] The rain streak feature map, the background feature map and the initial rainy day image are fused to obtain a preliminary derained image, and the preliminary derained image is fused with the residual channel map to obtain a target derained image.

[0008] The beneficial effects of an image deraining method of the present invention are as follows:

[0009] The method of the present invention improves the clarity restoration effect of rainy day images in real scenes, so that the result after rain removal is closer to a real rain-free image, thereby better meeting the image requirements for clarity and accuracy.

[0010] Based on the above solution, an image deraining method of the present invention can also be improved as follows.

[0011] In an optional manner, the step of obtaining a rain streak feature map of an initial rainy day image includes: splitting the initial rainy day image into three-channel grayscale images, and extracting the grayscale feature maps of the three-channel grayscale images respectively by convolution processing; using an attention mechanism to process the grayscale feature map of the three-channel grayscale image to obtain a rain streak attention map; and fusing the convolution feature map obtained by convolution processing the initial rainy day image with the rain streak attention map to obtain the rain streak feature map.

[0012] In the above optional method, by splitting the initial rainy day image into three-channel grayscale images and extracting grayscale feature maps respectively, and then combining the attention mechanism, the characteristics of rain streaks can be captured more accurately, thereby improving the accuracy and efficiency of the subsequent rain removal step.

[0013] In an optional manner, the step of obtaining a background feature map of an initial rainy day image includes: downsampling the initial rainy day image to obtain multiple downsampled images; using deep convolution to extract local features of any downsampled image to obtain local background features, and using dilated convolution to extract global features of any downsampled image to obtain global background features; using channel attention to fuse the local background features with the global background features to obtain a predicted background feature map of any downsampled image, until the predicted background feature maps of the multiple downsampled images are obtained; fusing the predicted background feature maps of the multiple downsampled images and the predicted background feature map corresponding to the initial rainy day image to obtain the background feature map of the initial rainy day image.

[0014] In the above optional method, by combining downsampling and deep convolution with dilated convolution, local and global features of the image can be extracted simultaneously, and then the channel attention mechanism is used for feature fusion, which can more comprehensively retain the background information of the image, and is conducive to retaining background details in the subsequent rain removal process.

[0015] In an optional manner, the step of obtaining the residual channel map of the initial rainy day image includes: performing residual channel priori processing on the initial rainy day image to obtain the residual channel map.

[0016] In the above optional method, the residual channel prior processing is performed on the initial rainy day image to obtain the residual channel map of the image, which can help to better remove rain streaks and retain the original information of the image through residual learning in the subsequent rain removal process.

[0017] In an optional manner, the step of fusing the rain streak feature map, the background feature map and the initial rainy day image to obtain a preliminary derained image includes: convolving the rain streak feature map and the background feature map to obtain an intermediate feature map, and fusing the intermediate feature map with the initial rainy day image to obtain a preliminary derained image.

[0018] In the above optional method, by first convolving the rain streak feature map and the background feature map to obtain an intermediate feature map, and then fusing it with the initial rainy day image, more background information can be retained while removing the rain streaks, thereby improving the deraining effect.

[0019] In an optional manner, the step of fusing the preliminary derained image with the residual channel map to obtain a target derained image includes: performing convolution processing on the residual channel map and the preliminary derained image respectively to obtain a residual channel feature map corresponding to the residual channel map and a preliminary derained feature map corresponding to the preliminary derained image; and performing feature fusion on the residual channel feature map and the preliminary derained feature map to obtain the target derained image.

[0020] In the above optional manner, by performing convolution processing on the residual channel map and the preliminary derained image respectively and then performing feature fusion, the information in the residual channel map can be used to further correct and optimize the preliminary derained image, thereby obtaining a more accurate derained result.

[0021] In a second aspect, the present invention provides an image deraining system, and the technical solution of the system is as follows:

[0022] The image deraining system comprises: an acquisition module and a fusion module;

[0023] The acquisition module is used to: acquire a rain streak feature map, a background feature map and a residual channel map of an initial rainy day image;

[0024] The fusion module is used to: fuse the rain streak feature map, the background feature map and the initial rainy day image to obtain a preliminary derained image, and fuse the preliminary derained image with the residual channel map to obtain a target derained image.

[0025] The beneficial effects of an image deraining system of the present invention are as follows:

[0026] The system of the present invention improves the clarity restoration effect of rainy day images in real scenes, so that the result after rain removal is closer to a real rain-free image, thereby better meeting the image requirements for clarity and accuracy.

[0027] Based on the above solution, an image deraining system of the present invention can also be improved as follows.

[0028] In an optional manner, the acquisition module is specifically used to:

[0029] Splitting the initial rainy day image into three-channel grayscale images, and extracting grayscale feature maps of the three-channel grayscale images respectively by using a convolution processing method;

[0030] Using the attention mechanism, the grayscale feature map of the three-channel grayscale image is processed to obtain a rain streak attention map;

[0031] The convolution feature map obtained by convolution processing the initial rainy day image is fused with the rain streak attention map to obtain the rain streak feature map.

[0032] In the above optional method, by splitting the initial rainy day image into three-channel grayscale images and extracting grayscale feature maps respectively, and then combining the attention mechanism, the characteristics of rain streaks can be more accurately captured, thereby improving the accuracy and efficiency of the subsequent rain removal step.

[0033] In a third aspect, a technical solution of an electronic device of the present invention is as follows:

[0034] The invention comprises a memory, a processor and a program stored in the memory and running on the processor. When the processor executes the program, the steps of the image deraining method of the present invention are realized.

[0035] In a fourth aspect, the present invention provides a computer-readable storage medium having the following technical solution:

[0036] Instructions are stored in the computer-readable storage medium. When the computer-readable storage medium reads the instructions, the computer-readable storage medium executes the steps of the image deraining method of the present invention.

[0037] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0039] Figure 1 A schematic diagram of a process of an image deraining method of the present invention;

[0040] Figure 2 It is a processing logic diagram of the rain streak extraction network in the present invention;

[0041] Figure 3 It is a processing logic diagram of the multi-scale convolutional network in the present invention;

[0042] Figure 4 A processing logic diagram of an image deraining method of the present invention;

[0043] Figure 5 This is a rain removal effect diagram of the present invention;

[0044] Figure 6 A schematic structural diagram of an embodiment of an image rain removal system of the present invention;

[0045] Figure 7The figure is a schematic structural diagram of an embodiment of an electronic device of the present invention. DETAILED DESCRIPTION

[0046] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0047] Figure 1 FIG. 1 is a flow chart of an image deraining method provided by the present invention, and the method is executed by an image deraining device. Figure 1 As shown, the following steps are included:

[0048] S1. Obtain the rain streak feature map, background feature map and residual channel map of the initial rainy day image.

[0049] In S1:

[0050] ① The initial rainy day image is the original rainy day image with rain streaks. The image is the image to be processed. The initial rainy day image contains rain streaks and background content. The clarity of the image needs to be optimized.

[0051] ②The rain streak feature map is an image extracted from the initial rainy day image that specifically describes the characteristics of rain streaks. The rain streak feature map extracted from the initial rainy day image through convolution processing and attention mechanism is used for subsequent deraining processing.

[0052] ③The background feature map is an image describing the background content extracted from the initial rainy day image. It is a feature map describing the background information obtained by downsampling the initial rainy day image, extracting local features and extracting global features, and combining the channel attention mechanism.

[0053] ④ The residual channel map is an image obtained from the initial rainy day image through residual channel prior processing. The residual channel map is a grayscale image with a clear target structure and is used for subsequent rain removal processing.

[0054] Specifically, the image deraining device obtains a rain streak feature map, a background feature map, and a residual channel map of an initial rainy day image.

[0055] S2. Fusing the rain streak feature map, the background feature map and the initial rainy day image to obtain a preliminary derained image, and fusing the preliminary derained image with the residual channel map to obtain a target derained image.

[0056] In S2:

[0057] ① The preliminary derained image is the intermediate result obtained by fusing the rain streak feature map, the background feature map and the initial rainy day image. The preliminary derained image has removed or weakened the rain streaks in the initial rainy day image to a certain extent, but it still contains some rain streaks that have not been completely removed. Therefore, the preliminary derained image is not the final derained result, but an intermediate image that needs further processing and optimization. In the subsequent steps, the preliminary derained image will be fused with the residual channel map to obtain the final target derained image.

[0058] ② The target derained image is the final image without rain streaks obtained by fusing the preliminary derained image with the residual channel map.

[0059] Specifically, the image deraining device further fuses the rain streak feature map, the background feature map and the initial rainy day image to obtain a preliminary derained image, and fuses the preliminary derained image with the residual channel map to obtain a target derained image.

[0060] The technical solution of this embodiment improves the clarity restoration effect of rainy day images in real scenes, making the result after rain removal closer to a real rain-free image, thereby better meeting the image requirements for clarity and accuracy.

[0061] In an optional manner, the step of obtaining the rain streak feature map of the initial rainy day image includes: splitting the initial rainy day image into three-channel grayscale images, and extracting the grayscale feature maps of the three-channel grayscale images respectively by convolution processing. Using an attention mechanism, the grayscale feature map of the three-channel grayscale image is processed to obtain a rain streak attention map. The convolution feature map obtained by convolution processing the initial rainy day image is fused with the rain streak attention map to obtain a rain streak feature map.

[0062] Figure 2 The processing logic diagram of the rain streak extraction network in the present invention is shown, wherein the color channels of the initial rainy day image are split to obtain the corresponding R, G, B three-channel grayscale images, and the convolutional layers are used to extract features for the obtained three-channel grayscale images to obtain the grayscale feature map I of the corresponding grayscale image. R ,I G ,I B .

[0063] The attention mechanism is used to estimate the feature similarity of the grayscale feature map of the three-channel grayscale image and generate the rain streak attention map. The expression of this process is as follows:

[0064] M streak =softmax(I R *Conv(I G ,I B ))

[0065] Among them, Conv represents the convolution layer; I R is the grayscale feature map of the R channel; I G is the grayscale feature map of the G channel; I B is the grayscale feature map of the B channel; M streak is the rain streak attention map; (I G ,I B ) indicates that I G and I B The first feature map obtained by channel concatenation; Conv(I G ,I B ) represents a second feature map obtained by convolution processing on the first feature map; I R *Conv(I G ,I B ) indicates that I R The third feature map obtained by pixel-by-pixel multiplication of the second feature map, softmax(I R *Conv(I G ,I B )) indicates that the third feature map is activated by the softmax activation function to extract the rain streak attention map M of similar rain streak feature maps between different channels streak .

[0066] It should be noted that first of all, G and I B Perform channel splicing to obtain the first feature map, and then G and I B The first feature map after channel splicing is convolved to obtain the second feature map, and then I R The third feature map is obtained by pixel-by-pixel multiplication with the second feature map. Finally, the third feature map is activated by the softmax activation function to extract the rain streak attention map M of similar rain streak feature maps between different channels. streak .

[0067] Specifically, the image deraining device further splits the initial rainy day image into three-channel grayscale images, and uses convolution processing to extract the grayscale feature maps of the three-channel grayscale images respectively, and uses the attention mechanism to process the grayscale feature maps of the three-channel grayscale images to generate a rain streak attention map, which can reflect similar rain streak features between different channels. The image deraining device then fuses the convolution feature map of the initial rainy day image with the rain streak attention map to obtain a rain streak feature map that only contains rain streak information. This process not only improves the accuracy of rain streak feature extraction, but also provides important basic information for subsequent image deraining processing.

[0068] In an optional manner, the step of obtaining a background feature map of an initial rainy day image includes: downsampling the initial rainy day image to obtain multiple downsampled images. Using deep convolution to extract local features of any downsampled image to obtain local background features, and using dilated convolution to extract global features of any downsampled image to obtain global background features. Using channel attention, the local background features and the global background features are fused to obtain a predicted background feature map of any downsampled image, until the predicted background feature maps of multiple downsampled images are obtained. The predicted background feature maps of multiple downsampled images and the predicted background feature map corresponding to the initial rainy day image are fused to obtain the background feature map of the initial rainy day image.

[0069] Figure 3 The processing logic diagram of the multi-scale convolutional network in this embodiment is shown, wherein the initial rainy day image is downsampled to obtain two downsampled images of sizes 1 / 2 and 1 / 4 respectively, which are recorded as the first downsampled image and the second downsampled image.

[0070] The background features of the first downsampled image and the second downsampled image are extracted using depth convolution and dilated convolution, respectively, so as to obtain local background features and global background features, respectively. Finally, the local background features and global background features are fused using channel attention to obtain the predicted background feature maps of the first downsampled image and the second downsampled image, respectively. The expression of this process is as follows:

[0071] I P =SCA(DWD_C(DW_C(I rain ))

[0072] Among them, I P Represents the predicted background feature map; I rain represents the initial rainy day image; DW_C represents deep convolution, which has a larger local receptive field, so that local feature information can be better extracted; DWD_C represents dilated convolution, which can obtain the global receptive field, which is crucial for background recovery; SCA represents channel attention, which is used to fuse local background features and global background features and generate a predicted background feature map; DW_C(I rain ) indicates that the background features of the initial rainy day image are extracted by deep convolution to obtain local background features; DWD_C(DW_C(I rain ) indicates that the local background features are extracted using dilated convolution to obtain the global background features; SCA(DWD_C(DW_C(I rain )) indicates that channel attention is used to fuse local background features and global background features to obtain a predicted background feature map.

[0073] It should be noted that the channel attention first performs a pooling operation on the global background features of size H×W×C to obtain a channel attention map of size 1×1×C, and then obtains the weights of different channels through convolutional layer, ReLU activation, convolutional layer, and softmax activation. Finally, the input global background features of size H×W×C are multiplied by the weights to obtain predicted background feature maps at different scales.

[0074] The predicted background feature maps of the multiple downsampled images and the predicted background feature map corresponding to the initial rainy day image are fused to obtain the background feature map of the initial rainy day image.

[0075] Specifically, the image deraining device can further extract the background feature map of the initial rainy day image through a multi-scale convolutional network. The multi-scale convolutional network first downsamples the initial rainy day image to obtain multiple downsampled images of different sizes. Then, the local features of any downsampled image are extracted by deep convolution to obtain local background features, and the global features of any downsampled image are extracted by dilated convolution to obtain global background features. Then, the local background features and the global background features are fused by channel attention to generate a predicted background feature map of any downsampled image until the predicted background feature maps of all downsampled images are obtained. Finally, the predicted background feature map of each downsampled image is fused to obtain a fused predicted background feature map, and then the fused predicted background feature map is fused with the predicted background feature map corresponding to the initial rainy day image to obtain the background feature map of the initial rainy day image. This process can effectively process rainy day images of different scales and improve the accuracy and efficiency of background recovery.

[0076] In an optional manner, the step of obtaining the residual channel map of the initial rainy day image includes: performing residual channel priori processing on the initial rainy day image to obtain the residual channel map.

[0077] Among them, the residual channel map is extracted through the residual channel prior, and the formula is as follows:

[0078] F rcp =max I (R,G,B) (x)-min I (R,G,B) (x)

[0079] Among them, F rcp is the residual channel map; x is the initial rainy day image; I (R,G,B) Represents the pixel value vector of the three channels of the initial rainy day image, that is, a three-channel grayscale image. The three values ​​of the vector correspond to the pixel values ​​of the three channels respectively; maxI (R,G,B) (x) and min I (R,G,B)(x) represents the maximum pixel value and the minimum pixel value of the three-channel grayscale image split by taking the initial rainy day image. Through this operation, a grayscale image with a clear target structure can be obtained, namely the residual channel image.

[0080] Specifically, the image deraining device further extracts the residual channel map of the initial rainy day image through residual channel prior processing. The purpose of this step is to separate the rain information from the initial rainy day image, so as to obtain a grayscale image containing only clear target structures, namely the residual channel map. This step enables the system to extract the residual channel map more accurately, providing important input information for subsequent image deraining processing.

[0081] In an alternative approach, Figure 4 As shown, the step of fusing the rain streak feature map, the background feature map and the initial rainy day image to obtain a preliminary derained image includes: convolving the rain streak feature map and the background feature map to obtain an intermediate feature map, and fusing the intermediate feature map with the initial rainy day image to obtain a preliminary derained image.

[0082] Specifically, the image deraining device further performs convolution processing on the rain streak feature map and the background feature map, and combines the rain streak information and the background information to obtain an intermediate feature map that includes the effect of removing rain streaks. Subsequently, the intermediate feature map is fused with the initial rainy day image, and the feature map with most rain streaks removed is combined with the original image to restore the original details and structure of the image, thereby obtaining a preliminary derained image. This step effectively solves the problem of rain streak interference in the image, improves the visual quality of the image and the accuracy of subsequent analysis.

[0083] In an alternative approach, Figure 4 As shown, the step of fusing the preliminary derained image with the residual channel map to obtain the target derained image includes: performing convolution processing on the residual channel map and the preliminary derained image respectively to obtain a residual channel feature map corresponding to the residual channel map and a preliminary derained feature map corresponding to the preliminary derained image. Performing feature fusion on the residual channel feature map and the preliminary derained feature map to obtain the target derained image.

[0084] Specifically, convolutional layers are used to extract features from the residual channel map and the preliminary derained image respectively, and finally channel connection and convolution operations are used to perform feature fusion to restore the over-removed background features similar to the rain streak features to obtain the target derained image.

[0085] Figure 5 This is a comparison diagram of the rain removal effect of the method of this embodiment and the prior art, where Figure 5 The left part is the initial rainy day image, located Figure 5 The right part shows the rain removal effect after being processed by the method of this embodiment;

[0086] Figure 6 FIG. 2 is a schematic diagram showing a structure of an embodiment of an image rain removal system 200 provided by the present invention. Figure 2 As shown, the system 200 includes: an acquisition module and a fusion module;

[0087] The acquisition module 210 is used to acquire a rain streak feature map, a background feature map and a residual channel map of an initial rainy day image.

[0088] The fusion module 220 is used to fuse the rain streak feature map, the background feature map and the initial rainy day image to obtain a preliminary derained image, and fuse the preliminary derained image with the residual channel map to obtain a target derained image.

[0089] The technical solution of this embodiment improves the clarity restoration effect of rainy day images in real scenes, making the result after rain removal closer to a real rain-free image, thereby better meeting the image requirements for clarity and accuracy.

[0090] In an optional manner, the acquisition module 210 is specifically used to:

[0091] The initial rainy day image is split into three-channel grayscale images, and the grayscale feature maps of the three-channel grayscale images are extracted by convolution processing. The grayscale feature maps of the three-channel grayscale images are processed using the attention mechanism to obtain the rain streak attention map. The convolution feature map obtained by convolution processing of the initial rainy day image is fused with the rain streak attention map to obtain the rain streak feature map.

[0092] The technical solution of this embodiment improves the clarity restoration effect of rainy day images in real scenes, making the result after rain removal closer to a real rain-free image, thereby better meeting the image requirements for clarity and accuracy.

[0093] The parameters and steps of each module in the image deraining system 200 of the present embodiment to implement the corresponding functions can refer to the parameters and steps in the above embodiment of the image deraining method, which will not be described in detail here.

[0094] like Figure 7 As shown, an electronic device 300 according to an embodiment of the present invention includes a processor 320, the processor 320 is coupled to a memory 310, and the memory 310 stores at least one computer program 330, and the at least one computer program 330 is loaded and executed by the processor 320, so that the electronic device 300 implements any of the above-mentioned image deraining methods, specifically:

[0095] The electronic device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors 320 (Central Processing Units, CPU) and one or more memories 310, wherein the one or more memories 310 store at least one computer program 330, and the at least one computer program 330 is loaded and executed by the one or more processors 320, so that the electronic device 300 implements any image deraining method provided in the above embodiments. Of course, the electronic device 300 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the electronic device 300 may also include other components for implementing device functions, which will not be described in detail here.

[0096] A computer-readable storage medium according to an embodiment of the present invention stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that a computer implements any of the above-mentioned image deraining methods.

[0097] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0098] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs any of the above-mentioned image deraining methods.

[0099] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects and represent the definition of a specific order or sequence. The order of use of similar objects can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0100] Those skilled in the art know that the present invention can be implemented as a system, method or computer program product. Therefore, the present disclosure can be specifically implemented in the following forms, namely: it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.

[0101] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0102] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. An image deraining method, characterized in that: include: Obtain rain streak feature map, background feature map and residual channel map of the initial rainy day image; The rain streak feature map, the background feature map and the initial rainy day image are fused to obtain a preliminary derained image, and the preliminary derained image is fused with the residual channel map to obtain a target derained image; The steps of obtaining the rain streak feature map of the initial rainy day image include: Splitting the initial rainy day image into three-channel grayscale images, and extracting grayscale feature maps of the three-channel grayscale images respectively by using a convolution processing method; The steps of splitting the initial rainy day image into three-channel grayscale images and respectively extracting grayscale feature maps of the three-channel grayscale images by using a convolution processing method include: The color channels of the initial rainy day image are split to obtain corresponding R, G, and B three-channel grayscale images. The three-channel grayscale images are respectively subjected to feature extraction using a convolutional layer to obtain the grayscale feature map I corresponding to the grayscale image. R ,I G ,I B ; Using the attention mechanism, the grayscale feature map of the three-channel grayscale image is processed to obtain a rain streak attention map; The step of processing the grayscale feature map of the three-channel grayscale image by using the attention mechanism to obtain the rain streak attention map comprises: The attention mechanism is used to estimate the feature similarity of the grayscale feature map of the three-channel grayscale image, and generate the rain streak attention map; this process is expressed as follows: M streak =softmax(I R *Conv(I G ,I B )) Wherein, Conv represents the convolutional layer; I R is the grayscale feature map of the R channel; I G is the grayscale feature map of the G channel; I B is the grayscale feature map of the B channel; M streak is the rain streak attention map; (I G ,I B ) indicates that I G and I B The first feature map obtained by channel concatenation; Conv(I G ,I B ) represents a second feature map obtained by performing convolution processing on the first feature map; I R *Conv(I G ,I B ) indicates that I R The third feature map obtained by pixel-by-pixel multiplication of the second feature map, softmax(I R *Conv(I G ,I B )) represents activating the third feature map using a softmax activation function to extract the rain streak attention map M of similar rain streak feature maps between different channels streak ; fusing the convolution feature map obtained by convolution processing the initial rainy day image with the rain streak attention map to obtain the rain streak feature map; The steps of obtaining the background feature map of the initial rainy day image include: Downsampling the initial rainy day image to obtain a plurality of downsampled images; The step of downsampling the initial rainy day image to obtain a plurality of downsampled images comprises: Downsampling the initial rainy day image to obtain a first downsampled image having a size of 1 / 2 of the initial rainy day image and a second downsampled image having a size of 1 / 4 of the initial rainy day image; Extracting local features of any downsampled image using deep convolution to obtain local background features, and extracting global features of any downsampled image using dilated convolution to obtain global background features; Using channel attention, the local background features and the global background features are fused to obtain a predicted background feature map of any down-sampled image, until the predicted background feature maps of the multiple down-sampled images are obtained; The step of fusing the local background features and the global background features by using channel attention to obtain a predicted background feature map of any down-sampled image, until the predicted background feature maps of the multiple down-sampled images are obtained, comprises: The local background features and the global background features are fused using the channel attention to obtain predicted background feature maps of the first down-sampled image and the second down-sampled image respectively; the process expression is as follows: I P =SCA(DWD_C(DW_C(I rain )) Among them, I P I represents the predicted background feature map; rain represents the initial rainy day image; DW_C represents the depth convolution; DWD_C represents the dilated convolution; SCA represents the channel attention; DW_C(I rain ) represents the use of the depth convolution to extract background features from the initial rainy day image to obtain the local background features; DWD_C(DW_C(I rain ) represents the use of the dilated convolution to extract the background features of the local background features to obtain the global background features; SCA(DWD_C(DW_C(I rain )) represents using the channel attention to fuse the local background features and the global background features to obtain the predicted background feature map; The predicted background feature maps of the multiple down-sampled images and the predicted background feature map corresponding to the initial rainy day image are fused to obtain the background feature map of the initial rainy day image.

2. The image deraining method according to claim 1, characterized in that: The steps of obtaining the residual channel map of the initial rainy day image include: The residual channel priori processing is performed on the initial rainy day image to obtain the residual channel map.

3. The image deraining method according to claim 1, characterized in that: The step of fusing the rain streak feature map, the background feature map and the initial rainy day image to obtain a preliminary rain-removed image comprises: The rain streak feature map and the background feature map are convolved to obtain an intermediate feature map, and the intermediate feature map is fused with the initial rainy day image to obtain a preliminary rain-removed image.

4. The image deraining method according to claim 1, characterized in that: The step of fusing the preliminary derained image with the residual channel map to obtain a target derained image comprises: Performing convolution processing on the residual channel map and the preliminary derained image respectively to obtain a residual channel feature map corresponding to the residual channel map and a preliminary derained feature map corresponding to the preliminary derained image; Feature fusion is performed on the residual channel feature map and the preliminary rain removal feature map to obtain the target rain removal image.

5. An image deraining system, characterized in that: include: Acquisition module and fusion module; The acquisition module is used to: acquire a rain streak feature map, a background feature map and a residual channel map of an initial rainy day image; The fusion module is used to: fuse the rain streak feature map, the background feature map and the initial rainy day image to obtain a preliminary derained image, and fuse the preliminary derained image with the residual channel map to obtain a target derained image; The acquisition module is specifically used for: Splitting the initial rainy day image into three-channel grayscale images, and extracting grayscale feature maps of the three-channel grayscale images respectively by using a convolution processing method; The steps of splitting the initial rainy day image into three-channel grayscale images and respectively extracting grayscale feature maps of the three-channel grayscale images by using a convolution processing method include: The color channels of the initial rainy day image are split to obtain corresponding R, G, and B three-channel grayscale images. The three-channel grayscale images are respectively subjected to feature extraction using a convolutional layer to obtain the grayscale feature map I corresponding to the grayscale image. R ,I G ,I B ; Using the attention mechanism, the grayscale feature map of the three-channel grayscale image is processed to obtain a rain streak attention map; The step of processing the grayscale feature map of the three-channel grayscale image by using the attention mechanism to obtain the rain streak attention map comprises: The attention mechanism is used to estimate the feature similarity of the grayscale feature map of the three-channel grayscale image, and generate the rain streak attention map; this process is expressed as follows: M streak =softmax(I R *Conv(I G ,I B )) Wherein, Conv represents the convolutional layer; I R is the grayscale feature map of the R channel; I G is the grayscale feature map of the G channel; I B is the grayscale feature map of the B channel; M streak is the rain streak attention map; (I G ,I B ) indicates that I G and I B The first feature map obtained by channel concatenation; Conv(I G ,I B ) represents a second feature map obtained by performing convolution processing on the first feature map; I R *Conv(I G ,I B ) indicates that I R The third feature map obtained by pixel-by-pixel multiplication of the second feature map, softmax(I R *Conv(I G ,I B )) represents activating the third feature map using a softmax activation function to extract the rain streak attention map M of similar rain streak feature maps between different channels streak ; fusing the convolution feature map obtained by convolution processing the initial rainy day image with the rain streak attention map to obtain the rain streak feature map; The steps of obtaining the background feature map of the initial rainy day image include: Downsampling the initial rainy day image to obtain a plurality of downsampled images; The step of downsampling the initial rainy day image to obtain a plurality of downsampled images comprises: Downsampling the initial rainy day image to obtain a first downsampled image having a size of 1 / 2 of the initial rainy day image and a second downsampled image having a size of 1 / 4 of the initial rainy day image; Extracting local features of any downsampled image using deep convolution to obtain local background features, and extracting global features of any downsampled image using dilated convolution to obtain global background features; Using channel attention, the local background features and the global background features are fused to obtain a predicted background feature map of any down-sampled image, until the predicted background feature maps of the multiple down-sampled images are obtained; The step of fusing the local background features and the global background features by using channel attention to obtain a predicted background feature map of any down-sampled image, until the predicted background feature maps of the multiple down-sampled images are obtained, comprises: The local background features and the global background features are fused using the channel attention to obtain predicted background feature maps of the first down-sampled image and the second down-sampled image respectively; the process expression is as follows: I P =SCA(DWD_C(DW_C(I rain )) Among them, I P I represents the predicted background feature map; rain represents the initial rainy day image; DW_C represents the depth convolution; DWD_C represents the dilated convolution; SCA represents the channel attention; DW_C(I rain ) represents the use of the depth convolution to extract background features from the initial rainy day image to obtain the local background features; DWD_C(DW_C(I rain ) represents the use of the dilated convolution to extract the background features of the local background features to obtain the global background features; SCA(DWD_C(DW_C(I rain )) represents using the channel attention to fuse the local background features and the global background features to obtain the predicted background feature map; The predicted background feature maps of the multiple down-sampled images and the predicted background feature map corresponding to the initial rainy day image are fused to obtain the background feature map of the initial rainy day image.

6. A computer device, characterized in that: The computer device includes a processor, the processor is coupled to a memory, at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor, so that the computer device implements the image deraining method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by the processor, so that the computer-readable storage medium implements the image deraining method according to any one of claims 1 to 4.

Images