A multi-module cooperative rain removal method and system for rainy day vehicle detection

Abstract

The application discloses a multi-module cooperative rain removal method and system for rainy day vehicle detection, and the multi-module cooperative rain removal method is given a rainy image as input. Convolution layers are used to complete original noise filtering, and a progressive information perception module is constructed to extract spatial details and output high-priority information. The features are down-sampled in the encoding unit to gradually reduce the resolution to a low resolution, and then mapped to the original high resolution by the decoding unit after intermediate layer processing, and finally output the features. After the features are mapped by a 3*3 convolution layer, a residual image is generated, and the final rain removal result is obtained by addition.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a multi-module cooperative rain removal method and system for rainy day vehicle detection. BACKGROUND

[0002] The current research on intelligent vehicles is becoming the focus, and intelligent vehicle technology plays an important role in ensuring traffic safety and promoting urban development. Among the many key subsystems of intelligent vehicles, the vehicle detection system plays a role in identifying vehicle targets in the traffic scene and calculating their spatial positions, so ensuring the stable operation of the vehicle detection system is of great significance to the braking and obstacle avoidance functions of intelligent vehicles.

[0003] Among the many vehicle detection methods, computer vision-based vehicle detection methods are more widely used than other methods due to their lower cost. However, the actual traffic environment is complex and variable, so it is necessary to continuously optimize the performance of benchmark vehicle detection methods represented by SSD and YOLO series models to ensure the safe driving of intelligent vehicles in various working conditions. To consolidate and improve the performance of vehicle detection methods, the current conventional approach is to use a general and powerful benchmark vehicle detection method as the underlying architecture, and then continuously optimize various components of the benchmark vehicle detection method to meet the needs of various detection tasks. Liu et al. improved the accuracy of SSD in detecting distant vehicles by designing feature fusion. Xu et al. improved the vehicle detection accuracy of YOLOX-S based on the strategy of integrating attention mechanisms and optimizing loss functions. At the same time, some scholars are committed to improving the accuracy of benchmark vehicle detection methods in specific scenarios. Ning et al. improved the vehicle detection performance of YOLOv3 in foggy scenarios by fusing DenseNet and attention mechanisms. Zhang et al. improved the vehicle detection accuracy of RetinaNet in night scenarios based on the method of optimizing loss functions and network structures. Although the above optimization methods improve the detection performance of benchmark vehicle detection methods, they may not be the optimal solution for intelligent vehicles. As we all know, images generated by vehicle-mounted cameras in complex scenarios may be degraded to varying degrees, which fundamentally restricts the vehicle feature information contained in the images. In the case of rainy scenarios, raindrops attached to the lens or windshield glass will directly interfere with the visibility of the background, raindrops suspended in the air will scatter the light, and even when the rain is heavier, dense rain lines will converge into obvious rain streaks and rain fog in the air, thereby blocking various objects in the vehicle background layer. These interference factors will cause problems such as loss of texture details and degradation of edge information in rainy images, which will reduce the accuracy of vehicle detection and directly threaten the safe driving of intelligent vehicles. Therefore, strengthening image rain removal research to enrich the feature information of the vehicle background layer is of great significance to improving the performance of vehicle detection methods.

[0004] Image deraining is a classic image restoration task, and it is becoming increasingly popular due to the growing demand for image quality in industries such as smart logistics and intelligent vehicles. Early image deraining methods were mostly model-driven, designing deraining models based on prior knowledge of the image, such as rain streak direction and density. Methods included discriminative sparse coding and Gaussian mixture models, but these methods did not consider other factors such as rain and fog. Benefiting from the rapid development of artificial intelligence, data-driven deraining methods have become mainstream. Li et al. proposed a nonlocal enhancement encoder-decoder architecture to learn complex features, thus accurately modeling rain streaks while preserving image details. Zhang et al. constructed a rain density-aware deraining method. Ren et al., considering network structure and loss function, built a simple yet efficient baseline deraining method. While these methods can achieve image deraining, they suffer from issues such as incomplete deraining leading to residual rain layers and over-deraining causing background damage. The MPRNet proposed by Zamir et al. decomposes the rain removal process into multiple stages to achieve progressive rain removal. Simultaneously, it extracts deep features based on channel attention and establishes a cross-stage feature fusion mechanism to enhance information interaction between stages. Therefore, MPRNet effectively removes rain while mitigating rain layer residue and background damage. However, the complex structural design of MPRNet leads to an increase in the number of network parameters and a longer runtime.

[0005] In summary, the various shortcomings of existing rain removal methods limit their direct application in vehicle inspection during rainy weather. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a multi-module collaborative rain removal method and system for vehicle detection in rainy weather, which addresses the shortcomings of the prior art and solves the technical problem of the decline in the accuracy of the benchmark vehicle detection method in rainy weather.

[0007] The present invention adopts the following technical solution:

[0008] A multi-module collaborative rain removal method for vehicle detection in rainy weather includes the following steps:

[0009] S1. Given a rainy image from a multi-module collaborative rain removal method, the original noise is filtered through convolutional layers. A progressive information perception module is constructed to enrich the extraction of spatial details and convey high-priority information, outputting features. ;

[0010] S2, Features After the encoding unit is downsampled and progressively reduced to a lower resolution, it is processed by intermediate layers and then mapped back to the original high resolution by the decoding unit, outputting the features. ;

[0011] S3、characteristics After mapping by a 3x3 convolutional layer, a residual image is generated, and the final rain removal result is obtained by adding it to the input image. .

[0012] Preferably, the characteristics are:

[0013]

[0014] wherein, represents a 3x3 convolutional layer responsible for dimensionality increasing operation of the input image, represents two consecutive progressive information perception modules, is the feature output by the shallow feature extraction unit.

[0015] Preferably, the progressive information perception module comprises:

[0016] a low-level detail information extraction module, which embeds the feature after layer normalization processing to obtain a normalized feature , then completes cross-dimensional information interaction by a convolutional layer, and performs channel-level equal split on ; based on the convolutional layer to complete feature extraction, without processing, directly passed to the next layer; and and are integrated at the channel level and refined by a convolutional layer; based on GeLU to generate an attention map and obtain a representative detail feature representation; the output feature of the low-level detail information extraction module is obtained by element addition.

[0017] a high-level semantic information extraction module, which outputs a feature after layer normalization processing to obtain a normalized feature , then sequentially completes deep feature extraction by multiple convolutional layers, the SCA adaptively completes the calibration of the rain layer information with higher priority; the feature is further refined by a convolutional layer, and the output feature is obtained by element addition.

[0018] Preferably, the output feature is:

[0019]

[0020] wherein, represents the feature of attention feedback, represents a 1x1 convolutional layer.

[0021] Preferably, the characteristics​ is:

[0022]

[0023] wherein, is the feature of the intermediate layer after processing, is a decoding unit in the encoder-decoder architecture.

[0024] Preferably, a multi-level feature capture module is embedded in each level of the encoder-decoder architecture, and the information flow process is as follows:

[0025] The embedded features are preprocessed based on layer normalization to stabilize the training process; then a convolutional layer is used for dimension transformation; a deep separable convolutional layer is used for deep feature extraction, and by aggregating three-dimensional attention, it can adaptively guide the preservation and transmission of rain layer information with higher priority, and further refined by a convolutional layer.

[0026] Preferably, the multi-level feature capture module aggregates a double-branch structure under the premise of acceptable computational overhead to explore rain layer features of different levels and different forms, as follows:

[0027]

[0028] wherein, represents a 3x3 deep separable convolutional layer, represents GeLU activation, and are the output features of the upper branch and the lower branch, respectively, represents element-wise addition, is the feature after fusion of the two branches.

[0029] Preferably, the high-quality rain-free image is:

[0030]

[0031] Preferably, the Charbonnier loss and edge loss are fused to complete the multi-module collaborative rain removal network training, and the total loss function is defined as is:

[0032]

[0033] wherein, is set to 0.05 by experience, is the edge loss, is the Charbonnier loss, represents the predicted rain-free image, represents the original rain-free image.​

[0034] In a second aspect, an embodiment of the present application provides a multi-module collaborative rain removal system for rainy day vehicle detection, comprising:

[0035] A shallow feature extraction unit, given a rainy image input to the multi-module collaborative rain removal method, filters the original noise through a convolution layer, constructs a progressive information perception module, and outputs features .

[0036] A deep feature extraction unit, the features are down-sampled by the encoding unit to be gradually reduced to a low resolution, and after intermediate layer processing, are mapped to the original high resolution by the decoding unit, and output features .

[0037] A rain removal result output unit, the features are mapped to generate a residual image after a 3x3 convolution layer, and the final rain removal result is obtained by adding the residual image and the features .

[0038] In a third aspect, a computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the multi-module collaborative rain removal method for rainy day vehicle detection described above when executing the computer program.

[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium comprising a computer program, which, when executed by a processor, implements the steps of the multi-module collaborative rain removal method for rainy day vehicle detection described above.

[0040] In a fifth aspect, a chip comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the multi-module collaborative rain removal method for rainy day vehicle detection described above when executing the computer program.

[0041] In a sixth aspect, an embodiment of the present application provides an electronic device comprising a computer program, which, when executed by the electronic device, implements the steps of the multi-module collaborative rain removal method for rainy day vehicle detection described above.

[0042] Compared with the prior art, the present application has at least the following beneficial effects:

[0043] A multi-module collaborative rain removal method for rainy day vehicle detection can efficiently remove rain layers and restore vehicles in images, and effectively improve the rainy day detection accuracy of various benchmark vehicle detection algorithms.

[0044] Further, given the rainy image of the input multi-module collaborative rain removal method, the original noise is filtered through the convolution layer, and the progressive information perception module is constructed for rich spatial detail extraction and high-priority information transmission, and the output feature The progressive information perception module works collaboratively through multi-scale processing and attention mechanism, enhancing the model's ability to recognize rain streak features. Multi-scale processing enables the model to capture raindrop and rainline features at different scales, which behave differently at different resolutions. Through multi-scale analysis, the distribution and morphology of rain streaks can be more comprehensively understood. The attention mechanism further enhances the model's ability to focus on key information, such as the edges and shapes of raindrops, while suppressing less important background information. The benefit of this design is that it not only improves rain removal effects, but also preserves image details and texture information. By progressively fusing features at different scales, the model can more accurately reconstruct rain-free images while reducing artifacts and blurring that may occur during the rain removal process. In addition, the preferential transmission of high-priority information means that the model can respond and process the most critical features more quickly, which is particularly important for real-time or near-real-time rain removal applications.

[0045] Further, the progressive information perception module has the advantage that the progressive information perception module includes a low-level detail information extraction module and a high-level semantic information extraction module, which mainly contribute to representative feature representation.

[0046] Further, the feature The encoding unit is gradually reduced in resolution by downsampling to a low resolution, and after intermediate layer processing, the decoding unit maps the low resolution to the original high resolution, and outputs the feature Through downsampling, the spatial dimension of the data can be reduced, which not only reduces the computational complexity of the model, but also helps the model to extract more abstract features, which are crucial for understanding the global structure of the data. At the same time, the expansion of the receptive field during downsampling allows the model to capture more extensive contextual information, which is particularly important for image recognition and other tasks. Secondly, although the spatial resolution is reduced during downsampling, the model preserves important features and spatial information of the input data through parameter sharing and convolution operations. The preservation of this information is crucial for subsequent tasks such as image segmentation or object detection. After intermediate layer processing, the decoding unit maps the low resolution feature map back to the original high resolution. This step allows the model to fuse feature information at different scales, enhancing the model's ability to capture details. This feature fusion not only improves the performance of the model, but also helps to restore spatial details that may be lost during downsampling.

[0047] Further, the multi-level feature capture module can explore the context information of the rain layer in a large receptive field and its bidirectional disturbance with the vehicle background layer.

[0048] Further, the total loss function can enhance the tolerance to errors and further improve the quality of the rain-removed image.

[0049] It can be understood that the beneficial effects of the above-mentioned second aspect can be referred to the related description in the above-mentioned first aspect, which will not be repeated here.

[0050] In summary, the present application can efficiently eliminate the rain layer and restore the vehicle in the image, and effectively improve the detection accuracy of various reference vehicle detection methods in rainy days.

[0051] The technical solutions of the present application will be further described in detail below with the help of the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 The method flowchart of the present application is shown in the figure;

[0053] Figure 2 The multi-level feature capture module diagram is shown in the figure;

[0054] Figure 3 The progressive information perception module diagram is shown in the figure;

[0055] Figure 4 The simplified channel attention diagram is shown in the figure;

[0056] Figure 5 The gain effect diagram of the algorithm proposed in the rain streak and rain fog scene on the reference YOLOv8m is shown in the figure;

[0057] Figure 6 The schematic diagram of the computer device provided by an embodiment of the present application is shown in the figure;

[0058] Figure 7 The block diagram of a chip provided by the present application according to an embodiment is shown in the figure. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present application will be described clearly and completely below with the help of the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0060] In the description of the present application, it should be understood that the terms "include" and "contain" indicate the existence of described features, whole, steps, operations, elements and / or components, but do not exclude the existence or addition of one or more other features, whole, steps, operations, elements, components and / or sets thereof.

[0061] It should also be understood that the terms used in the specification and the following claims are for the purpose of describing particular embodiments and are not intended to be limiting, as the specific scope of the invention is disclosed in the appended claims. As used in this specification and the appended claims, the singular forms "a," "an" and "the" encompass plural referents unless the context clearly dictates otherwise.

[0062] It should also be further understood that the term "and / or" as used in the specification and the following claims, means any one of the items, or combinations of items, listed are possible and includes all possible combinations, for example A and / or B can mean A alone, B alone, or A and B together. In addition, the character " / " in the present invention generally represents a "or" relationship between the objects before and after it.

[0063] It should be understood that, although the terms first, second, third, etc. can be used herein to describe various ranges or elements, these ranges or elements should not be limited by these terms. These terms are only used to distinguish one range or element from another. For example, a first range could be termed a second range without departing from the scope of the embodiments.

[0064] The word "if" can be interpreted to mean "upon" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [a stated condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]," depending on the context.

[0065] Various structural diagrams according to the disclosed embodiments of the present application are shown in the accompanying drawings. These diagrams are not drawn to scale, in which certain details are shown in a somewhat exaggerated manner for the purpose of clarity and understanding, and certain details can be omitted. The shapes and relative sizes of the various regions, layers, and their relative positions shown in the drawings are merely exemplary, and in actuality can be deviated due to manufacturing tolerances or technical limitations, and regions / layers with different shapes, sizes, and relative positions can be additionally designed by those skilled in the art according to actual needs.

[0066] The application provides a multiple block collaborative deraining net (MBCDNet) for rainy day vehicle detection, builds a multi-level feature capturing module, and is used for the insight of the context information of the rain layer in a large receptive field; an incremental information perception module is constructed, which contributes to the representation of representative features; efficient identification and removal of the rain layer, clear positioning and restoration of the vehicle background layer are realized. In the aspect of image deraining, the overall evaluation results in the Test1200 and Test2800 public synthetic data sets show that, under the condition that the image PSNR and SSIM after deraining of the method have some advantages compared with MPRNet, the average running time is reduced by 30.5% and 33.6% compared with MPRNet, which proves the advancement and real-time performance of the method. In the aspect of rainy day vehicle detection, the test results in the Rain-Vehicle self-built data set show that the mAP@0.5 gain of the method to the benchmark SSD, YOLOv5m, YOLOv8n and YOLOv8m is 6.1%, 8.8%, 12.4% and 7.8% respectively, which proves the effectiveness of the method for rainy day vehicle detection and has certain practical value.

[0067] According to the design principle of light weight, the network core module is assembled only based on two kinds of modules in the single-stage architecture. Considering that the rain layer in the rainy image has various shapes and complex positions, the MBCDNet builds a multi-level feature capturing module to understand the context information of the rain layer in a large-scale receptive field, and constructs an incremental information perception module to strengthen the representation of representative features in the redundant information flow. The two modules cooperate with each other, embed different units and flexibly cooperate to represent the context information and spatial features of the rain layer, so as to guide the MBCDNet to complete efficient identification and removal of the rain layer, clear positioning and restoration of the vehicle background layer.

[0068] The multiple block collaborative deraining net for rainy day vehicle detection comprises a shallow feature extraction unit and a deep feature extraction unit. The shallow feature extraction unit enhances the preservation of the spatial features of the rain layer. The deep feature extraction unit strengthens the accurate perception and efficient transmission of the context information and structural features of the rain layer. In addition, the encoder and the decoder correspond to each other through long skip connection to establish interaction and enrich the spatial features. The specific steps are as follows:

[0069] S1, a rainy image is given to the multiple block collaborative deraining net, in the shallow feature extraction unit, the convolutional layer completes the original noise filtering, and the incremental information perception module is used for the extraction of spatial details and the transmission of information with high priority;

[0070] The features output by the shallow feature extraction unit is:

[0071] (1)

[0072] wherein, represents a 3x3 convolutional layer responsible for dimensionality increasing operation of the input image , represents two consecutive progressive information perception modules, is the feature output by the shallow feature extraction unit.

[0073] Progressive information perception module

[0074] The rain image collected by the camera contains various complex rain layers and vehicle background layers, and the performance mode and flow mode of rain streaks and rain fog in the air have significant differences, so the disturbance caused by them to the vehicle background layer also has its own characteristics. Therefore, the network should enhance the recognition and perception ability of different forms of rain layers, avoid the confusion of rain layers and vehicle background layers, and cause rain layer residues and excessive rain removal, which in turn leads to damage to the details of the vehicle background layer. In order to enable the network to efficiently explore and remove the redundant rain layer information in the rain image, while perfectly preserving the fine details of the vehicle background layer, the present application designs a progressive information perception module, as shown in Figure 3 .

[0075] The progressive information perception module contains a low-level detail information extraction module and a high-level semantic information extraction module, and is a feature extraction module of a Transformer-like architecture. Since the multi-level feature capture module has strong context information extraction capability, this module serves as a supplement and mainly contributes to representative feature representation. The conventional Transformer uses a self-attention mechanism to calculate the correlation, but its reasoning cost is difficult to ignore, and there is feature redundancy.

[0076] In particular, in order to improve the running efficiency of the network, the module does not choose to aggregate self-attention mechanism and its variants, but incorporates the spatial adaptive modulation mechanism and the basic concept of feature capture in Res2Net, thereby establishing a low-level detail information extraction module. At the same time, the aggregated simplified channel attention (SCA) constructs a high-level semantic information extraction module. Given the embedding features of the progressive information perception module, the detailed mapping process is represented by the following formula

[0077] (8)

[0078] wherein, is the embedding feature, and are the output features of the low-level detail information extraction module and the high-level semantic information extraction module, respectively, , representative feature operations of low-level detail information and high-level semantic information, respectively.

[0079] Specifically, in the low-level detail information extraction module of the progressive information perception module, the embedded features After layer normalization processing, the normalized features Then, the cross-dimensional information interaction is completed by the convolution layer

[0080] (9)

[0081] wherein, represents the features mapped by the 1x1 convolution layer.

[0082] Secondly, the channel-level equal split is performed on

[0083] (10)

[0084] wherein, represents channel segmentation, and represent the split features.

[0085] Next, Based on the convolution layer, the feature extraction is completed, without processing, directly passed to the next layer

[0086] (11)

[0087] wherein, represents the features mapped by the 3x3 depth separable convolution layer.

[0088] Subsequently, the channel-level integration is performed on and and refined by the convolution layer

[0089] (12)

[0090] wherein, represents channel-level splicing, represents the features refined by the 1x1 convolution layer.

[0091] After that, the attention map is generated based on GeLU and the representative detail feature representation is obtained

[0092] (13)

[0093] Finally, the output features of the low-level detail information extraction module are obtained by element addition.

[0094] ​ (14)

[0095] In particular, to strengthen the screening and transmission ability of the progressive information perception module for key rain layer features, the application designs a high-level semantic information extraction module. In this module, After layer normalization processing, the normalized features are obtained , and then deep feature extraction is completed by multiple convolution layers in turn

[0096] (15)

[0097] wherein, represents the deep semantic features.

[0098] Then, the SCA adaptively completes the calibration of rain layer information with higher priority

[0099] (16)

[0100] wherein, and represent the attention correction and its feedback features, respectively, and the SCA network structure is shown in Figure 4 .

[0101] Finally, the features are further refined by the convolution layer, and the output features are obtained by element addition

[0102] (17)

[0103] Based on the above structure and mapping process, the progressive information perception module, as a supplement to the multi-level feature capture module, realizes the exploration and perception of representative rain layer low-level detail information and high-level semantic information.

[0104] S2, in the deep feature extraction unit, the encoding unit is gradually reduced to low resolution by downsampling, and after intermediate layer processing, it is mapped to the original high resolution by the decoding unit;

[0105] (2)

[0106] wherein, and represent the encoding unit and the decoding unit in the encoder-decoder architecture, respectively, and are the features input and output of the encoding unit, is the feature after intermediate layer processing, is the feature output by the decoding unit, represents the intermediate layer composed of six consecutive progressive information perception modules.

[0107] Multi-level feature capturing module

[0108] There are complex spatial relationships in rainy images, such as a rain layer in a region may contain different depth information. Therefore, limited receptive fields may not be able to excavate and deliver sufficient information to enable the network to identify the relationship between rain layers and the belonging relationship of the rain layers, as well as the interaction between the rain layers and the vehicle background layer, which will lead to the residual rain marks in the restored image and the damage of the details of the vehicle background layer, which may eventually have an insurmountable disadvantage on the subsequent vehicle detection task. In summary, the complete removal of the rain layer and the complete recovery of the vehicle background layer must consider multi-scale information. Therefore, in order to further explore the context information of the rain layer in the large receptive field and its bidirectional disturbance with the vehicle background layer,

[0109] Please refer to Figure 2 , a multi-level feature capturing module is adopted to embed each level of the encoder-decoder architecture to understand the context information of the rain layer in the multi-scale space, while also dynamically adjusting the spatial features transmitted by the encoding unit to the decoding unit through the skip connection. Given the embedded features, the information flow process in the module is as follows:

[0110] First, given the effectiveness of layer normalization has been proven in multiple image restoration studies, the embedded features are preprocessed based on layer normalization to stabilize the training process

[0111] (4)

[0112] wherein, is the embedded feature, and represent the layer normalization operation and its normalized features, respectively.

[0113] Then use the convolution layer to change the dimension of

[0114] (5)

[0115] wherein, and represent the 1x1 convolution layer and its mapped features, respectively.

[0116] Next, in order to meet the design requirements of lightweight, deep separable convolution layers are used for deep feature extraction. At the same time, this module aggregates a double-branch structure to explore different levels and different forms of rain layer features under the premise of acceptable computational overhead

[0117] (6)

[0118] wherein, represents a 3x3 deep separable convolution layer,​ represent GeLU activation, and are the output features of the upper branch and the lower branch, respectively, represent element-wise addition, is the feature after the fusion of the two branches.

[0119] Subsequently, the three-dimensional attention is aggregated to adaptively guide the preservation and transmission of rain layer information with higher priority, and the convolution layer is further refined

[0120] (7)

[0121] wherein, represents a three-dimensional attention mechanism, is the rain layer feature finally mined by the multi-level feature capture module.

[0122] S3, in the rain removal result output unit, a residual image is generated after mapping through a 3x3 convolution layer, and then the residual image is added to the rain layer feature to obtain the final rain removal result.

[0123] high-quality rain-free image is:

[0124] (3)

[0125] wherein, is the high-quality rain-free image finally output by the method.

[0126] loss function

[0127] In order to further improve the quality of the image after rain removal, the Charbonnier loss and the edge loss are fused to complete the network training, the Charbonnier loss , and the edge loss are respectively defined as

[0128] (18)

[0129] (19)

[0130] wherein, represents a Laplacian operator, represents a predicted rain-free image, represents an original rain-free image, is set to 1x10 -3 .

[0131] total loss function is defined as

[0132] ​ (20)

[0133] wherein, is empirically set to 0.05.

[0134] In still another embodiment of the present application, a multi-module collaborative rain removal system for rainy day vehicle detection is provided, which can be used to implement the multi-module collaborative rain removal method for rainy day vehicle detection described above. Specifically, the multi-module collaborative rain removal system for rainy day vehicle detection includes a shallow feature extraction unit, a deep feature extraction unit, and a rain removal result output unit.

[0135] The shallow feature extraction unit is configured to input a rainy image of the multi-module collaborative rain removal method, filter out original noise through a convolution layer, construct a progressive information perception module, extract spatial details, and output features .

[0136] The deep feature extraction unit is configured to extract features The encoding unit is down-sampled to a low resolution, processed by an intermediate layer, mapped to an original high resolution by a decoding unit, and output features .

[0137] The rain removal result output unit is configured to output features After being mapped by a 3x3 convolution layer, a residual image is generated, and a final rain removal result is obtained by adding the residual image and the rain removal result of the shallow feature extraction unit.

[0138] In still another embodiment of the present application, a terminal device is provided, which includes a processor and a memory. The memory is configured to store a computer program, and the computer program includes program instructions. The processor is configured to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor is the computing core and control core of the terminal, and is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement a corresponding method process or a corresponding function. The processor described in the embodiments of the present application can be used for the operation of the multi-module collaborative rain removal method for rainy day vehicle detection, including: ​

[0139] Given the rainy image of the input multi-module collaborative rain removal method, the original noise is filtered through the convolution layer, the progressive information perception module is constructed, which is used to enrich the extraction of spatial details and the transmission of high-priority information, and the feature ; Feature The encoding unit is gradually reduced to low resolution by downsampling, and is mapped to the original high resolution by the decoding unit after intermediate layer processing, and the feature ; Feature The residual image is generated after being mapped by the 3x3 convolution layer, and the final rain removal result is obtained by adding .

[0140] Please refer to Figure 6 , the terminal device is a computer device, the computer device 60 of the embodiment includes: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61, the computer program 63 is executed by the processor 61 to realize the multi-module collaborative rain removal method for rain vehicle detection in the embodiment, to avoid repetition, not here. Or, the computer program 63 is executed by the processor 61 to realize the function of each model / unit in the multi-module collaborative rain removal system for rain vehicle detection in the embodiment, to avoid repetition, not here. Not one by one.

[0141] The computer device 60 can be a desktop computer, a notebook computer, a palm computer and a cloud server, etc. The computer device 60 can include, but is not limited to, a processor 61, a memory 62. Those skilled in the art can understand that Figure 6 is only an example of the computer device 60 and does not constitute a limitation on the computer device 60, and can include more or fewer components than the illustration, or combine certain components, or different components, for example, the computer device can also include an input / output device, a network access device, a bus, etc.

[0142] The processor 61 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0143] The memory 62 can be an internal storage unit of the computer device 60, such as a hard disk or a memory of the computer device 60. The memory 62 can also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like.

[0144] Further, the memory 62 can include both an internal storage unit and an external storage device of the computer device 60. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 can also be used to temporarily store data that has been output or will be output.

[0145] Referring to Figure 7 , the terminal device is a chip, and the chip 600 of the embodiment includes a processor 622, the number of which can be one or more, and a memory 632 used to store computer programs executable by the processor 622. The computer programs stored in the memory 632 can include one or more than one module each corresponding to a set of instructions. In addition, the processor 622 can be configured to execute the computer programs to perform the multi-module collaborative rain removal method for rain day vehicle detection described above.

[0146] In addition, the chip 600 can also include a power supply component 626 which can be configured to perform power management of the chip 600, and a communication component 650 which can be configured to implement communication of the chip 600, such as wired or wireless communication. In addition, the chip 600 can also include an input / output interface 658. The chip 600 can operate based on an operating system stored in the memory 632.

[0147] In another embodiment of the present application, the present application also provides a storage medium, specifically a computer readable storage medium, which is a memory device in a terminal device and is used to store programs and data. It can be understood that the computer readable storage medium herein can include a built-in storage medium in the terminal device, and of course can also include an expansion storage medium supported by the terminal device. The computer readable storage medium provides a storage space, and the storage space stores an operating system of the terminal. In addition, one or more than one instruction suitable for being loaded and executed by a processor is also stored in the storage space, and the instruction can be one or more than one computer program. It should be noted that the computer readable storage medium herein can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory.

[0148] The one or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the multi-module collaborative rain removal method for rain day-oriented vehicle detection in the above embodiments; the one or more instructions stored in the computer readable storage medium are loaded and executed by the processor to implement the following steps:

[0149] Given a rainy image input into the multi-module collaborative rain removal method, the original noise is filtered through a convolution layer, and a progressive information perception module is constructed for extracting spatial details and transmitting information with high priority, and outputting features ; the features are mapped to the original high resolution by the decoding unit after being down-sampled to low resolution by the encoding unit, and the features are output. After being mapped by a 3x3 convolution layer, a residual image is generated, and the final rain removal result is obtained by adding .

[0150] To make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0151] The multi-module collaborative rain removal method and the integrated benchmark vehicle detection method are tested on an RTX 3090. The batch is set to 4, the epoch is set to 400, the Adam optimizer is used, the initial learning rate is set to 2x10 -4 , the Cosine Annealing strategy is used, and the decay is to 1x10 -6 .

[0152] Advanced verification experiment of the multi-module collaborative rain removal method: this subsection follows the method of document [20,21] , and the data set provided in document [13,17,34-36] is re-divided as shown in Table 1.

[0153] Table 1: Division and renaming of rain removal data set

[0154]

[0155] Effectiveness and generalization verification experiment of multi-module collaborative rain removal method: Since the focus of the present application is to consolidate and enhance the vehicle detection accuracy of the baseline vehicle detection method in rainy days by designing a rain removal method suitable for various rain layer patterns, a Rain-Vehicle data set for rainy day vehicle detection experiments is constructed. The data in the data set is mainly derived from the COCO data set and the vehicle images collected by the author. Specifically, the training set and the validation set are 2150 normal clear images and 500 normal clear images, respectively, and the test set is 696 degraded rainy images. The distribution patterns of the rain layer mainly include synthetic rain streaks, real rain streaks and rain fog. Since the data set contains various scenes such as cities, suburbs and rural areas at home and abroad, the generalization and practicality of the multi-module collaborative rain removal method can be fully verified. In the rainy day vehicle detection experiment in Chapter 4, the data containing various rain layer distribution patterns are used for training to obtain a rain removal model that can be used for rainy day vehicle detection. The Rain-Vehicle data set is used to complete the training of four baseline vehicle detection methods, and the trained weights are used for comparative experiments in the test set.

[0156] To prove the advancement of the MBCDNet proposed in the present application, this subsection compares the rain removal performance of the MBCDNet with other excellent rain removal methods in five public data sets. The rain removal performance evaluation indexes are Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM). The higher the PSNR and the closer to 1 the SSIM, the better the rain removal performance of the corresponding method. The specific results are shown in Table 2, where the bold display is the optimal result, and the underlined display is the suboptimal result.

[0157] Table 2 Rain removal performance comparison results of different rain removal methods

[0158]

[0159] As can be seen from Table 2, compared with other classic excellent rain removal methods RESCAN, PReNet and MSPFN, MBCDNet has achieved the optimal PSNR and SSIM on the Test100, Rain100H, Rain100L, Test2800 and Test1200 data sets. Compared with the recent excellent rain removal method MPRNet

[21] In contrast, MBCDNet performs best in PSNR on Test1200 dataset, and performs best in SSIM on Test100, Rain100L and Test1200 datasets. Taking the average of five datasets, the PSNR of MBCDNet is 32.56dB, and the SSIM is 0.924, which is better than MPRNet. In summary, the above comparisons objectively prove that MBCDNet has excellent rain removal performance.

[0160] To fully illustrate the advancement and real-time performance of the method MBCDNet of the present application, this section compares the main competitive method MPRNet in the scene content rich and rain pattern diverse Test1200 and Test2800 datasets as shown in Table 3, and the optimal results are shown in bold.

[0161] Table 3 Comparison of rain removal performance, average running time and parameter amount of the method and MPRNet

[0162]

[0163] As shown in Table 3, MBCDNet has similar PSNR and SSIM to MPRNet on Test1200 and Test2800 datasets. The average running time is 0.098s and 0.079s, respectively, which is 30.5% and 33.6% lower than MPRNet, respectively. At the same time, the parameter amount of MBCDNet is 1.18M, which is 67.6% lower than MPRNet. The above experiments fully show that MBCDNet achieves a balance in rain removal performance and computational efficiency.

[0164] To illustrate the advantages of MBCDNet in running efficiency compared with recent excellent rain removal methods, 100 rain images with image sizes of 256x256, 384x384 and 512x512 are selected, respectively, and named as Image256, Image384 and Image512. The average running time of the recent excellent rain removal methods is compared on the above three datasets, and the test results are shown in Table 4, wherein the optimal results are shown in bold, and the suboptimal results are shown in underlined.

[0165] Table 4 Comparison of average running time of different rain removal methods

[0166]

[0167] As shown in Table 4, in the Image256 dataset, i.e. the image size is 256x256, the average running time of MBCDNet is reduced by 21.4% compared with MPRNet, and the average running time of MBCDNet is reduced by 21.4% compared with Restormer [The descending rate is 49.4%. In the Image384 dataset, that is, the image size is 384*384, the single-image processing speed of the rain removal method Restormer is 0.216s, which cannot meet the real-time requirement, while the processing speed of the method is 0.067s, which still has low real-time performance. In summary, for rain images of various sizes, MBCDNet has obvious speed advantage compared with the recent excellent rain removal methods, and thus is more likely to meet the real-time rain removal requirements under various conditions.

[0168] The comprehensive advantages of the method MBCDNet in rain removal performance and average running time have been fully proved, and thus the method is more suitable for vehicle detection tasks in rainy days compared with other excellent rain removal methods.

[0169] Further, in order to prove the effectiveness of the method for vehicle detection in rainy days, the benchmark vehicle detection method SSD [3] , YOLOv5 and YOLOv8 are integrated.

[0170] Scheme one is the result of the benchmark vehicle detection method directly performing vehicle detection in the test set (without rain removal processing),

[0171] Scheme two is the vehicle detection result of the benchmark vehicle detection method after the test set is subjected to rain removal processing by MBCDNet.

[0172] The evaluation index is mAP@0.5, and the higher the value is, the better it is, and the detailed comparison is shown in Table 5.

[0173] Table 5: Gain result of the method of the application on the benchmark vehicle detection method

[0174]

[0175] As shown in Table 5, the mAP@0.5 gain of MBCDNet on the benchmark SSD, YOLOv5m, YOLOv8n and YOLOv8m in rainy day vehicle detection is 6.1%, 8.8%, 12.4% and 7.8% respectively, which fully proves that the MBCDNet proposed in the application can improve the detection accuracy of various benchmark vehicle detection methods in rainy days.

[0176] In order to more intuitively illustrate the generalization and effectiveness of MBCDNet, five traffic scene images containing different rain amounts are selected for visual display. In the synthetic rain streak and rain fog scenes (a), (b) and (c) and the real rain streak and rain fog scenes (d), (e), the gain result of MBCDNet on the benchmark YOLOv8m is shown in Table 6. Figure 5

[0177] As shown in Table 6, the gain of MBCDNet on the benchmark YOLOv8m in the synthetic rain streak and rain fog scenes (a), (b) and (c) is 6.1%, 8.8% and 12.4% respectively, and in the real rain streak and rain fog scenes (d), (e) is 7.8% and 7.8% respectively. Figure 5 ​It is observed that for synthetic rainfall scenes (a), (b) and (c), although the raindrop falling angle and the number of raindrops are significantly different, MBCDNet can effectively remove the raindrops therein, not only the vehicle background layer in the figure is clearly restored, but also the restored image is real and natural. Specifically, from scene (a), due to the interference of dense raindrops, only scheme one using the benchmark YOLOv8m misdetected roadside plants as vehicles, while scheme two introduced MBCDNet, benefiting from the excellent image rain removal capability of the method, the misdetection phenomenon not only disappeared but also the confidence of the vehicle detection result was improved from 0.84 to 0.94. From scene (c), benefiting from MBCDNet, the benchmark YOLOv8m avoided missed detection, and the detection result had high confidence. For real rainfall scenes (d) and (e), the raindrops in scene (e) are more complex, but the restoration effect of MBCDNet on scene (e) is still excellent, after removing most of the raindrops in the image, the confidence of vehicle detection is improved from 0.88 to 0.94.

[0178] In summary, the multi-module collaborative rain removal method and system for rainy day vehicle detection has the following characteristics:

[0179] (1) The rain removal method MBCDNet designed in the application has a simple architecture, and the multi-level feature capture module and the progressive information perception module complement each other and cooperate with each other, realizing accurate perception and capture of the position, shape of the rain layer and the interference degree of the vehicle background layer in the rainy image, thereby guiding the whole network to complete the core task of removing the rain layer and restoring the vehicle background layer. The test results on multiple synthetic rain datasets show that the method has excellent rain removal performance. The test results on Test1200 and Test2800 synthetic rain datasets show that the method has the comprehensive advantages of excellent rain removal performance and fast average running time compared with the recent excellent rain removal method MPRNet.

[0180] (2) According to the driving environment of intelligent vehicles, the Rain-Vehicle dataset is constructed, and MBCDNet is fused with multiple benchmark vehicle detection methods and tested in the dataset. The results show that the mAP@0.5 gain of the proposed method to the benchmark SSD, YOLOv5m, YOLOv8n and YOLOv8m is 6.1%, 8.8%, 12.4% and 7.8% respectively, proving the generalization and effectiveness of the method.

[0181] The method can efficiently and accurately restore the vehicle target in the rainy image, thereby effectively improving the detection accuracy of the benchmark vehicle detection method in the intelligent automobile in rainy days. The results of the application can provide a reference for improving the detection accuracy of the vehicle detection system of the intelligent automobile in rainy days and even other complex weather conditions. However, the rain removal method still has optimization space, such as the processing speed should be further improved to realize real-time rain removal in high-resolution images. In addition, due to the influence of various light sources on the rainy images collected by the vehicle-mounted camera in the night scene, the rain pattern is more complex and diverse, so the method for removing rain in the night scene should be studied in the subsequent work.

[0182] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or software. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0183] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0184] Those skilled in the art can realize that the units and method steps of each example described in combination with the embodiments disclosed in the application can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the application.

[0185] In the embodiments of the present application, it should be understood that the disclosed apparatus / terminal and method can be implemented in other manners. For example, the embodiments of the apparatus / terminal described above are merely schematic, and the division of the modules or units is merely logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0186] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0187] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can be a physically independent unit, or two or more units can be integrated into a unit. The integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0188] The integrated module / unit, if implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the computer readable medium can include appropriate contents according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0189] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps in one or more flow or blocks

[0190] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps in one or more flow or blocks

[0191] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps in one or more flow or blocks

[0192] The above merely provides the technical idea of the present application, and cannot be used to limit the protection scope of the present application. Any modification made according to the technical idea of the present application, on the basis of the technical solutions, falls within the protection scope of the claims of the present application.

Claims

1. A multi-module collaborative rain removal method for rainy day vehicle detection, characterized in that, Comprising the steps of: S1, given the rainy image of the input multi-module collaborative rain removal method, the original noise is filtered through the convolution layer, and the progressive information perception module is constructed for rich spatial detail extraction and high priority information transmission, output features The progressive information perception module comprises: Low-level detail information extraction module, embedded features After layer normalization processing, get standard features Then cross-dimensional information interaction is completed by convolution layer to get 1×1 convolution layer mapped features , channel-level equal split is performed on to get split features and ; Based on the convolution layer, feature extraction is completed to get 3×3 depth separable convolution layer mapped features , Without processing, directly passed to the next layer; channel-level integration is performed on and , and refinement is performed through the convolution layer to get 1×1 convolution layer refined features ; based on GeLU, attention map is generated and representative detail feature representation is obtained ; through element addition, the output features of the low-level detail information extraction module are obtained ; The high-level semantic information extraction module outputs features. Normalized features are obtained after layer normalization. Then, multiple convolutional layers sequentially extract deep features to obtain deep semantic features. , The SCA adaptively performs calibration of high-priority rain layer information to obtain features. Features are processed through convolutional layers. Further refinement and element-wise addition to obtain output features Output features for: wherein, a feature representing attention feedback, represents a 1 x 1 convolutional layer; S2, feature In the encoding unit, the image is down-sampled and progressively reduced to a low resolution, processed by an intermediate layer, and mapped to the original high resolution by the decoding unit, outputting the feature ; The multi-level feature capturing module is embedded in each level of the encoder-decoder architecture, and the information flow process is as follows: layer normalization is used to preprocess the embedding features to stabilize the training process, for the normalized features; then a convolutional layer is used to dimension transformation, 1x1 convolutional layer mapping features; using a depth separable convolutional layer to extract deep features, by aggregating three-dimensional attention to adaptively guide the preservation and transmission of rain layer information with higher priority, and further refined by convolutional layer, multi-level feature capture module aggregates double branch structure to explore different levels and different forms of rain layer features under the premise of acceptable computational overhead, as follows: wherein, represents a 3x3 depthwise separable convolution layer, represents a GeLU activation, and are the output features of the upper branch and the lower branch, respectively, represents element-wise addition, is the feature after the fusion of the two branches; S3, Features The residual image is generated after mapping through a 3×3 convolutional layer and then compared with the input image. The sums are then added together to obtain the final result of the rain.

2. The multi-module collaborative rain removal method for rainy day vehicle detection according to claim 1, characterized in that, Features are: wherein, represents a 3x3 convolutional layer responsible for the upscaling operation of the input image represents two successive progressive information perception modules, feature output by the shallow feature extraction unit.​ 3. The multi-module collaborative rain-removal method for rainy-day vehicle detection according to claim 1, wherein, Features are: wherein, is a feature after intermediate layer processing, is a decoding unit in an encoder-decoder architecture.

4. The multi-module collaborative rain-removal method for rainy-day vehicle detection according to claim 1, wherein, High quality rain free images To: 。 5. The multi-module collaborative rain-removal method for rainy-day vehicle detection according to claim 1, wherein, The training of the multi-module collaborative rain removal network is completed by fusing the Charbonnier loss and the edge loss, and a total loss function is defined is defined as: wherein, is empirically set to 0.05, is an edge loss, is a Charbonnier loss, represents a predicted rain-free image, represents an original rain-free image.

6. A multi-module collaborative rain removal system for rainy day vehicle detection, characterized in that, Comprising: The shallow feature extraction unit is configured to filter original noise through a convolution layer, construct a progressive information perception module, and output features The progressive information perception module includes: Low-level detail information extraction module, embedded features After layer normalization processing, get standard features Then cross-dimensional information interaction is completed by convolution layer to get 1×1 convolution layer mapped features , low-level detail information extraction module, embedded features Channel-level equal split is performed to get split features And ; Based on convolution layer, feature extraction is completed to get 3×3 depth separable convolution layer mapped features , Without processing, directly passed to the next layer; channel-level integration is performed on And Through convolution layer, refinement is performed to get 1×1 convolution layer refined features ; based on GeLU, attention map is generated and representative detail feature representation is obtained ; through element addition, output features of low-level detail information extraction module are obtained ; A high-level semantic information extraction module outputs a feature After layer normalization processing, a standard feature is obtained , and then deep layer feature extraction is sequentially completed by multiple convolution layers to obtain deep semantic features , The SCA adaptively completes calibration of rain layer information with higher priority to obtain a feature ; the feature is further refined by a convolution layer and an output feature is obtained by element addition , and the output feature is: wherein, a feature representing attention feedback, represents a 1 x 1 convolutional layer; deep feature extraction unit, feature in the encoding unit is down-sampled to be progressively reduced to a low resolution, after being processed by an intermediate layer, mapped to the original high resolution by a decoding unit, and output features ; The multi-level feature capturing module is embedded in each level of the encoder-decoder architecture, and the information flow process is as follows: The embedded features are preprocessed based on layer normalization to stabilize the training process, to normalize the features; then a convolutional layer is used to to perform dimension transformation, to map the features of the 1x1 convolutional layer; a deep separable convolutional layer is used to extract deep features, and by aggregating three-dimensional attention, the preservation and transmission of rain layer information with higher priority are adaptively guided, and the features are further refined by a convolutional layer. The multi-level feature capture module aggregates the double-branch structure to explore rain layer features at different levels and in different forms under acceptable computational overhead, as follows: wherein, represents a 3x3 depthwise separable convolution layer, represents a GeLU activation, and are the output features of the upper branch and the lower branch, respectively, represents element-wise addition, is the feature after the fusion of the two branches; The rain removal result output unit is characterized in that The residual image is generated after being mapped through the 3*3 convolution layer, and the final rain removal result is obtained by adding the residual image and the rain image. The residual image is generated after being mapped through the 3*3 convolution layer, and the final rain removal result is obtained by adding the residual image and the rain image.

Images