Rainwater removing method and system based on two-stage progressive mode

Abstract

The invention discloses a two-stage progressive rain removal method and system, and aims to solve the problem that the vehicle detection precision of an intelligent connected vehicle is reduced in a rainy day scene. According to the method, a special vehicle detection data set MCGVD is constructed, the MCGVD is divided into a training set and a test set, and a two-stage progressive rain removal network TSPNet including an improved U-Net feature extraction network and a feature fusion mechanism is used for training. Inputting the test set into the trained TSPNet, performing channel dimension raising through 1 * 1 convolution, extracting shallow rain stripe information, extracting deep rain stripe information through an improved encoder-decoder architecture, and finally recovering to obtain a vehicle image, thereby realizing rainy day vehicle detection. According to the system, through the lightweight feature extraction and weighting module and the high-efficiency feature transmission and fusion module, the rain removal effect and the image quality are improved, the performance of the vehicle detection system under complex weather conditions is remarkably improved, and the system has important practical application value.

Description

Background Art

[0002] Intelligent and connected vehicle (ICV) is a high-tech complex that integrates key systems such as environmental perception and information interaction. As the "eyes" of ICV, the environmental perception system undertakes the important task of detecting and identifying the semantic categories, positions and directions of surrounding objects. Among the many subsystems of the environmental perception system, the vehicle detection system (VDS) based on computer vision has stable performance under clear weather conditions, but under harsh conditions such as rainy days, the images collected by VDS have problems such as rain streak occlusion and loss of detail information, and the detection accuracy of VDS is affected by the vehicle feature information in the image. Therefore, its detection accuracy will decline when rain streaks occlude the vehicle background layer, which seriously threatens the driving safety of ICV.

[0003] In view of this, improving the detection accuracy of VDS under adverse weather conditions is of great research significance. At present, many scholars have conducted relevant research on this issue. Ning et al. improved the vehicle recognition accuracy of YOLOv3 in foggy scenes by integrating DensNet and attention mechanism into YOLOv3. Yuan et al. optimized YOLOv5 based on the idea of ​​improving data enhancement algorithm and introducing attention module, and enhanced the vehicle detection accuracy of YOLOv5 in foggy environment. Wang et al. integrated millimeter wave radar and machine vision to ensure the vehicle detection performance in adverse environments such as rain and fog. Although the above studies have made certain progress, they all focus on the improvement of detectors and the use of auxiliary equipment, ignoring the objective fact that image information is closely related to detection accuracy. Therefore, the work on the image side needs to be strengthened.

[0004] At present, there are relevant research results on image deraining. Li et al. built a model based on deep convolution and recurrent neural network to remove rain marks. Ren et al. comprehensively considered the network architecture, loss function and other modules, and created a progressive deraining network to remove the rain layer. Jiang et al. proposed a multi-scale progressive fusion network based on pyramid structure to realize the perception and removal of rain streak distribution. Although the above algorithms have excellent deraining performance, there are still problems such as residual rain streaks and loss of detail information to varying degrees. In order to solve the long-standing problems in deraining work, Zamir et al. achieved effective removal of rain streaks based on channel attention and supervised attention, but the simplicity of the network architecture needs to be improved. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a two-stage progressive rain removal method and system to address the deficiencies in the above-mentioned prior art, so as to solve the technical problems of vehicle detection systems of intelligent connected vehicles suffering from deterioration in detection accuracy, incomplete rain removal and serious loss of details in rainy scenes.

[0006] The present invention adopts the following technical solutions: A two-stage progressive rain removal method includes the following steps: S1, construct the vehicle detection dataset MCGVD, and divide the vehicle detection dataset MCGVD into a training set and a test set; S2, constructing a two-stage progressive deraining network TSPNet including an improved U-Net feature extraction network and a feature fusion mechanism, and training the two-stage progressive deraining network TSPNet using the training set obtained in step S1; S3. Input the test set obtained in step S1 into the two-stage progressive rain removal network TSPNet trained in step S2. After channel dimension increase by 1x1 convolution, shallow rain streak information is extracted. The feature map enters the improved encoder-decoder architecture to extract deep rain streak information. Finally, the vehicle image is restored to achieve vehicle detection in rainy days.

[0007] Preferably, in step S2, the two-stage progressive rain removal network TSPNet includes two stages, the first stage is the initial image rain removal and complex sub-network, and the second stage is the terminal image rain removal and complex sub-network. The sub-networks of the first stage and the second stage both include an encoder-decoder architecture, and generate information interaction through a cross-stage fusion mechanism.

[0008] Preferably, in the first stage, the initial image is derained and the subnet is restored. , the specific calculation is divided into: The first part is Conventional convolutional layer, which performs dimension-upgrading operation on the input rainy image; The second part is the encoder-decoder architecture, which is responsible for mining the global rain map feature information and reconstructing the restored image from the feature map output by the first part; The third part is The conventional convolutional layer performs dimensionality reduction on the feature map output by the second part.

[0009] Preferably, the feature map output by the second part is subjected to a dimensionality reduction operation as follows:

[0010] in, express Convolution operation, represents the feature map output by the second part and input to the third part, represent The feature map after the convolution operation is also the restored image output by the first stage.

[0011] Preferably, in the second stage, the end image is derained and the subnet is complexed. , the specific calculation is divided into: The first part is Conventional convolutional layer: firstly, the de-rained image output from the first stage and the rainy image input from the second stage are feature-fused, and then this part performs dimension-upgrading operation; The second part is the encoder-decoder architecture, which is responsible for the fusion of the rain streak feature information output by the first part of this stage and the second part of the previous stage, as well as the mining of the global rain map feature information and the reconstruction of the restored image; The third part is The conventional convolution layer is responsible for the dimensionality reduction operation of the output feature map of the second part.

[0012] Preferably, the dimension reduction operation on the second part output feature map is specifically:

[0013] in, represents the feature map output by the second part and input to the third part, represent Feature map after convolution operation.

[0014] Preferably, the two-stage progressive rain removal network TSPNet includes a lightweight feature extraction and weighting module and a high-efficiency feature transfer and fusion module; the lightweight feature extraction and weighting module integrates batch normalization to accelerate network training, while using deep convolution to replace part of the conventional convolution, and integrating a lightweight channel attention mechanism.

[0015] Preferably, the lightweight feature extraction and weighting module is specifically: Batch normalization layer, responsible for improving the generalization of the module and accelerating network training; The feature extraction unit GRSPB includes a shallow feature extraction unit and a deep feature extraction unit. In the shallow feature extraction unit, the input feature map through The conventional convolutional layer performs dimensionality increase and realizes cross-channel information fusion; then The deep convolutional layer fully extracts the channel dimension rain image feature information, and then uses Activate to enhance nonlinear features, and finally The conventional convolutional layer completes the global shallow rain image feature mining; in the deep feature extraction unit, Sequentially Conventional convolutional layer, The depth of the convolutional layer and After activating the layer, The conventional convolutional layer completes the mining of global deep rain image features; Feature weighting unit, using adaptive average pooling layer Process it in both width and height dimensions to make it equal to , followed by The conventional convolutional layer fuses cross-channel features and then uses Activate to obtain the weight matrix , and Point multiplication obtains the output feature map of the unit ; Finally adopted Conventional convolution refines the output feature map And output the result.

[0016] Preferably, the total loss function is defined as:

[0017] in, for loss, is the edge loss, represents the predicted rain-free image, Represents a clear, rain-free image. Represents the network stage, According to experience, .

[0018] In a second aspect, an embodiment of the present invention provides a two-stage progressive rain removal system, comprising: Data module, constructs the vehicle detection dataset MCGVD, and divides the vehicle detection dataset MCGVD into a training set and a test set; The training module builds a two-stage progressive deraining network TSPNet that includes an improved U-Net feature extraction network and a feature fusion mechanism, and uses the training set to train the two-stage progressive deraining network TSPNet; The output module inputs the test set into the trained two-stage progressive rain removal network TSPNet. After 1x1 convolution for channel dimension increase, shallow rain streak information is extracted. The feature map enters the improved encoder-decoder architecture to extract deep rain streak information. Finally, the vehicle image is restored to achieve vehicle detection in rainy days.

[0019] Compared with the prior art, the present invention has at least the following beneficial effects: A two-stage progressive rain removal method is proposed. The proposed two-stage progressive rain removal method ensures that the model can be trained and verified on a variety of data by constructing a special vehicle detection dataset and dividing it into a training set and a test set, thereby improving the generalization ability of the model. By designing a two-stage network structure that includes an improved U-Net feature extraction network and a feature fusion mechanism, this method can first extract shallow rain streak information and then extract more complex rain streak features through a deep network. This hierarchical feature extraction and fusion mechanism enables the model to more finely identify and process raindrops, improving the rain removal effect. Ultimately, this method can effectively restore rain-free vehicle images and significantly improve the accuracy of vehicle detection in rainy days, which is of great significance for improving the performance of vehicle detection systems under complex weather conditions.

[0020] Furthermore, the patented two-stage progressive rain removal network TSPNet optimizes the image rain removal effect through staged processing. The first stage quickly removes most raindrops and restores the image infrastructure, while the second stage performs in-depth processing to finely remove residual raindrops and enhance image details. Each stage adopts an encoder-decoder architecture to effectively capture and reconstruct image features. The cross-stage fusion mechanism enables each stage to share information, collaboratively optimize the rain removal effect, and fully restore the original image quality.

[0021] Furthermore, the first-stage rain removal and restoration subnet described in the patent processes rainy images through three main parts working together. First, the conventional convolution layer performs dimensionality increase operations on the input image, which helps to extract richer feature information. Next, the encoder-decoder architecture deeply explores the global rain map features in these feature maps and uses this information to reconstruct the restored image. Finally, another set of conventional convolution layers performs dimensionality reduction operations on the feature maps output by the decoder to optimize the feature representation in preparation for subsequent stages or final output. This design enables the network to effectively restore the image step by step from coarse to fine while maintaining the hierarchy and details of the features, improving the rain removal effect and image quality.

[0022] Furthermore, the second-stage rain removal network TSPNet focuses on improving the processing of image details and deep features. It accelerates network training and improves the efficiency of feature extraction through lightweight feature extraction and weighting modules, combined with batch normalization and deep convolution technology. At the same time, the introduction of the lightweight channel attention mechanism allows the network to pay more attention to key features in the image, especially in areas with greater impact from raindrops. This mechanism helps the network to more accurately identify and remove raindrops in the second stage, while retaining the subtle texture and contours of the image, thereby achieving higher-quality image restoration. Through such a design, the second stage not only optimizes the rain removal effect, but also provides strong support for improving the accuracy of vehicle detection in rainy environments.

[0023] Furthermore, the lightweight feature extraction and weighting module is designed to improve the network's computational efficiency and feature expression capabilities. Through batch normalization, the network training process is accelerated, while the stability and convergence speed of the model are enhanced. The use of deep convolution replaces some traditional convolutions, which helps the network capture deeper features in the image, while the lightweight channel attention mechanism allows the network to focus more on features that are critical to the rain removal task, thereby improving the accuracy of rain removal. The purpose of the high-efficiency feature transfer and fusion module is to optimize the flow of features in the network and ensure that the information transfer from the first stage to the second stage is efficient and accurate. This module enables the network to share and supplement information between different stages through an effective feature fusion strategy, which is crucial for capturing and removing raindrop features, especially in complex scenes. This design helps the network to achieve fine restoration of image details while maintaining lightweight, improving the rain removal effect and image quality.

[0024] It can be understood that the beneficial effects of the second aspect mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0025] In summary, the present invention provides an efficient two-stage progressive rain removal system, which ensures the training and verification of the model on diversified data by constructing a special vehicle detection data set and dividing it into a training set and a test set, thereby improving the generalization ability of the model. The system adopts a feature extraction network and feature fusion mechanism including an improved U-Net, so that the model can extract shallow and deep rain streak information hierarchically, finely identify and process raindrops, and effectively improve the rain removal effect. In addition, the system captures and reconstructs image features through an encoder-decoder architecture, and shares information through a cross-stage fusion mechanism to collaboratively optimize the rain removal effect and fully restore the original quality of the image. In particular, in the second stage, the system uses a lightweight feature extraction and weighting module, combined with batch normalization and deep convolution technology, to accelerate network training and improve the efficiency of feature extraction, while introducing a lightweight channel attention mechanism, so that the network pays more attention to the key features in the image, especially in areas with greater raindrop influence, to achieve higher quality image restoration. This design not only optimizes the rain removal effect, but also provides strong support for improving the accuracy of vehicle detection in rainy environments, and significantly improves the performance of the vehicle detection system under complex weather conditions.

[0026] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a two-stage progressive rain removal network architecture diagram; Figure 2 It is an encoder-decoder architecture based on U-net; Figure 3 It is a lightweight feature extraction and weighting module; Figure 4 For lightweight channel attention; Figure 5 High-efficiency feature transfer and fusion module diagram; Figure 6 The comparison results of different deraining algorithms are shown in Figure 1, where (a) is the PSNR comparison result; (b) is the SSIM comparison result; Figure 7 The deraining effects in different scenes, where (a) is the coast; (b) is a partial enlargement of the coast; (c) is a residence; (d) is a partial enlargement of the residence; Figure 8 The rain removal effects in different scenes are shown below. (a) is a lawn; (b) is a partial enlargement of the lawn; (c) is a road; (d) is a partial enlargement of the road; Fig. 9 This is a flow chart of the vehicle detection method in rainy days; Fig.10 This is a comparative experimental diagram of Scheme A and Scheme B; Fig.11 The following are the vehicle detection results in different scenarios, where (a) is a rural road; (b) is an urban road; (c) is a suburban road; Fig.12 A schematic diagram of a computer device provided by an embodiment of the present invention; Fig.13 The present invention is a block diagram of an electronic device provided according to an embodiment. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0030] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0031] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects are in an "or" relationship.

[0032] It should be understood that, although the terms first, second, third, etc. may be used to describe preset ranges, etc. in the embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0033] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0034] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0035] The present invention provides a two-stage progressive rain removal method, comprising the following steps: S1. Construct a vehicle detection dataset (MCGVD) based on the operating environment of ICV, and divide the vehicle detection dataset MCGVD into a training set and a test set; The experimental results on MCGVD show that in rainy traffic scenes, the deraining method proposed in the present invention achieves gains of 3.0%, 8.9% and 7.6% on the Precision, Recall and mAP@0.5 of the benchmark vehicle detector YOLOv5, respectively, significantly improving the vehicle detection accuracy of the benchmark vehicle detector YOLOv5 in rainy traffic scenes.

[0036] S2, constructing a two-stage progressive deraining network TSPNet including an improved U-Net feature extraction network and a feature fusion mechanism, and training the two-stage progressive deraining network TSPNet using the training set obtained in step S1; The morphological distribution of raindrops in an image is affected by many factors at the moment of imaging, such as camera parameters, wind, gravity, and background brightness. In the rainy traffic scene images acquired by VDS, the real three-dimensional scene is mapped onto the two-dimensional plane of the image. Therefore, the neighborhood pixels are very likely to contain depth and brightness features of different magnitudes. If different pixels are far apart, the probability that they belong to the same rain streak is smaller. When the ICV is driving, the vehicles that are relatively close to its own ICV are large in size, occupy more pixels, and are located in a large local area of ​​the image. Rain streaks may block local vehicle feature information; vehicles that are relatively far from its own ICV are small in size, occupy fewer pixels, and are located in a small local area of ​​the image. Rain streaks may block global vehicle feature information. Combined with the above objective fact analysis, the present invention is based on the idea of ​​progressive recovery and builds a system such as Figure 1 The two-stage progressive deraining network TSPNet is shown. The network includes two stages, the first stage is the initial image deraining and complex sub-network, and the second stage is the terminal image deraining and complex sub-network. Considering the simplicity of the network architecture, the present invention makes the two sub-network structures of TSPNet similar. In view of the significant advantages of the encoder-decoder architecture in mining and capturing feature information and the outstanding advantages of the cross-stage fusion mechanism in transmitting and aggregating feature information, the sub-networks of the two stages both include an encoder-decoder architecture and generate information interaction through the cross-stage fusion mechanism.

[0037] See also Figure 1 The two-stage progressive deraining network TSPNet is based on the Lightfeature extraction and weighting block (LFEWB) and the Efficient feature transfer and fuse block (EFTFB). It captures and fuses the feature information of rain patterns at different scales and stages in rainy images, thereby completing the complex task of accurately removing rain streaks and restoring the background layer, solving the long-standing problems of rain streak residue and detail information loss in existing deraining algorithms.

[0038] Suppose the rainy image input to TSPNet is , represents the network stage of the sub-image, Represents the position of the sub-image in the complete image.

[0039] In the first stage, the initial image is derained and the subnet is restored. , the specific calculation process can be divided into three parts. The first part is Conventional convolutional layer performs dimensionality increase on the input rainy image.

[0040] (1) in, express Convolution operation, represents the original input image, represent Feature map after convolution operation.

[0041] The second part is the encoder-decoder architecture, which is responsible for mining the global rain map feature information of the output feature map of the first part and reconstructing the restored image. This process is shown in Equations (2), (3) and (4).

[0042] (2) (3) (4) in, represents the encoding operation, represents the decoding operation, Indicates splicing processing, represents the feature map output by the first part and input to the second part, Represents the feature map after encoding operation, Represents the feature map after splicing. Represents the feature map after decoding operation.

[0043] The third part is The conventional convolutional layer performs dimensionality reduction on the feature map output by the second part.

[0044] (5) in, express Convolution operation, represents the feature map output by the second part and input to the third part, represent The feature map after the convolution operation is also the restored image output by the first stage.

[0045] The second stage is to remove rain from the end image and reconstruct the subnet. , the specific calculation process is similar to that of the first stage.

[0046] The first part is In the conventional convolution layer, the derained image output from the first stage and the rainy image input from the second stage are firstly feature-fused, and then the dimension is increased by this part.

[0047] (6) in, represents the original input image, represents the derained image output from the first stage, Representatives passed Feature map after convolution operation.

[0048] The second part is the encoder-decoder architecture, which is responsible for the fusion of the rain streak feature information output from the first part of this stage and the second part of the previous stage, as well as the mining of the global rain map feature information and the reconstruction of the restored image. This process is shown in Equations (7), (8) and (9).

[0049] (7) (8) (9) in, represents the feature map output by the first part and input to the second part, Represents the feature map after encoding operation, Represents the feature map after splicing. Represents the feature map after decoding operation.

[0050] The third part is The conventional convolution layer is responsible for the dimensionality reduction operation of the output feature map of the second part.

[0051] (10) in, represents the feature map output by the second part and input to the third part, represent The feature map after the convolution operation is also the final derained image output by TSPNet.

[0052] In order to accurately perceive the rain streak area and fully explore the local and global rain streak features of different scales, firstly, the present invention adopts an encoder-decoder architecture based on the improved U-Net, such as Figure 2 shown.

[0053] Secondly, a lightweight feature extraction and weighting module (LFEWB) based on the attention mechanism is proposed, such as Figure 3 The encoder-decoder architecture uses LFEWB to achieve deep extraction of rain image features at different scales, and uses skip connections to achieve fine fusion of rain image features at corresponding scales in the encoding and decoding areas, alleviating detail loss during the sampling process.

[0054] LFEWB is mainly composed of a feature mining unit, the Global Rain Streaks Perception Block (GRSPB), and a feature weighting unit, the Light Channel Attention (LCA). Figure 3 As shown in Figure 4 Compared with other studies that generally use dilated convolution to increase the receptive field, the present invention designs LFEWB by extending the network depth to expand the receptive field, thereby ensuring that the characteristic information of the rain image is fully mined and captured; Secondly, LFEWB incorporates batch normalization to accelerate network training, and uses deep convolution to replace some conventional convolutions to reduce computational costs. Finally, a lightweight channel attention mechanism is incorporated to ensure that the key information of the rain map in the channel dimension is effectively retained, eliminating redundant information while further improving the network reasoning efficiency.

[0055] In LFEWB, for a given feature map , let its channel, height and width be , , The specific calculation process is divided into four parts.

[0056] The first part is the batch normalization layer, which is responsible for improving the generalization of the module and accelerating network training.

[0057] (11) in, represents batch normalization processing, Represents the input feature map, Represents the feature map after batch normalization.

[0058] The second part is the feature extraction unit GRSPB, which includes a shallow feature extraction unit and a deep feature extraction unit. Considering the simplicity of the network, the structures of the two units are consistent. In the shallow feature extraction unit, the input feature map through The conventional convolutional layer performs dimensionality increase and realizes cross-channel information fusion. The deep convolutional layer fully extracts the channel dimension rain image feature information, and then uses Activate to enhance nonlinear features, and finally The conventional convolutional layer completes the global shallow rain image feature mining. In the deep feature extraction unit, similarly, Sequentially Conventional convolutional layer, The depth of the convolutional layer and After activating the layer, The conventional convolutional layer completes the mining of global deep rain image features. The above process is shown in Equation (12) and Equation (13).

[0059] (12) (13) in, express Convolution operation, express Depthwise convolution operation, express activation, express Convolution operation, Represents the feature map output by the first part and input into the shallow feature extraction unit of the second part, Represents the feature map output by the shallow feature extraction unit and input to the deep feature extraction unit, Represents the feature map output by the deep feature extraction unit.

[0060] The third part is the feature weighting unit, which is responsible for focusing the network’s attention on key feature channels. This unit uses an adaptive average pooling layer to Process it in both width and height dimensions to make it equal to , followed by The conventional convolutional layer fuses cross-channel features and then uses Activate to obtain the weight matrix ,at last and Point multiplication obtains the output feature map of the unit , this process is shown in equation (14) and equation (15).

[0061] (14) (15) in, represents the adaptive average pooling operation, express Convolution operation, represents the dot product operation, express activation, Represents the feature map output by the feature extraction unit and input to the feature weighting unit, represents the weight matrix, Represents the output of the feature weighting unit.

[0062] Finally, adopt Conventional convolution refinement And output the result.

[0063] (15) in, express Convolution operation, represents the feature map output by the feature weighting unit, Feature map representing the final output of LFEWB.

[0064] Feature fusion mechanism Aiming at the loss of detail information caused by multiple convolutions and the important contribution of the first-stage subnet in extracting small local feature information, the present invention proposes an efficient feature transfer and fusion module (EFTFB), such as Figure 5 As shown in Figure 2, thanks to the LCA mechanism, the features passed from the first-stage encoder and decoder to the second-stage encoder focus more on the key rain image channel information, effectively enriching the rain image feature information of the second stage.

[0065] Compared with other methods that only use MSE loss, which will inevitably cause image blurring, the present invention integrates the outlier-tolerant Loss, edge loss constrains network training that can improve image visual quality. loss , edge loss They are defined as: (17) (18) in, represents the Laplace operator, represents the predicted rain-free image, Represents a clear, rain-free image Represents the network stage, penalty coefficient According to experience, .

[0066] The total loss function is defined as: (19) in, According to experience, .

[0067] S3. Input the test set obtained in step S1 into the two-stage progressive rain removal network TSPNet trained in step S2, and output the rainy day vehicle detection result.

[0068] The benchmark vehicle detector YOLOv5 is integrated into the image deraining algorithm TSPNet proposed in this paper to complete the detection of vehicles in rainy days. First, the rainy traffic scene image is input into TSPNet, and then the derained image processed by TSPNet is passed to YOLOv5 for vehicle detection, achieving high-precision detection and recognition of vehicles.

[0069] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0070] In yet another embodiment of the present invention, a two-stage progressive rain removal system is provided, which can be used to implement the above-mentioned two-stage progressive rain removal method. Specifically, the two-stage progressive rain removal system includes a data module, a training module and an output module.

[0071] Among them, the data module constructs the vehicle detection dataset MCGVD and divides the vehicle detection dataset MCGVD into a training set and a test set; The training module builds a two-stage progressive deraining network TSPNet that includes an improved U-Net feature extraction network and a feature fusion mechanism, and uses the training set to train the two-stage progressive deraining network TSPNet; The output module inputs the test set into the trained two-stage progressive rain removal network TSPNet. After 1x1 convolution for channel dimension increase, shallow rain streak information is extracted. The feature map enters the improved encoder-decoder architecture to extract deep rain streak information. Finally, the vehicle image is restored to achieve vehicle detection in rainy days.

[0072] In another embodiment of the present invention, a terminal device is provided, the terminal device includes a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for operations based on a two-stage progressive deraining method, including: A vehicle detection dataset MCGVD is constructed and divided into a training set and a test set. A two-stage progressive rain removal network TSPNet, which includes an improved U-Net feature extraction network and a feature fusion mechanism, is constructed and trained using the training set. The step test set is input into the trained two-stage progressive rain removal network TSPNet, and the shallow rain streak information is extracted after channel dimension increase by 1x1 convolution. The feature map enters the improved encoder-decoder architecture, and the deep rain streak information is extracted. Finally, the vehicle image is restored to achieve vehicle detection in rainy days.

[0073] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a terminal device for storing programs and data. It is understandable that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device, and can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that more specific examples (non-exhaustive list) of the computer-readable storage medium here include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0074] Computer readable storage media also include data signals propagated in baseband or as part of a carrier wave, which carry readable program codes. Such propagated data signals can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than a readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0075] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0076] The processor may load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the two-stage progressive rain removal method in the above embodiment; the processor may load and execute the following steps: A vehicle detection dataset MCGVD is constructed and divided into a training set and a test set. A two-stage progressive rain removal network TSPNet, which includes an improved U-Net feature extraction network and a feature fusion mechanism, is constructed and trained using the training set. The step test set is input into the trained two-stage progressive rain removal network TSPNet, and the shallow rain streak information is extracted after channel dimension increase by 1x1 convolution. The feature map enters the improved encoder-decoder architecture, and the deep rain streak information is extracted. Finally, the vehicle image is restored to achieve vehicle detection in rainy days.

[0077] See also Fig.12 The computer device 60 of this embodiment includes: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When the computer program 63 is executed by the processor 61, the two-stage progressive rain removal method in the embodiment is implemented. To avoid repetition, it is not described here one by one. Alternatively, when the computer program 63 is executed by the processor 61, the functions of each model / unit in the fluid composition calculation system in the reservoir transformation wellbore of the embodiment are implemented. To avoid repetition, it is not described here one by one.

[0078] The computer device 60 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will appreciate that Fig.12 This is only an example of the computer device 60 and does not constitute a limitation of the computer device 60. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, etc.

[0079] The processor 61 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0080] The memory 62 may be an internal storage unit of the computer device 60, such as a hard disk or memory of the computer device 60. The memory 62 may also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the computer device 60.

[0081] Furthermore, the memory 62 may include both an internal storage unit of the computer device 60 and an external storage device. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 may also be used to temporarily store data that has been output or is to be output.

[0082] Any reference to a memory, database or other medium used in the embodiments provided in the present application may include at least one of a non-volatile and a volatile memory. Non-volatile memory may include a read-only memory (ROM), a magnetic tape, a floppy disk, a flash memory, an optical memory, a high-density embedded non-volatile memory, a resistive random access memory (ReRAM), a magnetic random access memory (MRAM), a ferroelectric random access memory (FRAM), a phase change memory (PCM), a graphene memory, etc. Volatile memory may include a random access memory (RAM) or an external cache memory, etc. As an illustration and not limitation, RAM may be in various forms, such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), etc.

[0083] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0084] See also Fig.13 The terminal device 600 is an electronic device, and the electronic device is in the form of a general computing device. The components of the electronic device may include but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0085] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above method section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 Follow the steps shown in .

[0086] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0087] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0088] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0089] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0090] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can usually be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0091] Experimental results and analysis All experiments were completed under the Linux operating system, the deep learning framework used Pytorch, and the graphics card used NVIDA RTXA5000.

[0092] Table 1 Dataset Table 1 The datasets

[0093] Quantitative and qualitative comparative experiments on synthetic datasets Peak signal to noise ratio (PSNR) and structural similarity (SSIM) are used as quantitative evaluation indicators. The higher the PSNR and the closer the SSIM is to 1, the better the deraining performance. The quantitative comparison of TSPNet and other advanced deraining algorithms is shown in Table 2 below. The best results are shown in bold and the suboptimal results are shown in underline. Figure 6 This is a visual representation of Table 2.

[0094] Table 2 Comparison results of different rain removal algorithms

[0095] From Table 2 and Figure 6 It can be seen that compared with MSPFN and MPRNet, TSPNet has the best PSNR and SSIM in Test100, Rain100L and Test2800. In Rain100H, PSNR performs best and SSIM performs second best. Taking the average PSNR and SSIM of the five test sets, TSPNet obtains all the superior results. To further intuitively prove the superiority of TSPNet, the deraining effects of different algorithms in different scenarios are selected for visualization, as shown in the figure below. Figure 7 shown.

[0096] Depend on Figure 7 It can be seen that MSPFN, MPRNet and TSPNet can effectively remove most rain streaks. However, a small amount of rain streaks still remain in MSPFN and MPRNet, as shown in the local enlarged part (d) of the residential scene. In contrast, TSPNet has almost no rain streaks. At the same time, as shown in the local enlarged part (b) of the coastal scene, MSPFN has blurred background layer structure and texture loss, while the method of the present invention has a significant rain removal effect, restoring a clean and rain-free background layer without losing details.

[0097] To further demonstrate the generalization of the proposed algorithm, the Realrain485 dataset is constructed, which contains a total of 485 real rainy day images of different scenes. Since Realrain485 does not have a corresponding clear rain-free dataset, the quantitative analysis uses spatial spectral entropy-based quality (SSEQ) and naturalness image quality evaluator (NIQE) as evaluation indicators. The smaller the SSEQ and NIQE values ​​of the derained image, the better the performance of the corresponding deraining algorithm. The quantitative comparison results of different algorithms are shown in Table 3, and the qualitative comparison method is consistent with the method used in Section 3.1.1.

[0098] Table 3 Comparison results of different rain removal algorithms

[0099] As shown in Table 3, the SSEQ and NIQE values ​​of Realrain485 processed by the method of the present invention are the lowest. Compared with MSPFN and MPRNet, TSPNet is 2.0192 and 1.2003 lower than their SSEQ values, and 0.0507 and 0.0779 lower than their NIQE values, respectively. Therefore, TSPNet still has superior deraining performance in real rainy scenes. The deraining effects of different algorithms in different scenes are shown in the figure. Figure 8 shown.

[0100] Depend on Figure 8 It can be seen that TSPNet has a more significant effect in removing rain in real rainy scenes than MPRNet and MSPFN. For example, it can be clearly seen from the local magnification of the lawn (b) that TSPNet has the least rain streak residue. At the same time, it can be observed from the local magnification of the road scene (d) that compared with MPRNet and MSPFN, TSPNet can still effectively remove rain streaks even when the brightness of the image background layer and the rain streak layer is similar.

[0101] Ablation experiment To verify the contribution of LFEWB and EFTFB to the deraining performance of TSPNet, this section conducts an ablation test on Rain100L. The following four schemes are used to evaluate the impact of LFEWB and EFTFB on the deraining performance of TSPNet, and the results are shown in Table 4.

[0102] Table 4 Comparison of ablation experiment results on Rain100L

[0103] As shown in Table 4, the values ​​of the two indicators of scheme 4 with LFEWB and EFTFB are superior, the values ​​of the two indicators of scheme 2 with LFEWB removed and scheme 3 with EFTFB removed are significantly reduced, and the values ​​of the two indicators of scheme 1 with LFEWB and EFTFB removed are the lowest. Therefore, the coordinated use of LFEWB and EFTFB makes an important contribution to ensuring the deraining performance of TSPNet.

[0104] Vehicle detection experiment The rainy day vehicle detection method integrated into the baseline vehicle detector YOLOv5 is as follows Fig. 9 As shown in the figure, it includes a rainy traffic scene image input module, a rain removal algorithm TSPNet module, a reference vehicle detector YOLOv5 module, and a vehicle detection result output module. The rainy traffic scene image is first input into TSPNet, then the rain removal image is passed into YOLOv5, and YOLOv5 outputs the final vehicle detection result.

[0105] According to the working environment of ICV, a mixed vehicle dataset MCGVD is constructed, which contains 2000 clear rain-free traffic scene images and 500 synthetic rainy traffic scene images, and the label category is set to: Vehicle. Taking into account the existing synthetic rain image methods, the method of synthesizing rain images in the present invention is specifically set to: Raindrop size range: ,Rain streak direction angle: , Rain streak length range: Since MCGVD contains scenes with large feature differences in cities, suburbs, and villages at home and abroad (2357 traffic scene images are provided by the COCO dataset, and 143 traffic scene images are collected by the authors of the present invention using a GoPro10BLACK device in Xi'an city and surrounding towns), it can be effectively used to verify the generalization and robustness of the proposed deraining algorithm.

[0106] In order to verify the effectiveness of TSPNet in improving the detection accuracy of YOLOv5, this paper selects 2000 clear rain-free traffic scene images from MCGVD to train YOLOv5. The input image size during training is 640 640, batchsize is set to 32, epoch is set to 300, and the initial learning rate is 0.01. The trained weights are used to conduct comparative experiments on 500 synthetic rainy traffic scene images. The specific experimental methods are as follows Fig.10 shown.

[0107] Experimental Results The precision rate P, recall rate R, and average precision mAP are used as evaluation indicators. The three evaluation indicators are defined as (twenty one) (twenty two) (twenty three) in, represents the number of correctly identified vehicles, represents the number of vehicles that were misidentified, Represents the number of vehicles that were not identified. Represents the number of vehicle types, Representative mAP@0.5\mAP@0.95 respectively represent that when the overlap area between the predicted box and the true box exceeds 50%\95%, the detection result is considered correct. The experimental results are shown in Table 5.

[0108] Table 5 Comparison results of different vehicle detection schemes

[0109] As shown in Table 5, compared with solution A, solution B's P, R, mAP@0.5 and mAP@0.95 are increased by 3%, 8.9%, 7.6% and 6.7% respectively. This shows that the vehicle feature information obtained by the benchmark vehicle detector YOLOv5 in solution B is richer than that in solution A, indicating the positive contribution of TSPNet to improving vehicle detection accuracy and proving the effectiveness of the deraining algorithm proposed in this invention.

[0110] In order to more intuitively demonstrate the superiority of the vehicle detection method proposed in the present invention, the test results of different scenarios are selected for visualization, such as Fig.11 shown.

[0111] Depend on Fig.11 It can be seen that in the local zoomed-in area of ​​the rural road, Scheme A identified the flowers blocked by rain streaks as vehicles, and Scheme B benefited from TSPNet and did not make any false detections. In the local zoomed-in area of ​​the urban road, the vehicle detection confidence of Scheme B was 0.96, which was 0.18 higher than that of Scheme A (0.78). In the local zoomed-in area of ​​the suburban road, because the rain streaks seriously blocked the vehicle background layer, Scheme A missed detection and did not detect any vehicles. Scheme B successfully detected the vehicle and maintained a high detection confidence because TSPNet achieved effective perception and removal of rain streaks.

[0112] In summary, the present invention is a two-stage progressive deraining method and system, which achieves the goal of efficient deraining and detection of vehicle images under complex weather conditions through its innovative structure and processing flow. This method ensures the training and verification of the model in a variety of scenarios by constructing and dividing a special vehicle detection data set, and enhances the generalization ability of the model. By improving the feature extraction network and feature fusion mechanism of U-Net, the model can process images hierarchically, first extract shallow rain streak information, and then deeply extract deep features, and finely identify and process raindrops, which significantly improves the deraining effect. The encoder-decoder architecture of the system effectively captures and reconstructs image features, while the cross-stage fusion mechanism ensures the efficient transmission and sharing of information between different stages, further optimizes the deraining effect, and fully restores the original quality of the image. Especially in the second stage, the system uses lightweight feature extraction and weighting modules, combined with batch normalization and deep convolution technology, which not only accelerates network training, but also improves the efficiency and accuracy of feature extraction. The introduction of the lightweight channel attention mechanism enables the network to identify and remove raindrops more accurately while retaining the subtle textures and contours of the image, achieving higher quality image restoration.

[0113] In general, the present invention not only improves the rain removal effect, but also provides strong support for improving the accuracy of vehicle detection in rainy environments, significantly improves the performance of the vehicle detection system under complex weather conditions, and has important practical application value.

[0114] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0115] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0116] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0117] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0118] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0119] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0120] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0121] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0122] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0123] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0124] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A two-stage progressive rain removal method, characterized in that: The following steps are involved: S1, construct the vehicle detection dataset MCGVD, and divide the vehicle detection dataset MCGVD into a training set and a test set; S2, constructing a two-stage progressive deraining network TSPNet including an improved U-Net feature extraction network and a feature fusion mechanism, and training the two-stage progressive deraining network TSPNet using the training set obtained in step S1; S3. Input the test set obtained in step S1 into the two-stage progressive rain removal network TSPNet trained in step S2. After channel dimension increase by 1x1 convolution, shallow rain streak information is extracted. The feature map enters the improved encoder-decoder architecture to extract deep rain streak information. Finally, the vehicle image is restored to achieve vehicle detection in rainy days.

2. The two-stage progressive rain removal method according to claim 1 is characterized in that: In step S2, the two-stage progressive rain removal network TSPNet includes two stages. The first stage is the initial image rain removal and complex sub-network, and the second stage is the terminal image rain removal and complex sub-network. The sub-networks of the first and second stages both contain an encoder-decoder architecture and generate information interaction through a cross-stage fusion mechanism.

3. The two-stage progressive rain removal method according to claim 2 is characterized in that: In the first stage, the initial image is derained and the subnet is restored. , the specific calculation is divided into: The first part is Conventional convolutional layer, which performs dimension-upgrading operation on the input rainy image; The second part is the encoder-decoder architecture, which is responsible for mining the global rain map feature information and reconstructing the restored image from the feature map output by the first part; The third part is The conventional convolutional layer performs dimensionality reduction on the feature map output by the second part.

4. The two-stage progressive rain removal method according to claim 3 is characterized in that: The dimensionality reduction operation of the feature map output from the second part is as follows: in, express Convolution operation, represents the feature map output by the second part and input to the third part, represent The feature map after the convolution operation is also the restored image output by the first stage.

5. The two-stage progressive rain removal method according to claim 2 is characterized in that: The second stage is to remove rain from the end image and reconstruct the subnet. , the specific calculation is divided into: The first part is Conventional convolutional layer: firstly, the de-rained image output from the first stage and the rainy image input from the second stage are feature-fused, and then this part performs dimension-upgrading operation; The second part is the encoder-decoder architecture, which is responsible for the fusion of the rain streak feature information output by the first part of this stage and the second part of the previous stage, as well as the mining of the global rain map feature information and the reconstruction of the restored image; The third part is The conventional convolution layer is responsible for the dimensionality reduction operation of the output feature map of the second part.

6. The two-stage progressive rain removal method according to claim 5, characterized in that: The dimensionality reduction operation of the output feature map of the second part is as follows: in, represents the feature map output by the second part and input to the third part, represent Feature map after convolution operation.

7. The two-stage progressive rain removal method according to claim 2 is characterized in that: The two-stage progressive deraining network TSPNet includes a lightweight feature extraction and weighting module and a high-efficiency feature transfer and fusion module; the lightweight feature extraction and weighting module integrates batch normalization to accelerate network training, while using deep convolution to replace part of the conventional convolution and integrating a lightweight channel attention mechanism.

8. The two-stage progressive rain removal method according to claim 7, characterized in that: The lightweight feature extraction and weighting module is as follows: Batch normalization layer, responsible for improving the generalization of the module and accelerating network training; The feature extraction unit GRSPB includes a shallow feature extraction unit and a deep feature extraction unit. In the shallow feature extraction unit, the input feature map through The conventional convolutional layer performs dimensionality increase and realizes cross-channel information fusion; then The deep convolutional layer fully extracts the channel dimension rain image feature information, and then uses Activate to enhance nonlinear features, and finally The conventional convolutional layer completes the global shallow rain image feature mining; in the deep feature extraction unit, Sequentially Conventional convolutional layer, The depth of the convolutional layer and After activating the layer, The conventional convolutional layer completes the mining of global deep rain image features; Feature weighting unit, using adaptive average pooling layer Process it in both width and height dimensions to make it equal to , followed by The conventional convolutional layer fuses cross-channel features and then uses Activate to obtain the weight matrix , and Point multiplication obtains the output feature map of the unit ; Finally adopted Conventional convolution refines the output feature map And output the result.

9. The two-stage progressive rain removal method according to claim 7, characterized in that: The total loss function is defined as: in, for loss, is the edge loss, represents the predicted rain-free image, Represents a clear, rain-free image. Represents the network stage, According to experience, .

10. A two-stage progressive rain removal system, characterized in that: include: Data module, constructs the vehicle detection dataset MCGVD, and divides the vehicle detection dataset MCGVD into a training set and a test set; The training module builds a two-stage progressive deraining network TSPNet that includes an improved U-Net feature extraction network and a feature fusion mechanism, and uses the training set to train the two-stage progressive deraining network TSPNet; The output module inputs the test set into the trained two-stage progressive rain removal network TSPNet. After 1x1 convolution for channel dimension increase, shallow rain streak information is extracted. The feature map enters the improved encoder-decoder architecture to extract deep rain streak information. Finally, the vehicle image is restored to achieve vehicle detection in rainy days.

Images