An image detection method and system under rainy conditions

Abstract

The application discloses a kind of image detection method and system under rain condition, it is related to target detection technical field.The application carries out image detection under rain condition using cascade network, the cascade network includes rain removal network and target detection network.After training data is sent into rain removal network, the reconstruction loss of clean image calculated from rain-free image is output with the result, and the rain-free image and the image after rain removal are sent into target detection network.The initialization model of target detection network is the model trained using rain-free image.The feature map is obtained after rain-free image and reconstructed rain-free image after rain removal pass through feature extraction layer in target detection network, and the feature map of the two is used to calculate perception loss.The joint loss of reconstruction loss, perception loss and target detection loss is added as network, and the gradient of joint loss is fed back to rain removal network to update network weight.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, and in particular relates to an image detection method and system under rainy conditions. Background Technology

[0002] Image denoising is a fundamental technical field in image processing, aiming to recover clean image signals from noisy images. Traditional image denoising methods mostly utilize local or non-local image features. In recent years, due to the widespread use of deep learning in the image field, many image denoising algorithms based on deep neural networks have emerged. In real life, some objective factors, such as rain, fog, and sandstorms, can cause significant noise in acquired images. Currently, algorithms for fog denoising are relatively mature, and the atmospheric scattering physical model for handling foggy image noise has been widely used. However, rainy conditions are more complex; the density and direction of rain streaks pose challenges to image denoising. While some effective algorithms for rain denoising have been proposed, some algorithms suffer from loss of image details and neglect of semantic information while eliminating rain streaks, leading to a decline in the quality of the de-rained image.

[0003] Object detection algorithms have made significant progress, evolving from traditional methods to deep learning-based algorithms. Existing deep learning-based object detection algorithms are mainly divided into two categories: one-stage and two-stage. One-stage is an end-to-end detection method that directly extracts object features to obtain the object's type and location, achieving faster detection results while maintaining accuracy. The widely used YOLO series has undergone continuous updates and demonstrates good performance in both accuracy and speed. It defines object detection as a regression problem, applying a single convolutional neural network to the entire image, segmenting the image into a grid, and predicting the class probability and bounding box for each grid. Its detection speed far exceeds that of two-stage algorithms like R-CNN and Fast R-CNN, making it more suitable for scenarios requiring real-time object detection.

[0004] Foggy or rainy weather conditions often significantly reduce road visibility, severely impacting drivers' ability to observe road conditions and enabling authorities to monitor them, potentially leading to a series of accidents. Therefore, a method is needed to eliminate the influence of rain on target detection performance, ensuring accuracy in rainy weather and improving driving and road safety.

[0005] Researchers have proposed many solutions for dealing with noise in rainy images from various perspectives. Traditional methods for improving target detection performance in rainy conditions often include enhancing the visibility of blurred images as a preprocessing step. Image deblurring is beneficial for the visual perception of image quality by the human eye or surveillance equipment, but it does not guarantee that the preprocessed image will necessarily improve target detection performance. Currently, some researchers have proposed algorithms for target detection in rainy conditions, but few methods combine image deblurring algorithms with target detection algorithms. Current research can be divided into three strategic directions:

[0006] (1) Cascade the image filter and the target detection algorithm to achieve target detection in rainy weather;

[0007] (2) The image deraining algorithm and the target detection algorithm are cascaded together. The two networks adjust their weights according to the loss function to achieve target detection in rainy weather.

[0008] (3) The image deraining algorithm and the target detection algorithm are cascaded together. The loss functions of the two networks are added together to form a joint loss function, which is used as the loss function of the entire network to update the weights of the deraining and target detection networks, thereby realizing target detection in rainy weather.

[0009] The first strategy, which uses filters to remove rain streaks from rainy images, also removes original image details, significantly reducing the image quality entering the object detection network. The second strategy, while simpler, involves two networks updating independently without interaction, and cannot guarantee that weight updates will improve detection accuracy. The third strategy, which jointly minimizes the image reconstruction loss and the object detection loss, improves object detection performance under the guidance of image semantic information, but increases computational complexity and may also cause a loss of semantically perceptual details needed for training the object detection network. Summary of the Invention

[0010] Rain is one of the most common types of special weather. Rain can blur the vision of drivers and affect real-time road monitoring by relevant departments. Although some vehicles and road monitoring equipment are now capable of handling special weather conditions, rainy weather scenarios are complex. Different levels of rainfall and the direction of rain streaks can all affect the detection results of the equipment, and the ability to detect targets in rainy weather still needs to be improved.

[0011] To address the aforementioned technical problems, the present invention aims to improve the target detection accuracy of vehicle and road monitoring equipment for pedestrians and other vehicles under rainy conditions, thereby proposing an image detection scheme for rainy conditions.

[0012] The first aspect of this invention discloses an image detection method under rainy conditions. The method employs a cascaded network to perform image detection under rainy conditions, the cascaded network including a rain removal network and a target detection network, the method comprising:

[0013] Step S1: Input the standard rainless image and the synthetic rainy day image into the rain removal network. The rain removal network performs rain removal operation on the synthetic rainy day image to obtain a first image, and calculates the reconstruction loss based on the first image and the standard rainless image.

[0014] Step S2: Input the first image and the standard rainless image into the target detection network. The target detection network extracts features from the first image and the standard rainless image respectively to obtain a first feature map and a second feature map, and calculates the perceptual loss based on the first feature map and the standard feature map.

[0015] Step S3: The target detection network performs target detection on the first feature map and the second feature map respectively to obtain a first detection result and a second detection result, and calculates the target detection loss based on the first detection result and the second detection result;

[0016] Step S4: The reconstruction loss, the perception loss, and the target detection loss are used as the joint loss of the cascaded network. The joint loss is fed back to the rain removal network to update the network parameters. The cascaded network with updated network parameters is then used to detect images under rainy conditions.

[0017] According to the method of the first aspect, the structure of the rain removal network, in sequence according to the image input direction, includes: a low-pass filter, a first convolutional layer, a first batch regularization layer, an activation function, a second convolutional layer, and a second batch regularization layer; the rain removal network uses the low-pass filter to extract the base layer of the synthesized rainy day image, subtracts the base layer from the synthesized rainy day image to obtain the high-frequency detail layer of the synthesized rainy day image, and the reconstruction loss L... D for:

[0018]

[0019] Among them, X i Let Y represent the i-th composite rainy day image. i Let X represent the i-th standard rainless image, F represent the norm, N represent the number of images, f(·) represent the output of the second convolutional layer in the rain removal network, and X represent the standard rainless image. i，detail This refers to the high-frequency detail layer.

[0020] According to the method of the first aspect, the object detection network is a YOLO-v51 model, including an input module, a backbone module, a neck module, and a prediction module; wherein:

[0021] At the input end, Mosaic is used to perform data augmentation on the input image, and adaptive anchor box calculation and adaptive image scaling are performed.

[0022] The Backbone module slices the input image and concentrates the image size information into the channel space without losing image information, thus obtaining a feature map with double downsampling. The Backbone module adopts a CSP structure.

[0023] The Neck module uses an FPN+PAN structure and CSPNet to fuse the features in the feature map;

[0024] When performing the target detection, the prediction module determines the approximate location of the target based on the confidence of the prediction box, filters the prediction boxes containing the target according to the MSE algorithm, removes duplicate prediction boxes containing the same target, and takes the classification index corresponding to the highest classification probability based on the classification probability of the filtered prediction boxes, thereby obtaining the detection result.

[0025] The target detection loss consists of the prediction box loss, confidence loss, and classification loss.

[0026] According to the method of the first aspect, the perceptual loss is:

[0027]

[0028] Where g(·) represents the output of the feature extraction layer of the target detection network with size H*W, z represents the label required for training the target detection network, and F(x) represents the first image.

[0029] A second aspect of the present invention discloses an image detection system under rainy conditions; the system employs a cascaded network to perform image detection under rainy conditions, the cascaded network including a rain removal network and a target detection network, the system comprising:

[0030] The first processing unit is configured to: input a standard rainless image and a synthetic rainy day image into the rain removal network, the rain removal network performs a rain removal operation on the synthetic rainy day image to obtain a first image, and calculate a reconstruction loss based on the first image and the standard rainless image;

[0031] The second processing unit is configured to: input the first image and the standard rainless image into the target detection network, wherein the target detection network performs feature extraction on the first image and the standard rainless image respectively to obtain a first feature map and a second feature map, and calculates a perceptual loss based on the first feature map and the standard feature map;

[0032] The third processing unit is configured to: perform target detection on the first feature map and the second feature map respectively to obtain a first detection result and a second detection result, and calculate the target detection loss based on the first detection result and the second detection result;

[0033] The fourth processing unit is configured to: use the reconstruction loss, the perception loss, and the target detection loss as the joint loss of the cascaded network, feed the joint loss back to the rain removal network to update the network parameters, and use the cascaded network with the updated network parameters to detect images under rainy conditions.

[0034] According to the system of the second aspect, the structure of the rain removal network, in sequence according to the image input direction, includes: a low-pass filter, a first convolutional layer, a first batch regularization layer, an activation function, a second convolutional layer, and a second batch regularization layer; the rain removal network uses the low-pass filter to extract the base layer of the synthesized rainy image, subtracts the base layer from the synthesized rainy image to obtain the high-frequency detail layer of the synthesized rainy image, and the reconstruction loss L... D for:

[0035]

[0036] Among them, X i Let Y represent the i-th composite rainy day image. i Let X represent the i-th standard rainless image, F represent the norm, N represent the number of images, f(·) represent the output of the second convolutional layer in the rain removal network, and X represent the standard rainless image. i，detail This refers to the high-frequency detail layer.

[0037] According to the system in the second aspect, the object detection network is a YOLO-v51 model, including an input module, a backbone module, a neck module, and a prediction module; wherein:

[0038] At the input end, Mosaic is used to perform data augmentation on the input image, and adaptive anchor box calculation and adaptive image scaling are performed.

[0039] The Backbone module slices the input image and concentrates the image size information into the channel space without losing image information, thus obtaining a feature map with double downsampling. The Backbone module adopts a CSP structure.

[0040] The Neck module uses an FPN+PAN structure and CSPNet to fuse the features in the feature map;

[0041] When performing the target detection, the prediction module determines the approximate location of the target based on the confidence of the prediction box, filters the prediction boxes containing the target according to the MSE algorithm, removes duplicate prediction boxes containing the same target, and takes the classification index corresponding to the highest classification probability based on the classification probability of the filtered prediction boxes, thereby obtaining the detection result.

[0042] The target detection loss consists of the prediction box loss, confidence loss, and classification loss.

[0043] According to the system of the second aspect, the perceptual loss is:

[0044]

[0045] Where g(·) represents the output of the feature extraction layer of the target detection network with size H*W, z represents the label required for training the target detection network, and F(x) represents the first image.

[0046] A third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the image detection method under rainy conditions as described in any one of the first aspects of the present disclosure.

[0047] A fourth aspect of this invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the image detection method under rainy conditions as described in any one of the first aspects of this disclosure.

[0048] In summary, the technical solution proposed in this invention has low computational complexity and strong generalization ability. It integrates image deraining and object detection into a single network, achieving end-to-end object detection. This patent focuses solely on rainy scenes. It uses rainy images synthesized from clean, rain-free images as the dataset input to the deraining network. The resulting clean, rain-free images are then input into a cascaded object detection network for training. The loss function of the entire network comprises the reconstruction loss of the clean image from the deraining network, the object detection loss, and the perception loss. Only the gradient of this loss function is used to update the weights of the deraining network, keeping the object detection network unaffected. This preserves as much semantic perception detail as possible, reduces the overall computational complexity of the network, and improves object detection accuracy. Attached Figure Description

[0049] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0050] Figure 1 This is a schematic diagram of a cascaded network according to an embodiment of the present invention;

[0051] Figure 2 This is a comparative schematic diagram of pixel value histograms according to an embodiment of the present invention;

[0052] Figure 3 This is a schematic diagram of a rain removal network according to an embodiment of the present invention;

[0053] Figure 4 This is a flowchart illustrating the execution of target detection according to an embodiment of the present invention;

[0054] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] The first aspect of this invention discloses an image detection method under rainy conditions. The method employs a cascaded network to perform image detection under rainy conditions, the cascaded network including a rain removal network and a target detection network, the method comprising:

[0057] Step S1: Input the standard rainless image and the synthetic rainy day image into the rain removal network. The rain removal network performs rain removal operation on the synthetic rainy day image to obtain a first image, and calculates the reconstruction loss based on the first image and the standard rainless image.

[0058] Step S2: Input the first image and the standard rainless image into the target detection network. The target detection network extracts features from the first image and the standard rainless image respectively to obtain a first feature map and a second feature map, and calculates the perceptual loss based on the first feature map and the standard feature map.

[0059] Step S3: The target detection network performs target detection on the first feature map and the second feature map respectively to obtain a first detection result and a second detection result, and calculates the target detection loss based on the first detection result and the second detection result;

[0060] Step S4: The reconstruction loss, the perception loss, and the target detection loss are used as the joint loss of the cascaded network. The joint loss is fed back to the rain removal network to update the network parameters. The cascaded network with updated network parameters is then used to detect images under rainy conditions.

[0061] Specifically, cascaded network structures are as follows: Figure 1 As shown, the network mainly consists of two subnetworks: a rain removal network and an object detection network. The input to the network is a rainy day image synthesized from rainless images. This is because real-world rainy day images are difficult to collect; therefore, synthesized rainy day images are used as the network's training dataset. After the training data passes through the rain removal network, the output is used to calculate the reconstruction loss of the clean image using the rainless images. Simultaneously, the rain-removed image and the rainless image are fed into the object detection network. The initial model of the object detection network is trained using rainless images. The rainless image and the reconstructed rainless image pass through the feature extraction layer in the object detection network to obtain feature maps. These feature maps are then used to calculate the perceptual loss. The reconstruction loss, perceptual loss, and object detection loss are summed to form the network's joint loss. The gradient of this joint loss is fed back to the rain removal network to update the network weights.

[0062] In some embodiments, since rainy weather scenarios are complex and variable, data collection in actual scenarios would lack diversity and incur high human and physical costs. Therefore, the data used in this invention comes from:

[0063] (1) Collect images of roads in rainless weather, generate random noise in the images, and perform Gaussian filtering, motion blur, and transparency assignment to synthesize the rain map required for training.

[0064] (2) Download the existing rainy day dataset from the Internet, and select images including people and vehicles to add to the training dataset;

[0065] (3) Collect real rainy day images as test datasets.

[0066] In some embodiments, since the YOLO-v5 network needs to be initialized, it needs to be trained to obtain an initial model. A dataset for training is created using rainless images. Pedestrians and vehicles on the images used as the training set are labeled using the labelimg software to obtain a label file for each image. A text file containing the category and location information of the detected objects in all images of the training set is then generated based on the label files.

[0067] The deraining network structure, arranged sequentially according to the image input direction, includes: a low-pass filter, a first convolutional layer, a first batch regularization layer, an activation function, a second convolutional layer, and a second batch regularization layer. The deraining network uses the low-pass filter to extract the base layer of the synthesized rainy image, subtracts this base layer from the synthesized rainy image to obtain the high-frequency detail layer of the synthesized rainy image, and the reconstruction loss L... D for:

[0068]

[0069] Among them, X i Let Y represent the i-th composite rainy day image. i Let X represent the i-th standard rainless image, F represent the norm, N represent the number of images, f(·) represent the output of the second convolutional layer in the rain removal network, and X represent the standard rainless image. i，detail This refers to the high-frequency detail layer.

[0070] Specifically, the rain removal network used in this invention is a deep residual network improved based on the characteristics of residual networks. It enhances deep learning by changing the mapping range, dividing the input into a basic layer and a high-frequency detail layer, and directly mapping the input to the output to enhance image features for rain removal. This network can effectively distinguish between edges and rain streaks. By using the residual network as a parameter layer, it obtains deeper image features, solving the problem that removing rainwater may affect image detail features, thus achieving a better rain removal effect.

[0071] The input image to the network is a rainy image, and its corresponding clean image is Y. If we train a deep convolutional network directly, the loss function L is:

[0072]

[0073] Where f(·) represents the output of the convolutional neural network, N is the number of images used for network training, and F is the norm. However, the final training result obtained according to equation (1) has a serious color deviation. This is because the network learns a mapping range from X to Y that includes all possible pixel values, making it difficult to learn a well-fitting regression function; at the same time, directly feeding images into the network for training may lead to the gradient vanishing problem. This invention optimizes the learning effect by reducing the mapping range. Figure 2As shown, compared to a rain image X and a clean image Y, the residual (YX) has a significantly smaller range of pixel values. Therefore, inputting the residual (YX) into the network parameter layer helps the network learn the mapping between images. Since rain streaks are usually white, (YX) is typically negative, hence it is defined here as a negative residual mapping. Using a regular residual network might lose some image details, but adding a negative residual mapping can preserve color and object details.

[0074] To further improve the rain removal effect, the complete image input to the network parameter layer is replaced with the high-frequency detail layer of the image. First, the rainy image is passed through a low-pass filter to obtain the base layer. Then, the base layer is subtracted from the input rainy image to obtain the high-frequency detail layer. This processing method utilizes the sparsity of the image; the detail layer has even greater sparsity than the original image, further reducing the mapping range. This allows the network to learn only the mapping between high-frequency input details and high-frequency outputs, greatly simplifying the network's learning process.

[0075] Based on the above, the final loss function model is obtained, as shown in equation (2):

[0076]

[0077] Where X i,detail It is the detail layer of the i-th rain map.

[0078] In summary, we obtain the following: Figure 3 The network architecture is shown. ReLU is the activation function used in the network, and BN stands for Batch Normalization, which is used to regularize the data.

[0079] In some embodiments, the object detection network is a YOLO-v51 model, including an input module, a backbone module, a neck module, and a prediction module; wherein:

[0080] At the input end, Mosaic is used to perform data augmentation on the input image, and adaptive anchor box calculation and adaptive image scaling are performed.

[0081] The Backbone module slices the input image and concentrates the image size information into the channel space without losing image information, thus obtaining a feature map with double downsampling. The Backbone module adopts a CSP structure.

[0082] The Neck module uses an FPN+PAN structure and CSPNet to fuse the features in the feature map;

[0083] When performing the target detection, the prediction module determines the approximate location of the target based on the confidence of the prediction box, filters the prediction boxes containing the target according to the MSE algorithm, removes duplicate prediction boxes containing the same target, and takes the classification index corresponding to the highest classification probability based on the classification probability of the filtered prediction boxes, thereby obtaining the detection result.

[0084] The target detection loss consists of the prediction box loss, confidence loss, and classification loss.

[0085] The object detection network used in this invention is YOLO-v5. YOLO-v5 is the latest version of the YOLO series of object detection algorithms. This network combines various mainstream data augmentation, model training, and model post-processing methods. YOLO-v5 is extremely fast, achieving 140 FPDS, and also has very high accuracy, reaching 0.895 mPA. Furthermore, it is relatively small in size, around 27 MB, which is a significant advantage among existing object detection algorithms.

[0086] Currently, there are four versions of YOLO-v5, progressively increasing in network depth and width. In real-world applications, there are certain requirements for real-time pedestrian and vehicle detection, and performance is also needed for detecting small targets, ensuring the detection of distant pedestrians and vehicles. Considering both speed and accuracy, this invention selects YOLO-v5l from the four versions. This model shows a significant improvement in accuracy compared to the version with the smallest depth and width, and is significantly faster than the version with the largest depth and accuracy. The model mainly consists of four parts: input, backbone, neck, and prediction.

[0087] On the input side, YOLO v5 uses Mosaic data augmentation to improve the training speed and accuracy of the model, and employs adaptive anchor box calculation and adaptive image scaling to simplify the data processing steps on the input side and improve the overall inference speed of the algorithm.

[0088] The backbone uses a focus module to slice the image, concentrating image size information into the channel space without information loss, ultimately resulting in a double-downsampled feature map. Furthermore, YOLO-v5 uses a CSP structure in the backbone network, enhancing the learning capabilities of the convolutional neural network while reducing computational cost.

[0089] Like YOLO-v4, the Neck section uses an FPN+PAN structure. The difference is that YOLO-v5 uses CSPNet, which improves the network's feature fusion capabilities.

[0090] When detecting targets, the YOLO-v5 network first determines whether the confidence score of each predicted bounding box exceeds a threshold, thus obtaining the approximate location of the target. Then, it uses the MSE algorithm to filter the predicted bounding boxes containing the target, removing duplicate rectangles containing the same target. Finally, based on the classification probabilities of the filtered predicted bounding boxes, it takes the classification index corresponding to the highest probability to obtain the target's category. Therefore, its loss function consists of bounding box loss, confidence score loss, and classification loss.

[0091] In some embodiments, the perceived loss is:

[0092]

[0093] Where g(·) represents the output of the feature extraction layer of the target detection network with size H*W, z represents the label required for training the target detection network, and F(x) represents the first image.

[0094] In this invention, the network updates its parameters based on the gradient of the joint loss function. The cascaded network trained with the joint loss function not only improves the perceptual quality of de-rained images through image semantic guidance, solving the problem that the accuracy of target detection might decrease due to the loss of detailed features after de-raining, but also helps improve the accuracy of target detection tasks in rainy weather. The joint loss function is shown in equation (3), including the reconstruction loss L of the de-raining network on the rainless image. D (X,Y), the loss L of the object detection network O (F(x),Z) and perceptual loss L P (F(X),Y).

[0095] L(X,Y,Z)=L D (X,Y)+α P L P (F(X),Y)+α O L O (F(x),Z)

[0096] Where F(X) is the reconstructed rain-free map output by the rain removal network, Z is the label required for training the detection network, and α P α O These are the coefficients of the corresponding loss function.

[0097] In this invention, perceptual loss is defined as the Euclidean distance between the features of the reconstructed image and the clean reference image, expressed as follows:

[0098]

[0099] Where g(·) is the output of the feature extraction layer of the target detection network with size H*W.

[0100] In the entire network, only the weights of the rain removal network are updated based on the backpropagation error. This strategy is adopted to make the trained rain removal network more robust, while also generalizing the rain removal module to other high-level vision tasks without requiring fine-tuning of other high-level vision networks. This approach not only gives the rain removal network some generalization ability but also maintains the consistent performance of the object detection network for both rainy and rainless images.

[0101] In some embodiments, the method flow proposed in this invention is as follows: Figure 4 As shown, the first step is to create the dataset required for network training. Road images are collected in both rainy and dry weather. The rainy images are then combined with the dry images to create a rainy image. A portion of the publicly available rainy day dataset is used as the training set, and the collected real rainy images are used as the test set. The rainless images are labeled using the labelimg software to create the dataset needed for pre-training the object detection network.

[0102] The object detection network is trained using rainless images and corresponding label files to obtain an initialized model. Synthesized rainy images and their corresponding rainless images are then fed into a cascaded network for training. The rain removal network is updated based on the joint loss function until the object detection network converges and reaches its highest accuracy.

[0103] A second aspect of the present invention discloses an image detection system under rainy conditions; the system employs a cascaded network to perform image detection under rainy conditions, the cascaded network including a rain removal network and a target detection network, the system comprising:

[0104] The first processing unit is configured to: input a standard rainless image and a synthetic rainy day image into the rain removal network, the rain removal network performs a rain removal operation on the synthetic rainy day image to obtain a first image, and calculate a reconstruction loss based on the first image and the standard rainless image;

[0105] The second processing unit is configured to: input the first image and the standard rainless image into the target detection network, wherein the target detection network performs feature extraction on the first image and the standard rainless image respectively to obtain a first feature map and a second feature map, and calculates a perceptual loss based on the first feature map and the standard feature map;

[0106] The third processing unit is configured to: perform target detection on the first feature map and the second feature map respectively to obtain a first detection result and a second detection result, and calculate the target detection loss based on the first detection result and the second detection result;

[0107] The fourth processing unit is configured to: use the reconstruction loss, the perception loss, and the target detection loss as the joint loss of the cascaded network, feed the joint loss back to the rain removal network to update the network parameters, and use the cascaded network with the updated network parameters to detect images under rainy conditions.

[0108] According to the system of the second aspect, the structure of the rain removal network, in sequence according to the image input direction, includes: a low-pass filter, a first convolutional layer, a first batch regularization layer, an activation function, a second convolutional layer, and a second batch regularization layer; the rain removal network uses the low-pass filter to extract the base layer of the synthesized rainy image, subtracts the base layer from the synthesized rainy image to obtain the high-frequency detail layer of the synthesized rainy image, and the reconstruction loss L... D for:

[0109]

[0110] Among them, X i Let Y represent the i-th composite rainy day image. i Let X represent the i-th standard rainless image, F represent the norm, N represent the number of images, f(·) represent the output of the second convolutional layer in the rain removal network, and X represent the standard rainless image. i，detail This refers to the high-frequency detail layer.

[0111] According to the system in the second aspect, the object detection network is a YOLO-v51 model, including an input module, a backbone module, a neck module, and a prediction module; wherein:

[0112] At the input end, Mosaic is used to perform data augmentation on the input image, and adaptive anchor box calculation and adaptive image scaling are performed.

[0113] The Backbone module slices the input image and concentrates the image size information into the channel space without losing image information, thus obtaining a feature map with double downsampling. The Backbone module adopts a CSP structure.

[0114] The Neck module uses an FPN+PAN structure and CSPNet to fuse the features in the feature map;

[0115] When performing the target detection, the prediction module determines the approximate location of the target based on the confidence of the prediction box, filters the prediction boxes containing the target according to the MSE algorithm, removes duplicate prediction boxes containing the same target, and takes the classification index corresponding to the highest classification probability based on the classification probability of the filtered prediction boxes, thereby obtaining the detection result.

[0116] The target detection loss consists of the prediction box loss, confidence loss, and classification loss.

[0117] According to the system of the second aspect, the perceptual loss is:

[0118]

[0119] Where g(·) represents the output of the feature extraction layer of the target detection network with size H*W, z represents the label required for training the target detection network, and F(x) represents the first image.

[0120] A third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the image detection method under rainy conditions as described in any one of the first aspects of the present disclosure.

[0121] Figure 5 This is a structural diagram of an electronic device according to an embodiment of the present invention, such as... Figure 5 As shown, the electronic device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, Near Field Communication (NFC), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0122] Those skilled in the art will understand that Figure 5 The structure shown is merely a structural diagram of the part related to the technical solution of this disclosure and does not constitute a limitation on the electronic device to which the solution of this application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0123] A fourth aspect of this invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the image detection method under rainy conditions as described in any one of the first aspects of this disclosure.

[0124] In summary, this invention proposes a method for pedestrian and vehicle detection under rainy conditions. The proposed network uses a cascaded improved deep residual deraining network and a YOLO-v5 object detection network to quickly and accurately detect pedestrians and vehicles on roads in rainy scenarios. The deraining network uses a deep residual network to directly reduce the mapping range from input to output, simplifying the network learning process. It also utilizes prior image domain knowledge to focus on high-frequency details during training, thereby eliminating background interference and focusing the model on the rain structure in the image, preserving as much image detail as possible while restoring the image. Addressing the issue that image denoising may cause the restored noise-free image to lose details, affecting object detection accuracy, this invention uses the deraining network loss, object detection loss, and perceptual loss as a joint loss of the entire network to update the weights of the deraining network. Guided by image semantic information, the deraining network can further improve visual quality, demonstrating the importance of semantic information for image denoising.

[0125] As can be seen, the technical solution proposed in this invention has low computational complexity and strong generalization ability, integrating image deraining and object detection into a single network to achieve end-to-end object detection. This patent focuses solely on rainy scenes, using rainy images synthesized from clean, rain-free images as the dataset input to the deraining network. The resulting clean, rain-free images are then input into a cascaded object detection network for training. The loss function of the entire network comprises the reconstruction loss of the clean image from the deraining network, the object detection loss, and the perception loss. Only the gradient of this loss function is used to update the weights of the deraining network, keeping the object detection network unaffected, preserving as many semantic perception details as possible, reducing the overall computational complexity of the network, and improving object detection accuracy.

[0126] Please note that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An image detection method under rainy conditions, characterized in that, The method employs a cascaded network to perform image detection under rainy conditions. The cascaded network includes a rain removal network and a target detection network. The method includes: Step S1: Input the standard rainless image and the synthetic rainy day image into the rain removal network. The rain removal network performs rain removal operation on the synthetic rainy day image to obtain a first image, and calculates the reconstruction loss based on the first image and the standard rainless image. Step S2: Input the first image and the standard rainless image into the target detection network. The target detection network extracts features from the first image and the standard rainless image respectively to obtain a first feature map and a second feature map, and calculates the perceptual loss based on the first feature map and the second feature map. Step S3: The target detection network performs target detection on the first feature map and the second feature map respectively to obtain a first detection result and a second detection result, and calculates the target detection loss based on the first detection result and the second detection result; Step S4: The reconstruction loss, the perception loss, and the target detection loss are used as the joint loss of the cascaded network. The joint loss is fed back to the rain removal network to update the network parameters. The cascaded network with updated network parameters is used to detect images under rainy conditions. The structure of the rain removal network, arranged sequentially according to the image input direction, includes: a low-pass filter, a first convolutional layer, a first batch regularization layer, an activation function, a second convolutional layer, and a second batch regularization layer. The target detection network is a YOLO-v51 model, which includes an input module, a Backbone module, a Neck module, and a prediction module. The Backbone module adopts a CSP structure. The Neck module uses an FPN+PAN structure and CSPNet to fuse features in the feature map. Wherein, the perceptual loss is: Where g(·) represents the output of the feature extraction layer of the target detection network with size H*W, z represents the label required for training the target detection network, and F(x) represents the first image.

2. The image detection method under rainy conditions according to claim 1, characterized in that: The rain removal network uses the low-pass filter to extract the base layer of the synthesized rainy day image, and subtracts the base layer from the synthesized rainy day image to obtain the high-frequency detail layer of the synthesized rainy day image. The reconstruction loss L D for: Among them, X i Let Y represent the i-th composite rainy day image. i Let X represent the i-th standard rainless image, F represent the norm, N represent the number of images, f(·) represent the output of the second convolutional layer in the rain removal network, and X represent the standard rainless image. i，detail This refers to the high-frequency detail layer.

3. The image detection method under rainy conditions according to claim 2, characterized in that: At the input end, Mosaic is used to perform data augmentation on the input image, and adaptive anchor box calculation and adaptive image scaling are performed. The Backbone module slices the input image and concentrates the image size information into the channel space without losing image information, thus obtaining a feature map with double downsampling. When performing the target detection, the prediction module determines the approximate location of the target based on the confidence of the prediction box, filters the prediction boxes containing the target according to the MSE algorithm, removes duplicate prediction boxes containing the same target, and takes the classification index corresponding to the highest classification probability based on the classification probability of the filtered prediction boxes, thereby obtaining the detection result. The target detection loss consists of the prediction box loss, confidence loss, and classification loss.

4. An image detection system for rainy weather conditions, characterized in that, The system employs a cascaded network to perform image detection under rainy conditions. The cascaded network includes a rain removal network and a target detection network. The system comprises: The first processing unit is configured to: input a standard rainless image and a synthetic rainy day image into the rain removal network, the rain removal network performs a rain removal operation on the synthetic rainy day image to obtain a first image, and calculate a reconstruction loss based on the first image and the standard rainless image; The second processing unit is configured to: input the first image and the standard rainless image into the target detection network, wherein the target detection network performs feature extraction on the first image and the standard rainless image respectively to obtain a first feature map and a second feature map, and calculates a perceptual loss based on the first feature map and the second feature map; The third processing unit is configured to: perform target detection on the first feature map and the second feature map respectively to obtain a first detection result and a second detection result, and calculate the target detection loss based on the first detection result and the second detection result; The fourth processing unit is configured to: use the reconstruction loss, the perception loss, and the target detection loss as the joint loss of the cascaded network, the joint loss is fed back to the rain removal network to update the network parameters, and use the cascaded network with the updated network parameters to detect images under rainy conditions; The structure of the rain removal network, arranged sequentially according to the image input direction, includes: a low-pass filter, a first convolutional layer, a first batch regularization layer, an activation function, a second convolutional layer, and a second batch regularization layer. The target detection network is a YOLO-v51 model, which includes an input module, a Backbone module, a Neck module, and a prediction module. The Backbone module adopts a CSP structure. The Neck module uses an FPN+PAN structure and CSPNet to fuse features in the feature map. Wherein, the perceptual loss is: Where g(·) represents the output of the feature extraction layer of the target detection network with size H*W, z represents the label required for training the target detection network, and F(x) represents the first image.

5. The image detection system under rainy conditions according to claim 4, characterized in that: The rain removal network uses the low-pass filter to extract the base layer of the synthesized rainy day image, and subtracts the base layer from the synthesized rainy day image to obtain the high-frequency detail layer of the synthesized rainy day image. The reconstruction loss L D for: Among them, X i Let Y represent the i-th composite rainy day image. i Let X represent the i-th standard rainless image, F represent the norm, N represent the number of images, f(·) represent the output of the second convolutional layer in the rain removal network, and X represent the standard rainless image. i，detail This refers to the high-frequency detail layer.

6. The image detection system under rainy conditions according to claim 5, characterized in that: At the input end, Mosaic is used to perform data augmentation on the input image, and adaptive anchor box calculation and adaptive image scaling are performed. The Backbone module slices the input image and concentrates the image size information into the channel space without losing image information, thus obtaining a feature map with double downsampling. When performing the target detection, the prediction module determines the approximate location of the target based on the confidence of the prediction box, filters the prediction boxes containing the target according to the MSE algorithm, removes duplicate prediction boxes containing the same target, and takes the classification index corresponding to the highest classification probability based on the classification probability of the filtered prediction boxes, thereby obtaining the detection result. The target detection loss consists of the prediction box loss, confidence loss, and classification loss.

7. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the image detection method under rainy conditions as described in any one of claims 1-3.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the image detection method under rainy conditions as described in any one of claims 1-3.

Images