A night driving detection method, system, medium and electronic device

Abstract

This invention belongs to the field of image processing technology and provides a method, system, medium, and electronic device for nighttime vehicle detection. Addressing the problems of few feature points and blurred vision in nighttime images, this invention first performs supplementary lighting processing on low-light images to achieve preprocessing of nighttime images. Then, it extracts texture feature maps of nighttime vehicles from the preprocessed nighttime images and uses both the preprocessed nighttime images as input to a nighttime vehicle detection model for detection. Simultaneously, to improve detection accuracy, it replaces the Yolov5 backbone network with a Mobile Netv2 network and adds an attention mechanism to the residual block to enhance channel and spatial feature information in the feature maps. This allows the nighttime vehicle detection model to focus on the target area, extracting more effective features and further improving the feature extraction capability of the nighttime vehicle detection model.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a method, system, medium and electronic device for nighttime vehicle detection. Background Technology

[0002] With the continuous increase in the number of cars in cities, especially the development and deployment of autonomous vehicles, the challenges to traffic management and the urban environment have increased dramatically. To address these issues, intelligent transportation systems have become a key research area for urban development. Among these systems, vehicle identification technology is particularly important.

[0003] The inventors discovered that when vehicles are driving at night, the nighttime images of the vehicles being detected cannot meet the recognition requirements of the nighttime vehicle detection model due to problems such as fewer feature points and blurred vision. This results in low recognition accuracy and poor performance for nighttime vehicles. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes a nighttime vehicle detection method, system, medium, and electronic device. Specifically, it addresses the issues of limited feature points and blurred vision in nighttime images by utilizing the SSR (Single Scale Retinex) algorithm to enhance nighttime images and supplementing low-light images with additional illumination, thus preprocessing the nighttime images. The preprocessed nighttime images are then processed together with the extracted Local Binary Patterns (LBP) texture feature maps of nighttime vehicles, which are then fed into a nighttime vehicle detection model. Furthermore, to improve detection accuracy, the nighttime vehicle detection model is improved by replacing the Yolov5 backbone network with the Mobile Netv2 network and incorporating an attention mechanism in the residual block. This enhances the channel and spatial features in the feature maps, allowing the nighttime vehicle detection model to focus on the target area and extract more effective features, further improving its feature extraction capabilities.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0006] In a first aspect, the present invention provides a method for detecting nighttime driving, comprising:

[0007] Acquire nighttime images of vehicles driving at night;

[0008] The nighttime image is then illuminated to obtain a pre-processed nighttime image.

[0009] Extract texture feature maps of vehicles driving at night;

[0010] Based on the texture feature map, the preprocessed nighttime image, and the preset nighttime vehicle detection model, the detection results are obtained.

[0011] The nighttime driving detection model is a model that replaces the Yolov5 backbone network with the Mobile Netv2 network, and an attention mechanism is added to the residual block.

[0012] Furthermore, the video stream of nighttime driving is acquired, and the video stream is decoded into video frames to obtain nighttime images of nighttime driving.

[0013] Furthermore, the SSR algorithm is used to enhance the nighttime image. If the nighttime image is a grayscale image, the grayscale values ​​of each pixel in the nighttime image are converted from integers to floating-point numbers and then converted to the logarithmic domain. If the nighttime image is a color image, the color is processed by channel, and the pixel value of each component is converted from integers to floating-point numbers and then converted to the logarithmic domain.

[0014] Furthermore, a logarithmic transformation is performed on the enhanced night image, and the logarithmically transformed night image is then Gaussian blurred. The difference between the original night image and the Gaussian blurred night image is then used for supplementary lighting processing.

[0015] Furthermore, the preprocessed nighttime image is divided into several image sub-blocks, and the local binary value of each pixel in each image sub-block is calculated;

[0016] Histogram statistics are performed on each image sub-block to obtain histograms of several image sub-blocks;

[0017] Normalize the histograms of all image sub-blocks;

[0018] By connecting the normalized histograms of all image sub-blocks, the texture features of the entire image are obtained.

[0019] Furthermore, the backbone network of Yolov5 is replaced with the Mobile Netv2 network. The Mobile Netv2 network introduces a residual structure to enhance gradient propagation and removes ReLU after the Narrow layer.

[0020] Furthermore, the training of the nighttime vehicle detection model includes:

[0021] Acquire video streams of nighttime driving and decode the video streams into video frames;

[0022] Valid data in video frames are filtered and vehicles are labeled to obtain nighttime images of nighttime driving;

[0023] Preprocess the nighttime images;

[0024] Texture feature maps are extracted from the preprocessed nighttime images. These texture feature maps, along with the preprocessed nighttime images, are used as input images to train the nighttime vehicle detection model.

[0025] Secondly, the present invention also provides a nighttime vehicle detection system, comprising:

[0026] The data acquisition module is configured to acquire nighttime images of the vehicle during nighttime driving.

[0027] The preprocessing module is configured to perform supplemental lighting on the night image to obtain a preprocessed night image;

[0028] The feature map extraction module is configured to extract texture feature maps of vehicles driving at night.

[0029] The detection module is configured to obtain detection results based on the texture feature map, the preprocessed nighttime image, and the preset nighttime driving detection model.

[0030] The nighttime driving detection model is a model that replaces the Yolov5 backbone network with the Mobile Netv2 network, and an attention mechanism is added to the residual block.

[0031] Thirdly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the nighttime driving detection method described in the first aspect.

[0032] Fourthly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the night driving detection method described in the first aspect.

[0033] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0034] This invention addresses the problems of few feature points and blurred vision in nighttime images. First, it performs supplemental lighting on low-light images to preprocess them. Then, it extracts texture feature maps of nighttime driving from the preprocessed nighttime images and uses both the preprocessed and non-preprocessed nighttime images as input to a nighttime driving detection model. Simultaneously, to improve detection accuracy, it replaces the Yolov5 backbone network with a Mobile Netv2 network and adds an attention mechanism to the residual block to enhance channel and spatial features in the feature maps. This allows the nighttime driving detection model to focus on the target area, extracting more effective features and further improving its feature extraction capabilities, thus enhancing recognition accuracy and performance. Attached Figure Description

[0035] The accompanying drawings, which form part of this embodiment, are used to provide a further understanding of this embodiment. The illustrative embodiments and their descriptions are used to explain this embodiment and do not constitute an improper limitation of this embodiment.

[0036] Figure 1 This is a flowchart of Embodiment 1 of the present invention;

[0037] Figure 2 This is a structural diagram of Mobile Netv2 according to Embodiment 1 of the present invention;

[0038] Figure 3 This is a network structure diagram of the nighttime driving detection model in Embodiment 1 of the present invention. Detailed Implementation

[0039] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0040] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0041] Example 1:

[0042] This implementation provides a method for nighttime vehicle detection, including:

[0043] S1. Nighttime images of driving at night can be obtained by decoding the video stream captured by the camera into video frames;

[0044] S2. Based on the characteristics of nighttime images, preprocess the nighttime images;

[0045] S3. Extract the texture feature map of the night image, and send it together with the preprocessed night image as the input image to the preset night driving detection model for detection;

[0046] S4. Improve the Yolov5 detection model and train it.

[0047] In step S2, considering the characteristics of nighttime images, preprocessing is performed. Specifically, this includes using the SSR algorithm to eliminate the influence of light sources on nighttime images, making the features of nighttime images more prominent. The implementation process of the single-scale Retinex algorithm SSR can be summarized as follows:

[0048] S2.1 Read the original graph S(x, y):

[0049] If the original image of the nighttime image is a grayscale image: convert the grayscale values ​​of each pixel in the nighttime image from integer type int to floating-point number float, and then convert it to the logarithmic field;

[0050] If the original image of the nighttime image is a color image: process the colors by channel, convert the pixel value of each component from an integer (int) to a floating-point number (float), and then convert it to the logarithmic domain; where S(x, y) is the original image; x and y are the coordinates of each point in the image on the x-axis and y-axis, respectively.

[0051] S2.2 Input Gaussian surround scale C, discretize the integral operation, and convert it into a summation operation:

[0052] If the original image of the nighttime image is a grayscale image, then there is only one r(x, y);

[0053] If the original image of the nighttime image is a color image, then each channel has a corresponding r(x, y);

[0054] r(x,y)=logS(x,y)-log[F(x,y)·S(x,y)] This transforms the logarithmic domain to the real domain, resulting in the output image R(x,y).

[0055] Here, F(x, y) is the original image; at this time, the range of the output image R(x, y) value is not 0-255, so it is necessary to perform linear stretching and convert it into the corresponding format for output display.

[0056] The center wrap function F(x, y) = λe-(x² + y²)c² uses a low-pass function, which allows the algorithm to estimate the low-frequency components of the original image corresponding to the incident image; where c is a constant, and to balance the two enhancement effects, c is generally taken between 80 and 100; λ is a scale parameter; and e is a constant.

[0057] In step S3, the texture feature map of the nighttime image is extracted and used as input along with the preprocessed nighttime image. This is then fed into the nighttime vehicle detection model for detection. Specifically, this includes:

[0058] S3.1 Divide the preprocessed night image into several N×N image sub-blocks, such as 16×16, and calculate the LBP value of each pixel in each image sub-block;

[0059] S3.2 Perform histogram statistics on each image sub-block to obtain the histogram of N×N image sub-blocks;

[0060] S3.3 Normalize the histograms of all image sub-blocks;

[0061] S3.4 Connect the normalized histograms of all image sub-blocks to obtain the texture features of the entire image;

[0062] S3.5. The obtained texture feature map and the preprocessed night image are used as input images and sent together to the night driving detection model for detection.

[0063] In step S4, the Yolov5 detection model is improved and trained, specifically by replacing the Yolov5 backbone network with the Mobile Netv2 network. Mobile Netv2 introduces a residual structure, which enhances gradient propagation and significantly reduces the memory usage required during inference. ReLU after removing the Narrow layer (low dimension or depth) is removed to preserve feature diversity and enhance the network's expressive power.

[0064] The training steps for the nighttime vehicle detection model include:

[0065] S4.1 Collect video streams of nighttime driving and decode the acquired video streams into video frames;

[0066] S4.2 Filter valid data and label vehicles;

[0067] S4.3 Perform SSR preprocessing on nighttime images;

[0068] S4.4 Extract the LBP texture feature map from the nighttime image, and use the texture feature map and the preprocessed nighttime image as input images to train the nighttime vehicle detection model.

[0069] S4.5. The backbone network of Yolov5 is replaced with Mobile Netv2, which reduces the number of parameters and speeds up inference. An attention mechanism is added to the residual block to tune the parameters of intermediate training results until the optimal model parameters are obtained.

[0070] Example 2:

[0071] This embodiment provides a nighttime vehicle detection system, including:

[0072] The data acquisition module is configured to acquire nighttime images of the vehicle during nighttime driving.

[0073] The preprocessing module is configured to perform supplemental lighting on the night image to obtain a preprocessed night image;

[0074] The feature map extraction module is configured to extract texture feature maps of vehicles driving at night.

[0075] The detection module is configured to obtain detection results based on the texture feature map, the preprocessed nighttime image, and the preset nighttime driving detection model.

[0076] The nighttime driving detection model is a model that replaces the Yolov5 backbone network with the Mobile Netv2 network, and an attention mechanism is added to the residual block.

[0077] The working method of the system is the same as that of the nighttime driving detection method in Embodiment 1, and will not be repeated here.

[0078] Example 3:

[0079] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the nighttime vehicle detection method described in Embodiment 1.

[0080] Example 4:

[0081] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the nighttime driving detection method described in Embodiment 1.

[0082] The above description is merely a preferred embodiment of this practice and is not intended to limit the scope of this practice. Various modifications and variations can be made to this practice by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this practice should be included within the protection scope of this practice.

Claims

1. A method for detecting nighttime vehicle operation, characterized in that, include: Acquire nighttime images of vehicles driving at night; The nighttime image is then illuminated to obtain a pre-processed nighttime image. The SSR algorithm is used to enhance nighttime images; Read the original graph S(x, y); Input the Gaussian wrapper scale C, discretize the integral operation, and convert it into a summation operation: If the nighttime image is a grayscale image, then there is only one r(x, y); convert the grayscale values ​​of each pixel in the nighttime image from integers to floating-point numbers, and then convert them to the logarithmic domain; If the nighttime image is a color image, then each channel has a corresponding r(x, y); the color is processed by channel, and the pixel value of each component is converted from an integer to a floating-point number and then converted to the logarithmic domain; r(x,y)=logS(x,y)-log[F(x,y)·S(x,y)] transforms from the logarithmic domain to the real domain, resulting in the output image R(x,y); F(x, y) is the original image; at this time, the range of the output image R(x, y) value is not 0–255, so it is necessary to perform linear stretching and convert it into the corresponding format for output display; The central surrounding function F(x, y) = λe-(x 2 +y 2 )c 2 A low-pass function is used, which allows the algorithm to estimate the low-frequency components of the original image corresponding to the incident image; where c is a constant, and to balance the two enhancement effects, c is generally taken between 80 and 100; λ is the scale parameter; and e is a constant. Extract texture feature maps of vehicles driving at night; The preprocessed night image is divided into several image sub-blocks, and the local binary value of each pixel in each image sub-block is calculated; Histogram statistics are performed on each image sub-block to obtain histograms of several image sub-blocks; Normalize the histograms of all image sub-blocks; By connecting the normalized histograms of all image sub-blocks, the texture features of the entire image are obtained; Based on the texture feature map, the preprocessed nighttime image, and the preset nighttime vehicle detection model, the detection results are obtained. The nighttime driving detection model is a model that replaces the Yolov5 backbone network with the Mobile Netv2 network, and an attention mechanism is added to the residual block.

2. The nighttime vehicle detection method as described in claim 1, characterized in that, The video stream of nighttime driving is acquired, and the video stream is decoded into video frames to obtain nighttime images of the vehicle.

3. A nighttime vehicle detection method as described in claim 1, characterized in that, The backbone network of Yolov5 is replaced by the Mobile Netv2 network. The Mobile Netv2 network introduces a residual structure to enhance gradient propagation and removes ReLU after the Narrowlayer is removed.

4. The nighttime vehicle detection method as described in claim 1, characterized in that, Nighttime vehicle detection model training includes: Acquire video streams of nighttime driving and decode the video streams into video frames; Valid data in video frames are filtered and vehicles are labeled to obtain nighttime images of nighttime driving; Preprocess the nighttime images; Texture feature maps are extracted from the preprocessed nighttime images. These texture feature maps, along with the preprocessed nighttime images, are used as input images to train the nighttime vehicle detection model.

5. A nighttime vehicle detection system, characterized in that, include: The data acquisition module is configured to acquire nighttime images of the vehicle during nighttime driving. The preprocessing module is configured to perform supplemental lighting on the night image to obtain a preprocessed night image; The SSR algorithm is used to enhance nighttime images; Read the original graph S(x, y); Input the Gaussian wrapper scale C, discretize the integral operation, and convert it into a summation operation: If the nighttime image is a grayscale image, then there is only one r(x, y); convert the grayscale values ​​of each pixel in the nighttime image from integers to floating-point numbers, and then convert them to the logarithmic domain; If the nighttime image is a color image, then each channel has a corresponding r(x, y); the color is processed by channel, and the pixel value of each component is converted from an integer to a floating-point number and then converted to the logarithmic domain; r(x,y)=logS(x,y)-log[F(x,y)·S(x,y)] transforms from the logarithmic domain to the real domain, resulting in the output image R(x,y); F(x, y) is the original image; at this time, the range of the output image R(x, y) value is not 0–255, so it is necessary to perform linear stretching and convert it into the corresponding format for output display; The central surrounding function F(x, y) = λe-(x 2 +y 2 )c 2 A low-pass function is used, which allows the algorithm to estimate the low-frequency components of the original image corresponding to the incident image; where c is a constant, and to balance the two enhancement effects, c is generally taken between 80 and 100; λ is the scale parameter; and e is a constant. The feature map extraction module is configured to extract texture feature maps of vehicles driving at night. The preprocessed night image is divided into several image sub-blocks, and the local binary value of each pixel in each image sub-block is calculated; Histogram statistics are performed on each image sub-block to obtain histograms of several image sub-blocks; Normalize the histograms of all image sub-blocks; By connecting the normalized histograms of all image sub-blocks, the texture features of the entire image are obtained; The detection module is configured to obtain detection results based on the texture feature map, the preprocessed nighttime image, and the preset nighttime driving detection model. The nighttime driving detection model is a model that replaces the Yolov5 backbone network with the Mobile Netv2 network, and an attention mechanism is added to the residual block.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the nighttime vehicle detection method as described in any one of claims 1-4.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the nighttime vehicle detection method as described in any one of claims 1-4.

Images