An automatic detection method for black smoke vehicle based on superpixel segmentation

Abstract

The application discloses a kind of black smoke vehicle automatic detection methods based on superpixel segmentation, including steps as follows: obtaining motor vehicle image containing black smoke exhaust and moving shadow, using superpixel segmentation algorithm to obtain classification training sample;Classification model is built based on lightweight network MobileNetV2, and the superpixel image of motor vehicle, black smoke exhaust and moving shadow is input into classification model learning training;Three-frame difference method is used to extract the moving target area of motor vehicle to be measured, and the superpixel segmentation algorithm is used to segment the moving target area image, to obtain the superpixel image input into classification model identification classification;If there is black smoke exhaust superpixel image in classification result, then determine that the motor vehicle is black smoke vehicle and identify its license plate number information.The application is based on the idea of "segmentation-classification", with small detection error, high universality and high accuracy.

Description

Technical Field

[0001] This invention belongs to the field of intelligent traffic management technology, specifically relating to an automatic detection method for black smoke vehicles based on superpixel segmentation. Background Technology

[0002] Statistics show that controlling vehicle exhaust pollution is urgently needed. Old engines and the use of substandard fuel lead to incomplete combustion, resulting in black smoke in the exhaust. Black smoke not only causes particulate matter and photochemical smog pollution, but also harms the respiratory and cardiovascular systems. Therefore, strengthening the regulation of vehicle exhaust emissions is a crucial aspect of air pollution control. How to effectively monitor vehicles emitting black smoke on the road is a pressing issue that needs to be addressed.

[0003] Early methods for identifying vehicles emitting black smoke mainly included random roadside checks, vehicle-mounted detection technology, road remote sensing monitoring, and manual review of road surveillance videos. These methods generally suffered from low accuracy and high costs. Random roadside checks and manual review of road traffic surveillance videos required significant manpower and resources, and the accuracy of black smoke vehicle identification was easily affected by the work status of the personnel. Vehicle-mounted detection technology and road remote sensing monitoring improved the accuracy of black smoke vehicle identification, but the precision instruments such as laser emitters inside the detection devices required substantial R&D and maintenance costs, and the detection devices were generally large, making them unsuitable for large-scale deployment.

[0004] Existing methods for identifying vehicles emitting black smoke can be categorized into traditional machine learning-based methods and deep learning-based methods. Traditional machine learning-based methods consist of two parts: a discriminative model and a generative model. Most research on discriminative models combines features such as the color and texture of black smoke exhaust with classifiers like Support Vector Machines (SVMs) and K-nearest neighbors (KNNs) for identification. SVMs maximize the margin between samples and the decision boundary, offering excellent classification performance and wide applicability, but training on large-scale datasets presents challenges. Deep learning-based methods automatically learn target features containing higher-level semantic information from data. Their performance ceiling is determined by the classification performance of the backbone network, which typically employs the convolutional neural network architecture prevalent in image classification.

[0005] The main problems currently facing deep learning-based methods for identifying vehicles emitting black smoke are as follows: First, existing intelligent black smoke vehicle identification methods define the region of interest (ROI) as a certain size area at the rear of the vehicle, but easily dispersed black smoke exhaust can introduce errors into the defined ROI. Second, defining the ROI as the rear of the vehicle means that towing vehicles with exhaust outlets located in the middle of the vehicle will not be detected. Third, vehicles emitting small amounts of black smoke exhaust may be missed, and the moving shadows cast by vehicles on sunny days may lead to false detections. Fourth, deep learning-based network models require a large number of training samples, but there are currently no publicly available black smoke exhaust datasets, requiring the collection and labeling of road surveillance video data. Summary of the Invention

[0006] Technical problem solved: To address the above-mentioned technical problems, this invention provides an automatic detection method for black smoke vehicles based on superpixel segmentation. Based on the "segmentation-classification" concept, it has small detection error, high universality, and high accuracy.

[0007] Technical solution: An automatic detection method for black smoke vehicles based on superpixel segmentation, comprising the following steps:

[0008] Step 1: Obtain images of motor vehicles containing black smoke exhaust and motion shadows, and use superpixel segmentation algorithm to obtain classification training samples;

[0009] Step 2: Build a classification model based on the lightweight network MobileNetV2, and input superpixel images of motor vehicles, black smoke exhaust, and motion shadows into the classification model for learning and training.

[0010] Step 3: Use the three-frame difference method to extract the moving target region of the image of the motor vehicle under test, use the superpixel segmentation algorithm to segment the moving target region image, and obtain the superpixel image input to the classification model for recognition and classification.

[0011] Step 4: If the classification results contain a superpixel image of black smoke exhaust, then the vehicle is determined to be a black smoke vehicle and its license plate number information is identified.

[0012] Preferably, the vehicle images in step one are obtained from road traffic monitoring videos.

[0013] Preferably, the superpixel segmentation algorithm includes the following steps: color space conversion, initial cluster centers, adjusting seed point positions, distance metric similarity calculation, local iterative clustering, and connectivity merging.

[0014] Furthermore, the color space conversion specifically involves converting the RGB color space to the CIELab color space.

[0015] Preferably, the specific steps for extracting the moving target region of the vehicle image under test using the three-frame difference method in step three are as follows:

[0016] Acquire images of the vehicle under test, and perform grayscale conversion and Gaussian filtering;

[0017] The current frame of the processed image is subjected to a difference operation with the previous and next frames. A threshold is set to binarize the difference operation result. Finally, the intersection of the binarized results of the difference images of the previous and next frames is taken as the moving target region.

[0018] Furthermore, the threshold is set to 30.

[0019] Preferably, the classification results in step three are divided into the following four categories: motor vehicles with no shadow and no black smoke, motor vehicles with shadow and no black smoke, motor vehicles with no shadow and black smoke, and motor vehicles with shadow and black smoke.

[0020] Preferably, in step four, the license plate number information is identified through a modular vehicle identification system.

[0021] Beneficial effects: (1) The automatic detection method for black smoke vehicles based on superpixel segmentation algorithm proposed in this invention has the advantage of requiring less manpower and resources in practical applications and using artificial intelligence and computer vision technology to achieve the automatic detection of black smoke vehicles. At the same time, the automatic detection process will not affect the smooth flow of road traffic. The monitoring camera is small in size and can be deployed and monitored over a wide area.

[0022] (2) Based on the idea of ​​“segmentation-classification”, this invention uses the three-frame difference method to extract the moving target region as the region of interest for research. Compared with the existing intelligent black smoke vehicle detection methods, the error caused by determining the rear of the motor vehicle as the region of interest is smaller. This invention determines the region of interest for research as the moving target region, which can also detect some tractor vehicles whose exhaust outlets are located in the middle of the motor vehicle, effectively improving the universality of the intelligent black smoke vehicle automatic detection method.

[0023] (3) This invention proposes an automatic black smoke vehicle detection method based on superpixel segmentation algorithm. Compared with existing intelligent black smoke vehicle detection methods, this method fully considers the impact of motion shadows generated by motor vehicles on the accuracy of black smoke vehicle recognition under clear weather conditions. The superpixel segmentation algorithm is used to obtain superpixel images of motor vehicles, black smoke exhaust, and motion shadows as training samples, which are then input into a classification model built on the lightweight MobileNetV2 network for training. The model classification results are divided into the following four cases: motor vehicles with no shadows and no black smoke, motor vehicles with shadows and no black smoke, motor vehicles with no shadows and black smoke, and motor vehicles with shadows and black smoke. Using motion shadows as a classification category can effectively improve the recognition accuracy of automatic black smoke vehicle detection. Attached Figure Description

[0024] Figure 1 This is a schematic diagram illustrating the segmentation of a motor vehicle containing black smoke exhaust and motion shadows using a superpixel segmentation algorithm:

[0025] Figure 2 This is a schematic diagram of training samples of motor vehicles obtained using a superpixel segmentation algorithm;

[0026] Figure 3 This is a schematic diagram of training samples of black smoke exhaust obtained using a superpixel segmentation algorithm;

[0027] Figure 4 This is a schematic diagram of training samples for moving shadows obtained using a superpixel segmentation algorithm;

[0028] Figure 5 This is a schematic diagram illustrating the use of the three-frame difference method to extract moving targets from road traffic surveillance videos;

[0029] Figure 6 This is a schematic diagram of the automatic detection results of vehicles emitting black smoke according to the present invention, wherein "0" represents a vehicle emitting black smoke and "1" represents a motor vehicle that does not emit black smoke exhaust gas;

[0030] Figure 7 It is a modular structure composed of depthwise separable convolutions and standard convolutions;

[0031] Figure 8 This is the structural diagram of the inverted residual module;

[0032] In particular, considering privacy issues, this invention Figure 5 and Figure 6 The license plate number has been obscured. Detailed Implementation

[0033] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0034] Example 1

[0035] An automatic detection method for black smoke vehicles based on superpixel segmentation includes the following steps:

[0036] Step 1: Obtain images of motor vehicles containing black smoke exhaust fumes and moving shadows from road traffic surveillance videos, and use a superpixel segmentation algorithm to obtain classification training samples:

[0037] The research data originated from road traffic surveillance videos, and the research subjects were motor vehicles traveling on the road. The research objective was to automatically detect motor vehicles emitting black smoke exhaust fumes. Since the motion shadows cast by motor vehicles in clear weather can interfere with the identification of vehicles emitting black smoke, images of motor vehicles containing both black smoke exhaust fumes and motion shadows were selected as the original images for the study.

[0038] The implementation process of the superpixel segmentation algorithm includes color space conversion, initial cluster centers, adjusting seed point positions, distance metric similarity calculation, local iterative clustering, and connectivity merging. The detailed steps are as follows:

[0039] Color space conversion: Converting RGB color space to CIELab color space involves first converting RGB color space to XYZ color space, and then converting XYZ color space to CIELab color space. During the color space conversion, a 5-dimensional vector V = [L, a, b, x, y] is generated, which consists of the color vector (L, a, b) and position vector (x, y) of each pixel.

[0040]

[0041]

[0042]

[0043]

[0044] In the formula, the Gamma function is used to perform non-linear tonal editing of the image, with the aim of improving image contrast; the RGB color space is converted to the CIELab color space via the XYZ color space, and the conversion formula is shown below:

[0045]

[0046]

[0047] In the formula, X, Y, and Z are the calculated values ​​for converting the RGB color space to the XYZ color space; L*, a*, and b* are the final values ​​of the three channels in the CIELab color space.

[0048]

[0049]

[0050]

[0051]

[0052] Initial k cluster centers: Based on actual application requirements, k superpixels can be initially set. The seed points of the superpixels are evenly distributed across an image with N pixels. Then, each superpixel contains N / k pixels, and the distance between adjacent seed points is...

[0053] Adjusting seed point positions: The generated seed points may fall on the edges of superpixels with large gradients and noisy pixels. Adjust these initial seed points to the positions with the smallest gradient values ​​in the 3×3 neighborhood.

[0054] Distance metric similarity calculation: The similarity between a pixel and a seed point can be calculated using color distance d. Lab Spatial distance d xy Here, m represents the weighting factor that measures the relative importance between color distance and spatial distance; S represents the distance between adjacent seed points; and D represents the similarity between two pixels, with a larger value indicating greater similarity between the two pixels.

[0055]

[0056]

[0057]

[0058] Local iterative clustering: To improve the computation speed of the algorithm, similar pixels are searched in a 2S×2S region centered on the seed point. The distance metric from all pixels to the seed point is calculated within this range. The process is repeated iteratively and assigned, and pixels with similar features are generated into superpixels.

[0059] Connectivity merge algorithm: After local iterative clustering, there may be situations where the segmented superpixel blocks are too small or there are other categories of superpixel blocks with small areas in the superpixel image. The connectivity merge algorithm is used to handle these situations.

[0060] Step 2: Build a classification model based on the lightweight network MobileNetV2, and input superpixel images of motor vehicles, black smoke exhaust, and motion shadows into the classification model for learning and training.

[0061] Among them, the lightweight network model MobileNet can achieve good classification performance while significantly reducing the inference time of the network model, meeting the real-time requirements of practical application scenarios. The lightweight network MobileNetV1 introduces depthwise separable convolutions to replace standard convolutional layers to reduce the computational cost of the model, splitting a standard convolutional layer into depthwise convolutions and pointwise convolutions. For example... Figure 7 The diagram shows the modular structure of depthwise separable convolution and standard convolution. In depthwise convolution, the number of input channels and output channels are the same, and each input channel has an independent convolution kernel. Point convolution is a special type of convolution operation with a 1×1 kernel size, which collects features from each pixel.

[0062] The lightweight network model MobileNetV2 continues the use of depthwise separable convolutions instead of standard convolutions from version V1. It also introduces linear bottlenecks and inverse residual structures to improve the efficiency of the network layer structure. Shortcut connections are only used when stride = 1 and the input and output feature matrices have the same shape. The inverse residual module structure diagram is shown below. Figure 8 As shown;

[0063] In the inverse residual module structure design, a 1×1 pointwise convolution is used to increase the dimensionality of the feature map channels before the depthwise separable convolution, and finally a 1×1 convolution is used to reduce the dimensionality. The inverse residual structure uses the ReLU6 activation function, but the final 1×1 convolutional layer uses a linear activation function because using ReLU6 at this point would cause significant loss of low-dimensional feature information. The inverse residual structure is generally small at both ends and large in the middle; applying a linear activation function can reduce the information loss in the output.

[0064] Step 3: Use the three-frame difference method to extract the moving target area of ​​the vehicle under test, use the superpixel segmentation algorithm to segment the moving target area image, obtain the superpixel image and input it into the classification model for recognition and classification. The classification results are divided into the following 4 cases: no shadow and no black smoke, shadow and no black smoke, no shadow and black smoke, and shadow and black smoke.

[0065] The specific steps for extracting the moving target region of the vehicle under test using the three-frame difference method are as follows:

[0066] Real-time raw images are obtained from road surveillance videos. Next, image preprocessing operations such as grayscale conversion and Gaussian filtering are performed. Grayscale conversion can significantly reduce the amount of image data, thereby improving the algorithm's processing speed. The input video stream may generate noise due to natural vibrations, changes in lighting, or the camera itself. Gaussian filtering can smooth and blur the image.

[0067] Inter-frame differencing is simple to implement, computationally inexpensive, and exhibits strong adaptability and robustness in dynamic environments. However, the moving target contours obtained by inter-frame differencing often contain holes. Three-frame differencing, an improvement on inter-frame differencing, is more suitable for fast-moving targets and applications with high environmental noise. To extract motor vehicles from video images using three-frame differencing, assuming the grayscale values ​​of the current frame and the preceding and following frames are I(x,y,t), I(x,y,t-1), and I(x,y,t+1), respectively, the formula for calculating the difference image is:

[0068]

[0069] In the formula D t,t-1 (x,y,t) and D t,t+1 (x, y, t) represents the result of the grayscale difference operation between the current frame and the previous and next frames. An adaptive segmentation threshold T, adapted to changes in lighting conditions, is set based on the actual application scenario. The difference results from the previous and next frames are then binarized using the following formula:

[0070]

[0071] R(x,y,t)=R t-1 (x,y,t)∩R t+1 (x,y,t)

[0072] In the formula R k (x,y,t) represents the result of the differential image binarization process, k=t-1 and k=t+1 represent the differential image binarization results of the previous and next frames respectively, and T represents the threshold. In this invention, it has been experimentally confirmed that the threshold is set to a fixed value of 30. The final result is the intersection of the differential image binarization results of the previous and next frames.

[0073] Step 4: If the classification results contain a superpixel image of black smoke exhaust, then the vehicle is determined to be a black smoke vehicle and its license plate number information is identified.

[0074] The license plate number information is identified using an existing highly modular vehicle recognition system. The identification process is as follows:

[0075] Image preprocessing: Performing image preprocessing such as image denoising, grayscale conversion, contrast enhancement, and image edge recognition;

[0076] License plate localization can employ methods such as the horizontal line search-based localization method proposed by J. Barroso et al.; the frequency domain analysis method based on DFT transform proposed by R. Parisi et al.; the localization method based on Niblack binarization algorithm and adaptive boundary search algorithm proposed by Charl Coetzee; or the license plate extraction algorithm based on scan lines proposed by Barroso J. Buls-Cruz et al.

[0077] License plate character segmentation: Hough transform can be used to correct license plate tilt, and a combination of connected component analysis and projection can be used for license plate character segmentation.

[0078] License plate number recognition: Traditional manual feature extraction combined with classifiers can be used, such as HOG features + support vector machine (SVM); or a method based on backpropagation neural network can be used to recognize characters.

Claims

1. An automatic detection method for black smoke vehicles based on superpixel segmentation, characterized in that, The steps include the following: Step 1: Obtain images of motor vehicles containing black smoke exhaust and motion shadows, and use superpixel segmentation algorithm to obtain classification training samples; Step 2: Build a classification model based on the lightweight network MobileNetV2, and input superpixel images of motor vehicles, black smoke exhaust, and motion shadows into the classification model for learning and training. Step 3: Use the three-frame difference method to extract the moving target region of the image of the motor vehicle under test, use the superpixel segmentation algorithm to segment the moving target region image, obtain the superpixel image input to the classification model for recognition and classification, and the classification results are divided into the following 4 cases: motor vehicles with no shadow and no black smoke, with shadow and no black smoke, with no shadow and black smoke, and with shadow and black smoke. Step 4: If the classification results contain a superpixel image of black smoke exhaust, then the vehicle is determined to be a black smoke vehicle and its license plate number information is identified.

2. The automatic detection method for black smoke vehicles based on superpixel segmentation according to claim 1, characterized in that, In step one, the images of motor vehicles are obtained from road traffic monitoring videos.

3. The automatic detection method for black smoke vehicles based on superpixel segmentation according to claim 1, characterized in that, The superpixel segmentation algorithm includes the following steps: color space conversion, initial cluster centers, adjusting seed point positions, distance metric similarity calculation, local iterative clustering, and connectivity merging.

4. The automatic detection method for black smoke vehicles based on superpixel segmentation according to claim 3, characterized in that, The color space conversion specifically involves converting the RGB color space to the CIELab color space.

5. The automatic detection method for black smoke vehicles based on superpixel segmentation according to claim 1, characterized in that, The specific steps for extracting the moving target region of the vehicle under test using the three-frame difference method in step three are as follows: Acquire images of the vehicle under test, and perform grayscale conversion and Gaussian filtering; The current frame of the processed image is subjected to a difference operation with the previous and next frames. A threshold is set to binarize the difference operation result. Finally, the intersection of the binarized results of the difference images of the previous and next frames is taken as the moving target region.

6. The automatic detection method for black smoke vehicles based on superpixel segmentation according to claim 5, characterized in that, The threshold is set to 30.

7. The automatic detection method for black smoke vehicles based on superpixel segmentation according to claim 1, characterized in that, In step four, the license plate number information is identified using a modular vehicle identification system.

Images