Deep learning-based black smoke vehicle detection method and system

Through deep learning-based methods, black smoke vehicle detection is carried out, and the interference problem of road vehicle black smoke detection in the prior art is solved, and accurate identification and quantitative detection is achieved.

CN119007135AActive Publication Date: 2024-11-22JIANGSU YUNJING INFORMATION TECHNOLOGY CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411030833.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-22
Estimated Expiration
2044-07-30

AI Technical Summary

Technical Problem

The prior art is difficult to effectively quantify black smoke on vehicles on the road, and it is impossible to eliminate interference and accurately identify black smoke vehicles.

Method used

Using a deep learning-based method, the monitoring and shooting images are periodically acquired, motion detection and shadow removal are performed, the black smoke pixel ratio is calculated, and the expansion optimization is performed, the optimized pixel ratio is compared to the preset standard ratio, and the black smoke vehicle is identified and uploaded. information.

Benefits of technology

Effective black smoke quantitative detection of vehicles on the road is realized, interference is eliminated, and black smoke vehicles are accurately identified.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007135A_ABST
    Figure CN119007135A_ABST
Patent Text Reader

Abstract

The invention is suitable for the technical field of black smoke vehicle detection, and provides a black smoke vehicle detection method and system based on deep learning. The method comprises the following steps: periodically acquiring monitoring shot images; shadow detection is carried out, shadow removal processing is carried out, and a target foreground image is obtained; graying processing is carried out, a black smoke pixel area is identified based on a deep learning technology, and a black smoke pixel ratio is calculated; performing expansion optimization on the black smoke pixel ratio, and calculating an optimized pixel ratio; when the optimized pixel ratio is greater than the standard black smoke ratio, black smoke vehicle information is identified and uploaded. According to the invention, shadow detection and shadow removal processing can be carried out on a monitoring foreground image to obtain a target foreground image, then graying processing is carried out, a black smoke pixel ratio is calculated, expansion optimization is carried out, an optimized pixel ratio is calculated, comparison with a black smoke standard ratio is carried out, and black smoke vehicle information is identified and uploaded. Therefore, effective black smoke quantitative detection can be performed on the vehicles on the road, and interference elimination and accurate identification of the black smoke vehicles are realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of black smoke vehicle detection, and in particular relates to a black smoke vehicle detection method and system based on deep learning. Background Art

[0002] Black smoke vehicle detection is the monitoring of motor vehicles that emit visible black smoke or other visible pollutants.

[0003] Black smoke emissions indicate that the vehicle's engine is experiencing incomplete combustion, which may be caused by poor fuel quality, improper engine maintenance, or vehicle aging.

[0004] The purpose of detecting smoky vehicles is to reduce environmental pollution and protect public health.

[0005] In the prior art, since fuel vehicles have more or less observable exhaust emissions, and other factors can easily interfere with black smoke detection, it is impossible to conduct effective black smoke quantitative detection on vehicles on the road, and it is impossible to eliminate interference and accurately identify black smoke vehicles. Summary of the invention

[0006] The purpose of the embodiments of the present invention is to provide a method and system for detecting black smoke vehicles based on deep learning, aiming to solve the technical problems existing in the prior art mentioned in the background technology.

[0007] The embodiment of the present invention is implemented as follows:

[0008] A method for detecting smoky vehicles based on deep learning, the method specifically comprising the following steps:

[0009] Periodically acquiring surveillance images, performing motion detection on the surveillance images based on a preset surveillance background image, and extracting a surveillance foreground image from the surveillance images;

[0010] Performing shadow detection on the monitored foreground image to determine the foreground shadow area, and performing shadow removal processing on the monitored foreground image to obtain a target foreground image;

[0011] Graying the target foreground image to obtain a gray foreground image, and based on deep learning technology, identifying a black smoke pixel area in the gray foreground image, and calculating a black smoke pixel ratio;

[0012] Calculating the region center coordinates of the black smoke pixel region, matching the black smoke expansion ratio, performing expansion optimization on the black smoke pixel ratio, and calculating the optimized pixel ratio;

[0013] The optimized pixel ratio is compared with a preset black smoke standard ratio, and when the optimized pixel ratio is greater than the black smoke standard ratio, the black smoke vehicle information is identified and uploaded.

[0014] As a further limitation of the technical solution of the embodiment of the present invention, the periodic acquisition of surveillance images, the motion detection of the surveillance images based on a preset surveillance background image, and the extraction of the surveillance foreground image from the surveillance images specifically include the following steps:

[0015] According to the preset monitoring shooting cycle, periodically generate monitoring shooting instructions;

[0016] According to the surveillance shooting instruction, surveillance shooting is performed to obtain surveillance shooting images;

[0017] By using a background difference method, the surveillance image is compared with a preset surveillance background image for motion detection to obtain a difference detection result;

[0018] Determining a monitoring foreground area according to the differential detection result;

[0019] According to the monitoring foreground area, a monitoring foreground image is extracted from the monitoring shot image.

[0020] As a further limitation of the technical solution of the embodiment of the present invention, the shadow detection of the monitoring foreground image, determining the foreground shadow area, and shadow removal processing of the monitoring foreground image to obtain the target foreground image specifically includes the following steps:

[0021] Performing vehicle target color recognition on the monitoring foreground image to determine a vehicle body area threshold;

[0022] Performing three-primary color analysis on the monitoring foreground image and the monitoring background image to obtain three-primary color component data;

[0023] Based on the vehicle body area threshold and the three primary color component data, calculating the shadow detection value of the monitored foreground image, and determining the corresponding shadow detection coordinates;

[0024] generating shadow detection data according to the shadow detection value and the corresponding shadow detection coordinate;

[0025] Analyzing the shadow detection data to determine a foreground shadow area;

[0026] According to the foreground shadow area, shadow removal processing is performed on the monitored foreground image to obtain a target foreground image.

[0027] As a further limitation of the technical solution of the embodiment of the present invention, the calculation formula for calculating the shadow detection value of the monitored foreground image based on the vehicle body area threshold and the three primary color component data and determining the corresponding shadow detection coordinate is:

[0028]

[0029] Among them, (x, y) is the shadow detection coordinate, S(x, y) is the shadow detection value at (x, y), V a1 、V a2 and V a3 is the body area threshold, V b1 、V b2 and V b3 is the preset boundary judgment threshold, R(x,y), G(x,y) and B(x,y) are the R component, G component and B component at (x,y) in the monitored foreground image, R b (x,y),G b (x,y) and B b (x, y) is the R component, G component and B component at (x, y) in the monitoring background image.

[0030] As a further limitation of the technical solution of the embodiment of the present invention, the grayscale processing of the target foreground image to obtain a grayscale foreground image, and based on the deep learning technology, identifying the black smoke pixel area in the grayscale foreground image, and calculating the black smoke pixel ratio specifically include the following steps:

[0031] grayscale the target foreground image to obtain a grayscale foreground image;

[0032] Based on deep learning technology, identifying black smoke pixel areas in the grayscale foreground image;

[0033] Counting the number of black smoke pixels in the black smoke pixel area;

[0034] Counting the total number of pixels of the surveillance image;

[0035] A black smoke pixel ratio is calculated according to the black smoke pixel number and the total pixel number.

[0036] As a further limitation of the technical solution of the embodiment of the present invention, the calculation formula for calculating the black smoke pixel ratio according to the number of black smoke pixels and the total number of pixels is:

[0037]

[0038] Among them, F a is the black smoke pixel ratio, N a is the number of black smoke pixels, N g is the total number of pixels.

[0039] As a further limitation of the technical solution of the embodiment of the present invention, the calculation formula for calculating the regional center coordinates of the black smoke pixel area is:

[0040]

[0041] Among them, (x o ,y o ) is the coordinate of the center of the region, (x i ,y i ) is the i-th coordinate in the black smoke pixel area;

[0042] The calculation formula for calculating the optimized pixel ratio is:

[0043]

[0044] Among them, F b To optimize the pixel ratio, k is the black smoke expansion ratio.

[0045] As a further limitation of the technical solution of the embodiment of the present invention, the comparing the optimized pixel ratio with the preset black smoke standard ratio, and when the optimized pixel ratio is greater than the black smoke standard ratio, identifying and uploading the black smoke vehicle information specifically includes the following steps:

[0046] Comparing the optimized pixel ratio with a preset black smoke standard ratio to determine whether the optimized pixel ratio is greater than the black smoke standard ratio;

[0047] When the optimized pixel ratio is greater than the black smoke standard ratio, determining a black smoke vehicle target from the grayscale foreground image;

[0048] intercepting a black-smoky vehicle image of the black-smoky vehicle target from the monitoring image;

[0049] Identify the black smoke vehicle image to obtain black smoke vehicle information;

[0050] The black smoke vehicle information is uploaded according to the preset information upload address.

[0051] A smoky vehicle detection system based on deep learning, the system comprises a monitoring shooting processing module, a shadow removal processing module, a pixel ratio calculation module, a ratio expansion optimization module and a smoky vehicle recognition module, wherein:

[0052] A monitoring shooting processing module is used to periodically acquire monitoring shooting images, perform motion detection on the monitoring shooting images based on a preset monitoring background image, and extract a monitoring foreground image from the monitoring shooting images;

[0053] A shadow removal processing module is used to perform shadow detection on the monitoring foreground image, determine the foreground shadow area, and perform shadow removal processing on the monitoring foreground image to obtain a target foreground image;

[0054] A pixel ratio calculation module is used to grayscale the target foreground image to obtain a grayscale foreground image, and based on deep learning technology, identify the black smoke pixel area in the grayscale foreground image and calculate the black smoke pixel ratio;

[0055] A ratio expansion optimization module, used to calculate the regional center coordinates of the black smoke pixel area, match the black smoke expansion ratio, perform expansion optimization on the black smoke pixel ratio, and calculate the optimized pixel ratio;

[0056] The black smoke vehicle identification module is used to compare the optimized pixel ratio with a preset black smoke standard ratio, and when the optimized pixel ratio is greater than the black smoke standard ratio, identify and upload the black smoke vehicle information.

[0057] As a further limitation of the technical solution of the embodiment of the present invention, the shadow removal processing module specifically includes:

[0058] A color recognition unit is used to perform vehicle target color recognition on the monitoring foreground image and determine a vehicle body area threshold;

[0059] A three-primary color analysis unit, used for performing three-primary color analysis on the monitoring foreground image and the monitoring background image to obtain three-primary color component data;

[0060] A shadow detection unit, used to calculate the shadow detection value of the monitoring foreground image based on the vehicle body area threshold and the three primary color component data, and determine the corresponding shadow detection coordinates;

[0061] A data generating unit, configured to generate shadow detection data according to the shadow detection value and the corresponding shadow detection coordinate;

[0062] A region determination unit, configured to analyze the shadow detection data to determine a foreground shadow region;

[0063] The shadow removal unit is used to perform shadow removal processing on the monitoring foreground image according to the foreground shadow area to obtain a target foreground image.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] The embodiment of the present invention periodically acquires surveillance images; performs shadow detection and shadow removal processing to obtain a target foreground image; performs grayscale processing, identifies black smoke pixel areas based on deep learning technology, and calculates black smoke pixel ratios; performs expansion optimization on the black smoke pixel ratios to calculate optimized pixel ratios; and identifies and uploads black smoke vehicle information when the optimized pixel ratio is greater than the black smoke standard ratio. Shadow detection and shadow removal processing can be performed on the surveillance foreground image to obtain a target foreground image, and then grayscale processing is performed to calculate the black smoke pixel ratio, and expansion optimization is performed to calculate the optimized pixel ratio, which is compared with the black smoke standard ratio, and the black smoke vehicle information is identified and uploaded, so that effective black smoke quantitative detection can be performed on vehicles on the road, and interference can be eliminated to accurately identify black smoke vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A flow chart of a method for detecting smoky vehicles based on deep learning provided by an embodiment of the present invention is shown;

[0067] Figure 2 A flowchart of extracting a monitoring foreground image in a method provided by an embodiment of the present invention is shown;

[0068] Figure 3 A flowchart of shadow removal processing in the method provided by an embodiment of the present invention is shown;

[0069] Figure 4 A flow chart showing the calculation of the black smoke pixel ratio in the method provided by an embodiment of the present invention is shown;

[0070] Figure 5 A flow chart showing the method for identifying and uploading black smoke vehicle information in an embodiment of the present invention is shown;

[0071] Figure 6 An application architecture diagram of a smoky vehicle detection system based on deep learning provided by an embodiment of the present invention is shown;

[0072] Figure 7 The structure block diagram of the shadow removal processing module in the system provided by the embodiment of the present invention is shown. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0074] It is understandable that in the prior art, since fuel vehicles have more or less observable exhaust emissions, and other factors can easily interfere with black smoke detection, it is impossible to conduct effective black smoke quantitative detection on vehicles on the road, and it is impossible to eliminate interference and accurately identify black smoke vehicles.

[0075] To solve the above problems, an embodiment of the present invention discloses a method and system for detecting black smoke vehicles based on deep learning. The method and system periodically acquire surveillance images, perform motion detection on the surveillance images based on a preset surveillance background image, and extract a surveillance foreground image from the surveillance images; perform shadow detection on the surveillance foreground image to determine the foreground shadow area, and perform shadow removal processing on the surveillance foreground image to obtain a target foreground image; perform grayscale processing on the target foreground image to obtain a grayscale foreground image, and identify the black smoke pixel area in the grayscale foreground image based on deep learning technology, and calculate the black smoke pixel ratio; calculate the regional center coordinates of the black smoke pixel area, match the black smoke expansion ratio, perform expansion optimization on the black smoke pixel ratio, and calculate the optimized pixel ratio; compare the optimized pixel ratio with the preset black smoke standard ratio, and when the optimized pixel ratio is greater than the black smoke standard ratio, identify and upload the black smoke vehicle information. It can perform shadow detection and shadow removal processing on the monitored foreground image to obtain the target foreground image, then perform grayscale processing, calculate the black smoke pixel ratio, and perform expansion optimization, calculate the optimized pixel ratio, compare it with the black smoke standard ratio, identify and upload the black smoke vehicle information, so as to effectively perform black smoke quantitative detection on vehicles on the road and eliminate interference to accurately identify black smoke vehicles.

[0076] Specifically, Figure 1 A flow chart of a smoky vehicle detection method based on deep learning provided by an embodiment of the present invention is shown.

[0077] In a preferred embodiment of the present invention, a method for detecting smoky vehicles based on deep learning is provided, and the method specifically comprises the following steps:

[0078] Step S101: periodically acquire surveillance images, perform motion detection on the surveillance images based on a preset surveillance background image, and extract a surveillance foreground image from the surveillance images.

[0079] In an embodiment of the present invention, according to a preset surveillance shooting cycle, a surveillance shooting instruction is periodically generated, and then surveillance shooting is performed according to the surveillance shooting instruction to obtain a surveillance shooting image, and a preset surveillance background image is imported. Through the background difference method, the surveillance shooting image and the surveillance background image are compared and analyzed for background and target to obtain a differential detection result, and then based on the differential detection result, the surveillance foreground area in the surveillance shooting image is determined, and then the surveillance foreground area in the surveillance shooting image is image-cut, and the surveillance foreground image is extracted from the surveillance shooting image.

[0080] It can be understood that the background difference method is to judge the background and the target by comparing the difference value and threshold between the monitoring image and the monitoring background image.

[0081] Specifically, Figure 2 A flow chart of extracting a monitoring foreground image in a method provided by an embodiment of the present invention is shown.

[0082] Among them, in the preferred embodiment provided by the present invention, the periodic acquisition of surveillance images, the motion detection of the surveillance images based on the preset surveillance background images, and the extraction of the surveillance foreground images from the surveillance images specifically include the following steps:

[0083] Step S1011: Generate surveillance shooting instructions periodically according to a preset surveillance shooting cycle.

[0084] Step S1012: perform surveillance shooting according to the surveillance shooting instruction to obtain surveillance shooting images.

[0085] Step S1013: using a background difference method, compare and analyze the motion detection of the surveillance image with a preset surveillance background image to obtain a difference detection result.

[0086] Step S1014: Determine a monitoring foreground area according to the differential detection result.

[0087] Step S1015: extracting a monitoring foreground image from the monitoring shot image according to the monitoring foreground area.

[0088] Furthermore, the smoky vehicle detection method based on deep learning also includes the following steps:

[0089] Step S102: perform shadow detection on the monitored foreground image to determine the foreground shadow area, and perform shadow removal processing on the monitored foreground image to obtain a target foreground image.

[0090] In an embodiment of the present invention, a target monitored vehicle is determined by performing vehicle target recognition on a monitored foreground image, and then color recognition is performed on the target monitored vehicle to determine a vehicle body area threshold, and three-primary color analysis is performed on the monitored foreground image and the monitored background image to determine R component, G component and B component at different pixel points in the monitored foreground image and the monitored background image, and three-primary color component data is obtained, and then based on the vehicle body area threshold and the three-primary color component data, a shadow detection value of the monitored foreground image is calculated, and a corresponding shadow detection coordinate is determined, and then shadow detection data is generated according to the shadow detection value and the corresponding shadow detection coordinate, and a shadow area is filled for the shadow detection coordinate whose shadow detection value in the shadow detection data is 1 to determine a foreground shadow area, and then, according to the foreground shadow area, a corresponding shadow removal process is performed on the monitored foreground image to obtain a target foreground image. Specifically, the shadow detection value of the monitored foreground image is calculated, and the calculation formula for determining the corresponding shadow detection coordinate is:

[0091]

[0092] Among them, (x, y) is the shadow detection coordinate, S(x, y) is the shadow detection value at (x, y), V a1 、V a2 and V a3 is the body area threshold, V b1 、V b2 and V b3 is the preset boundary judgment threshold, R(x,y), G(x,y) and B(x,y) are the R component, G component and B component at (x,y) in the monitored foreground image, R b (x,y),G b (x,y) and B b (x, y) is the R component, G component and B component at (x, y) in the monitoring background image.

[0093] It is understandable that if the body color of the target monitored vehicle is dark such as black or gray, the identity color can be distinguished from the shadow by limiting the body area threshold.

[0094] It can be understood that the shadow removal process is a process of replacing the image of the foreground shadow area in the monitored foreground image with the image of the foreground shadow area in the monitored background image.

[0095] Specifically, Figure 3 A flowchart of shadow removal processing in the method provided by an embodiment of the present invention is shown.

[0096] Among them, in the preferred embodiment provided by the present invention, the shadow detection is performed on the monitoring foreground image, the foreground shadow area is determined, and the shadow removal process is performed on the monitoring foreground image to obtain the target foreground image specifically includes the following steps:

[0097] Step S1021: perform vehicle target color recognition on the monitored foreground image to determine a vehicle body area threshold.

[0098] Step S1022: perform three-primary-color analysis on the monitored foreground image and the monitored background image to obtain three-primary-color component data.

[0099] Step S1023: Calculate the shadow detection value of the monitored foreground image based on the vehicle body area threshold and the three primary color component data, and determine the corresponding shadow detection coordinates.

[0100] Step S1024: Generate shadow detection data according to the shadow detection value and the corresponding shadow detection coordinates.

[0101] Step S1025: Analyze the shadow detection data to determine the foreground shadow area.

[0102] Step S1026: Perform shadow removal processing on the monitored foreground image according to the foreground shadow area to obtain a target foreground image.

[0103] Furthermore, the smoky vehicle detection method based on deep learning also includes the following steps:

[0104] Step S103: grayscale the target foreground image to obtain a grayscale foreground image, and based on deep learning technology, identify the black smoke pixel area in the grayscale foreground image and calculate the black smoke pixel ratio.

[0105] In the embodiment of the present invention, the target foreground image is grayed to obtain a gray foreground image. Based on the deep learning technology, the black smoke gray value range is obtained. According to the black smoke gray value range, the black smoke pixel area in the gray foreground image is identified. Then, the number of black smoke pixels in the black smoke pixel area is counted, and the total number of pixels in the monitoring image is counted. Then, according to the number of black smoke pixels and the total number of pixels, the black smoke pixel ratio is calculated. Specifically, the calculation formula of the black smoke pixel ratio is:

[0106]

[0107] Among them, F a is the black smoke pixel ratio, N a is the number of black smoke pixels, N g is the total number of pixels.

[0108] Specifically, Figure 4A flow chart of calculating the black smoke pixel ratio in the method provided by an embodiment of the present invention is shown.

[0109] Among them, in the preferred embodiment provided by the present invention, the grayscale processing is performed on the target foreground image to obtain a grayscale foreground image, and based on the deep learning technology, the black smoke pixel area in the grayscale foreground image is identified, and the black smoke pixel ratio is calculated, which specifically includes the following steps:

[0110] Step S1031 , grayscale the target foreground image to obtain a grayscale foreground image.

[0111] Step S1032: Based on deep learning technology, identify the black smoke pixel area in the grayscale foreground image.

[0112] Step S1033: Count the number of black smoke pixels in the black smoke pixel area.

[0113] Step S1034: Count the total number of pixels of the surveillance image.

[0114] Step S1035: Calculate the black smoke pixel ratio according to the number of black smoke pixels and the total number of pixels.

[0115] Furthermore, the smoky vehicle detection method based on deep learning also includes the following steps:

[0116] Step S104, calculating the region center coordinates of the black smoke pixel region, matching the black smoke expansion ratio, performing expansion optimization on the black smoke pixel ratio, and calculating the optimized pixel ratio.

[0117] In the embodiment of the present invention, the regional center coordinates of the black smoke pixel area are calculated, and the corresponding black smoke expansion ratio is matched from a preset expansion ratio database according to the regional center coordinates, and then the black smoke pixel ratio is expanded and optimized according to the black smoke expansion ratio, and the optimized pixel ratio is calculated. Specifically, the calculation formula of the regional center coordinates is:

[0118]

[0119] Among them, (x o ,y o ) is the coordinate of the center of the region, (x i ,y i ) is the i-th coordinate in the black smoke pixel area;

[0120] The formula for calculating the optimized pixel ratio is:

[0121]

[0122] Among them, F b To optimize the pixel ratio, k is the black smoke expansion ratio.

[0123] Step S105: compare the optimized pixel ratio with a preset black smoke standard ratio, and when the optimized pixel ratio is greater than the black smoke standard ratio, identify and upload black smoke vehicle information.

[0124] In an embodiment of the present invention, the optimized pixel ratio is compared with a preset black smoke standard ratio to determine whether the optimized pixel ratio is greater than the black smoke standard ratio. When the optimized pixel ratio is greater than the black smoke standard ratio, a black smoke vehicle target is determined from a grayscale foreground image, and then a black smoke vehicle image of the black smoke vehicle target is captured from a surveillance image. By performing license plate and vehicle model recognition on the black smoke vehicle image, black smoke vehicle information including license plate number, vehicle model, etc. is obtained, and the black smoke vehicle information is uploaded according to a preset information upload address.

[0125] Specifically, Figure 5 A flow chart of identifying and uploading black smoke vehicle information in a method provided by an embodiment of the present invention is shown.

[0126] Among them, in the preferred embodiment provided by the present invention, the said optimizing pixel ratio is compared with the preset black smoke standard ratio, and when the optimizing pixel ratio is greater than the black smoke standard ratio, identifying and uploading the black smoke vehicle information specifically comprises the following steps:

[0127] Step S1051: Compare the optimized pixel ratio with a preset black smoke standard ratio to determine whether the optimized pixel ratio is greater than the black smoke standard ratio.

[0128] Step S1052: When the optimized pixel ratio is greater than the black smoke standard ratio, determine the black smoke vehicle target from the grayscale foreground image.

[0129] Step S1053: capturing a black smoke vehicle image of the black smoke vehicle target from the monitoring image.

[0130] Step S1054: Identify the black smoke vehicle image to obtain black smoke vehicle information.

[0131] Step S1055: Upload the black smoke vehicle information according to the preset information upload address.

[0132] Furthermore, Figure 6 An application architecture diagram of a smoky vehicle detection system based on deep learning provided in an embodiment of the present invention is shown.

[0133] Among them, in another preferred embodiment provided by the present invention, a smoky vehicle detection system based on deep learning includes:

[0134] The monitoring shooting processing module 101 is used to periodically acquire monitoring shooting images, perform motion detection on the monitoring shooting images based on a preset monitoring background image, and extract a monitoring foreground image from the monitoring shooting images.

[0135] In an embodiment of the present invention, the monitoring shooting processing module 101 periodically generates a monitoring shooting instruction according to a preset monitoring shooting cycle, and then performs monitoring shooting according to the monitoring shooting instruction, obtains a monitoring shooting image, imports a preset monitoring background image, and compares and analyzes the background and target of the monitoring shooting image and the monitoring background image through the background difference method to obtain a differential detection result, and then determines the monitoring foreground area in the monitoring shooting image based on the differential detection result, and then performs image capture on the monitoring foreground area in the monitoring shooting image, and extracts the monitoring foreground image from the monitoring shooting image.

[0136] The shadow removal processing module 102 is used to perform shadow detection on the monitoring foreground image, determine the foreground shadow area, and perform shadow removal processing on the monitoring foreground image to obtain a target foreground image.

[0137] In the embodiment of the present invention, the shadow removal processing module 102 determines the target monitored vehicle by performing vehicle target recognition on the monitored foreground image, and then performs color recognition on the target monitored vehicle to determine the vehicle body area threshold, and performs three-primary color analysis on the monitored foreground image and the monitored background image to determine the R component, G component and B component at different pixel points in the monitored foreground image and the monitored background image, and obtains the three-primary color component data, and then calculates the shadow detection value of the monitored foreground image based on the vehicle body area threshold and the three-primary color component data, and determines the corresponding shadow detection coordinates, and then generates shadow detection data according to the shadow detection value and the corresponding shadow detection coordinates, and determines the foreground shadow area by filling the shadow area of ​​the shadow detection coordinates with a shadow detection value of 1 in the shadow detection data, and then performs corresponding shadow removal processing on the monitored foreground image according to the foreground shadow area to obtain the target foreground image. Specifically, the calculation formula for calculating the shadow detection value of the monitored foreground image and determining the corresponding shadow detection coordinates is:

[0138]

[0139] Among them, (x, y) is the shadow detection coordinate, S(x, y) is the shadow detection value at (x, y), V a1 、V a2 and V a3 is the body area threshold, V b1 、V b2 and V b3is the preset boundary judgment threshold, R(x,y), G(x,y) and B(x,y) are the R component, G component and B component at (x,y) in the monitored foreground image, R b (x,y),G b (x,y) and B b (x, y) is the R component, G component and B component at (x, y) in the monitoring background image.

[0140] Specifically, Figure 7 It shows a structural block diagram of the shadow removal processing module 102 in the system provided by an embodiment of the present invention.

[0141] Among them, in the preferred embodiment provided by the present invention, the shadow removal processing module 102 specifically includes:

[0142] The color recognition unit 1021 is used to perform vehicle target color recognition on the monitoring foreground image and determine a vehicle body area threshold.

[0143] The three-primary-color analysis unit 1022 is used to perform three-primary-color analysis on the monitoring foreground image and the monitoring background image to obtain three-primary-color component data.

[0144] The shadow detection unit 1023 is used to calculate the shadow detection value of the monitored foreground image based on the vehicle body area threshold and the three primary color component data, and determine the corresponding shadow detection coordinates.

[0145] The data generating unit 1024 is configured to generate shadow detection data according to the shadow detection value and the corresponding shadow detection coordinate.

[0146] The area determination unit 1025 is used to analyze the shadow detection data to determine the foreground shadow area.

[0147] The shadow removal unit 1026 is used to perform shadow removal processing on the monitoring foreground image according to the foreground shadow area to obtain a target foreground image.

[0148] Furthermore, the smoky vehicle detection system based on deep learning also includes:

[0149] The pixel ratio calculation module 103 is used to grayscale the target foreground image to obtain a grayscale foreground image, and identify the black smoke pixel area in the grayscale foreground image based on deep learning technology to calculate the black smoke pixel ratio.

[0150] In the embodiment of the present invention, the pixel ratio calculation module 103 performs grayscale processing on the target foreground image to obtain a grayscale foreground image, obtains the black smoke grayscale value range based on the deep learning technology, identifies the black smoke pixel area in the grayscale foreground image according to the black smoke grayscale value range, and then counts the number of black smoke pixels in the black smoke pixel area, and counts the total number of pixels of the monitoring image, and then calculates the black smoke pixel ratio according to the number of black smoke pixels and the total number of pixels. Specifically, the calculation formula of the black smoke pixel ratio is:

[0151]

[0152] Among them, F a is the black smoke pixel ratio, N a is the number of black smoke pixels, N g is the total number of pixels.

[0153] The ratio expansion optimization module 104 is used to calculate the region center coordinates of the black smoke pixel region, match the black smoke expansion ratio, perform expansion optimization on the black smoke pixel ratio, and calculate the optimized pixel ratio.

[0154] In the embodiment of the present invention, the ratio expansion optimization module 104 calculates the regional center coordinates of the black smoke pixel area, matches the corresponding black smoke expansion ratio from a preset expansion ratio database according to the regional center coordinates, and then performs expansion optimization on the black smoke pixel ratio according to the black smoke expansion ratio to calculate the optimized pixel ratio. Specifically, the calculation formula of the regional center coordinates is:

[0155]

[0156] Among them, (x o ,y o ) is the coordinate of the center of the region, (x i ,y i ) is the i-th coordinate in the black smoke pixel area;

[0157] The formula for calculating the optimized pixel ratio is:

[0158]

[0159] Among them, F b To optimize the pixel ratio, k is the black smoke expansion ratio.

[0160] The black smoke vehicle identification module 105 is used to compare the optimized pixel ratio with a preset black smoke standard ratio, and identify and upload the black smoke vehicle information when the optimized pixel ratio is greater than the black smoke standard ratio.

[0161] In the embodiment of the present invention, the black smoke vehicle identification module 105 compares the optimized pixel ratio with the preset black smoke standard ratio to determine whether the optimized pixel ratio is greater than the black smoke standard ratio. When the optimized pixel ratio is greater than the black smoke standard ratio, the black smoke vehicle target is determined from the grayscale foreground image, and then the black smoke vehicle image of the black smoke vehicle target is captured from the monitoring image. By performing license plate and vehicle model recognition on the black smoke vehicle image, black smoke vehicle information including license plate number, vehicle model, etc. is obtained, and the black smoke vehicle information is uploaded and processed according to the preset information upload address.

[0162] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0163] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0164] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A smoky vehicle detection method based on deep learning, characterized in that: The method specifically comprises the following steps: Periodically acquiring surveillance images, performing motion detection on the surveillance images based on a preset surveillance background image, and extracting a surveillance foreground image from the surveillance images; Performing shadow detection on the monitored foreground image to determine the foreground shadow area, and performing shadow removal processing on the monitored foreground image to obtain a target foreground image; Graying the target foreground image to obtain a gray foreground image, and based on deep learning technology, identifying a black smoke pixel area in the gray foreground image, and calculating a black smoke pixel ratio; Calculating the region center coordinates of the black smoke pixel region, matching the black smoke expansion ratio, performing expansion optimization on the black smoke pixel ratio, and calculating the optimized pixel ratio; The optimized pixel ratio is compared with a preset black smoke standard ratio, and when the optimized pixel ratio is greater than the black smoke standard ratio, the black smoke vehicle information is identified and uploaded.

2. The method for detecting smoky vehicles based on deep learning according to claim 1, characterized in that: The periodic acquisition of surveillance images, the motion detection of the surveillance images based on a preset surveillance background image, and the extraction of surveillance foreground images from the surveillance images specifically include the following steps: According to the preset monitoring shooting cycle, periodically generate monitoring shooting instructions; According to the surveillance shooting instruction, surveillance shooting is performed to obtain surveillance shooting images; By using a background difference method, the surveillance image is compared with a preset surveillance background image for motion detection to obtain a difference detection result; Determining a monitoring foreground area according to the differential detection result; According to the monitoring foreground area, a monitoring foreground image is extracted from the monitoring shot image.

3. The method for detecting smoky vehicles based on deep learning according to claim 1, characterized in that: The step of performing shadow detection on the monitored foreground image, determining the foreground shadow area, and performing shadow removal processing on the monitored foreground image to obtain the target foreground image specifically comprises the following steps: Performing vehicle target color recognition on the monitoring foreground image to determine a vehicle body area threshold; Performing three-primary color analysis on the monitoring foreground image and the monitoring background image to obtain three-primary color component data; Based on the vehicle body area threshold and the three primary color component data, calculating the shadow detection value of the monitored foreground image, and determining the corresponding shadow detection coordinates; generating shadow detection data according to the shadow detection value and the corresponding shadow detection coordinate; Analyzing the shadow detection data to determine a foreground shadow area; According to the foreground shadow area, shadow removal processing is performed on the monitored foreground image to obtain a target foreground image.

4. The method for detecting smoky vehicles based on deep learning according to claim 3 is characterized in that: The calculation formula for calculating the shadow detection value of the monitored foreground image based on the vehicle body area threshold and the three primary color component data and determining the corresponding shadow detection coordinates is: Where (x, y) is the shadow detection coordinate, S(x, y) is the shadow detection value at (x, y), and V a1 、V a2 and V a is the body area threshold, V b1 、V b2 and V b3 is the preset boundary judgment threshold, R(x, y), G(x, y) and B(x, y) are the R component, G component and B component at (x, y) in the monitored foreground image, R b (x, y), G b (x, y) and B b (x, y) is the R component, G component and B component at (x, y) in the monitoring background image.

5. The method for detecting smoky vehicles based on deep learning according to claim 1, characterized in that: The grayscale processing of the target foreground image to obtain a grayscale foreground image, and based on the deep learning technology, identifying the black smoke pixel area in the grayscale foreground image, and calculating the black smoke pixel ratio specifically include the following steps: grayscale the target foreground image to obtain a grayscale foreground image; Based on deep learning technology, identifying black smoke pixel areas in the grayscale foreground image; Counting the number of black smoke pixels in the black smoke pixel area; Counting the total number of pixels of the surveillance image; A black smoke pixel ratio is calculated according to the black smoke pixel number and the total pixel number.

6. The method for detecting smoky vehicles based on deep learning according to claim 5 is characterized in that: The calculation formula for calculating the black smoke pixel ratio according to the number of black smoke pixels and the total number of pixels is: Among them, F a is the black smoke pixel ratio, N a is the number of black smoke pixels, N g is the total number of pixels.

7. The method for detecting smoky vehicles based on deep learning according to claim 6 is characterized in that: The calculation formula for calculating the area center coordinates of the black smoke pixel area is: Among them, (x o ,y o ) is the coordinate of the center of the region, (x i ,y i ) is the i-th coordinate in the black smoke pixel area; The calculation formula for calculating the optimized pixel ratio is: Among them, F b To optimize the pixel ratio, k is the black smoke expansion ratio.

8. The method for detecting smoky vehicles based on deep learning according to claim 1, characterized in that: The step of comparing the optimized pixel ratio with a preset black smoke standard ratio, and when the optimized pixel ratio is greater than the black smoke standard ratio, identifying and uploading the black smoke vehicle information specifically includes the following steps: Comparing the optimized pixel ratio with a preset black smoke standard ratio to determine whether the optimized pixel ratio is greater than the black smoke standard ratio; When the optimized pixel ratio is greater than the black smoke standard ratio, determining a black smoke vehicle target from the grayscale foreground image; Intercepting a black-smoky vehicle image of the black-smoky vehicle target from the monitoring image; Identify the black smoke vehicle image to obtain black smoke vehicle information; The black smoke vehicle information is uploaded according to the preset information upload address.

9. A smoky vehicle detection system based on deep learning, characterized in that: The system includes a monitoring shooting processing module, a shadow removal processing module, a pixel ratio calculation module, a ratio expansion optimization module and a smoky vehicle identification module, wherein: A monitoring shooting processing module is used to periodically acquire monitoring shooting images, perform motion detection on the monitoring shooting images based on a preset monitoring background image, and extract a monitoring foreground image from the monitoring shooting images; A shadow removal processing module is used to perform shadow detection on the monitoring foreground image, determine the foreground shadow area, and perform shadow removal processing on the monitoring foreground image to obtain a target foreground image; A pixel ratio calculation module is used to grayscale the target foreground image to obtain a grayscale foreground image, and based on deep learning technology, identify the black smoke pixel area in the grayscale foreground image and calculate the black smoke pixel ratio; A ratio expansion optimization module, used to calculate the regional center coordinates of the black smoke pixel area, match the black smoke expansion ratio, perform expansion optimization on the black smoke pixel ratio, and calculate the optimized pixel ratio; The black smoke vehicle identification module is used to compare the optimized pixel ratio with a preset black smoke standard ratio, and when the optimized pixel ratio is greater than the black smoke standard ratio, identify and upload the black smoke vehicle information.

10. The smoky vehicle detection system based on deep learning according to claim 9 is characterized in that: The shadow removal processing module specifically includes: A color recognition unit, used to perform vehicle target color recognition on the monitoring foreground image and determine a vehicle body area threshold; A three-primary color analysis unit, used for performing three-primary color analysis on the monitoring foreground image and the monitoring background image to obtain three-primary color component data; A shadow detection unit, used to calculate the shadow detection value of the monitoring foreground image based on the vehicle body area threshold and the three primary color component data, and determine the corresponding shadow detection coordinates; A data generating unit, configured to generate shadow detection data according to the shadow detection value and the corresponding shadow detection coordinate; A region determination unit, configured to analyze the shadow detection data to determine a foreground shadow region; The shadow removal unit is used to perform shadow removal processing on the monitoring foreground image according to the foreground shadow area to obtain a target foreground image.

Citation Information

Patent Citations

  • Motion shadow detection method based on color and gradient features

    CN102298781A

  • Household service robot performing smoke detection through image

    CN105957295A

  • A black smoke vehicle detection method based on video analysis

    CN109165602A

  • Lampblack concentration identification method and device and range hood

    CN111928303A

  • Lampblack concentration identification method and device and range hood

    CN111928304A