A black smoke vehicle detection method and system based on deep learning

Abstract

The present invention is applicable to the field of black smoke vehicle detection technology and provides a deep learning-based black smoke vehicle detection method and system. The present invention periodically acquires surveillance camera images; performs shadow detection and shadow removal processing to obtain a target foreground image; performs grayscale processing, and based on deep learning technology, identifies black smoke pixel areas and calculates the black smoke pixel ratio; performs dilation optimization on the black smoke pixel ratio to calculate the optimized pixel ratio; and when the optimized pixel ratio exceeds the black smoke standard ratio, identifies and uploads black smoke vehicle information. Shadow detection and shadow removal processing can be performed on the surveillance foreground image to obtain a target foreground image, followed by grayscale processing, calculating the black smoke pixel ratio, performing dilation optimization, and calculating the optimized pixel ratio. This is then compared with the black smoke standard ratio, identifying and uploading black smoke vehicle information. This allows for effective black smoke quantitative detection of vehicles on the road, eliminating interference and accurately identifying black smoke vehicles.

Description

Technical Field

[0001] The present invention belongs to the technical field of black smoke vehicle detection, and in particular relates to a black smoke vehicle detection method and system based on deep learning. Background Art

[0002] Black smoke vehicle detection is the monitoring of motor vehicles that emit visible black smoke or other visible pollutants.

[0003] Black smoke emissions indicate incomplete combustion in the vehicle's engine, which may be caused by poor fuel quality, improper engine maintenance or vehicle aging.

[0004] The purpose of detecting smoky vehicles is to reduce environmental pollution and protect public health.

[0005] In the existing technology, since fuel vehicles have more or less observable exhaust emissions, and other factors can easily interfere with black smoke detection, it is impossible to effectively quantify black smoke detection on vehicles on the road, and it is impossible to eliminate interference and accurately identify black smoke vehicles. Summary of the Invention

[0006] The purpose of the embodiments of the present invention is to provide a method and system for detecting black smoke vehicles based on deep learning, aiming to solve the technical problems existing in the existing technology mentioned in the background technology.

[0007] The embodiment of the present invention is implemented as follows:

[0008] A method for detecting smoky vehicles based on deep learning, the method specifically comprising the following steps:

[0009] Periodically acquiring surveillance images, performing motion detection on the surveillance images based on a preset surveillance background image, and extracting a surveillance foreground image from the surveillance images;

[0010] Performing shadow detection on the monitoring foreground image to determine the foreground shadow area, and performing shadow removal processing on the monitoring foreground image to obtain a target foreground image;

[0011] grayscale the target foreground image to obtain a grayscale foreground image, and identify black smoke pixel areas in the grayscale foreground image based on deep learning technology, and calculate the black smoke pixel ratio;

[0012] Calculating the coordinates of the center of the black smoke pixel area, matching the black smoke expansion ratio, performing expansion optimization on the black smoke pixel ratio, and calculating the optimized pixel ratio;

[0013] The optimized pixel ratio is compared with a preset black smoke standard ratio, and when the optimized pixel ratio is greater than the black smoke standard ratio, the black smoke vehicle information is identified and uploaded.

[0014] As a further limitation of the technical solution of the embodiment of the present invention, the periodic acquisition of surveillance images, the performance of motion detection on the surveillance images based on a preset surveillance background image, and the extraction of a surveillance foreground image from the surveillance images specifically include the following steps:

[0015] Generate surveillance shooting instructions periodically according to the preset surveillance shooting cycle;

[0016] Perform surveillance shooting according to the surveillance shooting instruction and obtain surveillance shooting images;

[0017] By using a background difference method, the monitoring image is compared with a preset monitoring background image for motion detection to obtain a difference detection result;

[0018] determining a monitoring foreground area according to the differential detection result;

[0019] According to the monitoring foreground area, a monitoring foreground image is extracted from the monitoring shot image.

[0020] As a further limitation of the technical solution of the embodiment of the present invention, the shadow detection on the monitoring foreground image, determining the foreground shadow area, and shadow removal processing on the monitoring foreground image to obtain the target foreground image specifically includes the following steps:

[0021] Performing vehicle target color recognition on the monitoring foreground image to determine a vehicle body area threshold;

[0022] Performing three-primary color analysis on the monitoring foreground image and the monitoring background image to obtain three-primary color component data;

[0023] Calculating a shadow detection value of the monitored foreground image based on the vehicle body region threshold and the three primary color component data, and determining a corresponding shadow detection coordinate;

[0024] generating shadow detection data according to the shadow detection value and the corresponding shadow detection coordinates;

[0025] Analyzing the shadow detection data to determine a foreground shadow area;

[0026] According to the foreground shadow area, shadow removal processing is performed on the monitoring foreground image to obtain a target foreground image.

[0027] As a further limitation of the technical solution of the embodiment of the present invention, the calculation formula for calculating the shadow detection value of the monitored foreground image based on the vehicle body area threshold and the three primary color component data and determining the corresponding shadow detection coordinates is:

[0028]

[0029] Among them, (x, y) is the shadow detection coordinate, S(x, y) is the shadow detection value at (x, y), V a1 、V a2 and V a3 is the body area threshold, V b1 、V b2 and V b3 is the preset boundary judgment threshold, R(x,y), G(x,y) and B(x,y) are the R component, G component and B component at (x,y) in the monitored foreground image, R b (x,y),G b (x,y) and B b (x, y) is the R component, G component, and B component at (x, y) in the monitoring background image.

[0030] As a further limitation of the technical solution of the embodiment of the present invention, grayscale processing is performed on the target foreground image to obtain a grayscale foreground image, and based on deep learning technology, black smoke pixel areas in the grayscale foreground image are identified, and the black smoke pixel ratio is calculated, which specifically includes the following steps:

[0031] grayscale the target foreground image to obtain a grayscale foreground image;

[0032] Based on deep learning technology, identifying black smoke pixel areas in the grayscale foreground image;

[0033] Counting the number of black smoke pixels in the black smoke pixel area;

[0034] Counting the total number of pixels of the surveillance image;

[0035] A black smoke pixel ratio is calculated according to the number of black smoke pixels and the total number of pixels.

[0036] As a further limitation of the technical solution of the embodiment of the present invention, the calculation formula for calculating the black smoke pixel ratio based on the number of black smoke pixels and the total number of pixels is:

[0037]

[0038] Among them, F a is the black smoke pixel ratio, N a is the number of black smoke pixels, N g is the total number of pixels.

[0039] As a further limitation of the technical solution of the embodiment of the present invention, the calculation formula for calculating the regional center coordinates of the black smoke pixel area is:

[0040]

[0041] Among them, (x o ,y o ) is the coordinate of the center of the region, (x i ,y i ) is the i-th coordinate in the black smoke pixel area;

[0042] The calculation formula for calculating the optimized pixel ratio is:

[0043]

[0044] Among them, F b To optimize the pixel ratio, k is the smoke expansion ratio.

[0045] As a further limitation of the technical solution of the embodiment of the present invention, comparing the optimized pixel ratio with a preset black smoke standard ratio, and when the optimized pixel ratio is greater than the black smoke standard ratio, identifying and uploading black smoke vehicle information specifically includes the following steps:

[0046] Comparing the optimized pixel ratio with a preset black smoke standard ratio to determine whether the optimized pixel ratio is greater than the black smoke standard ratio;

[0047] When the optimized pixel ratio is greater than the black smoke standard ratio, determining a black smoke vehicle target from the grayscale foreground image;

[0048] intercepting a black smoke vehicle image of the black smoke vehicle target from the monitoring image;

[0049] Identifying the black smoke vehicle image to obtain black smoke vehicle information;

[0050] The black smoke vehicle information is uploaded according to the preset information upload address.

[0051] A deep learning-based black smoke vehicle detection system includes a monitoring and shooting processing module, a shadow removal processing module, a pixel ratio calculation module, a ratio expansion optimization module, and a black smoke vehicle recognition module, wherein:

[0052] A surveillance camera processing module is used to periodically acquire surveillance camera images, perform motion detection on the surveillance camera images based on a preset surveillance background image, and extract a surveillance foreground image from the surveillance camera images;

[0053] a shadow removal processing module, configured to perform shadow detection on the monitoring foreground image, determine a foreground shadow area, and perform shadow removal processing on the monitoring foreground image to obtain a target foreground image;

[0054] a pixel ratio calculation module, configured to grayscale the target foreground image to obtain a grayscale foreground image, and identify black smoke pixel areas in the grayscale foreground image based on deep learning technology to calculate the black smoke pixel ratio;

[0055] a ratio expansion optimization module, configured to calculate the coordinates of the center of the black smoke pixel area, match the black smoke expansion ratio, perform expansion optimization on the black smoke pixel ratio, and calculate the optimized pixel ratio;

[0056] The black smoke vehicle identification module is used to compare the optimized pixel ratio with a preset black smoke standard ratio, and when the optimized pixel ratio is greater than the black smoke standard ratio, identify and upload the black smoke vehicle information.

[0057] As a further limitation of the technical solution of the embodiment of the present invention, the shadow removal processing module specifically includes:

[0058] A color recognition unit is used to perform vehicle target color recognition on the monitoring foreground image and determine a vehicle body area threshold;

[0059] A three-primary color analysis unit is used to perform three-primary color analysis on the monitoring foreground image and the monitoring background image to obtain three-primary color component data;

[0060] a shadow detection unit, configured to calculate a shadow detection value of the monitored foreground image based on the vehicle body region threshold and the three primary color component data, and determine a corresponding shadow detection coordinate;

[0061] a data generating unit, configured to generate shadow detection data according to the shadow detection value and the corresponding shadow detection coordinate;

[0062] an area determination unit, configured to analyze the shadow detection data and determine a foreground shadow area;

[0063] The shadow removal unit is used to perform shadow removal processing on the monitoring foreground image according to the foreground shadow area to obtain a target foreground image.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] The embodiment of the present invention periodically acquires surveillance images; performs shadow detection and shadow removal processing to obtain a target foreground image; performs grayscale processing, identifies black smoke pixel areas based on deep learning technology, and calculates the black smoke pixel ratio; performs dilation optimization on the black smoke pixel ratio to calculate the optimized pixel ratio; and identifies and uploads black smoke vehicle information when the optimized pixel ratio exceeds the black smoke standard ratio. Shadow detection and shadow removal processing can be performed on the surveillance foreground image to obtain a target foreground image. Grayscale processing is then performed to calculate the black smoke pixel ratio, perform dilation optimization, calculate the optimized pixel ratio, compare it with the black smoke standard ratio, identify and upload black smoke vehicle information, thereby effectively performing black smoke quantitative detection on vehicles on the road and accurately identifying black smoke vehicles without interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A flowchart of a method for detecting smoky vehicles based on deep learning is shown in an embodiment of the present invention;

[0067] Figure 2 A flowchart of extracting a monitoring foreground image in a method provided by an embodiment of the present invention is shown;

[0068] Figure 3 A flowchart of shadow removal processing in the method provided by an embodiment of the present invention is shown;

[0069] Figure 4 A flowchart of calculating the black smoke pixel ratio in the method provided by an embodiment of the present invention is shown;

[0070] Figure 5 A flowchart of the method for identifying and uploading black smoke vehicle information in an embodiment of the present invention is shown;

[0071] Figure 6 The following is an application architecture diagram of a deep learning-based black smoke vehicle detection system provided by an embodiment of the present invention;

[0072] Figure 7 The figure shows a structural block diagram of a shadow removal processing module in a system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0074] It is understandable that in the existing technology, since fuel vehicles have more or less observable exhaust emissions, and other factors can easily interfere with black smoke detection, it is impossible to conduct effective black smoke quantitative detection on vehicles on the road, and it is impossible to eliminate interference and accurately identify black smoke vehicles.

[0075] To solve the above problems, an embodiment of the present invention discloses a method and system for detecting black smoke vehicles based on deep learning. The method and system periodically acquire surveillance images, perform motion detection on the surveillance images based on a preset surveillance background image, and extract a surveillance foreground image from the surveillance images; perform shadow detection on the surveillance foreground image to determine the foreground shadow area, and perform shadow removal processing on the surveillance foreground image to obtain a target foreground image; perform grayscale processing on the target foreground image to obtain a grayscale foreground image, and identify the black smoke pixel area in the grayscale foreground image based on deep learning technology, and calculate the black smoke pixel ratio; calculate the regional center coordinates of the black smoke pixel area, match the black smoke expansion ratio, perform expansion optimization on the black smoke pixel ratio, and calculate the optimized pixel ratio; compare the optimized pixel ratio with the preset black smoke standard ratio, and when the optimized pixel ratio is greater than the black smoke standard ratio, identify and upload the black smoke vehicle information. It can perform shadow detection and shadow removal on the monitored foreground image to obtain the target foreground image, then perform grayscale processing, calculate the black smoke pixel ratio, and perform expansion optimization. The optimized pixel ratio is calculated and compared with the black smoke standard ratio, and the black smoke vehicle information is identified and uploaded. In this way, it can effectively perform black smoke quantitative detection on vehicles on the road and eliminate interference to accurately identify black smoke vehicles.

[0076] Specifically, Figure 1 A flowchart of a method for detecting smoky vehicles based on deep learning provided by an embodiment of the present invention is shown.

[0077] In a preferred embodiment of the present invention, a method for detecting smoky vehicles based on deep learning is provided, and the method specifically comprises the following steps:

[0078] Step S101: periodically acquire surveillance images, perform motion detection on the surveillance images based on a preset surveillance background image, and extract a surveillance foreground image from the surveillance images.

[0079] In an embodiment of the present invention, according to a preset surveillance shooting cycle, a surveillance shooting instruction is periodically generated, and then surveillance shooting is performed according to the surveillance shooting instruction to obtain a surveillance shooting image, and a preset surveillance background image is imported. Through the background difference method, the surveillance shooting image and the surveillance background image are compared and analyzed for background and target to obtain a differential detection result, and then based on the differential detection result, the surveillance foreground area in the surveillance shooting image is determined, and then the surveillance foreground area in the surveillance shooting image is image-cut, and the surveillance foreground image is extracted from the surveillance shooting image.

[0080] It can be understood that the background difference method is to judge the background and the target by comparing the difference value between the monitoring image and the monitoring background image with a threshold.

[0081] Specifically, Figure 2 A flowchart of extracting a monitoring foreground image in a method provided by an embodiment of the present invention is shown.

[0082] In a preferred embodiment of the present invention, the periodic acquisition of surveillance images, the motion detection of the surveillance images based on a preset surveillance background image, and the extraction of the surveillance foreground image from the surveillance images specifically include the following steps:

[0083] Step S1011: periodically generate surveillance shooting instructions according to a preset surveillance shooting cycle.

[0084] Step S1012: Perform surveillance shooting according to the surveillance shooting instruction to obtain surveillance shooting images.

[0085] Step S1013: Using a background difference method, the surveillance image is subjected to comparative analysis for motion detection with a preset surveillance background image to obtain a difference detection result.

[0086] Step S1014: Determine a monitoring foreground area based on the differential detection result.

[0087] Step S1015: extracting a monitoring foreground image from the monitoring shot image according to the monitoring foreground area.

[0088] Furthermore, the deep learning-based black smoke vehicle detection method further includes the following steps:

[0089] Step S102: Perform shadow detection on the monitored foreground image to determine a foreground shadow area, and perform shadow removal processing on the monitored foreground image to obtain a target foreground image.

[0090] In an embodiment of the present invention, vehicle target recognition is performed on a monitoring foreground image to determine a target monitoring vehicle, and then color recognition is performed on the target monitoring vehicle to determine a vehicle body area threshold. Three-primary color analysis is performed on the monitoring foreground image and the monitoring background image to determine the R component, G component, and B component at different pixel points in the monitoring foreground image and the monitoring background image, and three-primary color component data is obtained. Then, based on the vehicle body area threshold and the three-primary color component data, a shadow detection value of the monitoring foreground image is calculated, and a corresponding shadow detection coordinate is determined. Then, shadow detection data is generated based on the shadow detection value and the corresponding shadow detection coordinate. Shadow area filling is performed on the shadow detection coordinate with a shadow detection value of 1 in the shadow detection data to determine a foreground shadow area. Then, based on the foreground shadow area, corresponding shadow removal processing is performed on the monitoring foreground image to obtain a target foreground image. Specifically, the calculation formula for calculating the shadow detection value of the monitoring foreground image and determining the corresponding shadow detection coordinate is:

[0091]

[0092] Among them, (x, y) is the shadow detection coordinate, S(x, y) is the shadow detection value at (x, y), V a1 、V a2 and V a3 is the body area threshold, V b1 、V b2 and V b3 is the preset boundary judgment threshold, R(x,y), G(x,y) and B(x,y) are the R component, G component and B component at (x,y) in the monitored foreground image, R b (x,y),G b (x,y) and B b (x, y) is the R component, G component, and B component at (x, y) in the monitoring background image.

[0093] It is understandable that if the body color of the target monitored vehicle is a dark color such as black or gray, the identity color can be distinguished from the shadow by limiting the body area threshold.

[0094] It can be understood that the shadow removal process is a process of replacing the image of the foreground shadow area in the monitored foreground image with the image of the foreground shadow area in the monitored background image.

[0095] Specifically, Figure 3 A flowchart of shadow removal processing in the method provided by an embodiment of the present invention is shown.

[0096] In a preferred embodiment of the present invention, the shadow detection on the monitoring foreground image, determining the foreground shadow area, and shadow removal processing on the monitoring foreground image to obtain the target foreground image specifically include the following steps:

[0097] Step S1021: perform vehicle target color recognition on the monitored foreground image to determine a vehicle body area threshold.

[0098] Step S1022: Perform three-primary-color analysis on the monitoring foreground image and the monitoring background image to obtain three-primary-color component data.

[0099] Step S1023 : Calculate the shadow detection value of the monitored foreground image based on the vehicle body area threshold and the three primary color component data, and determine the corresponding shadow detection coordinates.

[0100] Step S1024: Generate shadow detection data according to the shadow detection value and the corresponding shadow detection coordinates.

[0101] Step S1025: Analyze the shadow detection data to determine the foreground shadow area.

[0102] Step S1026: Perform shadow removal processing on the monitored foreground image according to the foreground shadow area to obtain a target foreground image.

[0103] Furthermore, the deep learning-based black smoke vehicle detection method further includes the following steps:

[0104] Step S103 : grayscale the target foreground image to obtain a grayscale foreground image, and based on deep learning technology, identify the black smoke pixel area in the grayscale foreground image and calculate the black smoke pixel ratio.

[0105] In an embodiment of the present invention, a grayscale foreground image is obtained by grayscale processing the target foreground image. Based on deep learning technology, a black smoke grayscale value range is obtained. According to the black smoke grayscale value range, a black smoke pixel area in the grayscale foreground image is identified. The number of black smoke pixels in the black smoke pixel area is then counted, and the total number of pixels in the monitored image is also counted. Then, based on the number of black smoke pixels and the total number of pixels, a black smoke pixel ratio is calculated. Specifically, the black smoke pixel ratio is calculated as follows:

[0106]

[0107] Among them, F a is the black smoke pixel ratio, N a is the number of black smoke pixels, N g is the total number of pixels.

[0108] Specifically, Figure 4A flow chart of calculating the black smoke pixel ratio in the method provided by an embodiment of the present invention is shown.

[0109] In a preferred embodiment of the present invention, grayscale processing is performed on the target foreground image to obtain a grayscale foreground image, and based on deep learning technology, black smoke pixel areas in the grayscale foreground image are identified, and the black smoke pixel ratio is calculated, which specifically includes the following steps:

[0110] Step S1031 : grayscale the target foreground image to obtain a grayscale foreground image.

[0111] Step S1032: Based on deep learning technology, identify the black smoke pixel area in the grayscale foreground image.

[0112] Step S1033: Count the number of black smoke pixels in the black smoke pixel area.

[0113] Step S1034: Count the total number of pixels in the surveillance image.

[0114] Step S1035 : Calculate the black smoke pixel ratio according to the number of black smoke pixels and the total number of pixels.

[0115] Furthermore, the deep learning-based black smoke vehicle detection method further includes the following steps:

[0116] Step S104 : calculating the region center coordinates of the black smoke pixel region, matching the black smoke expansion ratio, performing expansion optimization on the black smoke pixel ratio, and calculating the optimized pixel ratio.

[0117] In an embodiment of the present invention, the region center coordinates of the black smoke pixel region are calculated, and the corresponding black smoke expansion ratio is matched from a preset expansion ratio database according to the region center coordinates. Then, the black smoke pixel ratio is expanded and optimized according to the black smoke expansion ratio, and the optimized pixel ratio is calculated. Specifically, the region center coordinates are calculated using the following formula:

[0118]

[0119] Among them, (x o ,y o ) is the coordinate of the center of the region, (x i ,y i ) is the i-th coordinate in the black smoke pixel area;

[0120] The formula for calculating the optimized pixel ratio is:

[0121]

[0122] Among them, F b To optimize the pixel ratio, k is the smoke expansion ratio.

[0123] Step S105: Compare the optimized pixel ratio with a preset black smoke standard ratio. When the optimized pixel ratio is greater than the black smoke standard ratio, identify and upload black smoke vehicle information.

[0124] In an embodiment of the present invention, the optimized pixel ratio is compared with a preset black smoke standard ratio to determine whether the optimized pixel ratio is greater than the black smoke standard ratio. When the optimized pixel ratio is greater than the black smoke standard ratio, a black smoke vehicle target is determined from the grayscale foreground image, and then a black smoke vehicle image of the black smoke vehicle target is captured from the surveillance image. By performing license plate and vehicle model recognition on the black smoke vehicle image, black smoke vehicle information including license plate number, vehicle model, etc. is obtained, and the black smoke vehicle information is uploaded according to a preset information upload address.

[0125] Specifically, Figure 5 A flowchart of the method for identifying and uploading black smoke vehicle information in an embodiment of the present invention is shown.

[0126] In a preferred embodiment of the present invention, comparing the optimized pixel ratio with a preset black smoke standard ratio, and identifying and uploading black smoke vehicle information when the optimized pixel ratio is greater than the black smoke standard ratio, specifically includes the following steps:

[0127] Step S1051 : Compare the optimized pixel ratio with a preset black smoke standard ratio to determine whether the optimized pixel ratio is greater than the black smoke standard ratio.

[0128] Step S1052: When the optimized pixel ratio is greater than the black smoke standard ratio, determine a black smoke vehicle target from the grayscale foreground image.

[0129] Step S1053: capturing a black smoke vehicle image of the black smoke vehicle target from the monitoring image.

[0130] Step S1054: Identify the black smoke vehicle image to obtain black smoke vehicle information.

[0131] Step S1055: Upload the black smoke vehicle information according to the preset information upload address.

[0132] Further, Figure 6 The application architecture diagram of the deep learning-based black smoke vehicle detection system provided by an embodiment of the present invention is shown.

[0133] In another preferred embodiment of the present invention, a smoky vehicle detection system based on deep learning includes:

[0134] The surveillance shooting processing module 101 is used to periodically acquire surveillance shooting images, perform motion detection on the surveillance shooting images based on a preset surveillance background image, and extract a surveillance foreground image from the surveillance shooting images.

[0135] In an embodiment of the present invention, the monitoring shooting processing module 101 periodically generates a monitoring shooting instruction according to a preset monitoring shooting cycle, and then performs monitoring shooting according to the monitoring shooting instruction, obtains a monitoring shooting image, imports a preset monitoring background image, and compares and analyzes the background and target of the monitoring shooting image and the monitoring background image through the background difference method to obtain a differential detection result, and then determines the monitoring foreground area in the monitoring shooting image based on the differential detection result, and then performs image capture of the monitoring foreground area in the monitoring shooting image, and extracts the monitoring foreground image from the monitoring shooting image.

[0136] The shadow removal processing module 102 is configured to perform shadow detection on the monitored foreground image, determine a foreground shadow area, and perform shadow removal processing on the monitored foreground image to obtain a target foreground image.

[0137] In an embodiment of the present invention, the shadow removal processing module 102 performs vehicle target recognition on the monitoring foreground image to determine the target monitoring vehicle, then performs color recognition on the target monitoring vehicle to determine the vehicle body area threshold, and performs three-primary color analysis on the monitoring foreground image and the monitoring background image to determine the R component, G component, and B component at different pixel points in the monitoring foreground image and the monitoring background image, and obtains three-primary color component data. Then, based on the vehicle body area threshold and the three-primary color component data, the shadow detection value of the monitoring foreground image is calculated, and the corresponding shadow detection coordinates are determined. Then, shadow detection data is generated based on the shadow detection value and the corresponding shadow detection coordinates. The shadow area is filled by the shadow detection coordinates with a shadow detection value of 1 in the shadow detection data to determine the foreground shadow area. Then, based on the foreground shadow area, corresponding shadow removal processing is performed on the monitoring foreground image to obtain the target foreground image. Specifically, the calculation formula for calculating the shadow detection value of the monitoring foreground image and determining the corresponding shadow detection coordinates is:

[0138]

[0139] Among them, (x, y) is the shadow detection coordinate, S(x, y) is the shadow detection value at (x, y), V a1 、V a2 and V a3 is the body area threshold, V b1 、V b2 and V b3is the preset boundary judgment threshold, R(x,y), G(x,y) and B(x,y) are the R component, G component and B component at (x,y) in the monitored foreground image, R b (x,y),G b (x,y) and B b (x, y) is the R component, G component, and B component at (x, y) in the monitoring background image.

[0140] Specifically, Figure 7 FIG. 1 shows a structural block diagram of the shadow removal processing module 102 in the system provided by an embodiment of the present invention.

[0141] In a preferred embodiment of the present invention, the shadow removal processing module 102 specifically includes:

[0142] The color recognition unit 1021 is used to perform vehicle target color recognition on the monitoring foreground image and determine a vehicle body area threshold.

[0143] The three-primary-color analysis unit 1022 is configured to perform three-primary-color analysis on the monitoring foreground image and the monitoring background image to obtain three-primary-color component data.

[0144] The shadow detection unit 1023 is configured to calculate a shadow detection value of the monitored foreground image based on the vehicle body region threshold and the three primary color component data, and determine a corresponding shadow detection coordinate.

[0145] The data generating unit 1024 is configured to generate shadow detection data according to the shadow detection value and the corresponding shadow detection coordinate.

[0146] The region determination unit 1025 is configured to analyze the shadow detection data to determine a foreground shadow region.

[0147] The shadow removal unit 1026 is configured to perform shadow removal processing on the monitoring foreground image according to the foreground shadow area to obtain a target foreground image.

[0148] Furthermore, the deep learning-based black smoke vehicle detection system also includes:

[0149] The pixel ratio calculation module 103 is configured to grayscale the target foreground image to obtain a grayscale foreground image, identify black smoke pixel areas in the grayscale foreground image based on deep learning technology, and calculate the black smoke pixel ratio.

[0150] In the embodiment of the present invention, the pixel ratio calculation module 103 grayscales the target foreground image to obtain a grayscale foreground image. Based on deep learning technology, it obtains a black smoke grayscale value range. According to the black smoke grayscale value range, it identifies the black smoke pixel area in the grayscale foreground image. It then counts the number of black smoke pixels in the black smoke pixel area and the total number of pixels in the monitored image. Then, it calculates the black smoke pixel ratio based on the number of black smoke pixels and the total number of pixels. Specifically, the calculation formula for the black smoke pixel ratio is:

[0151]

[0152] Among them, F a is the black smoke pixel ratio, N a is the number of black smoke pixels, N g is the total number of pixels.

[0153] The ratio expansion optimization module 104 is used to calculate the coordinates of the center of the black smoke pixel area, match the black smoke expansion ratio, perform expansion optimization on the black smoke pixel ratio, and calculate the optimized pixel ratio.

[0154] In the embodiment of the present invention, the ratio expansion optimization module 104 calculates the region center coordinates of the black smoke pixel region, matches the corresponding black smoke expansion ratio from a preset expansion ratio database according to the region center coordinates, and then performs expansion optimization on the black smoke pixel ratio according to the black smoke expansion ratio to calculate the optimized pixel ratio. Specifically, the calculation formula for the region center coordinates is:

[0155]

[0156] Among them, (x o ,y o ) is the coordinate of the center of the region, (x i ,y i ) is the i-th coordinate in the black smoke pixel area;

[0157] The formula for calculating the optimized pixel ratio is:

[0158]

[0159] Among them, F b To optimize the pixel ratio, k is the smoke expansion ratio.

[0160] The black smoke vehicle identification module 105 is configured to compare the optimized pixel ratio with a preset black smoke standard ratio, and identify and upload black smoke vehicle information when the optimized pixel ratio is greater than the black smoke standard ratio.

[0161] In an embodiment of the present invention, the black smoke vehicle identification module 105 compares the optimized pixel ratio with a preset black smoke standard ratio to determine whether the optimized pixel ratio is greater than the black smoke standard ratio. When the optimized pixel ratio is greater than the black smoke standard ratio, the black smoke vehicle target is determined from the grayscale foreground image, and then a black smoke vehicle image of the black smoke vehicle target is captured from the surveillance image. By performing license plate and vehicle model recognition on the black smoke vehicle image, black smoke vehicle information including license plate number, vehicle model, etc. is obtained, and the black smoke vehicle information is uploaded and processed according to a preset information upload address.

[0162] It should be understood that, although the various steps in the flow chart of each embodiment of the present invention are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in order, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0163] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0164] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A smoky vehicle detection method based on deep learning, characterized in that: The method specifically comprises the following steps: Periodically acquiring surveillance images, performing motion detection on the surveillance images based on a preset surveillance background image, and extracting a surveillance foreground image from the surveillance images; Performing shadow detection on the monitoring foreground image to determine the foreground shadow area, and performing shadow removal processing on the monitoring foreground image to obtain a target foreground image; grayscale the target foreground image to obtain a grayscale foreground image, and identify black smoke pixel areas in the grayscale foreground image based on deep learning technology, and calculate the black smoke pixel ratio; The target foreground image is grayscaled to obtain a grayscale foreground image. Based on deep learning technology, the black smoke grayscale value range is obtained. According to the black smoke grayscale value range, the black smoke pixel area in the grayscale foreground image is identified. The number of black smoke pixels in the black smoke pixel area is then counted, and the total number of pixels in the monitoring image is counted. The black smoke pixel ratio is then calculated based on the number of black smoke pixels and the total number of pixels. Specifically, the black smoke pixel ratio is calculated as follows: ； in, is the black smoke pixel ratio, is the number of black smoke pixels, is the total number of pixels; Calculating the center coordinates of the black smoke pixel area, matching the black smoke expansion ratio, performing expansion optimization on the black smoke pixel ratio, and calculating the optimized pixel ratio; By calculating the regional center coordinates of the black smoke pixel area, matching the corresponding black smoke expansion ratio from the preset expansion ratio database according to the regional center coordinates, and then performing expansion optimization on the black smoke pixel ratio according to the black smoke expansion ratio, the optimized pixel ratio is calculated. Specifically, the calculation formula of the regional center coordinates is: ； ； in, are the coordinates of the region center, The first coordinates; The formula for calculating the optimized pixel ratio is: ； in, To optimize the pixel ratio, is the black smoke expansion ratio; The optimized pixel ratio is compared with a preset black smoke standard ratio, and when the optimized pixel ratio is greater than the black smoke standard ratio, the black smoke vehicle information is identified and uploaded.

2. The method for detecting smoky vehicles based on deep learning according to claim 1, characterized in that: The periodic acquisition of surveillance images, the performance of motion detection on the surveillance images based on a preset surveillance background image, and the extraction of a surveillance foreground image from the surveillance images specifically include the following steps: Generate surveillance shooting instructions periodically according to the preset surveillance shooting cycle; Perform surveillance shooting according to the surveillance shooting instruction and obtain surveillance shooting images; By using a background difference method, the monitoring image is compared with a preset monitoring background image for motion detection to obtain a difference detection result; determining a monitoring foreground area according to the differential detection result; According to the monitoring foreground area, a monitoring foreground image is extracted from the monitoring shot image.

3. The method for detecting smoky vehicles based on deep learning according to claim 1, characterized in that: The shadow detection is performed on the monitoring foreground image to determine the foreground shadow area, and the shadow removal is performed on the monitoring foreground image to obtain the target foreground image, which specifically includes the following steps: Performing vehicle target color recognition on the monitoring foreground image to determine a vehicle body area threshold; Performing three-primary color analysis on the monitoring foreground image and the monitoring background image to obtain three-primary color component data; Calculating a shadow detection value of the monitored foreground image based on the vehicle body region threshold and the three primary color component data, and determining a corresponding shadow detection coordinate; generating shadow detection data according to the shadow detection value and the corresponding shadow detection coordinates; Analyzing the shadow detection data to determine a foreground shadow area; According to the foreground shadow area, shadow removal processing is performed on the monitoring foreground image to obtain a target foreground image.

4. The method for detecting smoky vehicles based on deep learning according to claim 3, characterized in that: The calculation formula for calculating the shadow detection value of the monitored foreground image based on the vehicle body area threshold and the three primary color component data and determining the corresponding shadow detection coordinates is: ； in, is the shadow detection coordinate, for The shadow detection value at 、 and is the body area threshold, 、 and is the preset boundary judgment threshold, 、 and To monitor the foreground image at Quantity, Quantity and Quantity, 、 and To monitor the background image at Quantity, Quantity and Quantity.

5. The method for detecting smoky vehicles based on deep learning according to claim 1, characterized in that: The step of comparing the optimized pixel ratio with a preset black smoke standard ratio and identifying and uploading black smoke vehicle information when the optimized pixel ratio is greater than the black smoke standard ratio specifically includes the following steps: Comparing the optimized pixel ratio with a preset black smoke standard ratio to determine whether the optimized pixel ratio is greater than the black smoke standard ratio; When the optimized pixel ratio is greater than the black smoke standard ratio, determining a black smoke vehicle target from the grayscale foreground image; intercepting a black smoke vehicle image of the black smoke vehicle target from the monitoring image; Identifying the black smoke vehicle image to obtain black smoke vehicle information; The black smoke vehicle information is uploaded according to the preset information upload address.

6. A smoky vehicle detection system based on deep learning, characterized in that: The system includes a monitoring shooting processing module, a shadow removal processing module, a pixel ratio calculation module, a ratio expansion optimization module and a black smoke vehicle identification module, wherein: A surveillance camera processing module is used to periodically acquire surveillance camera images, perform motion detection on the surveillance camera images based on a preset surveillance background image, and extract a surveillance foreground image from the surveillance camera images; a shadow removal processing module, configured to perform shadow detection on the monitoring foreground image, determine a foreground shadow area, and perform shadow removal processing on the monitoring foreground image to obtain a target foreground image; a pixel ratio calculation module, configured to grayscale the target foreground image to obtain a grayscale foreground image, and identify black smoke pixel areas in the grayscale foreground image based on deep learning technology to calculate the black smoke pixel ratio; The pixel ratio calculation module grayscales the target foreground image to obtain a grayscale foreground image. Based on deep learning technology, it obtains the grayscale value range of black smoke. According to the black smoke grayscale value range, it identifies the black smoke pixel area in the grayscale foreground image. It then counts the number of black smoke pixels in the black smoke pixel area and the total number of pixels in the monitored image. Then, it calculates the black smoke pixel ratio based on the number of black smoke pixels and the total number of pixels. Specifically, the calculation formula for the black smoke pixel ratio is: ； in, is the black smoke pixel ratio, is the number of black smoke pixels, is the total number of pixels; a ratio expansion optimization module, configured to calculate the coordinates of the center of the black smoke pixel area, match the black smoke expansion ratio, perform expansion optimization on the black smoke pixel ratio, and calculate the optimized pixel ratio; The ratio expansion optimization module calculates the regional center coordinates of the black smoke pixel area, matches the corresponding black smoke expansion ratio from the preset expansion ratio database according to the regional center coordinates, and then optimizes the black smoke pixel ratio according to the black smoke expansion ratio to calculate the optimized pixel ratio. Specifically, the calculation formula of the regional center coordinates is: ； ； in, are the coordinates of the region center, The first coordinates; The formula for calculating the optimized pixel ratio is: ； in, To optimize the pixel ratio, is the black smoke expansion ratio; The black smoke vehicle identification module is used to compare the optimized pixel ratio with a preset black smoke standard ratio, and when the optimized pixel ratio is greater than the black smoke standard ratio, identify and upload the black smoke vehicle information.

7. The deep learning-based black smoke vehicle detection system according to claim 6 is characterized in that: The shadow removal processing module specifically includes: A color recognition unit is used to perform vehicle target color recognition on the monitoring foreground image and determine a vehicle body area threshold; A three-primary color analysis unit is used to perform three-primary color analysis on the monitoring foreground image and the monitoring background image to obtain three-primary color component data; a shadow detection unit, configured to calculate a shadow detection value of the monitored foreground image based on the vehicle body region threshold and the three primary color component data, and determine a corresponding shadow detection coordinate; a data generating unit, configured to generate shadow detection data according to the shadow detection value and the corresponding shadow detection coordinate; an area determination unit, configured to analyze the shadow detection data and determine a foreground shadow area; The shadow removal unit is used to perform shadow removal processing on the monitoring foreground image according to the foreground shadow area to obtain a target foreground image.

Images