A neural network-based black smoke detection method and system on a ring detection line

Abstract

The application discloses a kind of black smoke detection method and system based on neural network on ring inspection line, the present application includes collecting vehicle black smoke image with whiteboard as background in vehicle smoke pipe position addition;Whiteboard area black smoke image is obtained by cutting from vehicle black smoke image;Black smoke density feature map is obtained by inputting whiteboard area black smoke image into trained neural network model, and the Lignac blackness of black smoke is calculated according to black smoke density feature map, the neural network model includes multiple encoding layers, first convolutional layer and multiple decoding layers connected in turn in cascade, encoding layer and decoding layer one-to-one correspondence, and the feature map output by each encoder is input as the input of corresponding decoding layer after being strengthened by a feature enhancement module.The present application uses black smoke blackness feature neural network model to identify black smoke blackness, and then converts its Lignac blackness value, which can improve the discrimination accuracy of black smoke detection on ring inspection line and save manpower.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of black smoke detection on a ring detection line, and particularly relates to a black smoke detection method and system based on a neural network on a ring detection line. BACKGROUND

[0002] Currently, black smoke detection on a ring detection line adopts manual comparison of a black smoke area of a vehicle exhaust hole and a Lignacott blackness card, which is time-consuming and laborious. With the popularization of artificial intelligence (AI) technology, people begin to apply artificial intelligence (AI) technology to black smoke detection on a ring detection line, that is, an image of a black smoke area of a vehicle exhaust hole is input into a neural network model, and a corresponding Lignacott blackness is obtained by using the neural network model. However, on the one hand, the neural network model for extracting black smoke blackness features at the present stage mostly adopts an encoding-decoding architecture, and this kind of neural network model has a common shortcoming: lack of depth information, which makes it difficult to improve the accuracy of density feature recognition. On the other hand, there are often unfavorable factors such as background clutter, shadows, and reflections in the ring detection station, which affect the accuracy of visual recognition. Therefore, how to improve the accuracy of black smoke detection has become a key technical problem to be solved in the technical field of black smoke detection on a ring detection line. SUMMARY

[0003] The technical problem solved by the present application: in view of the above problems of the prior art, a black smoke detection method and system based on a neural network on a ring detection line are provided, which uses a black smoke blackness feature neural network model to recognize black smoke blackness, and then calculates the Lignacott blackness value, so as to improve the discrimination accuracy of black smoke detection on a ring detection line and save manpower.

[0004] To solve the above technical problems, the technical scheme adopted by the present application is:

[0005] A black smoke detection method based on a neural network on a ring detection line, comprising:

[0006] S101, collecting a vehicle black smoke image with a whiteboard added as a background at a position of a vehicle smoke pipe;

[0007] S102, cutting and obtaining a whiteboard area black smoke image from the vehicle black smoke image;

[0008] S103, input the whiteboard area black smoke image into the trained neural network model to obtain a black smoke density feature map, the neural network model comprising a plurality of encoding layers, a first convolutional layer and a plurality of decoding layers connected in sequence, an input end of the first encoding layer serving as an input end of the neural network model, a last encoding layer being connected with a first decoding layer through the first convolutional layer, and an output end of a last decoding layer serving as an output end of the neural network model, the encoding layers and the decoding layers corresponding one by one, and a feature map output by each encoding layer being input into a corresponding decoding layer after being enhanced by a feature enhancement module;

[0009] S104, calculating the Lunge blackness of the black smoke according to the black smoke density feature map.

[0010] Optionally, before the step S101, the method further comprises adding a whiteboard at a position of a vehicle smoke pipe, and adjusting a camera used to collect a vehicle black smoke image with the whiteboard added at the position of the vehicle smoke pipe as a background so that the whiteboard has no shadow and reflection in a picture taken by the camera.

[0011] Optionally, the step S102 of obtaining the whiteboard area black smoke image by cropping from the vehicle black smoke image refers to inputting the vehicle black smoke image into a trained target detection network model, so as to obtain the whiteboard area black smoke image.

[0012] Optionally, the trained target detection network model is a YOLOv4 target detection network.

[0013] Optionally, the feature enhancement module comprises a second convolutional layer, a max-pooling layer, a deconvolutional layer and a third convolutional layer connected in sequence, the input whiteboard area black smoke image or the encoding layer feature being input into the encoding layer, first performing a convolution operation of k=3x3 and p=(1,1), a batch normalization operation BN and an activation operation of a ReLU activation parameter through the second convolutional layer, then performing a max-pooling encoding of k=2x2 and s=(2,2), a batch normalization operation BN and an activation operation of a ReLU activation parameter through the max-pooling layer, then performing a deconvolution operation of k=2x2 and s=(2,2) through the deconvolutional layer, and finally performing a convolution operation of k=3x3 and p=(1,1) through the third convolutional layer to obtain a corresponding encoding layer feature, wherein k is a convolution kernel size, s is a step length, and p is a boundary expansion.

[0014] Optionally, each decoding layer of the neural network model comprises a decoder and a concatenation layer concat and a fourth convolutional layer in front of the decoder, the concatenation layer concat is configured to perform a connection operation on the decoder features from the first convolutional layer or the decoder of the previous decoding layer, and the corresponding feature enhancement module performs feature enhancement on the encoder features, and the fourth convolutional layer is configured to perform a convolution operation of k=3x3 and p=(1,1), a batch normalization operation BN and an activation operation of a ReLU activation parameter on the feature map obtained by the connection operation, where k is a convolution kernel size and p is a boundary expansion.

[0015] Optionally, the first convolutional layer of the neural network model is configured to perform a convolution operation of k=1 and s=1, where k is a convolution kernel size and s is a step size.

[0016] Optionally, the loss function used by the neural network model during training has a function expression as follows:

[0017] D f =β a P a +β b P b +β c P c ,

[0018] In the above formula, D f is a loss function, β a , β b and β c are weight parameters, P a is a mean square error between the black smoke density feature map and the black smoke darkness feature real map, P b is a mean square error between the black smoke single extraction image and the black smoke single extraction real image, P c is a mean square error between the background image and the background real image, the black smoke single extraction image and the background image are obtained by foreground extraction from the whiteboard area black smoke image, and the following conditions are met:

[0019]

[0020]

[0021]

[0022] In the above formula, N is the number of training samples, w is the width of the image, h is the height of the image, is a pixel value of a coordinate (i,j) in the black smoke density feature map obtained by the mth training sample, is a pixel value of a coordinate (i,j) in the black smoke darkness feature real map corresponding to the mth training sample, a pixel value of a coordinate (i, j) in a background image obtained for the mth training sample, a pixel value of a coordinate (i, j) in a real image of the black smoke extracted alone for the mth training sample, a pixel value of a coordinate (i, j) in a black smoke density feature map obtained for the mth training sample, a pixel value of a coordinate (i, j) in a background real image corresponding to the mth training sample.

[0023] Optionally, the calculation of the Lunge blackness of the black smoke according to the black smoke density feature map in step S104 comprises:

[0024] S201, determining a pixel set S in which pixel values in a black smoke density feature map exceed a preset threshold T;

[0025] S202, calculating a black smoke level score x of the black smoke density feature map according to the pixel set S according to the following formula:

[0026]

[0027] In the above formula, N s is the number of pixels in S, S is a pixel set in which pixel values in a black smoke density feature map exceed a preset threshold T, and p(i, j) is a black smoke density value at a coordinate (i, j).

[0028] S203, determining the Lunge blackness of the black smoke according to a mapping relationship between the black smoke level score x and the Lunge blackness.

[0029] In addition, the present application also provides a neural network-based black smoke detection system on a ring detection line, comprising a microprocessor and a memory connected to each other, and the microprocessor is configured to execute the neural network-based black smoke detection method on the ring detection line.

[0030] In addition, the present application also provides a computer readable storage medium, and the computer readable storage medium stores a computer program for being configured by a microprocessor to execute the neural network-based black smoke detection method on the ring detection line.

[0031] Compared with the prior art, the present application mainly has the following advantages:

[0032] 1. In the ring detection station scene, a whiteboard is added at the position of the smoke pipe of the vehicle to collect the image of the black smoke of the vehicle, the influence of the complex background is isolated, and the recognition accuracy is improved in the most direct way.

[0033] 2. The present application reconstructs the architecture of coding and decoding, adds a feature enhancement module, superimposes and fuses the coding features of each layer into the corresponding decoding layer through the feature enhancement module, so that the decoding layer fuses features of different levels, and thus the accuracy of black smoke blackness extraction is improved. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0035] Figure 2 This is a schematic diagram of the topology of the neural network model used in this embodiment of the invention.

[0036] Figure 3 This is a schematic diagram of the feature enhancement module in an embodiment of the present invention.

[0037] Figure 4 This is a schematic diagram of the feature fusion module in the decoding layer of this invention. Detailed Implementation

[0038] like Figure 1 As shown, the neural network-based black smoke detection method on the inspection line in this embodiment includes:

[0039] S101, Collect images of black smoke from vehicles with a whiteboard added to the vehicle's exhaust pipe as a background;

[0040] S102, Obtain the black smoke image of the whiteboard area by cropping from the black smoke image of the vehicle;

[0041] S103, Input the black smoke image of the whiteboard area into the trained neural network model to obtain the black smoke density feature map;

[0042] S104, calculate the Ringelmann opacity of the black smoke based on the black smoke density characteristic map.

[0043] In this embodiment, before step S101, a whiteboard is added at the vehicle's exhaust pipe location, and the camera used to capture images of vehicle black smoke against the background of the whiteboard is adjusted so that the whiteboard in the camera's captured image has no shadows or reflections. By adding a whiteboard at the vehicle's exhaust pipe location in the vehicle inspection station scenario to capture vehicle black smoke images, the influence of complex backgrounds is isolated, thereby improving recognition accuracy in the most direct way. In this embodiment, the camera used to capture images of vehicle black smoke against the background of the whiteboard can cruise around a preset point, thereby achieving position adjustment and ensuring that the whiteboard in the camera's captured image has no shadows or reflections.

[0044] In this embodiment, step S102, which involves cropping the whiteboard region black smoke image from the vehicle black smoke image, refers to inputting the vehicle black smoke image into a trained target detection network model to obtain the whiteboard region black smoke image.

[0045] As an optional implementation, the trained target detection network model in the embodiment is a YOLOv4 target detection network, and other target detection network models can also be used according to needs. In the process of training the YOLOv4 target detection network, on-line whiteboard video pictures are collected on the ring detection line, and the size is uniformly scaled. The whiteboard is calibrated through a label marking method. The data is input into the YOLOv4 target detection network to train the YOLOv4 target detection network until the output data result reaches the accuracy requirement, and then the training of the YOLOv4 target detection network is completed. After the training is completed, the vehicle black smoke image is input into the trained YOLOv4 target detection network, and the whiteboard area black smoke image can be obtained. It should be noted that the embodiment only relates to the application of the target detection network model, and does not relate to the improvement of the structure or training method of the target detection network model, so the details of the structure or training method of the target detection network model are not described here.

[0046] As shown in Figure 2 The neural network model includes a plurality of encoding layers, a first convolutional layer, and a plurality of decoding layers connected in sequence, the input end of the first encoding layer is the input end of the neural network model, the last encoding layer is connected with the first decoding layer through the first convolutional layer, and the output end of the last decoding layer is the output end of the neural network model, the encoding layers and the decoding layers are one-to-one corresponding, and the feature map output by each encoder is input into a feature enhancement module for feature enhancement and then used as the input of the corresponding decoding layer. The encoding feature map contains more shallow features, has a small receptive field, and obvious local features, and covers sufficient position information. The enhanced deep features have a large receptive field and sufficient semantic information. The method of the embodiment enhances the features in the encoding module to obtain deep network features, and then fuses the deep network features with the shallow features to obtain features with different levels of information fusion, which is helpful for full use of information.

[0047] As shown in Figure 2As shown, the neural network model in this embodiment includes five cascaded encoding layers, a first convolutional layer, and five cascaded decoding layers. The size of the encoding layer features obtained by the first encoding layer is w×h×16, the size of the encoding layer features obtained by the second encoding layer is w / 2×h / 2×32, the size of the encoding layer features obtained by the third encoding layer is w / 4×h / 4×64, the size of the encoding layer features obtained by the fourth encoding layer is w / 8×h / 8×128, and the size of the encoding layer features obtained by the fifth encoding layer is w / 16×h / 16×256. The size of the decoding layer features obtained by the first decoding layer is w / 16×h / 16×256, the size of the decoding layer features obtained by the second decoding layer is w / 8×h / 8×128, the size of the decoding layer features obtained by the third decoding layer is w / 4×h / 4×64, the size of the decoding layer features obtained by the fourth decoding layer is w / 2×h / 2×32, and the size of the decoding layer features obtained by the fifth decoding layer is w×h×16.

[0048] like Figure 3 As shown, the feature enhancement module of the neural network model in this embodiment includes a second convolutional layer (conv.(k=3×3,p=(1,1))), a max pooling layer, a deconvolutional layer (dconv), and a third convolutional layer (conv.(k=3×3,p=(1,1))) connected in sequence. The input whiteboard area black smoke image or coding layer features, after being input into the coding layer, first undergo a convolution operation of k=3×3,p=(1,1) through the second convolutional layer. The image is encoded using Batch Normalization (BN) and ReLU activation parameters. Then, it passes through a max pooling layer with k = 2 × 2 and s = (2,2) max pooling encoding, followed by BN and ReLU activation. Next, it passes through a deconvolution layer with k = 2 × 2 and s = (2,2) deconvolution. Finally, a third convolutional layer performs a k = 3 × 3 convolution with p = (1,1) convolution to obtain the corresponding encoded layer features. Here, k is the kernel size, s is the stride, and p is the boundary expansion. Max pooling encodes the image, and deconvolution maintains the image size, thus outputting deep features. Figure 4As shown, each decoding layer of the neural network model includes a decoder and a connection layer concat and a fourth convolutional layer (conv. (k=3, p=1)) in front of the decoder, both of which constitute a feature fusion module for feature fusion of the first convolutional layer or the decoder features of the previous decoding layer, the corresponding feature enhancement module after feature enhancement of the encoder features. The connection layer concat is used to connect the encoder features after feature enhancement of the first convolutional layer or the decoder features of the previous decoding layer, and the fourth convolutional layer is used to perform a convolution operation of k=3, p=1, a batch normalization operation BN and an activation operation of a ReLU activation parameter on the feature map obtained by the connection operation. Wherein k is the size of the convolution kernel, and p is the boundary expansion. Wherein k=3 means k=3x3, and p=1 means p=(1,1). In the feature enhancement module design of the neural network model in this embodiment, in order to increase the network depth, a convolution and a maximum pooling are added after each layer of the encoder to realize the reduction of the image size of the encoding part. A synchronous long deconvolution and an up-sampling operation are added in the decoder part to realize the expansion of the image size of the decoding part. Such design can increase the network depth of the encoding and decoding sub-blocks, reduce information loss, and improve accuracy.

[0049] As shown in Figure 2 , the first convolutional layer (conv. (k=1, s=1)) of the neural network model is used to perform a convolution operation of k=1, s=1, wherein k is the size of the convolution kernel and s is the step length. Wherein k=1 means k=1x1, and s=1 means s=(1,1).

[0050] In addition, the encoder and the decoder of the neural network model can adopt various existing encoders and decoders as needed. The key of the method in this embodiment is to add a feature enhancement module on the basis of the existing encoder and decoder. Therefore, the structure and specific implementation of the existing encoder and decoder will not be described in detail here.

[0051] In this embodiment, the function expression of the loss function used by the neural network model during training is:

[0052] D f =β a P a +β b P b +β c P c ,

[0053] In the above formula, D f is the loss function, β a , β b and β c are weight parameters, and P aP is the mean square error between the black smoke density feature map and the black smoke blackness feature real map, b P is the mean square error between the black smoke single extraction image and the black smoke single extraction real map, c P is the mean square error between the background image and the background real map, the black smoke single extraction image and the background image are obtained by foreground extraction from the whiteboard area black smoke image (existing method), and have:

[0054]

[0055]

[0056]

[0057] In the above formula, N is the number of training samples, w is the image width, h is the image height, is the pixel value of the coordinate (i, j) in the black smoke density feature map obtained by the mth training sample, is the pixel value of the coordinate (i, j) in the black smoke blackness feature real map corresponding to the mth training sample, is the pixel value of the coordinate (i, j) in the background image obtained by the mth training sample, is the pixel value of the coordinate (i, j) in the black smoke single extraction real map corresponding to the mth training sample, is the pixel value of the coordinate (i, j) in the black smoke density feature map obtained by the mth training sample, is the pixel value of the coordinate (i, j) in the background real map corresponding to the mth training sample. In this embodiment, the loss function used by the neural network model in training is based on the test accuracy of the three-image output model of the black smoke density feature map, the black smoke single extraction image and the background image, which is higher than that of the single-output black smoke density feature map. The three-image output model can accurately learn more information and improve the generalization ability of the model.

[0058] In this embodiment, the calculation of the Lunge blackness of the black smoke according to the black smoke density feature map in step S104 includes:

[0059] S201, determining a pixel set S in which the pixel value in the black smoke density feature map exceeds a preset threshold T;

[0060] S202, calculating the black smoke level score x of the black smoke density feature map according to the pixel set S according to the following formula:

[0061]

[0062] In the above formula, N s is the number of pixels in S, S is the pixel set in which the pixel value in the black smoke density feature map exceeds the preset threshold T, and p(i, j) is the black smoke density value at the pixel coordinate (i, j).

[0063] In S203, the Lignin blackness of the black smoke is determined according to the mapping relationship between the black smoke level score x and the Lignin blackness, for example, the function expression of the mapping relationship between the black smoke level score x and the Lignin blackness in the embodiment is as follows:

[0064]

[0065] In the above formula, G(ρ) represents the Lignin blackness of the black smoke.

[0066] In addition, the embodiment also provides a neural network-based black smoke detection system on a loop inspection line, which comprises a microprocessor and a memory connected with each other, and the microprocessor is configured to execute the neural network-based black smoke detection method on the loop inspection line. The embodiment also provides a computer readable storage medium, which stores a computer program for being configured by the microprocessor to execute the neural network-based black smoke detection method on the loop inspection line.

[0067] The above is only the preferred embodiment of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiment. Any technical solution falling within the concept of the present application shall fall within the protection scope of the present application. It should be noted that, for ordinary skilled in the art, some improvements and refinements without departing from the principles of the present application shall also be considered as falling within the protection scope of the present application.

Claims

1. A neural network-based black smoke detection method on a ring detection line, characterized in that, The method comprises the following steps: S101, collecting a vehicle black smoke image with a whiteboard added at the position of a vehicle smoke pipe as a background; S102, obtaining a whiteboard area black smoke image by cropping from the vehicle black smoke image; S103, inputting the whiteboard area black smoke image into a trained neural network model to obtain a black smoke density feature map, the neural network model comprising a plurality of encoding layers, a first convolutional layer and a plurality of decoding layers connected in sequence, the input end of the first encoding layer being the input end of the neural network model, the last encoding layer being connected to the first decoding layer through the first convolutional layer, and the output end of the last decoding layer being the output end of the neural network model, the encoding layers and the decoding layers being in one-to-one correspondence, and the feature map output by each encoder being input into a corresponding decoding layer after being enhanced by a feature enhancement module; S104, calculating the Lignac blackness of the black smoke according to the black smoke density feature map; The feature enhancement module comprises a second convolutional layer, a max pooling layer, a deconvolutional layer and a third convolutional layer connected in sequence, and the input whiteboard area black smoke image or the encoding layer feature is input into the encoding layer first, then performs a convolution operation of k=3*3 and p=(1,1), a batch normalization operation BN and an activation operation of a ReLU activation parameter through the second convolutional layer, then performs a max pooling encoding of k=2*2 and s=(2,2), a batch normalization operation BN and an activation operation of a ReLU activation parameter through the max pooling layer, then performs a deconvolution operation of k=2*2 and s=(2,2) through the deconvolutional layer, and finally performs a convolution operation of k=3*3 and p=(1,1) through the third convolutional layer to obtain a corresponding encoding layer feature, wherein k is a convolution kernel size, s is a step length, and p is a boundary expansion.

2. The neural network based soot detection method on a ring track according to claim 1, characterized in that, Before step S101, a whiteboard is added at the position of the vehicle smoke pipe, and a camera for collecting the vehicle black smoke image with the whiteboard added at the position of the vehicle smoke pipe as a background is adjusted so that the whiteboard has no shadow and reflection in the camera shot picture.

3. The neural network based soot detection method on a ring track according to claim 1, characterized in that, In step S102, the whiteboard area black smoke image is obtained by inputting the vehicle black smoke image into a trained target detection network model.

4. The neural network based soot detection method on a ring track according to claim 3, characterized in that, Each decoding layer of the neural network model comprises a decoder, a connection layer concat in front of the decoder and a fourth convolutional layer, the connection layer concat is used for connecting the decoder features from the first convolutional layer or the decoder features of the previous decoding layer and the encoder features after being enhanced by the corresponding feature enhancement module, and the fourth convolutional layer is used for performing a convolution operation of k=3*3 and p=(1,1), a batch normalization operation BN and an activation operation of a ReLU activation parameter on the feature map obtained by the connection operation, wherein k is a convolution kernel size and p is a boundary expansion.

5. The neural network based soot detection method on a ring track according to claim 3, characterized in that, The first convolutional layer of the neural network model is used for performing a convolution operation of k=(1,1) and s=(1,1), wherein k is a convolution kernel size and s is a step length.

6. The neural network based soot detection method on a ring track according to claim 5, characterized in that, The loss function used in the training of the neural network model has a function expression as follows: D f = β a P a + β b P b + β c P c , In the above formula, D f is a loss function, β a , β b and β c are weight parameters, P a is the mean square error between the black smoke density feature map and the black smoke blackness feature real map, P b is the mean square error between the black smoke single extraction image and the black smoke single extraction real image, P c is the mean square error between the background image and the background real image, the black smoke single extraction image and the background image are obtained by foreground extraction from the whiteboard area black smoke image, and it has: In the above formula, N is the number of training samples, w is the image width, h is the image height, the pixel value of the (i, j) coordinate in the black smoke density feature map obtained for the mth training sample, the pixel value of the (i, j) coordinate in the black smoke density feature map obtained for the mth training sample, the pixel value of the (i, j) coordinate in the background image obtained for the mth training sample, the pixel value of the (i, j) coordinate in the black smoke density feature map obtained for the mth training sample, the pixel value of the (i, j) coordinate in the black smoke density feature map obtained for the mth training sample, the pixel value of the (i, j) coordinate in the black smoke density feature map obtained for the mth training sample, 7. The neural network based soot detection method on a ring of inspection according to claim 1, characterized in that, In step S104, the Lignac blackness of the black smoke is calculated according to the black smoke density feature map. S201, determining a pixel set S in which the pixel value of the black smoke density feature map exceeds a preset threshold T; S202, calculating the black smoke level score x of the black smoke density feature map according to the pixel set S according to the following formula; In the above formula, N s is the number of pixels in S, S is a pixel set in which the pixel value in the black smoke density feature map exceeds a preset threshold T, and p(i, j) is the black smoke density value at the pixel coordinates (i, j). S203, determining the Lunge blackness of the black smoke according to the mapping relationship between the black smoke level score x and the Lunge blackness.

8. A neural network based black smoke detection system on a circular inspection line comprising a microprocessor and a memory connected to each other, characterized in that, The microprocessor is configured to execute the neural network-based black smoke detection method on the ring detection line according to any one of claims 1-7.

9. A computer-readable storage medium having stored therein a computer program, characterized in that, The computer program is used for the microprocessor configured to execute the neural network-based black smoke detection method on the ring detection line according to any one of claims 1-7.

Images