Tunnel fire smoke identification method based on target detection

Abstract

The invention discloses a tunnel fire smoke identification method based on target detection, and the method specifically comprises the steps: 1, collecting a tunnel image, screening the image in a data set, carrying out the detection frame labeling of a kindling region and a smoke region in the image, and constructing a deep learning data set; 2, preprocessing the images in the data set, and performing image augmentation on the images with fire areas or smoke areas in the preprocessed data set; 3, constructing a kindling smoke recognition model, and training the kindling smoke recognition model by adopting the data set augmented in the step 2; and 4, acquiring a scene image in the tunnel in real time, preprocessing the image, transmitting the preprocessed image to the trained kindling smoke identification model, and judging whether a kindling area and a smoke area exist in the image or not.

Description

Technical Field

[0001] The invention belongs to the technical field of intelligent detection, and in particular relates to a tunnel fire smoke recognition method based on target detection. Background Art

[0002] With the rapid development of tunnel construction, tunnel fires are also increasing. The essence of the earliest real-time fire smoke detection is to use sensors to sense temperature, smoke, light intensity, etc., and convert them into alarm signals through specific thresholds. Due to the large span space in the tunnel, a large number of sensors need to be laid. At the same time, given that the accuracy and sensitivity of the sensors are limited, and they are easily affected by the tunnel's own environment and passing vehicles, the alarm is often not issued until the fire in the tunnel has occurred to a certain extent, which is difficult to meet the current needs of tunnel operation and safety assurance.

[0003] Later, with the rapid development of computer vision image processing, the research center gradually shifted to fire smoke detection based on video images. Compared with traditional fire smoke detection devices, it has the advantages of wide monitoring range and high accuracy. However, traditional computer vision mainly focuses on extracting flame and smoke areas, and identifying the shape, color and motion data of flames and smoke. Usually, the color and brightness of the flame are the main features, from RGB color model, HSV color model, and YCbCr, but the detection effect of this method is not accurate, and the warning is also started only after the fire has spread widely. Summary of the invention

[0004] Purpose of the invention: In order to solve the problems existing in the above-mentioned prior art, the present invention provides a tunnel fire smoke recognition method based on target detection.

[0005] Technical solution: The present invention discloses a tunnel fire smoke recognition method based on target detection, comprising the following steps:

[0006] Step 1: Collect tunnel images, filter the images in the dataset, mark the fire and smoke areas in the images with detection boxes, and build a deep learning dataset;

[0007] Step 2: Preprocess the images in the data set, and perform image augmentation on the images in the fire area or smoke area in the preprocessed data set;

[0008] Step 3: Build a fire smoke recognition model and use the augmented data set in step 2 to train the fire smoke recognition model;

[0009] Step 4: Collect the scene image in the tunnel in real time, pre-process the image, and transmit the pre-processed image to the trained fire and smoke recognition model to determine whether there is a fire area and a smoke area in the image.

[0010] Furthermore, the preprocessing in step 2 and step 4 includes image denoising, image filtering and image enhancement.

[0011] Furthermore, the image enhancement is specifically as follows: the image is divided into three-channel images of R, G, and B, each channel image is converted into a grayscale image, each grayscale image is subjected to guided filtering, and the image brightness layer image L is obtained. c (x, y), where c represents the channel, the original image I c (x,y) minus the brightness layer image L c (x, y), get the detail layer image D c (x, y), multiply the brightness layer image and the detail layer image by different stretching factors and perform weighted summation to obtain the enhanced image Im_E.

[0012] Furthermore, the image augmentation in step 2 includes image flipping, image rotation, image translation, image cutmix and image mixup.

[0013] Furthermore, the image cutmix is ​​specifically as follows: a rectangular image is cut out from image A, and randomly pasted into image B to obtain a new image, and the ratio λ of the area of ​​the cut image to the area of ​​image B is calculated, the label of the original image A is multiplied by (1-λ), and the label of image B is multiplied by λ, and then the sum is calculated to obtain the label of the new image.

[0014] Furthermore, the fire smoke recognition model adopts the YOLO model as the architecture, uses the SPP module in the last layer of the backbone network of the YOLO model, adds an attention mechanism module in the Neck, and the entire Neck module adopts the FPN+PAN structure.

[0015] Furthermore, the method also includes identifying the fire level according to the detected fire area and smoke area, specifically: calculating the fire score according to the following formula:

[0016]

[0017] Among them, S is the image pixel area, area_F is the area of ​​the fire area, area_S is the area of ​​the smoke area, ω2 and ω1 are weights;

[0018] Set the fire level, each level corresponds to a different score range.

[0019] Beneficial effects:

[0020] 1. The present invention uses a combination of vision and deep learning to monitor the fire smoke in the tunnel in real time, and intelligently ensures the safety of the tunnel.

[0021] 2. The present invention designs an image enhancement method based on guided filtering, which separates the brightness layer and the detail layer of the collected image in a dark tunnel environment, and stretches them separately to achieve the purpose of brightening the image while enhancing the image detail information.

[0022] 3. Design a YOLO-based deep learning detection network and add CBAM, SPP and other modules to effectively improve the model's ability to recognize small targets and significantly improve the accuracy of identifying tiny fires and smoke, so that the algorithm has a higher sensitivity, prevents accidents before they happen, and ensures tunnel safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Attached Figure 1 It is a flow chart of the present invention.

[0024] Attached Figure 2 This is the structural diagram of the fire smoke recognition model.

[0025] Attached Figure 3 This is the structural diagram of the SPP module.

[0026] Attached Figure 4 This is the structural diagram of the CBAM module.

[0027] Attached Figure 5 Schematic diagram of FPN+PAN structure.

[0028] Attached Figure 6 The following are examples of the application of the present invention in real-life scenarios, wherein (a) is a schematic diagram of detected open flame and smoke areas, and (b) is a schematic diagram of detected smoke areas. DETAILED DESCRIPTION

[0029] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0030] like Figure 1 As shown, the specific steps of this embodiment are:

[0031] Module A: Data preprocessing module

[0032] To achieve the above purpose, the present invention first performs image preprocessing on the captured image, specifically including:

[0033] Image denoising. Use an image filtering algorithm to filter the image to remove noise in the image imaging and shooting process. Specifically, the filtering algorithm is generally selected as mean filtering, median filtering and Gaussian filtering. The image filtering algorithm can be selected specifically according to the depth image situation of the specific embodiment.

[0034] Image enhancement: In this step, the guided filter function is used to enhance the brightness and details of the image to obtain the enhanced RGB image Im_E.

[0035] First, the image is divided into RGB three-channel images, each channel image is regarded as a grayscale image, and then the channel image I is calculated according to the following formula c (x,y) is carried out by I c (x, y) is the guided filtering of the guided graph, and the image brightness layer image L is obtained. c (x,y):

[0036] L c (x,y)=f(I c (x,y)

[0037] Among them, c represents R, G, B channels, and f() is the guided filter function.

[0038] Original image I c (x, y) minus the brightness layer image to get the detail layer image:

[0039] D c (x,y)=I c (x,y)-L c (x,y)

[0040] Multiply the brightness layer and detail layer by different stretching factors, that is, brighten the brightness layer and enhance the detail layer respectively, and add them together to get the enhanced image Im_E:

[0041] Im_E=α c L c (x,y)+β c D c (x,y)

[0042] Among them, α c is the brightness layer stretch factor, β c is the detail layer enhancement factor, and their value range is [1, 5].

[0043] Module B: Fire smoke identification module

[0044] This module aims to realize the intelligent identification of fire and smoke, and its processing object is the enhanced RGB image Im_E. The steps described in this module mainly include:

[0045] (B1) Construct a deep learning dataset, filter the data and annotate the detection boxes according to the fire area and smoke area. The annotated data is used as the total dataset, and the total dataset is divided into a training set, a validation set, and a test set according to the ratio of 8:1:1. Preprocess the images in the total dataset according to step A.

[0046] (B2) Image augmentation. It is not difficult to find in the data set distribution that the amount of fire and smoke data is relatively small, while the background area is large. Therefore, image augmentation is performed on images containing fire and smoke, specifically including:

[0047] Image flip, image rotation, image translation. Image flip includes horizontal mirror flip and vertical mirror flip; the image rotation angle range is [-90°, 90°]; image translation refers to randomly shifting the image in one direction by a certain number of pixels, and the remaining area is replaced by a uniform grayscale value.

[0048] Image cutmix means cutting out a small area from an image and pasting it to other images, and the label will also be mixed. Specifically, a rectangular image is cut out from image A and randomly pasted to image B to get a new image. The proportion λ of the cut image to image B is calculated, and the label of the original image A is multiplied by (1-λ) and the label of image B is multiplied by λ to get the label of the new image.

[0049] Image Mixup: fuse images according to a certain ratio lam, where lam is a random number between [0,1].

[0050] (B3) Construct a fire smoke recognition model.

[0051] The present invention can train the segmentation model using the training set based on the YOLO series model architecture, use the validation set for the trained detection model, and save the detection model with the best performance on the validation set as the detection model.

[0052] The fire smoke recognition model constructed by the present invention is as follows: Figure 2 As shown in the figure, considering the large field of view of the tunnel scene, while the detection targets are mostly small areas, the SPP module is used in the last layer of the YOLO backbone network, and the multi-scale receptive field is used to improve the detection accuracy of small targets; the CBAM (attention mechanism) module is added to the Neck to extract the attention area from the background of the large area, so that the model can pay more attention to useful target objects; the entire Neck module adopts the FPN+PAN structure and integrates multi-scale features to make the model more conducive to capturing small target information.

[0053] The SPP module is a spatial pyramid pooling network, which is usually used in the last layer of the convolution layer to divide feature maps of different scales (output after multiple convolution layers) into different scales and perform pooling in a specific scale. Its structure is shown in the attached figure. Figure 3 shown.

[0054] The multi-scale in the SPP module not only provides a multi-scale receptive field, which is beneficial for enhancing the detection of small objects, but also can generate feature vectors of the same size from feature maps of different scales. The number and size of the multi-scale are pre-set, such as [1,5,9].

[0055] In this embodiment, the SPP module may be replaced by the SPPF module, which is a faster version of the SPP module and can effectively reduce the amount of model calculation and improve the model operation speed.

[0056] The CBAM (Convolutional Block Attention Module) module represents the attention mechanism module of the convolution module, which is an attention mechanism module that combines spatial and channel, and can achieve better results than the attention mechanism that focuses on the channel.

[0057] In the YOLO model structure, a CBAM module is connected after the CSP module to add attention weighting to the features at the spatial and channel levels, which plays a positive role in improving the detection of small targets. The module structure diagram is shown in the attached figure. Figure 4 shown.

[0058] The CBAM module compresses the feature map in the spatial dimension through the channel attention module, obtains a one-dimensional vector and then operates it. When compressing in the spatial dimension, not only average pooling (AveraggePooling) but also maximum pooling (Max Pooling) is considered. Average pooling and maximum pooling can be used to aggregate the spatial information of the feature map, send it to a shared network, compress the spatial dimension of the input feature map, and sum and merge element by element to generate the channel attention feature. Similarly, the spatial attention mechanism compresses the channel. First, the feature map output by the channel attention module is used as the input feature map of this module, and then average pooling and maximum pooling are performed on the channel dimension respectively, and the number of extractions is height multiplied by width; then the spatial attention feature is generated through convolution operation and activation function, and finally the feature is multiplied by the input feature map of the module to obtain the final generated feature.

[0059] The FPN+PAN structure is shown in the attached Figure 5 As shown in the figure, the FPN structure transfers and fuses high-level feature information from top to bottom through upsampling to obtain a feature map for prediction; the PAN structure is used to form a bottom-up feature pyramid, convey strong positioning features upward, and aggregate parameters for different detections from different backbone layers.

[0060] (B4) Use the augmented data set to train the model constructed in step B3, and select the model parameters with the highest accuracy on the validation set as the final fire and smoke recognition model parameters, recorded as detected_model.

[0061] (B5) The tunnel image to be detected is passed to the detected_model of step (B4). If there is fire or smoke, the detection box detected_Box of the corresponding category is obtained:

[0062] detected_Box=[x1,x2,y1,y2,conf,cls]

[0063] Where x1, y1, x2, y2 are the coordinates of the upper left vertex and the lower right vertex of the small box, conf represents the probability value of the existence of the disease, and the category of the disease cls; the fire area and smoke area detected by this implementation in the real scene are as follows Figure 6 As shown, (a) is a schematic diagram of the detected open fire and smoke areas, and (b) is a schematic diagram of the detected smoke area.

[0064] Module C: Fire and smoke alarm module

[0065] This module further determines whether there is a fire risk based on the fire detection frame and smoke detection frame obtained by module B, and issues an early warning based on the level. The main steps include:

[0066] Calculate the weighted scores of fire and smoke in the image. The scores are calculated as follows:

[0067]

[0068] Among them, S is the image pixel area, which is W×H, that is, the width×height of the image;

[0069] area_F represents the pixel area of ​​all detection boxes of the Fire category obtained by module B;

[0070] area_S represents the pixel area of ​​all detection boxes of the Smoke category obtained by module B;

[0071] ω1 represents the weight factor of the fire type, which depends on the importance of the fire type in the specific embodiment;

[0072] ω2 represents the weight factor of the smoke category, which depends on the importance of the smoke in the specific embodiment.

[0073] The higher the score, the higher the potential fire hazard. The hazard level is divided based on the score range. When the hazard level is greater than a certain threshold, a fire warning should be issued in time to alert relevant personnel to pay attention.

[0074] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

Claims

1. A tunnel fire smoke recognition method based on target detection, characterized in that: The steps include: Step 1: Collect tunnel images, filter the images in the dataset, mark the fire and smoke areas in the images with detection boxes, and build a deep learning dataset; Step 2: Preprocess the images in the data set, and perform image augmentation on the images in the fire area or smoke area in the preprocessed data set; Step 3: Build a fire smoke recognition model and use the augmented data set in step 2 to train the fire smoke recognition model; Step 4: Collect the scene image in the tunnel in real time, pre-process the image, and transmit the pre-processed image to the trained fire and smoke recognition model to determine whether there is a fire area and a smoke area in the image.

2. The method for identifying tunnel fire smoke based on target detection according to claim 1, characterized in that: The preprocessing in step 2 and step 4 includes image denoising, image filtering and image enhancement.

3. The method for identifying tunnel fire smoke based on target detection according to claim 3 is characterized in that: The image enhancement is specifically as follows: the image is divided into three-channel images of R, G, and B, each channel image is converted into a grayscale image, each grayscale image is subjected to guided filtering, and a brightness layer image L of the image is obtained. c (x, y), where c represents the channel, the original image I c (x,y) minus the brightness layer image L c (x, y), get the detail layer image D c (x, y), multiply the brightness layer image and the detail layer image by different stretching factors and perform weighted summation to obtain the enhanced image Im_E.

4. The method for identifying tunnel fire smoke based on target detection according to claim 1, characterized in that: The image augmentation in step 2 includes image flipping, image rotation, image translation, image cutmix and image mixup.

5. The method for identifying tunnel fire smoke based on target detection according to claim 4 is characterized in that: The image cutmix is ​​specifically as follows: a rectangular image is cut out from image A, and randomly pasted into image B to obtain a new image, the ratio λ of the area of ​​the cut image to the area of ​​image B is calculated, the label of the original image A is multiplied by (1-λ), the label of image B is multiplied by λ, and then the sum is performed to obtain the label of the new image.

6. The method for identifying tunnel fire smoke based on target detection according to claim 1, characterized in that: The fire smoke recognition model adopts the YOLO model as the architecture, uses the SPP module in the last layer of the backbone network of the YOLO model, adds an attention mechanism module to the Neck of the YOLO model, and the entire Neck module adopts the FPN+PAN structure.

7. The method for identifying tunnel fire smoke based on target detection according to claim 1, characterized in that: The method also includes identifying the fire level according to the detected fire area and smoke area, specifically: calculating the fire score according to the following formula: Among them, S is the image pixel area, area_F is the area of ​​the fire area, area_S is the area of ​​the smoke area, ω2 and ω1 are weights; Set the fire level, each level corresponds to a different score range.

Images