A tunnel fire intelligent detection method based on a video target detection network

Abstract

The application discloses a tunnel fire intelligent detection method based on a video target detection network, which is based on a constructed tunnel fire target data set, combines a video target detection network to train the data set, obtains a fire detection model, inputs a video stream to be detected, judges whether a smoke area exists or not by using a smoke area screening strategy, detects a video sequence with the smoke area by using the fire detection model, reduces false positives by using a fire false detection identification strategy based on an area change rate, and obtains an accurate tunnel fire detection result. The application can continuously analyze flame smoke movement in multiple frames of video, combines multiple strategies before and after the video target network detection of the constructed data set, improves the calculation efficiency, and reduces fire false positives. The application has accurate fire detection accuracy in various tunnel monitoring scenes, has strong practical value and market potential.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of intelligent transportation, and relates to a tunnel fire intelligent detection method based on a video target detection network. BACKGROUND

[0002] In the operation and daily supervision of highway tunnels, due to the strong closed nature, few entrances and exits, poor signal and other characteristics of the tunnel scene, once a fire event occurs in the highway tunnel, it will not only cause serious damage to the personnel and property in the tunnel, but also cause damage to the internal structure of the tunnel, the tunnel pavement and other infrastructure. Therefore, it is necessary to establish a tunnel dedicated dataset and detect tunnel fires by combining a deep network to achieve timely and reliable detection of tunnel fires, reduce the cost of personnel patrol and improve the utilization efficiency of monitoring video.

[0003] At present, using traffic monitoring video to detect fires has lower cost and higher precision compared with the traditional method of relying only on fire temperature sensors, smoke sensors and the like, can realize real-time display of fire conditions, and is the current mainstream fire detection method. Compared with the use of multi-frame image video target detection network for fire detection, the method relying only on single-frame image cannot analyze the motion state of smoke and flame, so the false positive rate is high and there is a defect in detection precision. SUMMARY

[0004] In view of the deficiencies in the prior art, the purpose of the present application is to provide a tunnel fire intelligent detection method based on a video target detection network, which solves the problems of low precision, high false positive rate and application defects of the current fire sensor and single-frame image detection network fire detection method.

[0005] In order to solve the above technical problems, the present application adopts the following technical solutions:

[0006] A tunnel fire intelligent detection method based on a video target detection network, which is based on a constructed tunnel fire target dataset, combines a video target detection network to train the dataset, obtains a fire detection model, inputs a video stream to be detected, uses a smoke area screening strategy to determine whether there is a smoke area, then uses the fire detection model to detect the video sequence with the smoke area, and uses a fire false detection identification strategy based on area change rate to reduce false positives, to obtain accurate tunnel fire detection results; comprising the following steps:

[0007] Step 1, constructing a tunnel fire target dataset: obtaining a tunnel scene fire smoke video, and labeling the flame and smoke targets of the video in consecutive frames, the labeling file including the size of the fire smoke image and the position of the rectangular labeling box, so as to obtain the tunnel fire target dataset;

[0008] Step two, using a video target detection network, training the tunnel fire target data set in step one, after training, the fire detection model is obtained;

[0009] Step three, for the tunnel monitoring video stream data, a smoke area screening strategy is used to judge whether there is a smoke area, if there is a smoke area, go to step four for fire detection, if there is no smoke area, do not perform fire detection;

[0010] Step four, using the fire detection model of step two to detect the video sequence with smoke area, get the flame and smoke target and mark its image position with a rectangular box, use the fire false detection identification strategy based on area change rate to judge whether it is false detection, if there is no false detection, report the fire event, if there is false detection, do not report the fire;

[0011] Step five, for the reported fire event, save the corresponding flame smoke video sequence to the local, and record the occurrence time of the corresponding fire event, complete the intelligent detection of fire.

[0012] The present application also includes the following technical features:

[0013] Specifically, in step one, 10 or more tunnel scene fire smoke videos are obtained.

[0014] Specifically, in step one, the annotated images of the flame and smoke target in the continuous frame are not less than 90000.

[0015] Specifically, the video target detection network in step two is YOLOV video target detection network, the training set and test set ratio is set to 8:2 during training, the training iteration is 150 times, and the batch size is 16.

[0016] Specifically, the step three includes:

[0017] Step 3.1, using Gaussian mixture background modeling method, modeling the background area in the video stream, extracting the motion area of smoke, using N×N window to screen the suspected smoke area in the motion area;

[0018] Step 3.2, according to the change of smoke image and non-smoke image in the three channels of HSV color model, using the method of color characteristics to determine whether there is a smoke area.

[0019] Specifically, the method of color characteristics includes: setting the threshold value of saturation S in HSV color model as 70, calculating the average brightness value of every 75 frames of video as the brightness threshold value; if the brightness value of the detected area is greater than the brightness threshold value and the value of saturation S is not higher than 70, the area has a smoke area, go to step four; otherwise, no smoke is found, and no fire detection is performed.

[0020] Specifically, the fire false detection identification strategy based on the area change rate comprises:

[0021] The area change rate of the rectangular frame is calculated by using the flame and smoke rectangular frame detected by the fire detection model, as shown in formula (1):

[0022]

[0023] Wherein, S' represents the area change rate of the flame and smoke detection rectangular frame in k frames, S k+t , S k respectively represent the detection area in the k+t frame and the k frame image;

[0024] The average area change rate of the flame and smoke target in the continuous 100 frames is counted, if the counting result is higher than 10, it is considered that the current detected flame and smoke target does not exist false detection, a fire event occurs, and the fire event is reported; if the counting result is lower than 10, it is considered that the current fire smoke detection exists false detection, at this time, the fire is not reported.

[0025] Compared with the prior art, the present application has the following technical effects:

[0026] The present application can be applied to tunnel fire conditions of various monitoring angles, and at the same time, combined with multi-frame fire detection analysis, the problem of missing flame smoke movement information by relying on single frame image is avoided, and the low precision and high cost of traditional fire sensor and artificial patrol are broken. The smoke area screening strategy and the fire false detection identification strategy based on the area change rate can improve the tunnel fire detection efficiency, reduce the tunnel fire detection false alarm, have the advantages of stable algorithm, real-time processing, etc., and have wide application potential.

[0027] The tunnel fire target data set constructed in the present application is derived from the actual tunnel monitoring angle camera, and is divided into 10 different tunnel scenes and more, which can more specifically solve the problem of tunnel fire detection.

[0028] The present application uses the Gaussian mixture modeling method, and uses an N*N window to screen the suspected smoke area in the motion area, adopts the video target detection network YOLOV to uniformly detect the flame and smoke, focuses on the motion characteristics of the flame and smoke, and combines the characteristics of multi-frame flame and smoke to detect the target, so that the category and the position of the flame and smoke can be obtained.

[0029] The present application only uses a convolutional neural network for video target detection, which can distinguish the flame and smoke, and also can obtain the position of the flame and smoke in the image and mark it with a rectangular frame, so as to realize the detection of the two types of targets of flame and smoke.

[0030] The YOLOV network adopted by the application is oriented to a dynamic video sequence, that is, a multi-frame combined target detection problem, and the YOLOV network is used for fire detection in a tunnel fire scene, so that the movement of flames and smoke in multiple frames can be analyzed, time clues of different frames can be aggregated, video context information can be fully utilized, and detection of flames and smoke can be performed not only on a certain frame image, but also on multiple frames, so that the detection precision of flames and smoke can be effectively improved, which plays a crucial role in improving the accuracy of fire detection in a tunnel special scene. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 A method flowchart of the application is shown in the figure.

[0032] Figure 2 A frame of fire image in a tunnel monitoring perspective video is shown in the figure.

[0033] Figure 3 Part of the scene covered by the tunnel fire target data set and the annotation file information are shown in the figure.

[0034] Figure 4 The network structure of the YOLOV video target detection network is shown in the figure.

[0035] Figure 5 The smoke area screened by the high-speed mixed background modeling is shown in the figure.

[0036] Figure 6 The HSV color information of smoke with different concentrations is shown in the figure.

[0037] Figure 7 The result of flame and smoke target detection is shown in the figure.

[0038] Figure 8 The area change rate comparison of different moving targets is shown in the figure. DETAILED DESCRIPTION

[0039] The application discloses a tunnel fire intelligent detection method based on a video target detection network, constructs a tunnel fire target data set, combines a video target detection network to train the data set, obtains a fire detection model, inputs a video stream to be detected, judges whether a smoke area exists or not by using a smoke area screening strategy, detects a video sequence in which the smoke area exists by using the fire detection model, reduces false positives by using a fire false detection identification strategy based on an area change rate, and obtains an accurate tunnel fire detection result.

[0040] As shown in the figure, the method specifically comprises the following steps: Figure 1

[0041] ​Step one, build a tunnel fire target data set: in order to carry out video target network based fire detection, it is necessary to prepare mass image and video data for network learning and training; When building a tunnel fire target data set, obtain fire smoke video of 10 or more tunnel scenes, and label the continuous frames of the fire and smoke targets, and the number of labeled fire smoke images is not less than 90000. When labeling, different labeling criteria need to be developed for different tunnel scenes and fire conditions; The flame target with flame layers needs to be labeled, and the lane line needs to be avoided when labeling the smoke area; The labeling file includes the size of the fire smoke image and the position of the rectangular labeling box, and the position of the rectangular labeling box includes length, width and center point;

[0042] The embodiment adopts the monitoring video of Chinese highway tunnel, the video frame rate is 25FPS, and the video image size is 1920*1080.

[0043] As shown in Figure 2 is a frame of fire image in the tunnel monitoring view video. Figure 3 For the part of the scene covered by the tunnel fire target data set and the information of the labeling file, the fire targets are divided into flame and smoke in this embodiment for labeling.

[0044] Step two, use the video target detection network, such as YOLOV, to train the tunnel fire target data set in step one, set the training set and test set ratio to 8:2, train for 150 times, and the batch size is 16. After training, the fire detection model is obtained; Figure 4 The network structure of YOLOV video target detection network.

[0045] Step three, for tunnel monitoring video stream data, adopt smoke area screening strategy to judge whether there is smoke area, if there is smoke area, go to step four for fire detection, if there is no smoke area, do not carry out fire detection;

[0046] Step 3.1, adopt Gaussian mixture background modeling method to model the background area in the video stream, extract the motion area of the smoke, use N*N window to screen the suspected smoke area in the motion area, N is generally 200, to include more smoke area, go to step 3.2; Figure 5 As shown in

[0047] Step 3.2, for the screening result of suspected smoke area, according to the change of smoke image and non-smoke image in the three channels of HSV color model, use the method of color characteristics to determine the video frame with smoke;

[0048] In HSV color space model, H, S, V represent hue, saturation, and brightness respectively; the saturation S of smoke area is relatively low, and the threshold of S is set to 70; meanwhile, the number of pixels with increased brightness value increases constantly after the appearance of smoke, and the average brightness value of every 75 frames of video is calculated as the threshold; if the brightness value of the detected area is greater than the threshold and the saturation S is not higher than 70, it is considered that the area has smoke, and step four is entered, otherwise, it is considered that no smoke is found, and no fire detection is performed. Figure 6 The HSV color information is for smoke of different concentrations, and the smoke in the three smoke images is from weak to strong, the smoke information in the left image is weak, and the corresponding HSV distribution graph is relatively scattered; the smoke information in the middle image and the right image is obvious, and the gradual increase of the V component indicates that the smoke in the current area is thicker, and it can be seen from the saturation S component in the graph that even in the case of different concentrations of smoke, the saturation of the smoke image is always relatively low.

[0049] Step four, using the fire detection model of step two to detect the video sequence with smoke area, obtaining the flame and smoke target and marking the image position with a rectangular frame, Figure 7 The result of flame and smoke target detection (the detected target is framed with a rectangular frame). In order to reduce the false positives in the detection result, a fire false detection identification strategy based on area change rate is used to judge whether there is false detection, if there is no false detection, the fire event is reported, if there is false detection, the fire is not reported;

[0050] The specific implementation method includes the following steps:

[0051] Step 4.1, using the flame and smoke rectangular frame detected by the fire detection model, calculating the area change rate of the rectangular frame, as shown in formula (1):

[0052]

[0053] Wherein, S' represents the area change rate of the flame and smoke detection rectangular frame in k frames, Sk +t , Sk represent the area of the detected area in the k+t frame and the k frame image respectively;

[0054] Step 4.2, statistics of the average area change rate of the flame and smoke target in 100 continuous frames, if the statistical result is higher than 10, it is considered that the detected flame and smoke target has no false detection, and a fire event occurs, and the fire event is reported; if the statistical result is lower than 10, it is considered that the current fire and smoke detection has false detection, and the false detection source is tunnel wall light flicker, vehicle double flash, etc., at this time, no fire is reported.

[0055] Figure 8For the area change rate of different moving targets, the area change rates of three kinds of moving targets, i.e., flame, smoke and moving pedestrian, in 1000 frames are recorded, and it can be seen that the change rates of smoke and flame basically remain above 10, while the area change rate of the pedestrian target is relatively low, so the area change rate can be used to describe the change of the fire smoke in the scene.

[0056] Step five, for the reported fire event, save the corresponding flame and smoke video sequence to the local, and record the occurrence time of the corresponding fire event, so as to facilitate the tunnel supervisor to check and make the next decision.

[0057] After the whole process of the present application is completed, the tunnel fire intelligent detection based on the video target detection network is completed.

[0058] The tunnel fire intelligent detection method based on the video target detection network provided by the present application uses the accuracy (AR) and false alarm rate (ER) of the fire smoke detection evaluation standard as two indexes to measure the accuracy of the fire detection.

[0059] The accuracy (AR) is the ratio of the number of correctly detected samples to the total number of samples in the video sequence, and the calculation formula is formula (2):

[0060]

[0061] The false alarm rate (ER) is the ratio of the number of false detection non-fire smoke samples to the total number of non-fire smoke samples in the video sequence, and the calculation formula is formula (3):

[0062]

[0063] In formula (2) and formula (3), TP, TN, FN and FP are respectively the number of positive samples predicted as positive class by the model, the number of negative samples predicted as negative class by the model, the number of negative samples predicted as positive class by the model, and the number of positive samples predicted as negative class by the model.

[0064] In order to evaluate the accuracy of the fire event reporting, a plurality of tunnel fire videos as shown in Table 1 are used for analysis:

[0065] Table 1 contains smoke test video detection comparison

[0066]

[0067] These videos cover a variety of tunnel fire situations, including tire fire, smoke dispersion, and difficult-to-distinguish tunnel light illumination, and vehicle light flicker interference, which pose a high challenge for accurate fire detection.

[0068] Table 2 statistically compares the average accuracy (AR) and false alarm rate (ER) of the fire detection method of the present application with the mainstream existing fire detection methods based on YOLOv4, YOLOv5 and other target detection networks.

[0069] Table 2 Comparison test results of fire smoke detection methods

[0070]

[0071] As can be seen in Table 2, the accuracy (AR) of the method of the present application is high, reaching 89.45%, and the false alarm rate (ER) is low, only 8.73%, which shows that by using the dynamic information of flame and smoke, aggregating the time clues of different frames, fully utilizing the video context information, and not relying on the fire detection result of a certain frame to consider that a fire event has occurred in the current scene, combined with the screening of suspected smoke area, the fire false detection identification strategy based on area change rate, the fire detection result can be effectively improved, which plays a crucial role in improving the accuracy of fire detection in the tunnel special scene.

Claims

1. An intelligent tunnel fire detection method based on a video target detection network, characterized in that: This method is based on a constructed tunnel fire target dataset and is trained with a video target detection network to obtain a fire detection model. The method inputs the video stream to be detected, uses a smoke region screening strategy to determine whether a smoke region exists, then uses the fire detection model to detect video sequences containing smoke regions, and uses a fire false detection identification strategy based on area change rate to reduce false alarms, thereby obtaining accurate tunnel fire detection results. The method includes the following steps: Step 1: Construct a tunnel fire target dataset: Obtain a fire smoke video of a tunnel scene and annotate consecutive frames of its flames and smoke targets. The annotation file includes the size of the fire smoke image and the position of the rectangular annotation box, thus obtaining a tunnel fire target dataset; Step 2: Use the video target detection network to train the tunnel fire target dataset in step 1. After the training is completed, a fire detection model is obtained. Step 3: Using the smoke area screening strategy to determine whether a smoke area exists in the tunnel monitoring video stream data, if a smoke area exists, proceed to step 4 for fire detection; if no smoke area exists, no fire detection is performed; Step 4: Use the fire detection model from step 2 to detect the video sequence with smoke areas, obtain flame and smoke targets, and mark their image locations with rectangular boxes. Use a fire false detection identification strategy based on area change rate to determine whether it is a false detection. If there is no false detection, report the fire event; if there is a false detection, do not report the fire event. Step 5: For reported fire events, the corresponding flame and smoke video sequence is saved locally, and the occurrence time of the corresponding fire event is recorded to complete the intelligent fire detection; The step three includes: Step 3.1: Use the Gaussian mixture background modeling method to model the background area in the video stream, extract the moving area of ​​the smoke, and use an N×N window to filter the suspected smoke area in the moving area; Step 3.2: Based on the screening results of suspected smoke areas, the color characteristics method is used to determine whether there is a smoke area based on the changes in the three channels of the HSV color model between the smoke image and the non-smoke image; The fire false detection identification strategy based on area change rate includes: The flame and smoke rectangles detected by the fire detection model are used to calculate the area change rate of the rectangles, as shown in formula (1): （1） in, express The rate of change of the flame and smoke detection rectangular frame area within the frame, 、 Respectively represent Frame and The area of ​​the detection region in the frame image; The average area change rate of flame and smoke targets within 100 consecutive frames is counted. If the statistical result is higher than 10, it is considered that there is no false detection of the flame and smoke targets currently detected, a fire event has occurred, and the fire event is reported. If the statistical result is lower than 10, it is considered that there is a false detection of the current fire and smoke detection, and the fire event is not reported.

2. The method for intelligent detection of tunnel fires based on a video target detection network according to claim 1, characterized in that: In the step 1, 10 or more fire smoke videos of tunnel scenes are obtained.

3. The intelligent tunnel fire detection method based on video target detection network according to claim 1, characterized in that: In the step 1, no less than 90,000 consecutive frames of images of the flame and smoke targets are annotated.

4. The method for intelligent detection of tunnel fires based on a video target detection network according to claim 1, wherein: The video object detection network in step 2 is the YOLOV video object detection network. During training, the ratio of the training set to the test set is set to 8:2, the training iterations are 150 times, and the batch size is 16.

5. The intelligent tunnel fire detection method based on video target detection network according to claim 1, characterized in that: The color characteristic method includes: setting the saturation S threshold in the HSV color model to 70, calculating the average brightness value of every 75 frames of the video as the brightness threshold; if the brightness value of the detected area is greater than the brightness threshold and the saturation S value is not higher than 70, then smoke exists in the area, and the process proceeds to step 4; otherwise, no smoke is found, and fire detection is not performed.

Images