Tunnel fire abnormal event detection method based on YOLO

By introducing a triple attention mechanism and feature pyramid network into the YOLO network, combined with the comprehensive confidence score, the problems of false detection and missed detection in tunnel flame detection are solved, and high-precision and high-sensitivity fire detection are achieved.

CN118968425AInactive Publication Date: 2024-11-15NANJING TUNNEL & BRIDGE ADMINISTRATION CO LTD

Patent Information

Application Number
CN202411444514.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-16
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing tunnel flame detection technology has serious mis-detection and missed detection in complex environments, making it difficult to accurately detect small-scale flames, resulting in poor early fire warning effects.

Method used

Using the YOLO-based tunnel fire abnormal event detection method, a triple attention mechanism and characteristic pyramid network are introduced, and the YOLO network model is optimized through cross-layer connections, and combined with the comprehensive confidence scoring mechanism, high-precision detection of tunnel fires is achieved.

Benefits of technology

It effectively reduces the false alarm rate, improves detection accuracy and sensitivity, can accurately capture key features of small targets, reduces false alarms, and improves the reliability and accuracy of the detection system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968425A_ABST
    Figure CN118968425A_ABST
Patent Text Reader

Abstract

The invention discloses a tunnel fire abnormal event detection method based on YOLO, and the method comprises the steps: collecting monitoring data including fire and smoke through a camera, extracting and marking a key frame picture, and building a data set; the method comprises the following steps: establishing a data set, introducing a triple attention mechanism on the basis of a convolution module attention mechanism of a YOLOv network model to improve the attention capability of the network on important features, training on the YOLOv model by using the established data set to obtain a target detection model, and detecting a fire event occurring in a tunnel based on the target detection model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and artificial intelligence, and specifically relates to tunnel monitoring image detection and processing, and in particular to a tunnel fire abnormal event detection method based on YOLO. Background Art

[0002] Tunnel fire abnormal event detection is a crucial task after the tunnel is built and put into operation. With the advancement of science and technology, most tunnels are currently equipped with various monitoring sensors, including temperature sensors, smoke detectors, etc. These sensors play an important role in the early warning and emergency response of tunnel fires. However, the existing flame detection technology still faces some challenges in the tunnel environment, resulting in certain limitations in its detection effect, including the following aspects:

[0003] At present, flame detection in tunnels mainly relies on smoke detectors and temperature sensors. However, due to the large volume of traffic in tunnels, there are many interference factors such as car lights and exhaust gas, which can easily cause false detection by sensors. In addition, tunnels have complex structures, dead corners and obstructions, and traditional sensor deployment is difficult to fully cover, which can easily lead to missed detection. These problems make it impossible to detect fires in a timely and accurate manner in the early stages, increasing the risk of fire spreading.

[0004] Traditional flame detection algorithms often have low detection accuracy when faced with complex lighting and environmental conditions in tunnels. Factors such as headlights and reflective materials in tunnels can easily lead to algorithm misjudgment. At the same time, existing algorithms have limited detection capabilities for small-scale flame targets and cannot accurately capture the characteristics of early flames, resulting in poor early fire warning effects. Summary of the invention

[0005] Purpose of the invention: In view of the above-mentioned deficiencies in the prior art, the present invention provides a tunnel fire abnormal event detection method based on YOLO, so that tunnel fire abnormal events can be detected efficiently and accurately.

[0006] Technical solution: A tunnel fire abnormal event detection method based on YOLO, the steps are as follows:

[0007] S1. Collect monitoring data including fire and smoke based on the cameras deployed in the tunnel, and annotate the collected monitoring data to generate a training data set;

[0008] S2. Introducing the triple attention mechanism, and designing a small-scale detection layer for flame detection based on the feature pyramid network, and obtaining an improved YOLO network model through cross-layer connections;

[0009] The triple attention mechanism is used to capture the dependency between the channel dimension and the spatial dimension of the input tensor through three parallel branches, thereby realizing cross-dimensional interaction, which specifically includes the following calculation process:

[0010] Calculation of height-width attention:

[0011]

[0012] in represents the sigmoid function, represents a two-dimensional convolutional layer, Represents the input features Perform pooling operations;

[0013] By performing maximum pooling and average pooling in the zero dimension, the input features The graph is compressed into two dimensions, which is specifically expressed as:

[0014] Z p o o l ( χ ) = [ M a x P o o l 0 d ( χ ) , A v g p o o l 0 d ( χ ) ]

[0015] The features to be input After transposition, channel-height attention is obtained according to the height-width attention calculation method and channel-width attention ;

[0016] The final output triple attention feature map is obtained by fusing three weighted feature maps:

[0017]

[0018] S3, using the YOLO network model described in step S2 to learn and train the constructed data set to build a deep learning model for identifying tunnel fire and other abnormal event characteristics;

[0019] S4, using a distributed computing method to capture and decode the real-time video streams of all surveillance cameras in the tunnel, and input them into several copies of the YOLO network model for parallel detection, thereby realizing the processing of the real-time video streams;

[0020] S5. Calculate a comprehensive confidence score based on the detection targets of flame and smoke, and determine whether an abnormal tunnel fire event occurs based on the comprehensive confidence score.

[0021] Furthermore, the monitoring data described in step S1 includes picture form and video form, and the content of the monitoring data includes fire abnormal event scenes and some scenes where no abnormal events occur.

[0022] Step S1 also includes using tools such as web crawlers to obtain other tunnel fire image data to expand the training data set, so as to improve the generalization performance of the YOLO network model; the training data set also includes scene data of different weather and time periods; and then annotating the collected data, including the bounding boxes of flames and smoke targets.

[0023] Furthermore, in step S2, the triple attention mechanism introduces cross-dimensional interaction based on the CBAM module, combining channel attention, spatial attention and cross-layer attention, so that the network can more accurately capture the key features of small targets.

[0024] The feature pyramid network in step S2 is optimized by increasing the depth of the feature pyramid and adopting weighted cross-layer bidirectional connections to promote the interaction between low-level feature detail information and high-level features, thereby improving feature reuse and information transmission capabilities.

[0025] Furthermore, in the feature pyramid network, one or more weight parameters are set to realize a weighted bidirectional feature pyramid network, and then the importance of different input features is learned and adjusted by a fast normalization fusion method; the fast normalization fusion method is calculated by the following formula:

[0026]

[0027] In the formula, is the output feature after fusion, is the input feature branch, is a learnable parameter weight, which is ensured by the ReLU function non-negative, =0.0001 is a value to prevent numerical instability.

[0028] Furthermore, step S3 includes generating a set of anchor frames suitable for flame detection using the K-means++ algorithm during the training process according to the characteristics of the flame data set; and also includes manually labeling part of the data captured in real time in the tunnel, and then iterating the training data set with the manually labeled data.

[0029] The calculation formula of the comprehensive confidence score in step S5 is as follows:

[0030]

[0031] The parameters and are the weights of flame and smoke respectively, and are the number of detected flame and smoke targets, respectively. and Respectively The confidence level of each flame and smoke target is used to reduce the frequency of false positives in the model detection results and make up for the shortcomings of the prediction performance of a single category.

[0032] Beneficial effects: Based on the YOLO network model, the present invention utilizes the existing tunnel video surveillance system and computer vision methods to perform real-time detection of abnormal fire events in the tunnel. By introducing a comprehensive confidence scoring mechanism, the comprehensive score is calculated according to the number and confidence of smoke and flame targets, thereby effectively reducing the false alarm rate. This mechanism can accurately distinguish real fire events from other abnormal situations, reduce unnecessary alarms and interference caused by false alarms, and improve the reliability of the detection system. Secondly, the present invention introduces a triple attention mechanism, including channel attention, spatial attention, and cross-layer attention, so that the network can more accurately capture the key features of small targets. This is particularly important in tunnel fire detection because the fire source is usually small and difficult to detect in the early stages. Through the triple attention mechanism, the model performs well in capturing and distinguishing subtle fire source features, improving the accuracy and sensitivity of detection. Finally, the present invention optimizes the feature pyramid design based on the YOLO network model by introducing a weighted bidirectional feature pyramid network, so that features of different resolutions interact in a top-down and bottom-up bidirectional path. This cross-layer connection method promotes the full integration of low-level detail information and high-level semantic features, thereby significantly enhancing the ability to extract key features in the detection of abnormal fire events in tunnels. At the same time, the weighted bidirectional feature pyramid network adopts a fast normalization fusion method, which significantly improves the calculation efficiency and reduces the calculation complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flowchart of the abnormal event detection method of tunnel fire based on YOLO;

[0034] Figure 2 It is a precision-recall curve diagram of the embodiment of the present invention on the verification data set;

[0035] Figure 3 It is a graph of the F1 score of the embodiment of the present invention on the validation data set;

[0036] Figure 4 The data samples are labeled data sets of the embodiments of the present invention;

[0037] Figure 5 This is a partial data sample of the validation set of the data set of the embodiment of the present invention;

[0038] Figure 6 This is a diagram of the improved YOLO network structure of the present invention. DETAILED DESCRIPTION

[0039] To illustrate the technical solution disclosed in the present invention in detail, further description is given below in conjunction with the accompanying drawings.

[0040] The present invention provides a method for detecting abnormal fire events in tunnels based on YOLO, referring to Figure 1 As shown, its specific implementation and application includes the following steps:

[0041] 1. Data Collection

[0042] First, install high-definition surveillance cameras inside the tunnel. The cameras should cover all key areas of the tunnel, such as entrances and exits, vents, emergency exits, etc. The cameras should have the function of working at night to ensure that they can work normally even in low light conditions. The collected data includes pictures and videos. The data content should cover scenes of abnormal events such as fire and smoke, as well as scenes under normal conditions. The cameras need to collect data at different time periods to ensure that the data covers scenes of daytime, nighttime, various weather conditions, and different traffic flows to improve the robustness of the model. Secondly, use tools such as web crawlers to obtain more tunnel fire pictures and video data from the Internet to enrich the diversity of the data set. Filter and clean the crawled data to remove fuzzy, duplicated, and irrelevant data to ensure data quality. Perform data enhancement processing on the original data, including rotation, scaling, flipping, adding noise, etc., to generate more diverse training samples and improve the generalization ability of the model.

[0043] 2. Data processing

[0044] The collected data is screened and cleaned to remove fuzzy, duplicate and irrelevant data to ensure data quality. The original data is enhanced by rotating, scaling, flipping, adding noise, etc. to generate more diverse training samples and improve the generalization ability of the model.

[0045] The collected images and video data are annotated using manual annotation tools. The image target annotation uses LabelImg software. The annotation content includes the bounding box of the flame and smoke, and records the relevant timestamp and location. Figure 4 and Figure 5 As shown in the figure. The pre-trained flame and smoke detection model is used for preliminary annotation, and then manually corrected to improve the annotation efficiency and accuracy. Finally, the dataset is randomly divided into a training set and a validation set in a ratio of 7:3.

[0046] (III) Optimizing the YOLO network model

[0047] The optimization of the YOLO network model includes the following steps:

[0048] Step 2-1, based on the original CBAM module, introduces cross-dimensional interaction, combines channel attention, spatial attention and cross-layer attention, and implements a triple attention mechanism, enabling the network to more accurately capture the key features of small targets. Specifically, it includes the following steps:

[0049] (1) The introduced cross-dimensional interactive triple attention mechanism captures the dependency between the channel dimension and the spatial dimension of the input tensor through three parallel branches. The three branches specifically include:

[0050] (a) Height-width attention, calculated by the following formula:

[0051]

[0052] in are the input features, is the sigmoid function, represents a two-dimensional convolutional layer, It is a pooling process that compresses the input feature map into two dimensions by performing maximum pooling and average pooling in zero dimension. It is specifically expressed as:

[0053] Z p o o l ( χ ) = [ M a x P o o l 0 d ( χ ) , A v g p o o l 0 d ( χ ) ]

[0054] (b) Channel - High Attention and channel-width attention Similar to the height-width attention calculation method, just transpose the dimension of the feature before and after the calculation.

[0055] (2) The final output triple attention feature is obtained by fusing three weighted feature maps and is calculated by the following formula:

[0056]

[0057] Step 2-2, optimize the feature pyramid network design. Specifically, increase the depth of the feature pyramid and use weighted cross-layer bidirectional connections to obtain a weighted bidirectional feature pyramid module to promote the interaction between low-level feature details and high-level features, with the goal of achieving more adequate feature reuse and information transfer. The weighted bidirectional feature pyramid module includes multiple learnable weight parameters, which are used to learn and adjust the importance of different input features through a fast normalization fusion method. The fast normalization fusion method is calculated by the following formula:

[0058]

[0059] The optimized YOLO network model structure is as follows Figure 6 shown.

[0060] (IV) Training the improved YOLO network model:

[0061] The YOLO network model is trained using the constructed data set, and the data set is expanded based on the manual feedback after the system is deployed. The specific steps are as follows:

[0062] Step 3-1: Use the K-means++ algorithm to generate anchor boxes specifically for flame detection: According to the characteristics of the flame dataset, the K-means++ algorithm is used to generate a set of anchor boxes suitable for flame detection. The algorithm minimizes the distance between the anchor box and the true bounding box to ensure that the generated anchor box can effectively cover flame targets of different sizes and shapes.

[0063] Step 3-2: Use the dataset constructed by the method described in step (ii) to train the improved YOLO network model. It is necessary to correctly set the learning rate, batch size, number of iterations and other training hyperparameters, and regularly use the validation set to evaluate the model performance during the training process, and adjust the hyperparameters and early stopping strategy to avoid overfitting or underfitting.

[0064] Step 3-3: Provide manual feedback on some of the data captured in real time in the tunnel, mark new flame and smoke targets, generate a new data augmentation training data set according to the data processing method in step 1, and use the augmented data set to iteratively train the improved YOLO network model to continuously improve the model performance and robustness.

[0065] Step 3-4: Experimentally verify the obtained YOLO network model to evaluate the performance of the proposed YOLO-based tunnel fire abnormal event detection method. The detection accuracy (Precision, P), recall (Recall, R) and average precision (Average Precision, AP) are mainly used as evaluation indicators. The accuracy rate indicates the proportion of data predicted as positive samples that are actually positive samples, and the recall rate indicates the proportion of data that are actually positive samples in all data that are correctly predicted. Among them, AP is one of the most important indicators for evaluating the performance of target detection algorithms. The larger the AP value, the higher the model detection performance. Sort the confidence of all target detection boxes of a certain category in the detection image from large to small, calculate the accuracy and recall rate accumulated to each detection box one by one, and draw the precision-recall curve. The results are shown in the figure. Figure 2 As shown. AP is the area between the precision-recall curve of a single category and the coordinate axis. On the validation set, the AP values ​​of fire and smoke are 0.912 and 0.891 respectively. In addition to AP, the F1 score can also reflect the performance of the target detection model. The F1 score is calculated by the following formula:

[0066]

[0067] The relationship between F1 score and confidence is shown in the figure below: Figure 3 As shown. The verification method shows that the method proposed in the present invention has high accuracy and robustness in detecting abnormal events of tunnel fire. In particular, the improved model has obvious advantages in capturing small target fire sources and extracting detailed features.

[0068] 5. Real-time video stream processing

[0069] The surveillance cameras installed in the tunnel capture real-time video streams, decode the video streams through distributed computing methods, and input the decoded frame data into the YOLO network model for processing. The specific steps are as follows:

[0070] a. Use the open interface to query the monitoring point information from the monitoring system platform and obtain the preview stream.

[0071] b. Use hardware decoders or efficient software decoding libraries (such as OpenCV) to decode the video stream in real time and generate image frame data.

[0072] c. Preprocess the decoded frame data, including image scaling, normalization and other operations, to generate a data format suitable for YOLO network model input.

[0073] Real-time video stream processing uses a distributed computing architecture to distribute video stream decoding and detection tasks to multiple computing nodes for parallel processing. Each node runs multiple copies of the YOLO network model to process the assigned frame data, improving detection efficiency and real-time performance. The specific steps are as follows:

[0074] a. Divide the decoded frame data by source and distribute it to different computing nodes. A load balancing algorithm can be used to ensure balanced computing load on each node.

[0075] b. Run multiple copies of the YOLO network model simultaneously on each computing node to process the assigned frame data for flame and smoke target detection.

[0076] c. Merge the detection results output by each computing node and save them in the relational database to generate a complete video detection result.

[0077] 6. Post-processing of detection targets

[0078] The flame and smoke targets output by the YOLO network model are counted, and the number and confidence of each target are counted. The comprehensive score is calculated based on the number and confidence of smoke and flame targets, using the following formula:

[0079]

[0080] The parameters and The weights for flame and smoke respectively. and are the number of detected flame and smoke targets, respectively. and Respectively The confidence level of each fire and smoke target.

[0081] When the comprehensive confidence score exceeds the threshold, it is determined to be a fire abnormality event, and the system immediately triggers the alarm mechanism to notify relevant personnel to handle the emergency. At the same time, the detection results and comprehensive scores are recorded, and subsequent data analysis is carried out to optimize the model and algorithm.

Claims

1. A tunnel fire abnormal event detection method based on YOLO, characterized in that: Here are the steps: S1. Collect monitoring data including fire and smoke based on the cameras deployed in the tunnel, and annotate the collected monitoring data to generate a training data set; S2. Introducing the triple attention mechanism, and designing a small-scale detection layer for flame detection based on the feature pyramid network, and obtaining an improved YOLO network model through cross-layer connections; The triple attention mechanism is used to capture the dependency between the channel dimension and the spatial dimension of the input tensor through three parallel branches, thereby achieving cross-dimensional interaction, including the following calculations: Calculation of height-width attention: ; in Indicates the exclusive or logical operation. represents the sigmoid function, represents a two-dimensional convolutional layer, Represents the input features Perform pooling operations; By performing maximum pooling and average pooling in the zero dimension, the input features The graph is compressed into two dimensions, which is specifically expressed as: ; The features to be input After transposition, channel-height attention is obtained according to the height-width attention calculation method and channel-width attention ; The final output triple attention feature map is obtained by fusing three weighted feature maps: ; S3, using the YOLO network model described in step S2 to learn and train the constructed data set to build a deep learning model for identifying tunnel fire and other abnormal event characteristics; S4, using a distributed computing method to capture and decode the real-time video streams of all surveillance cameras in the tunnel, and input them into several copies of the YOLO network model for parallel detection, thereby realizing the processing of the real-time video streams; S5. Calculate a comprehensive confidence score based on the detection targets of flame and smoke, and determine whether an abnormal tunnel fire event occurs based on the comprehensive confidence score.

2. The method for detecting abnormal fire events in tunnels based on YOLO according to claim 1, characterized in that: The monitoring data described in step S1 includes picture and video forms, and the content of the monitoring data includes fire abnormal event scenes and some scenes where no abnormal events occur.

3. The tunnel fire abnormal event detection method based on YOLO according to claim 1 is characterized in that: Step S1 also includes using tools such as web crawlers to obtain other tunnel fire image data to expand the training data set, so as to improve the generalization performance of the YOLO network model; the training data set also includes scene data of different weather and time periods; The collected data is then labeled, including the bounding boxes of the flame and smoke targets.

4. The method for detecting abnormal fire events in tunnels based on YOLO according to claim 1, characterized in that: In step S2, the triple attention mechanism introduces cross-dimensional interaction based on the CBAM module, combining channel attention, spatial attention and cross-layer attention, so that the network can capture the key features of small targets more accurately.

5. The method for detecting abnormal fire events in tunnels based on YOLO according to claim 1, characterized in that: The feature pyramid network in step S2 is optimized by increasing the depth of the feature pyramid and adopting weighted cross-layer bidirectional connections to promote the interaction between low-level feature detail information and high-level features, thereby improving feature reuse and information transmission capabilities.

6. The method for detecting abnormal fire events in tunnels based on YOLO according to claim 5, characterized in that: In the feature pyramid network, one or more weight parameters are set to implement a weighted bidirectional feature pyramid network, and then the importance of different input features is learned and adjusted through a fast normalization fusion method; The fast normalized fusion method is calculated by the following formula: ; In the formula, is the output feature after fusion, is the input feature branch, , is a learnable parameter weight, which is ensured by the ReLU function non-negative, =0.0001 is a value to prevent numerical instability.

7. The method for detecting abnormal fire events in tunnels based on YOLO according to claim 1, characterized in that: Step S3 includes generating a set of anchor frames suitable for flame detection using the K-means++ algorithm during the training process according to the characteristics of the flame dataset; it also includes manually labeling part of the data captured in real time in the tunnel, and then iterating the training dataset with the manually labeled data.

8. The method for detecting abnormal fire events in tunnels based on YOLO according to claim 1, characterized in that: The calculation formula of the comprehensive confidence score in step S5 is as follows: ; The parameters and are the weights of flame and smoke respectively, and are the number of detected flame and smoke targets, respectively. and Respectively The confidence level of each flame and smoke target is used to reduce the frequency of false positives in the model detection results and make up for the shortcomings of the prediction performance of a single category.

Citation Information

Patent Citations

  • Flame detection method based on improved YOLOv5 model

    CN114662605A

  • Insulator defect detection method fusing BiFPN and attention mechanism

    CN118196598A

  • Lightweight smoke and fire detection method based on YOLOv8

    CN118570721A

  • Fire detection method, apparatus and device based on deep learning, and medium

    WO2024109873A1

Cited By

  • Detection model training method, detection method, equipment, storage medium and product

    CN119251648A

  • Air conditioner recommendation method and device based on LSTM model and air conditioner

    CN120561514A

  • Air conditioner recommendation method and device based on lstm model and air conditioner

    CN120561514B