Road traffic fire smoke detection method and device and storage medium

Abstract

The invention provides a road traffic fire smoke detection method and device, and a storage medium. The method comprises the following steps: S1, constructing a YOLOv8-SE model; s2, constructing a data set and performing model training; and S3, performing fire smoke detection by using the trained model. The problem that an existing detection method is not high in accuracy and real-time performance is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a road traffic fire smoke detection method, equipment and storage medium. Background Art

[0002] Fire is a multi-disaster with a large time span and a large spatial span, which seriously threatens the natural environment and the safety of people's lives and property. Due to the rapid spread of fire, the rapid and accurate detection of fire points plays a vital role in the early warning and prevention of major disasters. Usually, smoke appears earlier than flames and is not easy to be blocked. At the same time, with the acceleration of urbanization and the continuous increase in road traffic, the frequency of fire and smoke incidents has gradually increased. This not only threatens people's lives and safety, but also may cause huge property losses. Therefore, adding smoke recognition can achieve early detection and early warning of fires, and establishing an efficient fire and smoke detection system is of great significance to ensuring traffic safety and reducing the accident rate.

[0003] Early detection of fire and smoke is a pressing concern for industrial safety and decision management. However, methods for early detection of fire and smoke usually rely on manual observation or sensors. Methods based on manual observation have their limitations, especially in the presence of intermittent interruptions or interference, which make these methods unable to perform continuous and reliable monitoring. On the other hand, traditional sensor-based systems rely on smoke particle sampling or relative heat to detect fire, but these methods usually have significant time delays and require strong fires or close distances to sense fires. In addition, due to environmental changes, these methods may not be able to fully cover the detection area, especially in open spaces. In general, current fire and smoke detection technologies are difficult to meet modern industrial processes and safety requirements. Moreover, these traditional fire detection methods mostly rely on manual monitoring or fixed equipment, and have defects such as slow response speed and limited monitoring range, which are difficult to meet the real-time monitoring needs in complex environments such as road traffic.

[0004] To solve this problem, many scholars have integrated deep learning into fire detection. As a novel method, deep learning can automatically extract and analyze deeper features. At present, many scholars have conducted relevant research on flame detection, mostly using deep learning methods. However, the computational cost of these CNN-based methods is very high, and real-time fire detection is relatively slow regardless of accuracy. Faced with complex detection tasks such as fire smoke, existing methods still have some limitations, such as insufficient ability to capture flame features in different environments and low bounding box regression accuracy. These problems have led to the need to improve the accuracy and real-time performance of fire smoke detection in practical applications. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a road traffic fire smoke detection method, device and storage medium to solve the problems of low accuracy and real-time performance of existing detection methods.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A road traffic fire smoke detection method comprises the following steps:

[0008] S1: Build the YOLOv8-SE model;

[0009] S2: Build a dataset and perform model training;

[0010] S3: Use the trained model for fire smoke detection.

[0011] Further, the S1 includes:

[0012] S11: In the backbone network of YOLOv8, an attention module SE is added after the last basic convolution module CBS, feature fusion module C2F, and spatial pyramid pooling SPPF;

[0013] S12: In the head network of YOLOv8, the loss function of the detection head is replaced by the MPDIoU loss function.

[0014] Furthermore, the S11 includes:

[0015] S111: In the attention module SE, global average pooling is performed on each channel of the input feature map; the calculation formula is:

[0016]

[0017] Among them, i and j represent the index of the height and width of the feature map, and u c (i, j) represents the pixel value of channel c at position (i, j), H and W are the height and width of the feature map, and z c is the global information of the cth channel;

[0018] S112: Use a neural network composed of fully connected layers to generate the weight of each channel through the Sigmoid function; the calculation formula is:

[0019] s c =σ(W2δ(W1z c )) (2)

[0020] Among them, W1 and W2 are the weight matrices of the two fully connected layers, δ represents the ReLU activation function; σ is the Sigmoid activation function, s c is the weight of channel c;

[0021] S113: Multiply the weights by the input feature map channel by channel, adjust the feature representation of each channel, and obtain a new feature map. The calculation formula is:

[0022]

[0023] in, is the feature map of the cth channel, X C is the original feature map of the cth channel.

[0024] Further, the S12 includes:

[0025] In the detection head, the calculation formula of the MPDIoU loss function is as follows:

[0026]

[0027] Among them, w and h are the length and width of the input feature image of the loss function, Indicates the coordinates of the upper left corner and lower right corner of A. represents the coordinates of the upper left corner and lower right corner of B, A represents the predicted box, B represents the real box, A∩B represents the intersection area of ​​the predicted box and the real box, A∪B represents the union area of ​​the predicted box and the real box, d1 represents the distance between the center point of the real box and the center point of the predicted box, d2 represents the distance between the projection point of the predicted box and the boundary of the target box, that is, the distance between the projection points of the predicted box and the target box on their respective boundary projection lines.

[0028] Further, the S2 includes:

[0029] S21: Build dataset;

[0030] S22: preprocess and annotate the data set, and then divide it into training set, validation set, and test set;

[0031] S23: Train the model using the training set.

[0032] Another technical solution adopted by the present invention is:

[0033] A fire smoke detection device adopts the above method to realize fire smoke detection.

[0034] Another technical solution adopted by the present invention is:

[0035] A fire smoke detection storage device adopts the above method to realize fire smoke detection.

[0036] The beneficial effects of the present invention are

[0037] 1. First, this paper introduces the SE attention mechanism to enhance the model's ability to capture fire and smoke features in different scenarios and conditions. The SE attention mechanism can help the model better identify the characteristics of fire and smoke by dynamically adjusting the importance of the feature map, thereby improving the detection accuracy.

[0038] 2. Secondly, this paper introduces the MPDIoULoss (Multi-Perspective Distance Intersection over Union) loss function to further improve the bounding box regression capability of the model. Compared with the traditional IoU loss, MPDIoULoss takes into account the scale, shape and position deviation of the bounding box more comprehensively, and optimizes the matching degree between the predicted box and the real box from multiple perspectives, making the model more accurate in detecting small targets and targets with complex shapes. Through these improvements, the accuracy and robustness of the model in fire smoke detection tasks can be improved, providing strong technical support for traffic safety management.

[0039] 3. The present invention not only focuses on improving the performance of the model, but also takes into account the user experience in practical applications. In order to facilitate user operation, the present invention has developed a visual interface that supports model training, detection and result display. Users can upload and detect images and videos through this interface, view the detection results in real time, and export relevant data. This functional design not only improves the usability of the system, but also provides a convenient technical means for traffic safety management.

[0040] In summary, this paper proposes an efficient fire smoke detection system by combining the improved YOLOv8-SE model, adaptive attention mechanism and MPDIoULoss loss function. Through experimental verification, the system is superior to the traditional YOLOv8 model in terms of accuracy and recall, and can effectively identify fire and smoke events. This research not only provides a new solution for fire smoke monitoring, improves the intelligence level of road traffic fire detection, but also lays the foundation for the future development of traffic safety management and intelligent monitoring systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 It is a process framework diagram of the present invention.

[0043] Figure 2This is a comparison chart of the average accuracy of the present invention and other detection methods.

[0044] Figure 3 This is a comparison chart of the recall rates of the present invention and other detection methods.

[0045] Figure 4 It is a comparison chart of the detection results of different fire scenes by the present invention and other detection methods. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] The complete steps of the method of the present invention are as follows:

[0048] S1: Build an improved YOLOv8-SE model.

[0049] See also Figure 1 YOLOv8 itself does not have a built-in attention mechanism. Adding an attention mechanism can not only improve the feature selection capability, but also help the model focus on the target area more effectively in the feature map, suppress irrelevant information, and improve the accuracy of target detection, especially in scenes with complex backgrounds. It can also enhance detail capture. The attention mechanism can help the model better capture the details of objects, especially small objects or objects with complex shapes, by adaptively adjusting the weights in different channels or spaces. Therefore, in order to enhance the model's ability to capture fire smoke features under different scenes and conditions, the present invention adds the SE attention mechanism on the basis of the YOLOv8 model.

[0050] The present invention adds the SE module to the basic convolution module CBS, the feature fusion module C2F and the spatial pyramid pooling SPPF respectively. This hierarchical and modular design enables different feature extraction stages to autonomously adjust their feature weights, thereby more effectively improving the performance of the model in complex scenarios. This is different from the holistic or single-level SE introduction in the prior art. At the same time, by introducing SE after multiple modules, not only can the feature expression ability of each module be enhanced, but also the fusion effect of features at different levels can be optimized. This method can better adapt to different task requirements, so that the model has higher flexibility and adaptability in specific application scenarios.

[0051] The specific steps are as follows:

[0052] S11: In the backbone network of the YOLOv8 detection network, an attention module SE is added after the last basic convolution module CBS, the feature fusion module C2F, and the spatial pyramid pooling SPPF, and a total of three attention modules SE are added to the backbone network.

[0053] Since YOLOv8 has the problems of low detection accuracy and high missed detection rate when detecting flames and smoke through cameras, the network structure is modified by adding an attention mechanism to improve the detection accuracy and reduce the missed detection rate. SEAttention is a channel-based attention mechanism. The SE module enhances the representation ability of the convolutional neural network by adaptively adjusting the importance of each feature channel. The added SE (Squeeze-and-Excitation) attention mechanism extracts feature maps from each layer of the network through convolution operations through feature extraction and global information compression, and then performs global average pooling on each feature map channel through the "Squeeze" process in the SE module. This operation compresses the two-dimensional feature map into a scalar, representing the global feature information of the channel. This step simplifies each channel into a global feature, which is convenient for subsequent weight calculation. Finally, adaptive weight generation is performed. The compressed global feature vector passes through two fully connected layers and is processed by a nonlinear activation function to generate the weight of each channel. These weight values ​​are normalized by the Sigmoid function to represent the importance of each channel in the convolutional features of this layer. This step ensures that the network can adaptively select more important channel features and ignore irrelevant or interfering information.

[0054] The working principle of the SE module mainly includes two steps: "Squeeze" and "Excitation".

[0055] S111: In the SE module, global average pooling is first performed on each channel of the input feature map to capture the global spatial information. This step compresses the two-dimensional feature map of each channel into a scalar, representing the global information of the channel as a whole. The calculation formula is:

[0056]

[0057] Among them, i and j represent the index of the height and width of the feature map, and u c (i, j) represents the pixel value of channel c at position (i, j), H and W are the height and width of the feature map, and z c is the global information of the cth channel, which is a scalar obtained by global average pooling of all pixel values ​​of the channel;

[0058] S112: Use a two-layer neural network consisting of a fully connected layer to generate the weight of each channel through the Sigmoid function. These weights represent the importance of the channel in the current task, and the calculation formula is:

[0059] s c =σ(W2δ(W1z c )) (2)

[0060] Among them, W1 and W2 are the weight matrices of the two fully connected layers, δ represents the ReLU activation function, which introduces nonlinear characteristics so that the model can learn more complex patterns; σ is the Sigmoid activation function, which compresses the output to the interval [0,1] as the weight of each channel, and the generated s c is the weight of channel c.

[0061] S113: Feature reweighting: Multiply these weights by the input feature map channel by channel, adjust the feature representation of each channel, and obtain a new feature map. The calculation formula is:

[0062]

[0063] in, is the adjusted feature map of the cth channel, s c is the weight of the cth channel, X C is the original feature map of the cth channel.

[0064] S12: In the YOLOv8 head network, the CIoU loss function is replaced by the MPDIoU loss function.

[0065] The present invention adopts a loss function that is highly related to a specific task or data distribution. This targeting can more effectively optimize model performance. For example, if applied to target detection, a loss function that considers bounding box accuracy and category imbalance can be introduced. This refined loss function design is often based on a deep understanding of the task itself, rather than just a simple replacement of the existing loss function. In addition, when replacing the loss function, it is necessary to consider multiple performance indicators (such as accuracy, recall, etc.), rather than just focusing on a single indicator. This multi-dimensional optimization strategy can make the model more robust in practical applications, while many loss functions in the prior art are still oriented towards a single goal.

[0066] YOLOv8 mainly uses CIoU (Complete Intersection over Union) loss as its bounding box loss function, combining error calculations in multiple dimensions to improve the positioning effect of the target box. However, CIoU has the following shortcomings: lack of multi-view distance measurement, unable to capture geometric features in more dimensions; poor optimization performance when the overlapping area is small; insufficient optimization of bounding boxes for shapes, rotations, and complex scenes.

[0067] Therefore, in dealing with complex scenes, such as fire and smoke, where geometric changes are large or high-precision bounding box matching is required, the MPDIoU loss function is introduced. The MPDIoU loss function is a loss function that integrates multiple bounding box measurement information. On the basis of the traditional IoU loss, it combines the comprehensive information of the distance, shape, and overlapping area between the target box and the prediction box, and has better optimization capabilities. Its core idea is to quantify the degree of matching between the target box and the prediction box from multiple angles.

[0068] First, the MPDIoU loss function not only considers IoU, but also the distance between two boxes. The position of the box can be further optimized by minimizing the distance between the upper left corner and the lower right point between the target box and the predicted box.

[0069] Secondly, the MPDIoU loss function proposes to measure the distance between the target box and the prediction box from multiple perspectives (such as the distance between the four corner points, the distance between the center point, etc.), and use this additional information to improve the accuracy of bounding box matching. These multi-perspective measurements can capture some information that IoU cannot measure, especially when the target boxes have almost no overlap or the overlapping area is very small, it can give better feedback to optimize the model.

[0070] In addition, the MPDIoU loss function also takes into account the difference in aspect ratio between the target box and the predicted box to ensure that not only the center positions of the boxes coincide, but also the shapes are as consistent as possible.

[0071] The calculation formula of MPDIoU loss function is as follows:

[0072]

[0073] Among them, S represents a set, which is R n A subset of ; R represents the real number set, including all real numbers; n represents the dimension;

[0074] w and h are the length and width of the input feature image of the loss function, Indicates the coordinates of the upper left corner and lower right corner of A. represents the coordinates of the upper left corner and lower right corner of B, A represents the predicted box, B represents the real box, A∩B represents the intersection area of ​​the predicted box and the real box, A∪B represents the union area of ​​the predicted box and the real box, d1 represents the distance between the center point of the real box and the center point of the predicted box, d2 represents the distance between the projection point of the predicted box and the boundary of the target box, that is, the distance between the projection points of the predicted box and the target box on their respective boundary projection lines.

[0075] S2: Build a dataset and perform model training.

[0076] S21: In fire and smoke detection tasks, datasets are crucial. An excellent dataset should have the following characteristics: First, it should contain various types of flame and smoke images and videos, covering different scenarios, such as factory fires, forest fires, building fires, and vehicle fires, to ensure that the model can effectively identify in various situations. Secondly, the dataset requires high-quality annotation information, including the location, size, and color of flames and smoke, to support the accurate identification and positioning of the model. In addition, the data should be captured by a variety of cameras and devices so that it can be verified in different environments. Finally, the dataset needs to be strictly screened to ensure the quality and reliability of the data, so as to facilitate model training and evaluation.

[0077] The data in this paper mainly comes from Google, Sogou, Baidu and other websites obtained through web crawler tools. The original data collected initially includes 4507 flame and smoke images.

[0078] S22: Since the automatic collection of crawlers may result in duplicate images, high similarity, low resolution or irrelevant data, image deduplication tools are used for cleaning, and low-resolution and irrelevant images also need to be manually deleted. Finally, the data set is divided into: training set (Train), validation set (Val), and test set (Test). The original images are stored in this category. The training and validation sets are used for model training, and the test set is mainly used to detect whether the model is overfitting and adjust the training parameters.

[0079] This task belongs to supervised learning, which requires annotation information to guide the model so that it can clearly understand the direction or goal of learning. However, the original image data obtained by the crawler does not contain annotations, so these images need to be manually annotated. Since this is an object detection task, annotation tools must be used to mark the objects of interest (flames and smoke) and define their bounding box positions and categories in the image. This article uses the labelimg tool to annotate image data. The annotated data represents the true value of an instance, including the category, the normalized horizontal and vertical coordinate values ​​of the center of the bounding box, and the normalized width and height of the target box.

[0080] S23: Train the model using the training set.

[0081] S3: Use the trained model for fire and smoke detection.

[0082] In addition, the present invention not only focuses on improving the performance of the model, but also takes into account the user experience in practical applications. In order to facilitate user operation, the present invention develops a visual interface that supports model training, detection and result display. Users can upload and detect images and videos through this interface, view the detection results in real time, and export relevant data. This functional design not only improves the usability of the system, but also provides a convenient technical means for traffic safety management.

[0083] Model performance evaluation:

[0084] The mean average precision (mAP), F1 score, and accuracy are used to evaluate the model performance. The calculation formula is as follows:

[0085]

[0086] Among them, K is the total number of categories, AP i is the average accuracy of the i-th category, P represents the precision, and R represents the recall;

[0087] TP represents true positive examples, that is, the number of times the true category of the sample is a positive example and the model predicts it to be a positive example; TN represents the number of times the true category of the sample is a negative example and the model predicts it to be a negative example; FP represents false positive examples, that is, the number of times the true category of the sample is negative but the model predicts it to be positive; FN represents false negative examples, that is, the number of times the true category of the sample is a positive example but the model predicts it to be a negative example.

[0088] Model comparison:

[0089] Training parameter settings:

[0090] The YOLOv8n and the improved YOLOv8-SE model of the present invention are compared. The same data set and training parameter settings are used to compare the specific performance of the two. The training parameters are shown in Table 1:

[0091] Table 1

[0092] parameter value Image-size 640*640 Batch-size 16 Epochs 300 Leaning-rate 0.01

[0093] This paper evaluates the performance of our proposed YOLOv8-SE and compares it with other single-stage detectors including YOLOv5, YOLOv6, and the original YOLOv8. Figure 2 and Figure 3 The results in the figure show that YOLOv8-SE outperforms YOLOv8 in terms of MAP_50 and recall, which are 0.596 and 0.568 respectively. These two indicators are 0.3% and 0.3% higher than YOLOv8.

[0094] Figure 4 The test results of the YOLOv8-SE model of the present invention and other algorithms in different fire scenes show that compared with the YOLOv5 and YOLOv6 models, the YOLOv8 model shows higher accuracy in the detection of fire smoke and can more effectively detect the fire smoke area. The YOLOv8-SE method proposed in this study is further optimized on the basis of YOLOv8. By introducing the attention mechanism, it significantly improves the attention to the edge area of ​​the smoke, making the detection frame more accurate, especially in the positioning of the boundary box of the fire smoke. Figure 4 For YOLOv5 and YOLOv6: In the fire scene in the third column, YOLOv5 and YOLOv6 failed to detect the target in the fire smoke area, indicating that these two models are insufficient in identifying low contrast. YOLOv8-SE significantly improves the sensitivity to subtle smoke areas through the improved attention mechanism, enabling it to accurately detect the boundaries of smoke. For YOLOv8: In the fire scene in the first column, the number of bounding boxes detected by YOLOv8 is relatively small, and some smoke areas are not accurately located. This shows that YOLOv8 does not pay enough attention to the smoke boundaries when encountering complex fire scenes. The YOLOv8-SE model enhances the ability to capture edge area features by adding the SE attention mechanism, ensuring more accurate detection of smoke boundaries and avoiding missed detection. In summary, the YOLOv8-SE model has higher detection accuracy and bounding box positioning accuracy than YOLOv5, YOLOv6 and YOLOv8 in fire smoke detection, especially in detecting difficult scenes (such as low contrast, complex smoke edges, etc.). This further verifies the effectiveness of the YOLOv8-SE model proposed in this study in the fire smoke detection task.

[0095] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0096] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A road traffic fire smoke detection method, characterized in that: The following steps are involved: S1: Build the YOLOv8-SE model; S2: Build a dataset and perform model training; S3: Use the trained model for fire smoke detection.

2. A road traffic fire smoke detection method according to claim 1, characterized in that: The S1 includes: S11: In the backbone network of YOLOv8, an attention module SE is added after the last basic convolution module CBS, feature fusion module C2F, and spatial pyramid pooling SPPF; S12: In the head network of YOLOv8, the loss function of the detection head is replaced by the MPDIoU loss function.

3. A road traffic fire smoke detection method according to claim 2, characterized in that: The S11 includes: S111: In the attention module SE, global average pooling is performed on each channel of the input feature map; the calculation formula is: Among them, i and j represent the index of the height and width of the feature map, and u c (i, j) represents the pixel value of channel c at position (i, j), H and W are the height and width of the feature map, and z c is the global information of the cth channel; S112: Use a neural network composed of fully connected layers to generate the weight of each channel through the Sigmoid function; the calculation formula is: s c =σ(W2δ(W1z c )) (2) Among them, W1 and W2 are the weight matrices of the two fully connected layers, δ represents the ReLU activation function; σ is the Sigmoid activation function, s c is the weight of channel c; S113: Multiply the weights by the input feature map channel by channel, adjust the feature representation of each channel, and obtain a new feature map. The calculation formula is: in, is the feature map of the cth channel, X C is the original feature map of the cth channel.

4. A road traffic fire smoke detection method according to claim 2, characterized in that: The S12 includes: In the detection head, the calculation formula of the MPDIoU loss function is as follows: Among them, w and h are the length and width of the input feature image of the loss function, Indicates the coordinates of the upper left corner and lower right corner of A. represents the coordinates of the upper left corner and lower right corner of B, A represents the predicted box, B represents the real box, A∩B represents the intersection area of ​​the predicted box and the real box, A∪B represents the union area of ​​the predicted box and the real box, d1 represents the distance between the center point of the real box and the center point of the predicted box, d2 represents the distance between the projection point of the predicted box and the boundary of the target box, that is, the distance between the projection points of the predicted box and the target box on their respective boundary projection lines.

5. A road traffic fire smoke detection method according to claim 1, characterized in that: The S2 includes: S21: Build dataset; S22: preprocess and annotate the data set, and then divide it into training set, validation set, and test set; S23: Train the model using the training set.

6. A road traffic fire smoke detection device, characterized in that: Road traffic fire smoke detection is achieved by using the method described in any one of claims 1 to 5.

7. A road traffic fire smoke detection storage device, characterized in that: Road traffic fire smoke detection is achieved by using the method described in any one of claims 1 to 5.

Images