Flame smoke detection method and system based on improved YOLOv5s

By fusing the coordinate attention mechanism, CSPCM module and bidirectional weighted feature pyramid network in the YOLOv5s model, replacing the loss function and activation function, the challenge of small-scale object detection in flame smoke detection is solved, the detection accuracy and stability are improved, and efficient Flame smoke recognition.

CN119027863APending Publication Date: 2024-11-26ZHEJIANG NORMAL UNIV
View PDF 0 Cites 7 Cited by

Patent Information

Application Number
CN202411064384.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-05
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The existing flame smoke detection method is affected when the background color is similar to the flame smoke color. The shape and motion-based methods are relatively stable under different lighting conditions and are difficult to capture the motion characteristics of flame smoke. Single-stage target detection Algorithms such as YOLO have challenges in small object detection.

Method used

Improve the YOLOv5s model by fusing coordinate attention mechanism in the backbone network, adding a CSPCM module composed of mixed convolutional layer and ordinary convolutional layer, adopting a bidirectional weighted feature pyramid network instead of the feature fusion module, replacing the CIOU loss function as MPDIoU, and using ReLU to activate it. Functions to improve the model's detection ability and detection accuracy of small objects.

Benefits of technology

The model's perception of image features is enhanced, the accuracy and detection accuracy of small object detection is improved, and the efficient flame smoke recognition in different scenarios is achieved to meet the real-time detection needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027863A_ABST
    Figure CN119027863A_ABST
Patent Text Reader

Abstract

The invention discloses a flame smoke detection method and system based on improved YOLOv5s, and belongs to the technical field of artificial intelligence, and the method comprises the steps: obtaining flame smoke data in different scenes; the YOLOv5s model is improved; and carrying out iterative training on the improved YOLOv5s model by using the data set, carrying out test and validity verification on the trained improved YOLOv5s model, and carrying out flame smoke identification by using the improved YOLOv5s model screened out by the test and validity verification to obtain an identification result. According to the invention, by improving the YOLOv5s model, the detection precision is improved, the performance and generalization ability of the model are improved, the convergence of the network model is accelerated, the missed detection and false detection of the YOLOv5s on small target flame smoke in different scenes are reduced, and the overall detection precision of the flame smoke is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more specifically to a flame and smoke detection method and system based on improved YOLOv5s. Background Art

[0002] In daily life and work, fire is a very harmful disaster. Timely detection of fire is the key to taking timely remedial measures. Therefore, how to accurately detect flame smoke is of great research significance.

[0003] The mainstream methods for flame smoke detection include methods based on color and texture features, methods based on shape and motion, and methods based on machine learning. Among them, the methods based on color and texture features detect flames through thresholds or color space conversion, but when the background color is similar to the color of the flame smoke, it will interfere with the RGB extraction of the flame smoke, thereby affecting the accuracy of detection. Since light usually has little effect on shape and motion features, shape and motion-based methods are more stable under different lighting conditions than color-based methods. However, the movement of flame smoke is usually affected by factors such as wind and air flow, making its motion features unstable or difficult to capture, thereby affecting the accuracy of detection. Due to its end-to-end learning ability, the advantages of representation learning, strong generalization ability and multi-task processing ability, the method based on deep learning has higher accuracy and adaptability in flame smoke detection tasks than the first two methods.

[0004] The flame and smoke detection algorithms based on deep learning are divided into two categories: single-stage target detection algorithms and two-stage target detection algorithms. Among them, the two-stage target detection algorithm first generates the candidate area, and then classifies and regresses each candidate box. Its main representative algorithms are R-CNN, Fast R-CNN, Faster R-CNN, etc. The mainstream algorithms for single-stage target detection are SSD and YOLO, which directly process the entire image, simplify the detection process, and improve the detection speed. YOLO can effectively handle multiple overlapping targets and perform accurate and precise measurements. In the flame and smoke detection task, Lei Jingsheng et al. integrated the CBAM attention mechanism to enhance the representation ability of feature maps at different layers and improve the attention to features at different scales. Wang Yixu et al. introduced the 3-D attention mechanism SimAM into the network, which significantly enhanced the feature extraction ability of the algorithm. In order to enable the deep network to directly obtain shallow features, Song Huawei et al. updated the original path aggregation network module to a bidirectional cross-scale fusion module to enhance the information transmission capability. Although the bidirectional cross-scale fusion module can enhance the network feature fusion capability, how to effectively transmit information and fuse features to achieve high-precision detection of flames and smoke is still a challenge. Summary of the invention

[0005] In view of this, the present invention provides a flame and smoke detection method and system based on improved YOLOv5s, which improves the detection effect of small targets by improving the model.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] On the one hand, the present invention provides a flame smoke detection method based on improved YOLOv5s, comprising the following steps:

[0008] S1. Obtain fire and smoke data in different scenarios and construct a data set;

[0009] S2. Improve the YOLOv5s model to obtain an improved YOLOv5s model;

[0010] S3. Iteratively train the improved YOLOv5s model using the data set, and test and verify the trained improved YOLOv5s model to obtain the optimal improved YOLOv5s model;

[0011] S4. Obtain image data in the detection scene, input the image data into the optimal improved YOLOv5s model, and obtain the flame and smoke recognition result.

[0012] Preferably, the YOLOv5s model is improved, specifically including:

[0013] Integrate the coordinate attention mechanism in the C3 module of the backbone network;

[0014] Add a CSPCM module consisting of a hybrid convolutional layer and a normal convolutional layer to the backbone network;

[0015] Use a bidirectional weighted feature pyramid network to replace the feature fusion module at the neck of the YOLOv5s model;

[0016] The CIOU loss function in the YOLOv5s model is replaced with the MPDIoU loss function, and the SiLU activation function is replaced with the ReLU activation function.

[0017] Preferably, the coordinate attention mechanism is integrated into the C3 module of the backbone network, specifically including:

[0018] Decompose the global average pooling in the X and Y dimensions to generate two feature maps of size C*1*W and C*H*1;

[0019] The two feature maps are fused, and the fused feature maps are convolved and activated to reduce the dimension, generating a feature map of size C / r*1*(W+H).

[0020] Preferably, a CSPCM module consisting of a hybrid convolutional layer and a normal convolutional layer is added to the backbone network, specifically comprising:

[0021] The hybrid convolution layer divides the feature map of the input CSPCM module into g groups of different feature maps in the channel dimension, and performs depth-separable convolution operations on the g groups of different feature maps, wherein the convolution kernel size of each group of depth-separable convolution operations is different;

[0022] The convolution layer concatenates the feature maps obtained through the depthwise separable convolution operation, and uses a point-by-point convolution operation to obtain the final output feature map.

[0023] Preferably, comparative experiments and ablation experiments are used to verify the effectiveness of the trained improved YOLOv5s model.

[0024] On the other hand, the present invention provides a flame and smoke detection system based on improved YOLOv5s, comprising:

[0025] The dataset module is used to obtain fire and smoke data in different scenarios and construct datasets;

[0026] A model building module is used to improve the YOLOv5s model to obtain an improved YOLOv5s model;

[0027] An optimization module, used to iteratively train the improved YOLOv5s model using the data set, and test and verify the trained improved YOLOv5s model to obtain an optimal improved YOLOv5s model;

[0028] The recognition module is used to obtain image data in the detection scene, input the image data into the optimal improved YOLOv5s model, and obtain the flame and smoke recognition result.

[0029] Preferably, the improved YOLOv5s model includes:

[0030] The coordinate attention mechanism is integrated into the C3 module of the backbone network, and a CSPCM module consisting of a hybrid convolutional layer and a normal convolutional layer is added;

[0031] The neck uses a bidirectional weighted feature pyramid network to replace the feature fusion module of the YOLOv5s model;

[0032] The CIOU loss function in the YOLOv5s model is replaced with the MPDIoU loss function, and the SiLU activation function is replaced with the ReLU activation function.

[0033] Through the above technical solutions, it can be known that compared with the prior art, the present invention discloses a flame smoke detection method and system based on improved YOLOv5s. First, by adding the CA attention mechanism, the model can capture the information of direction perception and position perception while capturing cross-channel information, enhance the relationship between different channels in the feature map, make the representation ability of each layer of feature map richer, and enhance the model's perception of image features. Secondly, a new module CSPCM composed of a convolutional layer and a mixed deep convolutional layer is inserted into the backbone network, which can perform multi-level and multi-scale feature extraction on the input image. Furthermore, based on the feature fusion operation of BiFPN, the information contained in the feature map is enriched, and the model's detection ability for small targets is increased, thereby improving the detection accuracy of the model. Finally, MPDIoU is used as the bounding box loss function, which can improve the training effect of the bounding box regression. Using ReLU as the activation function performs well in solving the gradient vanishing problem and is more suitable for flame smoke detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0035] Figure 1 A flowchart of a flame and smoke detection method based on improved YOLOv5s provided by the present invention;

[0036] Figure 2 This is the architecture diagram of the traditional YOLOv5s;

[0037] Figure 3 The architecture diagram of the improved YOLOv5s provided by the present invention;

[0038] Figure 4(a) is a detection effect diagram of the traditional YOLOv5s model, and Figure 4(b) is a detection effect diagram of the improved YOLOv5s model;

[0039] Figure 5 This is a framework diagram of the flame and smoke detection system provided by the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] The embodiment of the present invention discloses a flame smoke detection method based on improved YOLOv5s, referring to Figure 1 , including the following steps:

[0042] S1. Obtain fire and smoke data in different scenarios and build a dataset.

[0043] The constructed data set includes a training set, a test set and a validation set. The present invention uses the Fire and Smoke data set to obtain data. The Fire and Smoke data set comes from the Roboflow data set website and is a platform focused on computer vision. This platform simplifies the process from data annotation to model training. It allows developers to upload, download, annotate and convert their own data sets and models more conveniently. This platform is also officially recommended by YOLOv8 and is a data set management platform that can support many types of data formats and model architectures. This platform also provides users with a very diverse model library and free data sets. Users can easily obtain data sets in the corresponding data formats required for training models on this platform without having to convert the data format themselves. The overall data set has the following categories: Fire, Smoke.

[0044] S2. Improve the YOLOv5s model to obtain an improved YOLOv5s model;

[0045] S3. Iteratively train the improved YOLOv5s model using the data set, and test and verify the trained improved YOLOv5s model to obtain the optimal improved YOLOv5s model;

[0046] S4. Obtain image data in the detection scene, input the image data into the optimal improved YOLOv5s model, and obtain the flame and smoke recognition result.

[0047] YOLOv5 (You Only Look Once) was first proposed by Joseph Redmon and developed by the Ultralytics team. It is a one-stage target detector that can directly predict bounding boxes and category probabilities in a single forward pass. It is fast and suitable for real-time applications. YOLOv5 provides five scaled versions: YOLOv5n (nano), YOLOv5s (small), YOLOv5m (medium), YOLOv5l (large) and YOLOv5x (extra large). The YOLOv5 model is more lightweight, saving storage space while reducing the computational burden of the network. It is a further improved version of YOLOv4. Compared with the previous version, YOLOv5 has been improved in the following aspects:

[0048] 1. Model lightweight and high efficiency: YOLOv5 achieves model lightweight by simplifying the model structure and optimizing the network design. For example, YOLOv5 uses fewer network layers and parameters, but still maintains high detection accuracy. This lightweight design allows YOLOv5 to significantly reduce the model's storage requirements and computational burden while maintaining high performance.

[0049] 2. Improved training strategy and data enhancement: YOLOv5 introduces an improved training strategy, including more effective data enhancement techniques and enhanced dataset processing methods. These improvements help improve the generalization and robustness of the model, making the model more stable and reliable in different scenarios.

[0050] 3. Use more advanced feature extractors: YOLOv5 has optimized the selection of backbone network and used more advanced feature extractors, such as CSPDarknet53, which helps to improve the model's ability to capture and express image features.

[0051] 4. Faster reasoning speed: Although YOLOv5 maintains high detection accuracy, it is optimized in terms of reasoning speed. Through improved network design and computational optimization, YOLOv5 is able to achieve faster real-time detection speed, which is suitable for application scenarios that require fast response.

[0052] 5. Support for multiple detection scales: YOLOv5 improves the detection capability of targets of different scales, enabling the model to effectively handle targets of different sizes and proportions. This improvement makes YOLOv5 more widely used and practical in complex scenarios.

[0053] YOLOv5s is mainly composed of a backbone network (Backbone), a neck network (Neck) and a detection head (Head). Its model structure is as follows Figure 2 shown.

[0054] The role of the backbone network is to extract network features, that is, to convert the original input image into a multi-layer feature map. In YOLOV5s, Backbone uses a cross-class connection method to improve the efficiency of information flow and network effectiveness. The use of the CSP structure can effectively improve the feature extraction ability of the model.

[0055] The Neck layer is responsible for combining the feature maps between different levels extracted by Backbone to generate feature maps with multi-scale information, thereby improving the accuracy of target detection. In YOLOV5s, the Neck part combines the top-down FPN feature propagation method and the bottom-up PAN feature propagation method to process targets of different scales and sizes.

[0056] The head detection head is mainly composed of convolutional layers, pooling layers and fully connected layers, which are responsible for multi-scale target detection using the feature maps extracted by the backbone network.

[0057] Furthermore, in order to improve the model's detection accuracy for flame smoke, this application improves the YOLOv5s model, specifically including:

[0058] (1) The coordinate attention mechanism is integrated into the C3 module of the backbone network, denoted as C3_CA, to capture feature information in different dimensions.

[0059] In computer vision tasks, the attention mechanism is introduced to improve the model's attention to the areas worth noting in the input image and ignore the unimportant parts of the input image. The attention mechanism is usually divided into channel attention mechanism, spatial attention mechanism, temporal attention mechanism and branch attention mechanism. In practical applications, different attention mechanisms are combined to improve the detection ability of the model.

[0060] Since the CA (Coordinate Attention) attention mechanism integrates position information into the channel attention mechanism, it can capture direction perception and position perception information while capturing cross-channel information. In FS-YOLO, the C3 structure of the backbone network is improved, and the CA attention mechanism is integrated into each C3 module of the backbone network to enhance the relationship between different channels in the feature map, making the representation ability of each layer of feature map richer and enhancing the model's perception of image features.

[0061] (2) A CSPCM module consisting of a hybrid convolutional layer and a normal convolutional layer is added to the backbone network to improve the detection capability of the model.

[0062] In order to better extract and transform feature information, convolution and mixed deep convolution layer (MixConv) are combined to form a new module, named CSPCM. Its structural design enables multi-level and multi-scale feature extraction on the input image, which is effective in target detection tasks.

[0063] (3) Use a bidirectional weighted feature pyramid network to replace the feature fusion module at the neck of the YOLOv5s model.

[0064] In YOLOv5s, the feature fusion module uses the FPN+PAN structure. FPN combines feature maps through up and down sampling to generate a multi-scale feature pyramid to improve the recognition ability of small targets. FPN uses a top-down feature propagation method to fuse the feature values ​​of the high-level and low-level layers to obtain a new prediction feature map. PAN fuses feature maps of different levels to ensure that the information contained in the feature maps from each level can be fully utilized in the network. PAN uses a bottom-up feature propagation method to better achieve semantic segmentation effects with less computing resources, but the FPN+PAN structure does not perform well in smoke detection tasks.

[0065] In order to enhance the model's ability to fuse features of flames and smoke, the feature fusion module of the Neck part is replaced with the BiFPN structure. BiFPN is improved on the basis of FPN and fuses multi-scale features from P1 layer to P5 layer in a top-down path.

[0066] The traditional feature fusion operation simply superimposes the input feature maps without distinguishing the different features of the input. In the actual training process, different feature maps often have different resolutions, so the contribution to the feature map in the feature fusion process is also different. Therefore, the feature map obtained by simply superimposing them is not optimal. BiFPN uses a weighted feature fusion mechanism to ensure that the network can learn different input features and distinguish and combine different input features. Fast normalized fusion normalizes the input feature map to ensure that the values ​​of different input images or features will not have an unreasonable impact on the fusion result. By improving the original feature fusion of the Neck part to a feature fusion operation based on BiFPN, the information contained in the feature map is enriched, and the model's detection ability for small targets is increased, thereby improving the detection accuracy of the model.

[0067] By improving the original feature fusion of the Neck part into a feature fusion operation based on BiFPN, the information contained in the feature map is enriched, the model's detection ability for small targets is increased, thereby improving the detection accuracy of the model.

[0068] (4) Replace the CIOU loss function in the YOLOv5s model with the MPDIoU loss function, and replace the SiLU activation function with the ReLU activation function.

[0069] In the YOLOv5 algorithm, the loss function is usually composed of three parts: classification loss, location loss and confidence loss. Among them, the location loss is used to describe the gap between the predicted value and the actual value of the bounding box by the algorithm, and the intersection over union (IoU) is used to represent the ratio of the intersection and union of the target box and the real box. In the YOLOv5 algorithm, the location loss function used is the CIoU loss function improved on the basis of IoU. The CIoU calculation process involves many parameters, which may reduce the performance of the loss function and make the model unable to converge quickly. In order to obtain a faster convergence speed and more accurate regression effect for the model, the present invention uses a bounding box similarity comparison metric loss function MPDIoU based on the minimum point distance, which directly minimizes the distance between the upper left corner and the lower right corner of the predicted bounding box and the actual bounding box, which can improve the training effect of bounding box regression, thereby improving the detection accuracy.

[0070] ReLU (Rectified Linear Unit) performs well in solving the gradient vanishing problem. Since its gradient has only two possible values ​​0 or 1. Only when the input data is greater than 0, ReLU takes 1 and the neuron will not be saturated; when the input data is less than 0, ReLU takes 0. The activation function used in YOLOv5s is the SiLU activation function. Compared with the SiLU function, ReLU has simpler calculation steps and is more suitable for flame and smoke detection tasks. The ReLU function does not require exponential operations and has low computational complexity.

[0071] The improved algorithm is recorded as FS-YOLO (Fire Smoke-YOLO), and its network structure is as follows Figure 3 shown.

[0072] Furthermore, the coordinate attention mechanism is integrated into the C3 module of the backbone network, including:

[0073] Decompose the global average pooling in the X and Y dimensions to generate two feature maps of size C*1*W and C*H*1:

[0074]

[0075]

[0076] Among them, Z h c represents the output of the cth channel with height h, Z w c represents the output of the cth channel with width w, X c represents the input, W, H are the total width and height. (h,i) and (j,w) respectively encode each channel along the horizontal and vertical coordinate directions using pooling kernels of size (H,1) and (1,W).

[0077] The two feature maps are fused, and the fused feature maps are convolved and activated to reduce the dimension, generating a feature map of size C / r*1*(W+H). The specific operations are as follows:

[0078] f=δ(F1([z h ,z w ]))

[0079] Among them, F1 means transforming the feature map using 1*1 convolution, and δ means performing Concat operation on the transformed feature map. F1 is similar to the dimension reduction operation using convolution kernel in SE attention mechanism. The dimension of f is C / r*1*(W+H). The generated feature map of dimension f is split along the spatial dimension to obtain two feature maps of dimensions C*1*W and C*H*1. The convolution operation is used to increase the dimension, and then the Sigmoid activation operation is performed to obtain the final feature map:

[0080] g h =σ(F h (f h ))

[0081] g w =σ(F w (f w ))

[0082] Among them, σ represents the Sigmoid activation operation on the obtained feature map.

[0083] The CA attention mechanism comprehensively considers channel information and direction-related position information. It is highly flexible and lightweight, and can capture the feature information of the input image more accurately.

[0084] Preferably, a CSPCM module consisting of a hybrid convolutional layer and a normal convolutional layer is added to the backbone network, specifically comprising:

[0085] The hybrid convolution layer divides the feature map of the input CSPCM module into g groups of different feature maps in the channel dimension, and performs depth-wise separable convolution operations on the g groups of different feature maps, where the convolution kernel size of each group of depth-wise separable convolution operations is different;

[0086] The convolution layer concatenates the feature maps obtained by the depthwise separable convolution operation and uses point-by-point convolution operation to obtain the final output feature map.

[0087] Specifically, MixConv divides the input tensor of size (H, W, C) into g groups in the channel dimension. For any group of tensors, its output is calculated as follows:

[0088]

[0089] in, Represents the output calculation of the t-th group of tensors, and (x, y, z) represents the height, width, and number of channels of the t-th group of tensors, respectively. Represents the input of the t-th group of tensors, each group has the same spatial height and width, and the sum of the number of channels in each group is equal to the number of channels of the input tensor. m·c represents the number of channels of the output feature map, c represents the number of channels of the input feature map, and k*k represents the size of the convolution kernel.

[0090] The final output tensor is obtained by concatenating all groups, and the expression is as follows:

[0091]

[0092] Since MixConv uses convolution kernels of different sizes to replace the original convolution operation, the receptive field of the network is increased in the detection task, thereby improving the accuracy of the model in flame and smoke detection.

[0093] Preferably, comparative experiments and ablation experiments are used to verify the effectiveness of the trained improved YOLOv5s model.

[0094] On the other hand, the present invention provides a flame and smoke detection system based on improved YOLOv5s, such as Figure 5 As shown, including:

[0095] The dataset module is used to obtain fire and smoke data in different scenarios and construct datasets;

[0096] A model building module is used to improve the YOLOv5s model to obtain an improved YOLOv5s model;

[0097] The optimization module is used to iteratively train the improved YOLOv5s model using the data set, and to test and verify the trained improved YOLOv5s model to obtain the optimal improved YOLOv5s model;

[0098] The recognition module is used to obtain image data in the detection scene, input the image data into the optimal improved YOLOv5s model, and obtain the flame and smoke recognition results.

[0099] Preferably, the improved YOLOv5s model includes:

[0100] The coordinate attention mechanism is integrated into the C3 module of the backbone network, and a CSPCM module consisting of a hybrid convolutional layer and a normal convolutional layer is added;

[0101] The neck uses a bidirectional weighted feature pyramid network to replace the feature fusion module of the YOLOv5s model;

[0102] The CIOU loss function in the YOLOv5s model is replaced with the MPDIoU loss function, and the SiLU activation function is replaced with the ReLU activation function.

[0103] For the improved YOLOv5s model, the Windows 11 64-bit operating system was used, the Python version was 3.8.19, the PyTorch version was 2.0.1, the cuda version was 11.7, the CPU version was 13th Gen Intel(R) Core(TM) i5-13600KF, the memory was 32G, and the GPU version was NVIDIA GeForce RTX 3080Ti, with a video memory of 12G. The specific experimental parameters are shown in Table 1.

[0104] Table 1 FS-YOLO experimental parameter configuration table

[0105] Epochs 150 Input_Size 640 Batch_Size 16 Optimizer Adam lr0 0.001

[0106] The accuracy (Precision), recall rate (Recall), mean average precision when IoU is 0.5 (mAP@0.5), mean average precision when the intersection over union ratio is between 0.5 and 0.95 (mAP@0.5:0.95), model parameters (Parameters) and GFLOPs are used as evaluation indicators of the algorithm.

[0107] The precision is represented by P, R represents the recall rate, AP represents the average precision, and mAP represents the mean average precision of IoU under the feature value. The calculation formula is as follows:

[0108]

[0109]

[0110]

[0111]

[0112] In order to evaluate the performance of FS-YOLO, this embodiment conducts comparative experiments on the Fire and Smoke dataset. The experimental results are shown in Table 2.

[0113] Table 2 Comparative experiments of different models on the fire and smoke dataset

[0114]

[0115]

[0116] Faster-RCNN, Cascade R-CNN, and Mask-RCNN, as classic two-stage target detection models, achieved mAP@0.5:0.95 detection results of 0.131, 0.135, and 0.12, respectively, and were able to process and display 47, 34, and 36 frames of images per second, respectively. However, due to their huge computational complexity and low detection frame rate, they were unable to meet the real-time detection standards in the flame and smoke detection task. SSD, as a single-stage target detector, achieved mAP@0.5:0.95 of 0.137 and 0.145, respectively, when the input image size was 300 pixels and 512 pixels, which was an improvement over the two-stage target detection algorithm. However, the image processing speed of the SSD algorithm was slower.

[0117] The YOLO series of algorithms can achieve a high recognition accuracy rate with a small number of parameters. At the same time, its FPS is significantly higher than that of the SSD algorithm, CenterNet and FCOS algorithm. YOLOv5n significantly improves the model's image processing speed with its small number of parameters and calculations, achieving 357 frames of images per second, which can meet the standard of real-time detection. However, compared with the baseline model YOLOv5s, its detection accuracy is low and it cannot effectively identify flame smoke instances in the scene. YOLOv8n does not rely on predefined anchor boxes and directly predicts the location of flame smoke in the input image, achieving a mAP@0.5:0.95 detection effect of 0.266, which is a significant improvement over other algorithms. However, YOLOv8n processes 127 frames of images per second, which is not the best choice for real-time fire detection. Although YOLOv8s can process 417 frames of images per second and performs well in real-time detection, its mAP@0.50:0.95 is 2.76% lower than the baseline model in terms of detection accuracy, and it cannot achieve high-precision detection.

[0118] The mAP@0.5:0.95 index of YOLOv9c in this experiment dropped by 1.97% compared with the baseline model YOLOv5s. The reason for the negative growth may be that YOLOv9 has a low detection accuracy for larger targets. Due to the influence of various external factors and the non-fixed shape and size of the flame smoke, the shape of the detected target has a large uncertainty, which makes YOLOv9c prone to missed detection and false detection.

[0119] YOLOv10 is the latest target detector in the current YOLO algorithm. Its two variants, YOLOv10n and YOLOv10s, achieved mAP@0.5:0.95 detection results of 0.264 and 0.266 in the flame and smoke detection task, which increased by 3.9% and 4.7% compared with the baseline model YOLOV5s. Although YOLOv10 performed well in the experiment, compared with FS-YOLO, YOLOv10's performance in P, R, mAP@0.5, and mAP@0.5:0.95 indicators was not outstanding enough, and it could not accurately identify flames and smoke in complex scenes, and did not reach the expected performance level.

[0120] FS-YOLO has the highest detection accuracy in the comparative experiment. On the validation set, mAP@0.50 reaches 0.606, which is 4.84% higher than the baseline model YOLOv5s, and has the highest recognition accuracy in the YOLO series of algorithms. Its mAP@0.5:0.95 index reaches 0.275, which is 8.27% higher than the baseline model YOLOv5s. FS-YOLO can process 303 frames of images per second, which can achieve relatively reliable real-time detection, with good robustness and high recognition accuracy.

[0121] Table 3 Comparative experiments on different improvements of the backbone network

[0122]

[0123] In order to achieve higher detection accuracy for FS-YOLO, different improvements were made to the backbone network under the condition of replacing the Concat of the Neck part with BiFPN, using the same activation function ReLU and loss function MPDIoU, and comparative experiments were set up. The experimental results are shown in Table 3. Analysis of the experimental data shows that after integrating the CA attention mechanism into the backbone network and adding the CSPCM layer, FS-YOLO achieved 0.606mAP@0.5 and 0.275mAP@0.5:0.95. Compared with the baseline algorithm, it showed different degrees of improvement in mAP@0.5 and mAP@0.5:0.95 indicators, specifically, an increase of 3.06% and 6.59% respectively. This shows that after adding the CA attention mechanism and CAPCM module to the backbone network, FS-YOLO can effectively enhance the representation ability of features, which helps the model to more comprehensively consider and utilize feature information at different levels.

[0124] Table 4 Comparative experiments of different loss functions

[0125]

[0126] In order to make FS-YOLO converge quickly during training, comparative experiments with different loss functions were set up. By comparing the experimental results in Table 4, it is found that when the model uses the same activation function and network structure, the MPDIoU loss function performs best in the flame smoke detection task, with an accuracy of 0.616, a recall of 0.585, mAP@0.5 and mAP@0.5:0.95 of 0.606 and 0.275 respectively. Compared with other loss functions, it has the highest accuracy. Compared with the original CIoU, its mAP@0.5 increased by 4.3%, and mAP@0.5:0.95 increased by 3.5%. Experimental data show that MPDIoU can better adapt to complex environments in flame smoke detection tasks, and can effectively evaluate the difference between the predicted box and the actual box when dealing with overlapping targets.

[0127] Table 5 Comparative test of different activation functions

[0128]

[0129]

[0130] The same loss function MPDIoU is used on the improved FS-YOLO model to conduct comparative experiments on different activation functions. Analysis of the experimental data in Table 5 shows that the activation function used by the original YOLOv5s is the smooth SiLU activation function, which achieves a detection accuracy of 0.604 on mAP@0.5 and 0.273 on mAP@0.5:0.95. When using Sigmoid, Tanh, and LeakyReLU activation functions, FS-YOLO shows a decreasing trend in detection accuracy. Although the detection accuracy of the model is improved by using the nonlinear Hardswish activation function, which is 0.17% higher than that of the SiLU activation function, the experimental results show that the ReLU activation function shows better detection performance during training, which shows that under the current experimental conditions, the ReLU activation function can effectively improve the accuracy and stability of the model detection results.

[0131] When using the ReLU activation function to calculate the gradient, FS-YOLO's mAP@0.5 and mAP@0.5:0.95 are the highest, reaching 0.606 and 0.275 respectively. The experimental results show that the simple and efficient ReLU can make neurons more active and selective in responding to input features, effectively improving the stability and convergence speed of the model.

[0132] Table 6 Ablation experiment

[0133] CA CSPCM BiFPN P R mAP@0.5 mAP@0.5: 0.95 Parameters GFLOPs FPS 0.581 0.581 0.577 0.254 7.0 15.8 323 √ 0.594 0.584 0.588 0.264 7.4 17.3 286 √ 0.602 0.563 0.581 0.245 7.9 16.7 159 √ 0.610 0.586 0.588 0.258 7.3 16.8 313 √ √ 0.589 0.572 0.59 0.271 8.2 18.0 192 √ √ 0.610 0.575 0.587 0.262 7.4 17.3 196 √ √ 0.581 0.588 0.591 0.257 7.9 16.9 204 √ √ √ 0.616 0.585 0.606 0.275 8.0 17.8 303

[0134] To further evaluate the performance of each improved module of FS-YOLO, MPDIoU and ReLU were used to set up ablation experiments to evaluate it, and the results are shown in Table 6. Analysis shows that the original YOLOv5s network has a mAP@0.5 of 0.577 and a mAP@0.5:0.95 of 0.254. After improving the model, the CA attention mechanism is integrated in the C3 module of the backbone network of FS-YOLO, and then the CSPCM layer is added before SPPF, and the original feature fusion part of the Neck part is replaced with BiFPN. The mAP@0.5 and mAP@0.5:0.95 of the model have increased, which are 5.03% and 8.26% respectively compared with YOLOv5s. The experimental results show that after adding the CA attention mechanism, FS-YOLO can capture cross-channel information while capturing direction perception and position perception information, and effectively focus on the useful parts of the input features. After adding the CAPCM module, the model can perform multi-level and multi-scale feature extraction on the input image, increasing the network receptive field. After improving the feature fusion of the Neck part, the BiFPN module is responsible for using the weighted feature fusion mechanism for different feature information to ensure that the network can distinguish and combine different input features, which can effectively improve the detection accuracy of FS-YOLO for flame and smoke.

[0135] Figure 4 shows the detection of flames and smoke by FS-YOLO and YOLOv5s in different scenarios. Figure 4(a) is the detection effect diagram of the YOLOv5s model, and Figure 4(b) is the detection effect diagram of the FS-YOLO model. From the detection results, YOLOv5s has missed detection of small targets and smoke. Due to the introduction of multi-level feature fusion and cross-layer connection in the feature fusion stage, FS-YOLO makes up for this well and achieves more accurate detection of flames and smoke targets in the scene. From the detection results, it can be seen that FS-YOLO recognizes and locates flame and smoke images in different scenarios with higher reliability and confidence, reduces the deviation between the predicted frame and the actual frame, and shows a more accurate positioning capability for flames and smoke.

[0136] This embodiment mainly proposes a new model FS-YOLO for flame smoke detection tasks based on the YOLOv5s algorithm. In order to make the model converge quickly in the flame smoke detection task, the original loss function and activation function are modified, and the CSPCM module is introduced. The feature information is multi-scale fused by hybrid deep convolution and convolution, and the CA attention mechanism is integrated in all C3 modules of the backbone network, so that the model can obtain channel information while sensing position information. Finally, the feature fusion method of the neck network is improved by using the BiFPN idea to realize the weighted fusion of information, so that the model can process different feature information accordingly.

[0137] Experimental results on the Fire and Smoke dataset show that FS-YOLO has improved the mean average precision compared to the baseline. Specifically, the improvement of mAP@0.5 is 4.84%, while the improvement of mAP@0.5:0.95 is 8.27%, which can achieve high-precision detection in the fire and smoke detection task. At the same time, FS-YOLO processes 303 frames of images per second and can meet the standard of real-time detection when deployed on the device.

[0138] Subsequently, FS-YOLO will be lightweighted by using model compression and pruning techniques to remove unimportant connections or layers in the model, adjusting the model weights and activation values, and using knowledge distillation technology to transfer the knowledge of FS-YOLO, thereby reducing the number of model parameters and computing requirements while maintaining model performance, enabling efficient deployment at a lower resource cost.

[0139] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0140] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A flame smoke detection method based on improved YOLOv5s, It is characterized in that The following steps are involved: S1. Obtain fire and smoke data in different scenarios and construct a data set; S2. Improve the YOLOv5s model to obtain an improved YOLOv5s model; S3. Iteratively train the improved YOLOv5s model using the data set, and test and verify the trained improved YOLOv5s model to obtain the optimal improved YOLOv5s model; S4. Obtain image data in the detection scene, input the image data into the optimal improved YOLOv5s model, and obtain the flame and smoke recognition result.

2. A flame smoke detection method based on improved YOLOv5s according to claim 1, It is characterized in that Improvements to the YOLOv5s model include: Integrate the coordinate attention mechanism in the C3 module of the backbone network; Add a CSPCM module consisting of a hybrid convolutional layer and a normal convolutional layer to the backbone network; Use a bidirectional weighted feature pyramid network to replace the feature fusion module at the neck of the YOLOv5s model; The CIOU loss function in the YOLOv5s model is replaced with the MPDIoU loss function, and the SiLU activation function is replaced with the ReLU activation function.

3. A flame smoke detection method based on improved YOLOv5s according to claim 2, It is characterized in that The coordinate attention mechanism is integrated into the C3 module of the backbone network, including: Decompose the global average pooling in the X and Y dimensions to generate two feature maps of size C*1*W and C*H*1; The two feature maps are fused, and the fused feature maps are convolved and activated to reduce the dimension, generating a feature map of size C / r*1*(W+H).

4. A flame smoke detection method based on improved YOLOv5s according to claim 2, It is characterized in that A CSPCM module consisting of a hybrid convolutional layer and a normal convolutional layer is added to the backbone network, specifically including: The hybrid convolution layer divides the feature map of the input CSPCM module into g groups of different feature maps in the channel dimension, and performs depth-separable convolution operations on the g groups of different feature maps, wherein the convolution kernel size of each group of depth-separable convolution operations is different; The convolution layer concatenates the feature maps obtained through the depthwise separable convolution operation, and uses a point-by-point convolution operation to obtain the final output feature map.

5. A flame smoke detection method based on improved YOLOv5s according to claim 1, It is characterized in that Comparative experiments and ablation experiments are used to verify the effectiveness of the trained improved YOLOv5s model.

6. A flame and smoke detection system based on improved YOLOv5s, It is characterized in that include: The dataset module is used to obtain fire and smoke data in different scenarios and construct datasets; A model building module is used to improve the YOLOv5s model to obtain an improved YOLOv5s model; An optimization module, used to iteratively train the improved YOLOv5s model using the data set, and test and verify the trained improved YOLOv5s model to obtain an optimal improved YOLOv5s model; The recognition module is used to obtain image data in the detection scene, input the image data into the optimal improved YOLOv5s model, and obtain the flame and smoke recognition result.

7. A flame and smoke detection system based on improved YOLOv5s according to claim 6, It is characterized in that The improved YOLOv5s model includes: The coordinate attention mechanism is integrated into the C3 module of the backbone network, and a CSPCM module consisting of a hybrid convolutional layer and a normal convolutional layer is added; The neck uses a bidirectional weighted feature pyramid network to replace the feature fusion module of the YOLOv5s model; The CIOU loss function in the YOLOv5s model is replaced with the MPDIoU loss function, and the SiLU activation function is replaced with the ReLU activation function.

Citation Information

Cited By

  • Safety belt detection method and system based on novel YOLOv10

    CN119516175A

  • A safety belt detection method and system based on a new YOLOv10

    CN119516175B

  • Power transmission line insulator detection method and system based on YOLOv10

    CN119516176A

  • Visual smoke detection method and system based on spatial relation characteristics

    CN119763037A

  • Flame smoke identification method and equipment

    CN120612569A