Fire smoke detection method fused with improved Yolov8

Abstract

The invention belongs to the technical field of target detection, and particularly relates to a fire smoke detection method fusing and improving Yolov8, which comprises the following steps of: 1, adding a 160 * 160 small target detection layer on an original Yolov8 model to form FS-Yolov8, and performing Concat splicing on a feature map extracted from a second layer and a feature map sampled on a sixteenth layer by the FS-Yolov8 to obtain a target detection result; and finally outputting at the eighteenth layer as a fourth detection head. 2, generating a DLKAv3 module on the basis of the DLKA module, and forming a C2f-DLKAv3 module by combining the C2f module; according to the network, a small target detection layer is added on the basis of Yolov8, an initial small flame target can be detected more effectively, and early warning is carried out; according to the feature extraction part, a deformable module C2f-DLKAv3 is innovated to adapt to fire and smoke detection with irregular shapes, and meanwhile, deformable convolution is improved, so that the boundary of a detected object is more effectively determined; a novel lightweight dynamic up-sampling Dysample is introduced to replace an original Upsample module so as to realize performance improvement.

Description

Technical Field

[0001] The invention belongs to the technical field of target detection, and specifically is a fire smoke detection method integrating and improving Yolov8. Background Art

[0002] In recent years, large-scale fires have occurred mainly in forests, factories and residential buildings, causing immeasurable losses to the protection of ecological resources and the safety of people's lives and property. Fire safety hazards also exist in daily life scenarios, and once they occur, they will be extremely destructive, so they need to be discovered in time and measures must be taken to control them.

[0003] In the traditional fire smoke detection process, sensors are used to detect fire, but the detection capability of sensors is relatively poor and needs to meet many requirements. After a fire occurs, the smoke concentration, light and temperature need to reach a certain threshold to trigger the sensor, and it is easily affected by various interference factors, which reduces the accuracy of detection.

[0004] At present, the flame and smoke detection methods based on video images are divided into detection based on traditional classifiers and detection based on deep learning. The former mostly extracts the physical characteristics of flames and smoke, such as brightness, color, shape, texture, flutter, etc., and then passes the feature vector into the classifier for training, and finally uses the classifier for judgment. However, this method consumes a lot of time, and the detection performance of fire and smoke is not high, and the real-time performance is slow. And if there is interference information such as occlusion, false alarms and wrong alarms will occur. Labati et al. studied the color characteristics of smoke in YUV space to improve the smoke detection algorithm. However, the algorithm is not effective for smoke in complex backgrounds and smoke that diffuses slowly. Li Hongdi et al. used pooling operations and image pyramids to extract smoke feature vectors, and finally put them into SVM (support vector machine) for training to obtain a model. The above method has relatively good detection effect for video images with a single background, but for small and distant fire smoke images or scenes with slightly complex backgrounds, its detection effect is poor, and there are many missed detections, so it is not suitable for fire smoke detection in multiple scenes.

[0005] In recent years, computer vision technology has developed particularly rapidly, and deep learning has been increasingly used in the field of target detection. At present, the research on fire smoke detection is mainly based on the research of deep learning algorithms. These algorithms have the advantages of high speed and high precision, and can detect fires in real-time environments. Deep learning represented by convolutional neural networks can extract image features more efficiently, so it is applied to a wider range of computer vision task scenarios, such as semantic segmentation, image classification, target detection, etc. Zhang Dexiong proposed a regional division method, which can segment areas unrelated to forest scenes, shield them in subsequent fire smoke detection, and improve the effect of forest fire detection, but its real-time performance is difficult to guarantee.

[0006] Pan et al. designed a forest fire warning system that combines segmentation, detection and classification into one model, extracts corresponding features, and determines the fire category according to the algorithm. However, the model of this method is large and cannot meet the effect of real-time detection. He et al. optimized the network and integrated multi-scale features to improve the network's detection ability for small smoke and smoke-like scenes. However, its applicable scene is single and only suitable for applications in dense fog scenes. It cannot identify the specific location of the smoke and has limitations in real scenes. Li Yingjie et al. proposed a method for automatic classification of forestry business images based on self-learning features. When the identified image is similar to its similar image, it is correctly identified. However, this method can only be used for forest fire classification and cannot determine the location of the flame. Based on the target detection model, flame and smoke targets can be located and classified, and the detected objects can be marked with rectangular boxes. Lin et al. combined Faster RCNN and 3DCNN models to detect smoke, which improved the accuracy and detection precision. However, since the generation and classification of candidate boxes are carried out in two steps during the detection process, the model is large, generalization is insufficient, and the detection speed is slow.

[0007] Therefore, to address the above problems, a fire smoke detection method that integrates and improves Yolov8 is proposed. Summary of the invention

[0008] 1. Technical issues to be resolved

[0009] In view of the deficiencies in the prior art, the present invention provides a fire smoke detection method that integrates and improves Yolov8 to solve the problems raised in the background technology.

[0010] (II) Technical solution

[0011] To achieve the above object, the present invention provides the following technical solution: a fire smoke detection method integrating and improving Yolov8, comprising the following steps:

[0012] Step 1: Add a 160×160 small target detection layer to the original Yolov8 model to form FS-Yolov8. FS-Yolov8 concatenates the feature map extracted from the second layer with the upsampled feature map from the sixteenth layer, and finally outputs the eighteenth layer as the fourth detection head.

[0013] Step 2: Generate a DLKA_v3 module based on the DLKA module, and then combine it with the C2f module to form a C2f-DLKA_v3 module;

[0014] Step 3: Introduce a new lightweight dynamic upsampling Dysample module on the original Yolov8 model to replace the original Upsample module, build the FS-Yolov8 detection network, and achieve performance improvement.

[0015] Preferably, in step 1, on the Yolov8 original model, the head selects three feature maps of different sizes for detection, namely 80×80, 15th layer; 40×40, 18th layer; 20×20, 21st layer; the large-size feature map is used to identify relatively small targets, and the small-size feature map is used to identify larger targets.

[0016] Preferably, the DLKA module in step 2 is composed of a deformable convolution and a large convolution kernel, and the convolution part in the large convolution kernel is a deformable convolution, allowing the sampling network of the model to be flexibly deformed according to image features;

[0017] The DLKA module can be expressed as:

[0018] Attention=Conv1×1(DDW_D_Conv2d

[0019] (DDW_Conv2d(F'))(a)

[0020]

[0021] Where F' = GELU(Conv2d(F)), the input features are F∈R C×H×W express.

[0022] Preferably, the generation process in step 2 includes:

[0023] S1. Use deformable convolution to improve the algorithm's ability to model geometric deformations;

[0024] S2. Based on the original algorithm, a corresponding weight is added to each sampling point of the deformable convolution in formula (a) to determine whether the introduced area is an area of ​​interest. If the model is not interested in the sampling point area, the weight is learned to be 0.

[0025] (III) Beneficial effects

[0026] Compared with the prior art, the present invention provides a fire smoke detection method integrating and improving Yolov8, which has the following beneficial effects:

[0027] In order to improve the fire smoke detection performance of the algorithm in multiple scenarios, the present invention improves the Yolov8 algorithm and designs a lightweight FS-Yolov8 network adapted for fire smoke detection. The network adds a small target detection layer on the basis of Yolov8, which can more effectively detect the initial small flame target and make early warning; the feature extraction part innovates the deformable module C2f-DLKA_v3 to adapt to the irregular shape of fire and smoke detection, and at the same time improves the deformable convolution to more effectively determine the boundary of the detected object; the new lightweight dynamic upsampling Dysample is introduced to replace the original Upsample module to achieve performance improvement; experimental results show that the mAP@0.5 of the FS-Yolov8 fire smoke detection network reaches 94.1%, which is 5.4% higher than that of Yolov8, the detection speed reaches 190FPS, the model size is only 8.7MB, the comprehensive performance is significantly improved, and it can well meet the application requirements of being deployed to embedded devices for real-time monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0029] Figure 1 It is the FS-Yolov8 network structure diagram of the present invention;

[0030] Figure 2 It is a structural diagram of the DLKA module of the present invention;

[0031] Figure 3 It is a C2f-DLKA_v3 module structure diagram of the present invention;

[0032] Figure 4 It is a line graph comparing mAP@0.5 between FS-Yolov8 and Yolov8n in the present invention;

[0033] Figure 5 This is a thermal comparison diagram of Yolov8n in the present invention and after adding C2f-DLKA_v3;

[0034] Figure 6 It is a comparison chart of the detection results of Yolov8n and FS-Yolov8 in the present invention. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] Specific examples are given below.

[0037] See also Figure 1-Figure 6 The present invention provides a fire smoke detection method integrating and improving Yolov8, comprising the following steps:

[0038] Step 1: Add a 160×160 small target detection layer to the original Yolov8 model to form FS-Yolov8. FS-Yolov8 concatenates the feature map extracted from the second layer with the upsampled feature map from the sixteenth layer, and finally outputs the eighteenth layer as the fourth detection head.

[0039] Step 2: Generate a DLKA_v3 module based on the DLKA module, and then combine it with the C2f module to form a C2f-DLKA_v3 module;

[0040] Step 3: Introduce a new lightweight dynamic upsampling Dysample module on the original Yolov8 model to replace the original Upsample module, build the FS-Yolov8 detection network, and achieve performance improvement.

[0041] In the above scheme: the upsampler mainly adopts the point sampling method instead of the traditional kernel-based method, which not only simplifies its implementation in Pytorch, but also improves resource efficiency. Although kernel-based dynamic upsampling such as CARAFE has achieved impressive performance improvements, it has increased a lot of workload due to time-consuming dynamic convolution and additional sub-networks. Dysample can effectively reduce the computational workload and latency, mainly because it avoids time-consuming dynamic convolution and additional sub-networks to generate dynamic kernels. Its lightweight characteristics, fewer parameters and lower GPU requirements make it highly efficient. Dysample outperforms other upsamplers in multiple prediction tasks, including target detection and semantic segmentation, demonstrating its versatility and effectiveness in diverse applications, ensuring the speed and accuracy of reasoning, and reducing the model size, which has a good performance improvement effect on fire smoke detection;

[0042] The C2f-DLKA_v3 module in step 2 can better learn the irregular shape characteristics of fire smoke than the C2f module and is more sensitive to boundary information.

[0043] Further Figure 1-Figure 6 As shown in Figure 1, in step 1, on the Yolov8 original model, the head selects three feature maps of different sizes for detection, namely 80×80, 15th layer; 40×40, 18th layer; 20×20, 21st layer; large-size feature maps are used to identify relatively small targets, and small-size feature maps are used to identify larger targets.

[0044] In the above scheme: the fourth detection head can effectively extract fireworks image information with a resolution below 8×8, which is used to enhance the sensitivity to small target flames and smoke, thereby meeting the needs of early fire warning, and better solving the problem of Yolov8 detecting small target fireworks.

[0045] Further Figure 1-Figure 6 As shown, the DLKA module in step 2 consists of a deformable convolution and a large convolution kernel. The convolution part in the large convolution kernel is a deformable convolution, allowing the sampling network of the model to be flexibly deformed according to the image features.

[0046] The DLKA module can be expressed as:

[0047] Attention=Conv1×1(DDW_D_Conv2d

[0048] (DDW_Conv2d(F'))(a)

[0049] Where F' = GELU(Conv2d(F)), the input features are F∈R C×H×W express.

[0050] In the above scheme: deformable convolution is used to enhance the model's ability to extract information about objects of irregular shapes and sizes in medical images, mainly by adding additional offsets to adjust the sampling position of the standard convolution, so that the convolution kernel can dynamically adapt to the content of the image. Such a convolution kernel can more flexibly capture various irregular structures, so it can greatly improve the ability to obtain information about irregular objects such as flames and smoke. By learning the characteristics of flames and smoke themselves to determine these offsets, an adaptive kernel shape can be provided, which helps to improve the accuracy of detection; and the large convolution kernel is a mechanism for capturing a wide range of contextual information in an image. It imitates the feeling of the self-attention mechanism, but maintains less computation and parameters. The main reason is that its convolution part uses depthwise separable convolution, depthwise separable dilated convolution and point convolution to construct a large convolution kernel. This method enables the network to learn features in a larger receptive field while reducing the computational complexity by reducing the number of parameters.

[0051] Further Figure 1-Figure 6As shown, the generation process in step 2 includes:

[0052] S1. Use deformable convolution to improve the algorithm's ability to model geometric deformations;

[0053] S2. Based on the original algorithm, a corresponding weight is added to each sampling point of the deformable convolution in formula (a) to determine whether the introduced area is an area of ​​interest. If the model is not interested in the sampling point area, the weight is learned to be 0.

[0054] In the above scheme: the DLKA_v3 module is generated based on the DLKA module through the above method, and then combined with the C2f module to form the C2f-DLKA_v3 module. This module has a better understanding of the object boundary, reduces the introduction of irrelevant background, is more versatile, and can better meet the network's information extraction of irregular fire smoke;

[0055] The model size, accuracy, recall, FPS, average accuracy AP, and average precision mAP after model training are used as references for result comparison. The corresponding formulas are as follows:

[0056]

[0057] TP is a true positive example; FP is a false positive example; FN is a false negative example; mAP is the average classification performance of all categories. The higher the mAP, the higher the overall detection accuracy of the model. When the overlap between the predicted box and the true box is 0.5, it is expressed as mAP@0.5.

[0058] The model pre-training weights use Yolov8.pt, the input image resolution is uniformly adjusted to 640×640, the SGD function is used for parameter optimization during training, the initial learning rate is set to 0.01, the Batch-size is set to 16, and 300 epochs are set for each experiment.

[0059] Performance comparison experiment:

[0060] The present invention uses the original network Yolov8n and the improved network FS-Yolov8 for training on FS24, and other parameters such as batch remain the same. The total number of training epochs is set to 300, and the final change curve of mAP@0.5 is shown in the attached figure. Figure 4 As shown in the figure, we can see that the average classification performance of the FS-Yolov8 model is closer to 1, and the results are gradually becoming stable.

[0061] Classic algorithm comparison experiment

[0062] First, Yolov8n is used as the baseline and trained on the self-built dataset FS24. The classic detection algorithms SSD, Fire SSD, Faster-RCNN, YOLOv5, and YOLOv7 in the field of target detection are selected for comparative experiments with the algorithm of the present invention. All experimental environments are kept consistent. The results after training are shown in Table 1.

[0063] Compared with target detection algorithms such as SSD, Fire SSD, Faster-RCNN, YOLOv5 and YOLOv7, the present invention has very significant advantages. It can be seen from Table 1 that compared with traditional SSD and Faster-RCNN algorithms, the overall performance of the YOLO algorithm has been significantly improved, and the mAP@0.5 of FS-Yolov8 is also 16.8% and 14.8% higher respectively. The detection speed and model size are lightweight, which is convenient for deployment in daily real-time detection tasks.

[0064] Table 1 Performance comparison of different detection models

[0065]

[0066] Ablation experiment

[0067] In order to verify the experimental effects of different improvement strategies, the present invention conducts ablation experiments on the Yolov8n model. The experimental results are shown in Table 2. All experimental training parameters are the same and are all completed on the data set FS24. It can be seen from Table 2 that after adding the C2f-DLKA_v3 module to the network, all performances are improved, and mAP@0.5 increases by 5 percentage points based on the original model; after adding a small target detection layer and Dysample, the accuracy is improved to 95.7%, and mAP@0.5 finally reaches 94.1%; in terms of detection speed, since this paper adds a small target detection layer and the corresponding number of network layers is also increased, the detection speed is reduced, but it is still ahead of most of the current fire smoke detection models and meets the requirements for deployment to embedded devices.

[0068] Table 2 The impact of different improvement strategies on the Yolov8n model

[0069]

[0070] Note: “-” indicates that the method is not used in the model, and “√” indicates that the method is used in the model

[0071] Detection effect analysis

[0072] From the attached Figure 5From the changes in the hotspot area, we can see that for irregularly shaped flame smoke, the innovation of deformable convolution in the C2f-DLKA_v3 module makes the network more sensitive to the boundary information of flame smoke, pays less attention to irrelevant information (background), and can better focus on the detection target itself, thus improving the model's ability to extract features and making it very suitable for fire smoke detection tasks;

[0073] Multiple scene images were randomly collected and tested by the Yolov8n model and the FS-Yolov8 model. The results are shown in the attached figure. Figure 6 As shown in the figure, the target detected in the first picture is a small flame in the distance, and there is occlusion. The second picture is a thin smoke in the distance. These two targets are small and have less feature information. Yolov8n did not detect the target and missed detection, while FS-Yolov8 can effectively detect the target with a high confidence level. In the third group of pictures, there are many electric poles with colors similar to smoke, and they are mixed with the smoke. Yolov8n missed smoke detection and was insensitive to boundaries, resulting in multiple frames and repeated frames. FS-Yolov8 identified all the smoke, excluded interference and background, had clear boundaries, and improved confidence. The detection performance in complex scenes was greatly improved.

[0074] The present invention proposes a lightweight fire smoke detection algorithm FS-Yolov8 for shape-variable and irregular fire smoke. Aiming at the insufficient detection performance of Yolov8 on small targets, a small target detection layer is added to enhance the feature extraction capability, solve the problem that small target information is difficult to extract, and make FS-Yolov8 have better performance in small target detection; Aiming at the shape characteristics of fire smoke, the DLKA attention mechanism is introduced, but DLKA will introduce irrelevant information, so we innovate it, introduce sampling point weights, set the weight of irrelevant information to 0, and combine it with C2f to form the C2f-DLKA_v3 module, so that The network is more sensitive to irrelevant information and can better determine the boundaries of the detection targets; the present invention also introduces the latest ICCV2023 achievement Dysample to achieve lightweight and performance improvement of the network; at the same time, the present invention builds its own data set FS24, which includes fire smoke in various scenarios, and adds a large number of smoke and fire negative samples. The generated model can effectively filter out interference information such as sunset and clouds in actual tasks, and reduce the false detection rate caused by color features. Compared with the Yolov8n model, FS-Yolov8 has better accuracy performance on small targets and overall performance improvement, which meets the real-time detection and deployment requirements in actual environments.

[0075] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. The fire smoke detection method based on the improved Yolov8 is characterized by: The following steps are involved: Step 1: Add a 160×160 small target detection layer to the original Yolov8 model to form FS-Yolov8. FS-Yolov8 concatenates the feature map extracted from the second layer with the upsampled feature map from the sixteenth layer, and finally outputs the eighteenth layer as the fourth detection head. Step 2: Generate a DLKA_v3 module based on the DLKA module, and then combine it with the C2f module to form a C2f-DLKA_v3 module; Step 3: Introduce a new lightweight dynamic upsampling Dysample module on the original Yolov8 model to replace the original Upsample module, build the FS-Yolov8 detection network, and achieve performance improvement.

2. The fire smoke detection method integrating and improving Yolov8 according to claim 1 is characterized in that: In the step 1, on the Yolov8 original model, the head selects three feature maps of different sizes for detection, namely 80×80, 15th layer; 40×40, 18th layer; 20×20, 21st layer; the large-size feature map is used to identify relatively small targets, and the small-size feature map is used to identify larger targets.

3. The fire smoke detection method integrating and improving Yolov8 according to claim 1 is characterized in that: In step 2, the DLKA module is composed of a deformable convolution and a large convolution kernel, and the convolution part in the large convolution kernel is a deformable convolution, allowing the sampling network of the model to be flexibly deformed according to the image features; The DLKA module can be expressed as: Attention=Conv1×1(DDW_D_Conv2d (DDW_Conv2d(F'))(a) Where F' = GELU(Conv2d(F)), the input features are F∈R C×H×W express.

4. The fire smoke detection method integrating and improving Yolov8 according to claim 3 is characterized in that: The generation process in step 2 includes: S1. Use deformable convolution to improve the algorithm's ability to model geometric deformations; S2. Based on the original algorithm, a corresponding weight is added to each sampling point of the deformable convolution in formula (a) to determine whether the introduced area is an area of ​​interest. If the model is not interested in the sampling point area, the weight is learned to be 0.

Images