Fire detection and classification method and device based on improved YOLOv8 and storage medium

Abstract

The invention relates to the field of computer vision, in particular to a fire detection and classification method and device based on improved YOLOv8 and a storage medium. The method comprises the following steps: acquiring images of buildings, vehicles and forest fires; inputting the images of different types of fire into a fire detection and classification model to obtain a fire detection and classification result; the fire detection and classification model is improved based on a YOLOv8 model; the fire detection and classification model comprises a feature extraction network Backbone module, a feature fusion network Neck module and a feature recognition network Head module; a DGSS module for effectively capturing flame details and spatial information around a fire disaster is added into the Backbone module; in the Neck module, an SPPF module of a traditional YOLOv8 model is replaced by an SLSM module capable of effectively recognizing flame characteristics of different scales; according to the method, the fire detection and classification model based on the improved YOLOv8 model is adopted, fire in different scenes can be efficiently and accurately detected, the types of the fire are marked, and the fire emergency management efficiency is remarkably improved.

Description

Technical Field

[0001] The present invention provides a fire detection and classification method, device and storage medium based on improved YOLOv8, belonging to the technical field of fire monitoring. Background Art

[0002] Fire has always been regarded as a highly threatening disaster, which not only seriously disrupts people's daily lives, but also brings huge economic losses, endangers life safety, damages property and wildlife habitats, and may cause environmental pollution. In order to minimize the losses caused by fire, it is necessary to detect it in its early stages.

[0003] Recently, more and more researchers have applied computer vision technology to fire detection scenarios. Compared with traditional physical sensors, vision-based fire detection solutions have the advantages of wide coverage, low deployment cost, short response time, and strong robustness. Although some progress has been made in this field, there are still some limitations.

[0004] (1) Currently, most deep learning methods are mainly applied to classification tasks in the field of fire detection. These methods tend to classify the entire image as a single category for processing. However, the fire often occupies only a small part of the image. When the characteristics of the fire are not obvious enough, relying on full image features for detection will reduce the accuracy of detection and cause delayed response to the fire alarm.

[0005] (2) Most of the existing fire detection datasets only contain a single “fire” category, lacking a breakdown of fire types, such as building fires, vehicle fires, forest fires, etc. This limits managers’ ability to quickly identify fire types and take targeted rescue measures, thus affecting fire emergency response and rescue efficiency. Summary of the invention

[0006] In order to solve the above technical problems, the present invention proposes a fire detection and classification method, device and storage medium based on improved YOLOv8, which can accurately locate the fire area, output the type of fire in the image and transmit the information to the corresponding management department.

[0007] The technical solution adopted by the present invention is: a fire detection and classification method based on improved YOLOv8, comprising the following steps:

[0008] Step 1: Obtain different types of fire images, create a fire detection dataset, and divide the test set and training set according to the ratio;

[0009] Step 2: Construct a fire detection and classification model: The fire detection and classification model is based on the YOLOv8 model, including a feature extraction network Backbone module, a feature fusion network Neck module and a feature recognition network Head module. The feature extraction network Backbone module is added with a DGSS module that can effectively capture flame details and spatial information around the fire. The feature fusion network Neck module replaces the SPPF module of the traditional YOLOv8 model with an SLSM module that can effectively identify flame features of different scales. The input of the SLSM module is the output of the DGSS module, and the output of the SLSM module is input to the first Upsample module and the Concat module that connects the last CBS module and the C2F module.

[0010] The DGSS module includes the first Linear layer, after which the information flow is divided into x and z; the information flow x is divided into four parts and passes through DGC and SS2D in parallel; the information flow z passes through the first Concat module after being multiplied by the four parts of the information flow in the information flow x after passing through the SiLU activation function; after the first Concat module, a channel attention is connected to obtain the output feature;

[0011] The SLSM module includes a first DWConv. After the first DWConv, the information flow is divided into five information flows, one information flow passes through the second DWConv, and the remaining four information flows pass through four SDWConvs respectively, and then the five information flows are integrated together through the second Concat module; a first Conv is connected after the second Concat module; after the first Conv, the information flow is divided into four parts, which pass through Max Pool, Avg Pool, Avg Pool (H, 1) designed for height, and Avg Pool (1, W) designed for width respectively, and then are integrated together through the third Concat module; after the third Concat module, the information flow passes through a second Conv and a Sigmoid activation function; the information flow passing through the first Conv and the information flow passing through a second Conv and a Sigmoid activation function are element-wise multiplied and then passed through a third Conv to obtain the output feature;

[0012] Step 3: Use the training set images to train the model and save the parameters;

[0013] Step 4: Input the test set images into the trained model to obtain the final fire detection results and label their categories.

[0014] Furthermore, the first DWConv is a 5×5 DWConv, and the second DWConv is a 7×7 DWConv with a dilation coefficient of 3.

[0015] Furthermore, the expansion coefficients of the four SDWConvs are 1, 1, 2, and 3, respectively, and the corresponding convolution kernel sizes are 3, 7, 7, and 7, respectively.

[0016] Furthermore, the first Conv is a 1×1 Conv, the second Conv is a 7×7 Conv, and the third Conv is a 1×1 Conv.

[0017] Furthermore, the training process of the fire detection and classification model in step 3 is:

[0018] Obtain a number of different types of fire images and annotate them according to their categories; wherein all fires in the fire images have been annotated with corresponding borders and corresponding labels for indicating different fire types are set;

[0019] The fire images in the training set are input into the fire detection and classification model, and the flame prediction bounding box and the fire category in the corresponding fire image are output;

[0020] The annotated bounding boxes and category labels of the fire images are input into the fire detection and classification model for training.

[0021] Furthermore, the boundary loss function in the training process adopts MPDIoU loss, which is expressed as follows:

[0022]

[0023]

[0024] L MPDIoU =1-MPDIoU;

[0025] Among them: B pre represents the coordinates of the predicted bounding box, B gt represents the coordinates of the ground-truth annotation bounding box, represents the coordinates of the upper left corner of the predicted bounding box, represents the coordinates of the lower right corner of the predicted bounding box, represents the coordinate of the upper left corner of the ground-truth annotation bounding box, represents the lower right corner coordinate of the ground-truth annotation bounding box, is the distance between the upper left corner and the lower right corner of the predicted bounding box and the true labeled bounding box, w and h are the width and height of the image, and L MPDIoU Represents the MPDIoU boundary loss function.

[0026] Further, the categories of fire include forest fire, vehicle fire and structure fire.

[0027] Furthermore, the fire detection result obtained in step 4 includes the precise location of the fire area.

[0028] A fire detection and classification device based on improved YOLOv8, comprising an image acquisition module and an image detection module;

[0029] An image acquisition module, used to acquire different types of fire images;

[0030] An image detection module is used to input the fire image into the fire detection and classification model to obtain the fire detection and classification model's accurate positioning of the fire area and the category of the fire in the image;

[0031] The fire detection and classification model is based on the YOLOv8 model, including a feature extraction network Backbone module, a feature fusion network Neck module and a feature recognition network Head module. The feature extraction network Backbone module is added with a DGSS module that can effectively capture flame details and spatial information around the fire. The feature fusion network Neck module replaces the SPPF module of the traditional YOLOv8 model with an SLSM module that can effectively identify flame features of different scales. The input of the SLSM module is the output of the DGSS module. The output of the SLSM module is input to the first Upsample module and the Concat module that connects the last CBS module and the C2F module.

[0032] The DGSS module includes the first Linear layer, after which the information flow is divided into x and z; the information flow x is divided into four parts and passes through DGC and SS2D in parallel; the information flow z passes through the first Concat module after being multiplied by the four parts of the information flow in the information flow x after passing through the SiLU activation function; after the first Concat module, a channel attention is connected to obtain the output feature;

[0033] The SLSM module includes a first DWConv. After the first DWConv, the information flow is divided into five information flows, one information flow passes through the second DWConv, and the remaining four information flows pass through four SDWConvs respectively, and then the five information flows are integrated together through the second Concat module; a first Conv is connected after the second Concat module; after the first Conv, the information flow is divided into four parts, which pass through Max Pool, Avg Pool, Avg Pool (H, 1) designed for height, and Avg Pool (1, W) designed for width respectively, and then are integrated together through the third Concat module; after the third Concat module, the information flow passes through a second Conv and a Sigmoid activation function; the information flow passing through the first Conv and the information flow passing through a second Conv and a Sigmoid activation function are element-wise multiplied and then passed through a third Conv to obtain the output feature;

[0034] A storage medium includes a stored computer program, which controls a device where the computer-readable storage medium is located to execute a fire detection and classification method based on an improved YOLOv8 when the computer program is running.

[0035] The beneficial effects of the present invention compared with the prior art are as follows: the present invention adopts a fire detection and classification model based on the improved YOLOv8, which can improve the accuracy of flame detection, can efficiently and accurately detect fires in different scenarios, and mark their types, thereby significantly improving the efficiency of fire emergency management. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The present invention will be further described below in conjunction with the accompanying drawings:

[0037] Figure 1 A flow chart of a fire detection and classification method based on improved YOLOv8 in one or more embodiments of the present invention;

[0038] Figure 2 A structural diagram of a fire detection and classification model based on improved YOLOv8 in one or more embodiments of the present invention;

[0039] Figure 3 A schematic diagram of the structure of a DGSS module that can effectively capture flame details and spatial information around a fire, as proposed in one or more embodiments of the present invention;

[0040] Figure 4 A schematic diagram of the structure of an SLSM module that can effectively identify flame characteristics of different scales, proposed in one or more embodiments of the present invention;

[0041] Figure 5 is a schematic diagram of the training process of a fire detection and classification model in one or more embodiments of the present invention;

[0042] Figure 6 is a schematic diagram of fire detection and classification effects in one embodiment of the present invention;

[0043] Figure 7 This is a schematic diagram of fire detection and classification results using the traditional YOLOv8 model;

[0044] Figure 8 It is a schematic diagram of fire detection and classification results of the YOLOv8 model improved by using an embodiment of the present invention. DETAILED DESCRIPTION

[0045] like Figures 1 to 8 As shown, the present invention provides a fire detection and classification method based on improved YOLOv8, comprising the following steps:

[0046] Step 1: Obtain different types of fire images, create a fire detection dataset, and divide the test set and training set according to the ratio. The different types of fire images include building fire, vehicle fire and forest fire images;

[0047] Step 2: Build a fire detection and classification model;

[0048] Step 3: Use the training set images to train the model and save the parameters;

[0049] Step 4: Input the test set images into the trained model to obtain the final fire detection results and label their types.

[0050] The fire image adopts RGB image, and different types of fire images can be fire images captured by real-time monitoring in different regions; it should be noted that in order to ensure the calculation rate of the fire detection and classification model and ensure real-time performance, the present invention can adjust the size of different types of fire images according to samples, for example, adjusting the input image size to 640×640.

[0051] The fire detection and classification model is built based on the improved YOLOv8 model. The fire detection and classification model includes a feature extraction network Backbone module, a feature fusion network Neck module and a feature recognition network Head module. The Backbone module is added with the independently developed DGSS (Dynamic Ghost Conv with SS2D) module that can effectively capture flame details and spatial information around the fire. In the Neck module, the SPPF module of the traditional YOLOv8 model is replaced with the SLSM (Strip Large Kernel Module) module that can effectively identify flame features of different scales.

[0052] like Figure 3 As shown in the figure, the independently developed DGSS module that can effectively capture flame details and spatial information around the fire includes the first Linear layer. After the first Linear layer, the information is divided into x and z; the information flow x is divided into four parts and passes through DGC and SS2D in parallel; the information flow z is multiplied with the four parts of the information flow x after passing through the SiLU activation function and then passes through the Concat module; after the Concat module, a channel attention is connected to obtain the output feature.

[0053] like Figure 4As shown in the figure, the independently developed SLSM module that can effectively identify flame features of different scales includes the first 5×5 DWConv; after the first 5×5 DWConv, the information flow is divided into five information flows, one information flow passes through the 7×7 DWConv with a dilation coefficient of 3, and the other four information flows pass through the SDWConv with dilation coefficients of 1, 1, 2, 3 and convolution kernel sizes of 3, 7, 7, 7 respectively, and then the five information flows are integrated together through the Concat module; a 1×1 Conv is connected after the Concat module; after the 1×1 Conv, the information flow is divided into four parts and passes through Max Pool, Avg Pool, Avg Pool (H, 1) designed for height and Avg Pool (H, 2) designed for width respectively. After Pool(1,W), they are integrated together through the Concat module; after the Concat module, the information flow passes through a 7×7 Conv and a Sigmoid activation function; the information flow that passes through a 1×1 Conv is multiplied by the information flow that passes through a 7×7 Conv and a Sigmoid activation function, and then passes through a 1×1 Conv to obtain the output features.

[0054] In the embodiment of the present invention, Figure 5 As shown in Figure 1, the training process of the fire detection and classification model includes:

[0055] S1: Obtain several images of building fires, vehicle fires, and forest fires and annotate them according to their categories; wherein all fires in the image have been annotated with corresponding bounding boxes and corresponding labels indicating different fire types are set;

[0056] S2: Input several images of building fires, vehicle fires, and forest fires into the fire detection and classification model, and output the flame prediction bounding box in the corresponding fire image and the fire category;

[0057] S3: The annotated bounding boxes and category labels of several images of building fires, vehicle fires, and forest fires are input into the fire detection and classification model for training, where the boundary loss function adopts MPDIoU loss.

[0058] In the embodiment of the present invention, MPDIoU is defined as follows:

[0059]

[0060]

[0061] L MPDIoU =1-MPDIoU;

[0062] Among them: B pre represents the coordinates of the predicted bounding box, B gtrepresents the coordinates of the ground-truth annotation bounding box, represents the coordinates of the upper left corner of the predicted bounding box, represents the coordinates of the lower right corner of the predicted bounding box, represents the coordinate of the upper left corner of the ground-truth annotation bounding box, represents the lower right corner coordinate of the ground-truth annotation bounding box, is the distance between the upper left corner and the lower right corner of the predicted bounding box and the true labeled bounding box, w and h are the width and height of the image, and L MPDIoU Represents the MPDIoU boundary loss function.

[0063] The parameters during training are set as follows: 100 iterations, initial learning rate set to 0.01, minimum learning rate set to 0.001, batch_size set to 8, Adam optimizer, and weight decay set to 0.0005.

[0064] In a preferred embodiment of the present invention, the present invention also provides a fire detection and classification device based on improved YOLOv8, including an image acquisition module and an image detection module;

[0065] An image acquisition module, used to acquire different types of fire images;

[0066] The image detection module is used to input the fire image into the fire detection and classification model to obtain the fire detection and classification model's accurate positioning of the fire area and the category to which the fire in the image belongs.

[0067] Among them, the fire detection and classification model is built based on the improved YOLOv8 model. The fire detection and classification model includes a feature extraction network Backbone module, a feature fusion network Neck module and a feature recognition network Head module. The Backbone module is added with the independently developed DGSS module that can effectively capture flame details and spatial information around the fire. In the Neck module, the SPPF module of the traditional YOLOv8 model is replaced with the SLSM module that can effectively identify flame characteristics of different scales.

[0068] In a preferred embodiment of the present invention, the present invention further provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the fire detection and classification method based on the improved YOLOv8 proposed by the present invention.

[0069] It should be noted that the execution subject of the fire detection and classification method based on improved YOLOv8 provided by the present invention can be an electronic device, a component in an electronic device, an integrated circuit or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), etc., and the non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), etc., which is not specifically limited by the present invention.

[0070] Figure 6 FIG. 1 is a schematic diagram of fire detection and classification effects in an embodiment of the present invention. Figure 6 As shown, it includes multiple jpg images, and the fire types contained in the selected jpg images include building fire (Building Fire), vehicle fire (Vehicles Fire) and forest fire (Forest Fire). The present invention can effectively detect fires and mark their types.

[0071] Figure 7 This is a schematic diagram of fire detection and classification results using the traditional YOLOv8 model, where the average accuracy (AP) value of building fire is 68.1%, the AP value of vehicle fire is 93.4%, the AP value of forest fire is 59.4%, and the mean average accuracy (mAP) value of all types is 73.6%.

[0072] Figure 8 The figure is a schematic diagram of the fire detection and classification results of the YOLOv8 model improved by the embodiment of the present invention, in which the AP value of building fire is 72.6%, the AP value of vehicle fire is 94.5%, the AP value of forest fire is 63.3%, and the mAP value of all types is 76.8%. It can be seen that the present invention significantly improves the accuracy of fire detection in each category, and effectively verifies its improvement effect on fire detection and classification.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A fire detection and classification method based on improved YOLOv8, characterized by: The following steps are involved: Step 1: Obtain different types of fire images, create a fire detection dataset, and divide the test set and training set according to the ratio; Step 2: Construct a fire detection and classification model: The fire detection and classification model is based on the YOLOv8 model, including a feature extraction network Backbone module, a feature fusion network Neck module and a feature recognition network Head module. The feature extraction network Backbone module is added with a DGSS module that can effectively capture flame details and spatial information around the fire. The feature fusion network Neck module replaces the SPPF module of the traditional YOLOv8 model with an SLSM module that can effectively identify flame features of different scales. The input of the SLSM module is the output of the DGSS module, and the output of the SLSM module is input to the first Upsample module and the Concat module that connects the last CBS module and the C2F module. The DGSS module includes the first Linear layer, after which the information flow is divided into x and z; the information flow x is divided into four parts and passes through DGC and SS2D in parallel; the information flow z passes through the first Concat module after being multiplied by the four parts of the information flow in the information flow x after passing through the SiLU activation function; after the first Concat module, a channel attention is connected to obtain the output feature; The SLSM module includes a first DWConv. After the first DWConv, the information flow is divided into five information flows, one information flow passes through the second DWConv, and the remaining four information flows pass through four SDWConvs respectively. After that, the five information flows are integrated together through the second Concat module; after the second Concat module, a first Conv is connected; After the first Conv, the information flow is divided into four parts, which are respectively passed through Max Pool, Avg Pool, Avg Pool (H, 1) designed for height, and AvgPool (1, W) designed for width, and then integrated together through the third Concat module; after the third Concat module, the information flow passes through a second Conv and a Sigmoid activation function; the information flow that passes through the first Conv and the information flow that passes through a second Conv and a Sigmoid activation function are element-wise multiplied and then passed through a third Conv to obtain the output feature; Step 3: Use the training set images to train the model and save the parameters; Step 4: Input the test set images into the trained model to obtain the final fire detection results and label their categories.

2. A fire detection and classification method based on improved YOLOv8 according to claim 1, characterized in that: The first DWConv is a 5×5 DWConv, and the second DWConv is a 7×7 DWConv with a dilation factor of 3.

3. A fire detection and classification method based on improved YOLOv8 according to claim 2, characterized in that: The expansion coefficients of the four SDWConvs are 1, 1, 2, and 3, respectively, and the corresponding convolution kernel sizes are 3, 7, 7, and 7, respectively.

4. A fire detection and classification method based on improved YOLOv8 according to claim 3, characterized in that: The first Conv is a 1×1 Conv, the second Conv is a 7×7 Conv, and the third Conv is a 1×1 Conv.

5. A fire detection and classification method based on improved YOLOv8 according to claim 4, characterized in that: The training process of the fire detection and classification model in step 3 is: Obtain a number of different types of fire images and annotate them according to their categories; wherein all fires in the fire images have been annotated with corresponding borders and corresponding labels for indicating different fire types are set; The fire images in the training set are input into the fire detection and classification model, and the flame prediction bounding box and the fire category in the corresponding fire image are output; The annotated bounding boxes and category labels of the fire images are input into the fire detection and classification model for training.

6. A fire detection and classification method based on improved YOLOv8 according to claim 5, characterized in that: The boundary loss function during training uses MPDIoU loss, which is expressed as follows: L MPDIoU =1-MPDIoU; Among them: B pre represents the coordinates of the predicted bounding box, B gt represents the coordinates of the ground-truth annotation bounding box, represents the coordinates of the upper left corner of the predicted bounding box, represents the coordinates of the lower right corner of the predicted bounding box, represents the coordinate of the upper left corner of the ground-truth annotation bounding box, represents the lower right corner coordinate of the ground-truth annotation bounding box, is the distance between the upper left corner and the lower right corner of the predicted bounding box and the true labeled bounding box, w and h are the width and height of the image, and L MPDIoU Represents the MPDIoU boundary loss function.

7. A fire detection and classification method based on improved YOLOv8 according to any one of claims 1 to 6, characterized in that: The fire categories include forest fires, vehicle fires and structure fires.

8. A fire detection and classification method based on improved YOLOv8 according to claim 7, characterized in that: The fire detection results obtained in step 4 include the precise location of the fire area.

9. A fire detection and classification device based on improved YOLOv8, characterized in that: It includes an image acquisition module and an image detection module; An image acquisition module, used to acquire different types of fire images; An image detection module is used to input the fire image into the fire detection and classification model to obtain the fire detection and classification model's accurate positioning of the fire area and the category of the fire in the image; The fire detection and classification model is based on the YOLOv8 model, including a feature extraction network Backbone module, a feature fusion network Neck module and a feature recognition network Head module. The feature extraction network Backbone module is added with a DGSS module that can effectively capture flame details and spatial information around the fire. The feature fusion network Neck module replaces the SPPF module of the traditional YOLOv8 model with an SLSM module that can effectively identify flame features of different scales. The input of the SLSM module is the output of the DGSS module. The output of the SLSM module is input to the first Upsample module and the Concat module that connects the last CBS module and the C2F module. The DGSS module includes the first Linear layer, after which the information flow is divided into x and z; the information flow x is divided into four parts and passes through DGC and SS2D in parallel; the information flow z passes through the first Concat module after being multiplied by the four parts of the information flow in the information flow x after passing through the SiLU activation function; after the first Concat module, a channel attention is connected to obtain the output feature; The SLSM module includes a first DWConv. After the first DWConv, the information flow is divided into five information flows, one information flow passes through the second DWConv, and the remaining four information flows pass through four SDWConvs respectively. After that, the five information flows are integrated together through the second Concat module; after the second Concat module, a first Conv is connected; After the first Conv, the information flow is divided into four parts, which are respectively passed through Max Pool, Avg Pool, Avg Pool (H, 1) designed for height and AvgPool (1, W) designed for width, and then integrated together through the third Concat module; after the third Concat module, the information flow passes through a second Conv and a Sigmoid activation function; the information flow passing through the first Conv and the information flow passing through a second Conv and a Sigmoid activation function are element-wise multiplied and then passed through a third Conv to obtain the output features.

10. A storage medium, characterized in that: The invention comprises a stored computer program, which controls a device where a computer-readable storage medium is located to execute a fire detection and classification method based on improved YOLOv8 as claimed in any one of claims 1 to 8 when the computer program is running.

Images