Smoke and fire detection method based on enhanced YOLO algorithm

Abstract

The invention provides a smoke and fire detection method based on an enhanced YOLO algorithm, and the method comprises the steps: uploading obtained to-be-labeled data to a data labeling platform for labeling, so as to construct a training data set, and the to-be-labeled data is to-be-labeled smoke and fire image data; inputting the training data set into an enhanced YOLOv5-based smoke and fire detection model for training, wherein the smoke and fire detection model comprises an input module, a backbone network, a neck network and a prediction module; the convergence of the smoke and fire detection model during training is accelerated through the focus loss function, so that a trained smoke and fire detection model is obtained; and uploading to-be-predicted data to the model prediction platform, performing prediction by using the trained smoke fire detection model, and outputting a final prediction result. According to the invention, the accuracy and stability of fire and smoke detection are improved, and key support is provided for performance improvement of a fire monitoring system.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a smoke and fire detection method based on an enhanced YOLO algorithm. Background Art

[0002] Fire, as a highly destructive natural disaster, often causes serious damage and losses in a short period of time, especially to forests and urban buildings. Timely detection and effective control of fires are crucial, because fires spread very quickly and may spread rapidly to a wide area in a very short period of time, posing a huge threat to people's lives and property. The rapid spread of fires will not only cause losses of material property, but may also endanger people's lives and lead to immeasurable harm and tragedy. Therefore, establishing an efficient fire monitoring and early warning system and taking effective fire control measures are crucial to minimize the damage and harm caused by fires and protect people's lives and property.

[0003] In fire monitoring systems, smoke often plays an important role in early warning. As a precursor to fire, smoke usually appears earlier than flames and is more difficult to block. Its rapid appearance at the beginning of a fire provides the system with key clues to promptly detect potential dangers. The introduction of dynamic smoke recognition can respond quickly when signs of fire just appear, thereby improving the agility and accuracy of the fire warning system. By identifying smoke early, the system can more effectively monitor and respond to fire risks, providing important protection for protecting people's lives and reducing losses caused by fire.

[0004] Traditional fire detection methods mainly rely on collecting and analyzing data from temperature and smoke sensors to detect fires. However, this early warning method has disadvantages such as poor real-time performance, high investment cost, and susceptibility to external environmental interference. With the continuous advancement of science and technology, solutions based on video image processing technology have emerged. By using the physical characteristics of fireworks such as color, shape, texture, and movement for fire detection, it has the advantages of timely response and a wide range of applications. However, this solution also has disadvantages such as slow detection speed, high false detection rate, and weak generalization performance, and cannot fully meet actual needs.

[0005] In summary, the existing fire detection methods have the following disadvantages:

[0006] (1) Fire detection methods based on collecting and analyzing data have disadvantages such as poor real-time performance, high investment cost, and susceptibility to interference from the external environment.

[0007] (2) Solutions based on video image processing technology have disadvantages such as slow detection speed, high false detection rate, and weak generalization performance, and cannot meet actual requirements.

[0008] (3) The fire detection scheme based on deep learning uses less data or less diversity, which makes the trained model prone to overfitting or poor generalization and robustness. Summary of the invention

[0009] In view of this, the present invention provides a smoke and fire detection method based on an enhanced YOLO algorithm to at least solve the above problems.

[0010] The invention provides a smoke and fire detection method based on an enhanced YOLO algorithm, comprising: uploading acquired data to be labeled to a data labeling platform for labeling to construct a training data set, wherein the data to be labeled are smoke and fire image data to be labeled; inputting the training data set into a smoke and fire detection model based on enhanced YOLOv5 for training, wherein the smoke and fire detection model comprises an input module, a backbone network, a neck network and a prediction module; accelerating the convergence of the smoke and fire detection model during training by a focal loss function to obtain a trained smoke and fire detection model; uploading the data to be predicted to a model prediction platform, applying the trained smoke and fire detection model for prediction, and outputting a final prediction result.

[0011] In one implementation, the training process of the smoke and fire detection model based on enhanced YOLOv5 includes: the input module preprocesses the input training data set through a data enhancement strategy and a label smoothing strategy to obtain a processed data set; the backbone network uses a CSP1_X structure and a convolutional block attention module to extract features from the processed data set to obtain backbone features with different feature scales; the neck network uses a CSP2_X structure to fuse backbone features with different feature scales to obtain fused features of different feature scales; the prediction module screens and optimizes the fused features of different feature scales through a weighted non-maximum suppression operation, and outputs a prediction result.

[0012] In another implementation, the data enhancement strategy includes adaptive image scaling, adaptive anchor box calculation and mosaic, and the data enhancement strategy is used to increase the number of samples in the training data set, thereby increasing the recognition ability of the smoke and fire detection model for objects of different sizes.

[0013] In another implementation, the label smoothing strategy refers to introducing random erroneous labels into the training data set, and the label smoothing strategy is used to alleviate the overfitting problem of the smoke and fire detection model on the training data.

[0014] In another implementation, the convolutional block attention module is added before the classifier of the backbone network, and the convolutional block attention module includes a channel attention module and a spatial attention module. The channel attention module and the spatial attention module are used to compress and reweight the input features of the processed data set in the feature extraction stage.

[0015] In another implementation, the focus loss function is expressed as the following formula:

[0016]

[0017] Among them, y is the accurate sample, y' is the predicted class probability, α can change the ratio of positive and negative samples, set α = 0.25, and γ is used to reduce the loss value of simple samples, set γ = 2.

[0018] In summary, compared with the prior art, the beneficial effects of the present invention are:

[0019] (1) Compared with the traditional solution, the present invention has better real-time performance and stronger robustness, and can better resist the interference of the external environment.

[0020] (2) Compared with video-based solutions, the present invention improves the detection speed of fire and smoke.

[0021] (3) The amount of data used in the present invention is richer and larger, which can better solve the overfitting problem of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 The figure is a flowchart of the overall steps of the smoke and fire detection method based on the enhanced YOLO algorithm of the present invention;

[0023] Figure 2 It is a schematic diagram of image enhancement of the smoke and fire detection method based on the enhanced YOLO algorithm of the present invention;

[0024] Figure 3 A network architecture diagram of a smoke and fire detection model of a smoke and fire detection method based on an enhanced YOLO algorithm of the present invention;

[0025] Figure 4 It is a structural schematic diagram of the convolutional block attention module of the smoke and fire detection method based on the enhanced YOLO algorithm of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] See also Figure 1 The present invention provides a smoke and fire detection method based on an enhanced YOLO algorithm, comprising:

[0028] The acquired data to be labeled is uploaded to a data labeling platform for labeling to construct a training data set, wherein the data to be labeled is smoke and fire image data to be labeled;

[0029] The training data set is input into the smoke and fire detection model based on enhanced YOLOv5 for training, wherein the smoke and fire detection model includes an input module, a backbone network, a neck network and a prediction module;

[0030] The convergence of the smoke and fire detection model training is accelerated by the focal loss function to obtain a trained smoke and fire detection model;

[0031] Upload the data to be predicted to the model prediction platform, apply the trained smoke and fire detection model for prediction, and output the final prediction results.

[0032] Optionally, the training process of the smoke and fire detection model based on enhanced YOLOv5 includes: the input module preprocesses the input training data set through a data enhancement strategy and a label smoothing strategy to obtain a processed data set; the backbone network uses a CSP1_X structure and a convolutional block attention module to extract features from the processed data set to obtain backbone features with different feature scales; the neck network uses a CSP2_X structure to fuse backbone features with different feature scales to obtain fused features of different feature scales; the prediction module screens and optimizes the fused features of different feature scales through a weighted non-maximum suppression operation, and outputs a prediction result.

[0033] Optionally, the data enhancement strategy includes adaptive image scaling, adaptive anchor box calculation and mosaic, and the data enhancement strategy is used to increase the number of samples in the training data set, thereby increasing the recognition ability of the smoke and fire detection model for objects of different sizes.

[0034] For example, Figure 2 As shown in the figure, by performing operations such as translation, rotation and flipping on the original image, the number of samples was successfully expanded to three times the original amount.

[0035] It should be understood that during the data collection phase, different environmental factors and perspectives are integrated into the dataset, including various lighting conditions and building shadows, etc. This operation helps enable the model to recognize and learn from more aspects. By including data samples from various environmental conditions, the model can better adapt to different lighting, shadows, and perspective changes during training, thereby improving its recognition ability and generalization performance in diverse scenarios. This data collection method that comprehensively considers different environmental factors helps to enhance the robustness of the model, enabling it to more accurately respond to various challenges and changes in the real world.

[0036] Optionally, the label smoothing strategy refers to introducing random erroneous labels into the training data set, and the label smoothing strategy is used to alleviate the overfitting problem of the smoke and fire detection model on the training data.

[0037] It should be understood that the label smoothing strategy adopted is to introduce random wrong labels in the training data set. The purpose of this strategy is to help alleviate the problem of overfitting the model to the training data. By introducing a certain degree of noise and uncertainty, the model can better adapt to various data changes and noise interference during the training process, thereby improving its generalization ability and robustness. This strategy can make the model better adapt to data changes in the real world, and also help improve the performance of the model on unseen data.

[0038] For example, the real-time detection of fire and smoke targets needs to have higher accuracy. Since the YOLO series is the latest representative method, YOLOv5 has the characteristics of high recognition accuracy, small file size, and fast speed, and is easier to be transplanted into fire monitoring. Therefore, the present invention proposes a smoke and flame detection framework based on an improved YOLOv5, and reconstructs the traditional YOLOv5 network by introducing data enhancement strategy, CBAM attention module and focal loss function to effectively improve the detection efficiency and accuracy of fire disasters. The structure of YOLOv5 used in the present invention is as follows: Figure 3 As shown, it includes four parts, namely, input module, backbone network, neck network and prediction module.

[0039] The input part adopts adaptive image scaling, adaptive anchor box calculation and mosaic data enhancement strategies to expand the diversity of data.

[0040] (1) Through adaptive image scaling, the input image can be effectively processed at different scales, which increases the model's ability to recognize objects of different sizes;

[0041] (2) Adaptive anchor box calculation helps to dynamically generate anchor boxes according to the size and scale of the target, improving the accuracy and robustness of target detection;

[0042] (3) The mosaic data enhancement strategy introduces randomness and complexity by segmenting and mixing images, further increasing the diversity of the data and helping the training model to better adapt to various complex scenarios and changes.

[0043] The combined use of these strategies expands the diversity of data samples and improves the robustness and generalization ability of the model in various situations.

[0044] The backbone network not only performs slicing and convolution operations, but also adopts the CSP1_X structure to improve the learning ability of the feature network. At the same time, a convolutional block attention module (CBAM) is added before each classifier.

[0045] Optionally, the convolutional block attention module is added before the classifier of the backbone network, and the convolutional block attention module includes a channel attention module and a spatial attention module, and the channel attention module and the spatial attention module are used to compress and reweight the input features of the processed data set in the feature extraction stage.

[0046] For example, Figure 4 As shown in the figure, the convolutional block attention module infers a one-dimensional channel attention map and a two-dimensional spatial attention map, namely, M C 、M S The convolutional block attention module compresses and reweights the input features in two independent channels and spatial dimensions, and the CBAM module can increase the network's attention to key elements, thereby overcoming background interference and further improving model performance. This method of combining the CSP1_X structure and the CBAM module helps to enhance the network's perception and noise resistance, and improves the accuracy and stability of the model in complex environments.

[0047] In the neck of the network structure, the information extracted from the backbone is used to enhance the network's feature fusion capability. The neck adopts the CSP2_X design, which not only reduces the amount of calculation, but also enhances the network's ability to fuse features while retaining richer feature information.

[0048] In the target prediction stage, the present invention uses a weighted non-maximum suppression (NMS) operation to select multiple target anchor boxes, which helps to improve the accuracy of target recognition. Through the NMS operation, the bounding boxes with more overlaps can be effectively eliminated, and the most representative target bounding boxes can be retained, thereby improving the accuracy and robustness of target detection. This strategy of combining CSP2_X design and weighted NMS operation helps to improve the network's ability to identify and locate targets, thereby enhancing the performance of the overall system.

[0049] YOLOv5 provides various frame loss functions to compare the predicted frames and the actual frames to calculate the loss, such as CIOU loss and GIOU loss. However, in the basic training process, there is an extreme imbalance between positive and negative samples. Therefore, the present invention uses the focal loss function to reduce the weight of samples that are easy to classify.

[0050] The focal loss function is a technique for dealing with sample imbalance problems. It focuses on reducing the weight of samples that are easy to classify, thereby paying more attention to samples that are difficult to classify. By introducing the focal loss function, we can effectively deal with the imbalance between positive and negative samples, improve the model's learning ability for key samples, and accelerate the model's convergence process. This strategy can help the network better adapt to complex data distribution and improve the accuracy and stability of target detection.

[0051] Optionally, the focal loss function is expressed as the following formula:

[0052]

[0053] Among them, y is the accurate sample, y' is the predicted class probability, α can change the ratio of positive and negative samples, set α = 0.25, γ is used to reduce the loss value of simple samples, set γ = 2, then the model will focus on those examples that are difficult to classify during training, so that the model can handle various challenging situations more effectively, thereby improving the overall performance and robustness.

[0054] The experiments verified the feasibility and robustness of the invention, and it performed well on the test set. Compared with the existing solutions, all evaluation indicators were improved, achieving a balance between accuracy and detection speed.

[0055] In summary, the solution of the present invention firstly, the improved framework effectively expands the data set and solves the overfitting problem caused by insufficient number of samples; secondly, the present invention adds a convolutional block attention module (CBAM) before each backbone classifier, which can compress and reweight the input features of two independent channels and spatial dimensions, thereby enhancing the focus on smoke and fire feature information and improving the desired feature extraction capability; in addition, the present invention uses the focal loss function to enhance the weight of complex samples, solves the imbalance problem of positive and negative samples in single-stage detection, and solves the problem of a high proportion of easily separable samples in the loss function.

[0056] The combined effect of these optimization measures of the present invention further improves the accuracy and stability of fire and smoke detection, providing key support for the performance improvement of the fire monitoring system. Through this advanced algorithm and technology, the fire monitoring system can more reliably achieve real-time monitoring and early warning of fire and smoke targets, providing important protection for people's life safety and property safety.

[0057] It should be noted that the present invention may be a method, a system, an apparatus and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0058] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0059] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0060] The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, Python, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.

[0061] Various aspects of the present invention are described herein with reference to the flow charts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and / or block diagram and the combination of each box in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0062] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0063] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0064] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a part of a module, a program segment or an instruction, and a part of the module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0065] Embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims

1. A smoke and fire detection method based on enhanced YOLO algorithm, characterized in that: include: The acquired data to be labeled is uploaded to a data labeling platform for labeling to construct a training data set, wherein the data to be labeled is smoke and fire image data to be labeled; The training data set is input into the smoke and fire detection model based on enhanced YOLOv5 for training, wherein the smoke and fire detection model includes an input module, a backbone network, a neck network and a prediction module; The convergence of the smoke and fire detection model training is accelerated by the focal loss function to obtain a trained smoke and fire detection model; Upload the data to be predicted to the model prediction platform, apply the trained smoke and fire detection model for prediction, and output the final prediction results.

2. The method according to claim 1, characterized in that: The training process of the smoke and fire detection model based on enhanced YOLOv5 includes: The input module preprocesses the input training data set through data enhancement strategy and label smoothing strategy to obtain the processed data set; The backbone network uses the CSP1_X structure and the convolutional block attention module to extract features from the processed data set and obtain backbone features with different feature scales; The neck network uses the CSP2_X structure to fuse the backbone features with different feature scales to obtain fused features of different feature scales; The prediction module screens and optimizes the fusion features of different feature scales through weighted non-maximum suppression operation and outputs the prediction results.

3. The method according to claim 2, characterized in that The data enhancement strategy includes adaptive image scaling, adaptive anchor box calculation and mosaic, and the data enhancement strategy is used to increase the number of samples in the training data set, thereby increasing the recognition ability of the smoke and fire detection model for objects of different sizes.

4. The method according to claim 3, characterized in that The label smoothing strategy refers to introducing random erroneous labels into the training data set, and the label smoothing strategy is used to alleviate the overfitting problem of the smoke and fire detection model on the training data.

5. The method according to claim 2, characterized in that: The convolutional block attention module is added before the classifier of the backbone network. The convolutional block attention module includes a channel attention module and a spatial attention module. The channel attention module and the spatial attention module are used to compress and reweight the input features of the processed data set in the feature extraction stage.

6. The method according to claim 1, characterized in that The focal loss function is expressed as the following formula: Among them, y is the accurate sample, y' is the predicted class probability, α can change the ratio of positive and negative samples, set α = 0.25, and γ is used to reduce the loss value of simple samples, set γ = 2.

Images