A flame early smoke detection method based on improved DeeplabV3+

Abstract

This paper discloses a method for detecting early-stage smoke in flames based on a modified DeeplabV3+ algorithm. The method involves constructing a specific dataset and using the DeeplabV3+ model to identify smoke; then optimizing DeeplabV3+ with a Unet algorithm to improve smoke identification accuracy. This method effectively identifies smoke areas in images to prevent fires, providing new insights into optimizing the DeeplabV3+ model.

Description

Technical Field

[0001] The present invention relates to fire monitoring technology, and in particular to a method for detecting early flame smoke based on improved DeeplabV3+. Background Art

[0002] Fires are natural or unnatural combustion that is uncontrolled in space or time. They are a common disaster and can occur in a variety of settings. Fires can cause significant personal and social losses, even endangering lives. According to statistics, 219,000 fires were reported in the first quarter of 2022, resulting in nearly 1,000 casualties and approximately 1.52 billion yuan in direct property damage. Fires in residential areas cause the most casualties, while unattended locations such as warehouses and forests incur even greater economic losses. Therefore, whether in residential or uninhabited areas, early detection and identification of a fire's source can minimize the likelihood of large-scale combustion and explosions, minimizing losses. Firefighting during this early stage of a fire is characterized by low flames, heavy smoke, and slow movement. Minimizing the use of manpower and resources is crucial for firefighting efforts. This technology uses smoke recognition to detect early flames. There are two main reasons for using smoke recognition to detect early fires. First, in the early stages of a fire, it is generally difficult to clearly observe the flame in the video image. This is because flames are often very small in the flame stage and easily obscured by various objects. Even if the flame is not obscured, it may still be obscured by smoke, which will undoubtedly significantly delay the time of fire detection. Smoke is often observed in the image before flames. Second, in some cases, there is no fire in the early stages of a fire. For example, an unextinguished cigarette butt in a public place will only produce sporadic sparks and a small amount of smoke. Or, when aging electrical wires often first produce large-scale smoke, which then encounters flammable objects and causes a fire. In these cases, the early fire is only smoke-free. Sensor-based flame detection using cameras to capture images in real time is one of the mainstream research directions today and has high research value for the following three reasons: First, deploying cameras is more economical than deploying sensors and does not require a very dense deployment. In addition, most public places already have cameras, so the hardware deployment cost is very low. Secondly, maintainability is high. If the algorithm is proven to be effective, the maintenance cost of a camera is much lower than that of a sensor. Sensors require regular inspections for problems, while cameras can determine whether there are any problems simply by looking at the image. Finally, image-based recognition algorithms are highly scalable. This is because cameras can be installed in fixed locations in public places and can also be mounted on drones or unmanned patrol vehicles. This allows for monitoring of a larger area with fewer cameras. For example, vast areas such as forests are unlikely to be monitored through the installation of large-scale sensors, but drone patrols can be used for flame detection. In general, recognition methods based on camera-captured images are reliable and cost-effective.

[0003] The smoke-based early flame detection proposed in current research can be roughly divided into two categories: early flame detection through smoke image recognition and early flame detection based on gas sensors.

[0004] Early research indicated that various smoke gases, such as CO and CO, produced during the initial stages of flame combustion could serve as important indicators for early flame detection. Methods such as spectroscopy to detect gas concentrations sufficient for fire warning or CO sensors were used for flame detection. However, these CO and smoke sensors require a certain delay before the gas concentration in the room reaches a certain level before sounding an alarm. With the development of computer graphics, more economical camera image recognition has attracted the attention of researchers. In recent years, scholars have proposed various methods and solutions for smoke image detection, such as using pyramid texture and edge features for smoke detection and motion detection for smoke detection. It can be seen that the earliest smoke image detection methods primarily focused on smoke color and texture features (static features) and smoke motion features (dynamic features).

[0005] 1. Image-Based Smoke Recognition Based on Static Features: The main static features of smoke are color, shape, and texture. For color, image-based smoke recognition algorithms, like most algorithms, use thresholds within each channel of a color space model. Color space-based smoke detection methods based on image saturation information work well for gray smoke. For shape, smoke regions are typically determined by analyzing their area and using an ellipse approximation to analyze their proportions. For texture, a gray-level co-occurrence matrix is often used in conjunction with texture operators such as the Haar feature operator. These three traditional algorithms each have their own strengths and weaknesses, and traditional algorithms often combine them to ensure reliability.

[0006] 2. Image-Based Smoke Recognition Based on Dynamic Features: Smoke has distinct dynamic characteristics, with it tending to flow in a fixed direction. Based on this principle, background subtraction methods can be used for dynamic smoke detection, or Gaussian mixture models can be used for image detection. Some researchers have also proposed using multi-layer optical flow calculations and motion distribution entropy within a Bayesian classifier to detect the unique motion of smoke, based on the irregular motion caused by the non-hardness of smoke. This algorithm is computationally expensive but can effectively extract smoke features.

[0007] These static-based smoke recognition algorithms work well for grayish-white smoke with a distinct background. They can detect the early onset of fires by identifying smoke, and are currently the mainstream methods in the smoke recognition field. However, they are still limited in complex environments. For example, color-based models make it difficult to detect smoke mixed with flames or smoke with a similar background color, resulting in a high false detection rate. Smoke recognition algorithms based on dynamic features are only suitable for scenes with relatively static fires, where objects are relatively motionless except for the smoke. They are also unsuitable for scenes where the camera moves during drone loading. Detection algorithms based on optical flow calculations are computationally expensive and therefore cannot be effectively applied in actual production. In short, due to the diverse nature of fire scenarios and the high amount of interference, these traditional methods struggle to achieve the desired results. Summary of the Invention

[0008] The main purpose of this paper is to provide a method for early flame smoke detection based on a modified DeeplabV3+. This method constructs a suitable dataset for the characteristics of gray-black and gray-white smoke, and implements smoke detection using the DeeplabV3+ model. By addressing the shortcomings of DeeplabV3+, such as edge noise and overfitting, the method improves the model structure by combining the Unet model and adding an ECA layer to enhance recognition accuracy. This method effectively achieves smoke video recognition results, ultimately improving smoke recognition accuracy.

[0009] The technical solution adopted by the present invention is: a flame early smoke detection method based on improved DeeplabV3+, comprising:

[0010] Build a specific dataset and use the DeeplabV3+ model to identify smoke;

[0011] Combined with Unet to optimize DeeplabV3+ and improve the accuracy of smoke recognition.

[0012] Furthermore, the construction of a specific dataset and the use of the DeeplabV3+ model for smoke recognition include:

[0013] Construct a VOC dataset containing more than 400 images of gray-black and gray-white smoke for training and testing;

[0014] Using convolutional neural networks, for the integrable functions f(x,y) and g(x,y) in two-dimensional space, the definition of convolution is as follows:

[0015]

[0016] Use DeeplabV3+ model to identify smoke.

[0017] DeeplabV3+ is a semantic segmentation model consisting of an Encoder model and a Decoder model.

[0018] Furthermore, the Encoder model includes:

[0019] The first stage of the Encoder part: This stage performs a simple convolution and upsampling to generate a low-level feature as the input of the second stage and simultaneously passes it to the Decoder part for further processing;

[0020] The second stage of the Encoder part: using the void space convolution pooling pyramid model ASPP for processing,

[0021] ASPP consists of a 1×1 convolution, a pooling pyramid (three 3×3 dilated convolutions) and an ASPP.

[0022] pooling composition;

[0023] ASPP accepts low-level features as input in the first stage;

[0024] Perform a 1×1 convolution;

[0025] Three convolutions with different sampling rates were performed in parallel;

[0026] The dilated convolution allows neurons to extract features at different scales by controlling the filling and expansion during convolution.

[0027] The last layer is an ASPP Pooling layer, which consists of a 1×1 pooling layer, a 1×1 convolution layer and an upsampling layer.

[0028] After ASPP completes these five concurrent steps, the obtained features are concatenated and subjected to another 1×1 convolution to integrate the information, obtain high-level features and pass them to the decoder;

[0029] The output is then upsampled,

[0030] Then connect it with the output result of the original module ASPP.

[0031] Get the high-level features of the Encoder and pass this feature into the Decoder model as a replacement for the high-level features of the initial model Encoder.

[0032] Furthermore, the Decoder model includes:

[0033] The decoder receives the output of the two parts of the encoder. Similar to the decoder of Unet, the low-level features are resized by a 1x1 convolution and the stride is set to 4. The high-level features, i.e. the output of ASPP, are upsampled by bilinear interpolation to adjust the data size to 4 times and the stride is adjusted to 4.

[0034] After processing, the two features with the same size are concatenated, subjected to a 3x3 convolution, and then upsampled by 4 times to obtain the output of the decoder;

[0035] The upsampling algorithm is as follows: First, the y direction is sampled:

[0036]

[0037] Then the x direction is sampled to obtain the structure;

[0038] .

[0039] Advantages of the present invention:

[0040] This invention can better identify the smoke area in the image to prevent the occurrence of fire and provide new ideas for the optimization of the DeeplabV3+ model.

[0041] In addition to the above-described objects, features and advantages, the present invention has other objects, features and advantages. The present invention will be further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0043] Figure 1 is the VOC dataset of the present invention;

[0044] Figure 2 This is the original structure of DeeplabV3+ of the present invention;

[0045] Figure 3 It is the optimized Encoder structure of the present invention;

[0046] Figure 4 It is the final result of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0048] The present invention first constructs a VOC dataset, which contains more than 400 images of gray-black and gray-white smoke for training and testing. Figure 1 shown.

[0049] The present invention uses a convolutional neural network. For the integrable functions f(x, y) and g(x, y) in two-dimensional space, the definition of convolution is as follows:

[0050]

[0051] DeeplabV3+ is a semantic segmentation model composed of an Encoder and a Decoder. The improvement of DeeplabV3+ over DeeplabV3 is that DeeplabV3+ imitates Unet to perform an upsampling decoding, but the significant difference from Unet is that DeeplabV3+ has a pyramid structure with hollow convolution.

[0052] The Encoder model structure is as follows Figure 2 As shown:

[0053] The first stage of the Encoder part: This stage performs a simple convolution and upsampling to generate a low-level feature as the input of the second stage and simultaneously passes it to the Decoder part for further processing.

[0054] The second stage of the encoder part: Atrous Spatial Pyramid Pooling (ASPP) is a model inspired by the SPP (Spatial Pyramid Pooling) layer proposed by Kaiming He. Its model is a pyramid-like structure surrounded by two curly brackets in the figure below. As you can see,

[0055] ASPP consists of a 1×1 convolution, a pooling pyramid, i.e. three 3×3 dilated convolutions, and an ASPP pooling.

[0056] ASPP accepts the low-level features of the first-stage input, performs a 1×1 convolution, and then performs three convolutions with different sampling rates (dilation factors) in parallel. This type of convolution is called a dilated convolution. Dilated convolution controls the filling and dilation during convolution, allowing neurons to extract features at different scales.

[0057] The last layer is an ASPP Pooling layer, which consists of a 1×1 pooling, a 1×1 convolution layer and an upsampling layer.

[0058] After ASPP completes these five concurrent steps, the obtained features are concatenated and subjected to another 1×1 convolution to integrate the information. The high-level features are obtained and handed over to the decoder.

[0059] The decoder receives the outputs of both encoders. Similar to the decoder in Unet, the low-level features are resized using a 1x1 convolution with a stride of 4. The high-level features, i.e., the output of the ASPP, are upsampled using bilinear interpolation to 4 times the data size with a stride of 4. The two processed features of equal size are then concatenated, subjected to a 3x3 convolution, and upsampled again by a factor of 4 to produce the decoder output.

[0060] The upsampling algorithm is as follows: First, the y direction is sampled:

[0061]

[0062] Then the x direction is sampled to obtain the structure.

[0063]

[0064] The optimization part of DeeplabV3+ in this invention is mainly in the Encoder part.

[0065] To address the overfitting problem of the DeeplabV3+ model, the improved model after optimization absorbs the characteristics of Unet. After the output of the original ASPP, an ASPP module with a set of different expansion factors is added. The output is then upsampled and then connected with the output result of the original module ASPP to obtain the high-level features of the Encoder and pass this feature into the Decoder part as a replacement for the high-level features of the initial model Encoder.

[0066] Doing so can further connect the location information contained in the large feature map with the semantic information contained in the small feature map, bridging the gap between positioning accuracy and semantic information. For the training of the smoke dataset in DeeplabV3+,

[0067] The present invention proposes an optimized neural network structure model based on DeeplabV3+.

[0068] DeeplabV3+ has borrowed from Unet to a certain extent, which is also the source of the optimization idea of this invention. DeeplabV3+ borrows from Unet in that the low-level features and high-level features imitate unet and perform one upsampling and connection, but Unet has four upsampling and three connections. Therefore, this invention proposes an optimization idea of performing another dilated convolution after the dilated convolution and performing upsampling and connection.

[0069] The optimized model mainly adjusts the second part of the Encoder, such as Figure 3 The final recognition result is shown as Figure 4 shown.

[0070] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A flame early smoke detection method based on improved DeeplabV3+, characterized in that: include: Build a specific dataset and use the DeeplabV3+ model to identify smoke; Combined with Unet to optimize DeeplabV3+ and improve the accuracy of smoke recognition; The construction of a specific dataset and the use of the DeeplabV3+ model for smoke recognition include: Construct a VOC dataset containing more than 400 images of gray-black and gray-white smoke for training and testing; Using convolutional neural networks, for the integrable functions f(x,y) and g(x,y) in two-dimensional space, the definition of convolution is as follows: ， Use DeeplabV3+ model to identify smoke. DeeplabV3+ is a semantic segmentation model consisting of an Encoder model and a Decoder model; The Encoder model includes: The first stage of the Encoder part: This stage performs a simple convolution and upsampling to generate a low-level feature as the input of the second stage and is handed over to the Decoder part for further processing; The second stage of the Encoder part: using the void space convolution pooling pyramid model ASPP for processing, ASPP consists of a 1×1 convolution, a pooling pyramid (three 3×3 dilated convolutions) and an ASPP. pooling composition; ASPP accepts low-level features as input in the first stage; Perform a 1×1 convolution; Three convolutions with different sampling rates were performed in parallel; The dilated convolution allows neurons to extract features at different scales by controlling the filling and expansion during convolution. The last layer is an ASPP Pooling layer, which consists of a 1×1 pooling, a 1×1 convolution layer and an upsampling layer. After ASPP completes the concurrent steps, the obtained features are concatenated and a 1×1 convolution is performed again to integrate the information, obtain high-level features and pass them to the decoder; The output is then upsampled, Then connect it with the output result of the original module ASPP. Get the high-level features of the Encoder and use this feature as the initial model Encoder The replacement results of high-level features are passed into the Decoder model; The Decoder model includes: The decoder receives the output of the two parts of the encoder. Similar to the decoder of Unet, the low-level features are resized by a 1x1 convolution and the stride is set to 4. The high-level features, i.e. the output of ASPP, are upsampled by bilinear interpolation to adjust the data size to 4 times and the stride is adjusted to 4. After processing, the two features with the same size are concatenated, subjected to a 3x3 convolution, and then upsampled by 4 times to obtain the output of the decoder; The upsampling algorithm is as follows: First, the y direction is sampled: ， Then the x direction is sampled to obtain the structure; 。

Images