A lightweight smoke segmentation method and system based on deep learning

By building a lightweight smoke image segmentation network, the problems of unsatisfactory effects of traditional smoke segmentation methods and high hardware requirements of deep learning models are solved, and efficient smoke feature extraction and real-time segmentation on lightweight devices are realized.

CN113936241BActive Publication Date: 2024-11-26SHANGHAI NORMAL UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111245551.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-26
Publication Date
2024-11-26
Estimated Expiration
2041-10-26

AI Technical Summary

Technical Problem

Traditional smoke segmentation methods rely on manual design, resulting in unsatisfactory segmentation effect when the smoke appearance changes. The deep learning model is complex and has too high hardware requirements, making it not suitable for airborne use of lightweight equipment such as drones.

Method used

Design a lightweight smoke segmentation method based on deep learning. By building a lightweight smoke image segmentation network, including smoke space detail branches, deep semantic branches and feature aggregation modules, the model is trained using cross entropy loss and stochastic gradient descent algorithm. Realize efficient extraction and real-time segmentation of smoke characteristics.

Benefits of technology

It realizes efficient extraction of smoke characteristics with small model parameters and low calculation complexity, achieving the purpose of real-time smoke segmentation, balancing the segmentation accuracy and real-timeness, and is suitable for smoke detection and segmentation of lightweight equipment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113936241B_ABST
    Figure CN113936241B_ABST
Patent Text Reader

Abstract

The present invention relates to a lightweight smoke segmentation method and system based on deep learning. The method includes model training and model application. The model training specifically includes: constructing a smoke image data set; building a lightweight smoke segmentation network, including constructing a smoke spatial detail branch at the encoding end, constructing a smoke deep semantic branch at the encoding end, and constructing a feature aggregation module at the decoding end, setting training parameters, using cross entropy loss as the loss function, using Sigmoid as the activation function, and using a random gradient descent algorithm as the network optimization method; training the lightweight smoke segmentation network to obtain a trained lightweight smoke segmentation network. Compared with the prior art, the present invention uses a deep learning network architecture to build a lightweight real-time smoke image segmentation network model. While the network model has a small number of parameters and low computational complexity, it can efficiently extract smoke features, achieve the purpose of real-time smoke segmentation, and achieve a good balance between segmentation accuracy and real-time performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a lightweight smoke segmentation method and system based on deep learning. Background Art

[0002] Frequent fires seriously endanger public safety and life safety, not only causing economic losses, but also causing serious damage to the ecological environment. Timely and accurate fire detection can play a significant role in fire warning and fire fighting. Smoke is the most prominent phenomenon in the early stage of fire, so smoke segmentation is more practical.

[0003] Traditional smoke segmentation methods mainly use manual design to extract visual features of smoke, such as color, shape and texture, and perform feature recognition on the collected images to judge smoke. However, due to the ever-changing appearance of smoke, traditional algorithms mainly rely on the experience of designers, resulting in unsatisfactory segmentation results.

[0004] In recent years, deep learning has achieved great success in various fields, including fire and smoke detection. The technical achievements of applying convolutional neural networks to semantic segmentation have led researchers to apply this type of architecture to smoke segmentation, and have achieved considerable results in smoke segmentation. However, the network complexity and the number of model parameters have increased significantly, and the hardware requirements are too high, which is not conducive to the mounting of lightweight equipment, such as increasing the load of drones. As more and more practical applications require fast and efficient interaction and response, lightweight image segmentation has attracted more and more attention from researchers because it takes into account both segmentation effect and processing speed. Moreover, lightweight image segmentation technology has low hardware requirements, few model parameters, and excellent performance. Therefore, lightweight smoke segmentation methods have become a research hotspot. Summary of the invention

[0005] The purpose of the present invention is to provide a lightweight smoke segmentation method and system based on deep learning in order to overcome the defects of the above-mentioned prior art. A lightweight real-time smoke image segmentation network model is built by using a deep learning network architecture. While the network model has a small number of parameters and low computational complexity, it can efficiently extract smoke features, thereby achieving the purpose of real-time smoke segmentation and striking a good balance between segmentation accuracy and real-time performance.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A lightweight smoke segmentation method based on deep learning, including model training and model application. The model training is specifically as follows:

[0008] Construct a smoke image dataset with annotations and labels; build a lightweight smoke segmentation network, including constructing a smoke spatial detail branch at the encoding end, constructing a smoke deep semantic branch at the encoding end, and constructing a feature aggregation module at the decoding end, and then set the training parameters in the lightweight smoke segmentation network, the loss function uses cross entropy loss, the activation function uses Sigmoid, and the network optimization method uses a stochastic gradient descent algorithm. The encoding end extracts features from the input image to obtain a spatial detail feature map and a deep semantic feature map, and the decoding end obtains a segmentation result map based on the spatial detail feature map and the deep semantic feature map; use the smoke images in the smoke image dataset to train the lightweight smoke segmentation network to obtain a trained lightweight smoke segmentation network;

[0009] The model applications are as follows:

[0010] Equipped with a trained lightweight smoke segmentation network, smoke detection and segmentation are performed to obtain the segmentation result map.

[0011] Furthermore, constructing a smoke image database with annotations and labels is specifically as follows: collecting images with smoke caused by fire, adding labels to the images and annotating masks to obtain a smoke image dataset; flipping, cropping, translating and scaling the images in the smoke image dataset to expand the smoke image dataset.

[0012] Furthermore, it also includes: synthesizing multiple smoke images, adding labels and annotating masks, and then merging them into the smoke image dataset to further expand the smoke image dataset.

[0013] Furthermore, the encoding end constructs the smoke space detail branch as follows:

[0014] Build a smoke space detail branch, which includes the first stage layer, the second stage layer and the third stage layer. Each stage layer includes two connected convolutional layers. The convolution operation of each convolutional layer is connected to batch normalization and activation function. The input image is sent to the first stage layer, the output of the first stage layer is sent to the second stage layer, the output of the second stage layer is sent to the third stage layer, and the output of the third stage layer is sent to the feature aggregation module at the decoding end.

[0015] Furthermore, the encoding end constructs the deep semantic branches of smoke as follows:

[0016] A deep semantic branch of smoke is built, including an improved self-correction convolution layer, a first aggregation expansion layer, a second aggregation expansion layer and a context embedding block. The input image is sent to the improved self-correction convolution layer after convolution operation. The output of the improved self-correction convolution layer is sent to the first aggregation expansion layer together with the output of the second stage layer of the smoke space detail branch after convolution operation. The output of the first aggregation expansion layer is sent to the second aggregation expansion layer together with the output of the third stage layer of the smoke space detail branch after convolution operation. The output of the second aggregation expansion layer is sent to the context embedding block after convolution operation, and the output of the context embedding block is sent to the feature aggregation module at the decoding end.

[0017] Furthermore, the decoding end constructs a feature aggregation module specifically as follows:

[0018] Build a feature aggregation module, including a convolution layer and two parallel branches. The convolution layer has a 1×1 convolution kernel and a Sigmoid activation function. The first branch includes a 1×1 convolution layer, a 3×3 convolution layer, and a Sigmoid activation function layer. The second branch includes a 3×3 convolution layer and a Sigmoid activation function layer. The feature map Y output by the smoke deep semantic branch S After the 3×3 convolution layer and Sigmoid activation function layer of the second branch and up-sampled to the original size of the input image, the output segmentation prediction image Y is obtained. S ′; Feature map Y output by the deep semantic branch of smoke S After upsampling to the feature map Y output by the smoke space detail branch D The size of the smoke space detail branch output feature map Y D Perform the concatenation operation, and reduce the dimension of the result of the concatenation operation through the 1×1 convolution layer of the first branch, and then pass it through the 3×3 convolution layer and the Sigmoid activation function layer and upsample it to the original size of the input image to obtain the output segmentation prediction image Y D ′; Output segmentation prediction map Y D ′ and output segmentation prediction map Y S After performing the Element-wise Sum operation, the image is sent to a convolutional layer with a 1×1 convolution kernel and a Sigmoid activation function to obtain the segmentation result map Y O .

[0019] Furthermore, the training parameters in the lightweight smoke segmentation network are set to include:

[0020] The images in the smoke image dataset are used as input images, and the size of the input images is unified. The weight initialization method in the lightweight smoke segmentation network is Kaiming. The lightweight smoke segmentation network is trained using the stochastic gradient descent algorithm with momentum. The number of iterations is 100, the weight decay is 1e-5, the momentum is 0.9, the batch size is 32, the initial learning rate is 0.01, and the learning rate iteration strategy is "Poly".

[0021] A lightweight smoke segmentation system based on deep learning, including:

[0022] An image acquisition module is used to acquire scene images and pre-process the scene images;

[0023] The smoke segmentation module is equipped with a trained lightweight smoke segmentation network, which is used to perform real-time smoke detection and segmentation on scene images to obtain segmentation result images;

[0024] The training process of the lightweight smoke segmentation network is specifically as follows:

[0025] Construct a smoke image dataset with annotations and labels; build a lightweight smoke segmentation network, including constructing a smoke spatial detail branch at the encoding end, constructing a smoke deep semantic branch at the encoding end, and constructing a feature aggregation module at the decoding end, and then set the training parameters in the lightweight smoke segmentation network, the loss function uses cross entropy loss, the activation function uses Sigmoid, and the network optimization method uses a stochastic gradient descent algorithm. The encoding end extracts features from the input image to obtain a spatial detail feature map and a deep semantic feature map, and the decoding end obtains a segmentation result map based on the spatial detail feature map and the deep semantic feature map; use the smoke images in the smoke image dataset to train the lightweight smoke segmentation network to obtain a trained lightweight smoke segmentation network.

[0026] Furthermore, the encoding end constructs the smoke space detail branch as follows:

[0027] Build a smoke space detail branch, which includes the first stage layer, the second stage layer and the third stage layer. Each stage layer includes two connected convolutional layers. The convolution operation of each convolutional layer is connected to batch normalization and activation function. The input image is sent to the first stage layer, the output of the first stage layer is sent to the second stage layer, the output of the second stage layer is sent to the third stage layer, and the output of the third stage layer is sent to the feature aggregation module at the decoding end.

[0028] Furthermore, the encoding end constructs the deep semantic branches of smoke as follows:

[0029] A deep semantic branch of smoke is built, including an improved self-correction convolution layer, a first aggregation expansion layer, a second aggregation expansion layer and a context embedding block. The input image is sent to the improved self-correction convolution layer after convolution operation. The output of the improved self-correction convolution layer is sent to the first aggregation expansion layer together with the output of the second stage layer of the smoke space detail branch after convolution operation. The output of the first aggregation expansion layer is sent to the second aggregation expansion layer together with the output of the third stage layer of the smoke space detail branch after convolution operation. The output of the second aggregation expansion layer is sent to the context embedding block after convolution operation, and the output of the context embedding block is sent to the feature aggregation module at the decoding end.

[0030] Furthermore, the decoding end constructs a feature aggregation module specifically as follows:

[0031] Build a feature aggregation module, including a convolution layer and two parallel branches. The convolution layer has a 1×1 convolution kernel and a Sigmoid activation function. The first branch includes a 1×1 convolution layer, a 3×3 convolution layer, and a Sigmoid activation function layer. The second branch includes a 3×3 convolution layer and a Sigmoid activation function layer. The feature map Y output by the smoke deep semantic branch S After the 3×3 convolution layer and Sigmoid activation function layer of the second branch and up-sampled to the original size of the input image, the output segmentation prediction image Y is obtained. S ′; Feature map Y output by the deep semantic branch of smoke S After upsampling to the feature map Y output by the smoke space detail branch D The size of the smoke space detail branch output feature map Y D Perform the concatenation operation, and reduce the dimension of the result of the concatenation operation through the 1×1 convolution layer of the first branch, and then pass it through the 3×3 convolution layer and the Sigmoid activation function layer and upsample it to the original size of the input image to obtain the output segmentation prediction image Y D ′; Output segmentation prediction map Y D ′ and output segmentation prediction map Y S After performing the Element-wise Sum operation, the image is sent to a convolutional layer with a 1×1 convolution kernel and a Sigmoid activation function to obtain the segmentation result map Y O .

[0032] Compared with the prior art, the present invention improves the smoke feature extraction method in the encoding stage, which can achieve accurate positioning and detailed feature extraction of smoke targets of different scales and smoke with large differences in shape. The smoke spatial detail branch can accurately locate smoke puffs with large differences in scale and shape of smoke targets; the smoke deep semantic branch integrates the smoke spatial detail branch, which can achieve the extraction of more semantic features of smoke when the model parameters are low and the network is deep. The feature aggregation module at the decoding end enhances the expression of smoke feature information by reintegrating the spatial detail feature map and the deep semantic feature map, and obtains accurate smoke segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Schematic diagram of lightweight smoke segmentation network;

[0034] Figure 2 is a schematic diagram of the feature aggregation module;

[0035] Figure 3 A flowchart for the model application;

[0036] Figure 4 The smoke segmentation results obtained using the lightweight smoke segmentation network. DETAILED DESCRIPTION

[0037] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0038] Embodiment 1:

[0039] In some application scenarios, large equipment cannot enter, but convenient small equipment such as drones can detect in narrow and complex environments. In terms of fire detection, small equipment can detect fires in a timely and accurate manner. However, existing deep learning models often require a large amount of computing resources to achieve target detection. Small equipment has limited onboard capacity. In order to save computing resources, it is difficult to load deep learning models on small equipment, and it is impossible to rely on small equipment to achieve autonomous target detection.

[0040] This application uses a deep learning network architecture to build a lightweight real-time smoke image segmentation network model. While the network model has a small number of parameters and low computational complexity, it can efficiently extract smoke features, achieve the purpose of real-time smoke segmentation, and achieve a good balance between segmentation accuracy and real-time performance. The lightweight smoke segmentation network of this application can achieve fast and accurate smoke positioning in conjunction with small equipment, and promptly inform relevant monitoring personnel of the real-time feedback results.

[0041] A lightweight smoke segmentation method based on deep learning, including model training and model application. The model training is specifically as follows:

[0042] Construct a smoke image dataset with annotations and labels; build a lightweight smoke segmentation network, such as Figure 1 As shown, the encoding end constructs a smoke spatial detail branch, the encoding end constructs a smoke deep semantic branch, and the decoding end constructs a feature aggregation module, and then sets the training parameters in the lightweight smoke segmentation network. The loss function uses the cross entropy loss, the activation function uses the Sigmoid, and the network optimization method uses the stochastic gradient descent algorithm. The encoding end extracts features from the input image to obtain a spatial detail feature map and a deep semantic feature map. The decoding end obtains a segmentation result map based on the spatial detail feature map and the deep semantic feature map. The smoke image in the smoke image dataset is used to train the lightweight smoke segmentation network to obtain a trained lightweight smoke segmentation network.

[0043] like Figure 3 As shown, the model application is as follows:

[0044] Equipped with a trained lightweight smoke segmentation network, smoke detection and segmentation are performed. The image is input to the encoding end to extract smoke features, and then sent to the decoding end for feature aggregation and decoding to obtain an accurate smoke segmentation result image.

[0045] The construction of a smoke image database with annotations and labels is as follows: collect images with smoke caused by fire, add labels to the images and annotate masks to obtain a smoke image dataset, the format of which is: the original image is in jpg format, and the mask is in png format; flip, crop, translate and scale the images in the smoke image dataset to expand the smoke image dataset, which can improve the generalization ability of the lightweight smoke segmentation network in subsequent training. Finally, multiple smoke images are synthesized by computer, and then merged into the smoke image dataset after adding labels and annotating masks. The synthesized smoke images are mixed with the real smoke images, which can further prevent the network from overfitting in subsequent training.

[0046] When constructing a smoke image database, image collection should be performed for various scenes, covering smoke images in all scenes, such as indoors and outdoors. Similarly, computer-synthesized smoke images should also cover smoke images in all scenes.

[0047] The lightweight smoke segmentation network built is as follows Figure 1 As shown, the encoding end includes a parallel smoke spatial detail branch and a smoke deep semantic branch, and the decoding end is provided with a feature aggregation module.

[0048] 1) The encoding end constructs the smoke space detail branch as follows:

[0049] Build the smoke space detail branch, such as Figure 1 As shown, the smoke space detail branch includes the first stage layer, the second stage layer and the third stage layer. Each stage layer includes two connected convolutional layers. Each convolutional layer is used to perform convolution operations. After the convolution operation of each convolutional layer, a batch normalization function and an activation function are connected. The input image is sent to the first stage layer, the output of the first stage layer is sent to the second stage layer, the output of the second stage layer is sent to the third stage layer, and the output of the third stage layer is sent to the feature aggregation module at the decoding end.

[0050] For each stage layer, the convolution step size s of the first convolution layer is set to 2 to facilitate downsampling of the image sent to the stage layer, and the convolution kernel of each convolution layer is set to 3. The image input to the lightweight smoke segmentation network is converted from the initial RGB three channels through the first stage layer to obtain 1 / 2 of the original image with 32 channels. Similarly, the second stage layer obtains 1 / 4 of the original image with 64 channels, and the third stage layer obtains 1 / 8 of the original image with 128 channels. In the same stage layer, the convolution operations of the two convolution layers have the same number of channels and the same output size, and the second convolution operation in the first, second and third stage convolution layers can be repeated several times. As in this embodiment, the size of the input image is 256×256, and the parameter settings of the two convolution layers in the first stage layer D1 are shown in Table 1:

[0051] Table 1 Parameter settings of the two convolutional layers in the first stage

[0052]

[0053] After feature extraction is performed on the smoke spatial detail branch, a spatial detail feature map is obtained. The size of the output spatial detail feature map is 1 / 8 of the input image sent to the lightweight smoke segmentation network.

[0054] 2) The encoding end constructs the deep semantic branches of smoke as follows:

[0055] Build deep semantic branches of smoke, such as Figure 1 As shown, it includes an improved self-correction convolution layer, a first aggregation expansion layer, a second aggregation expansion layer and a context embedding block. The input image is sent to the improved self-correction convolution layer after the convolution operation. The output of the improved self-correction convolution layer is sent to the first aggregation expansion layer together with the output of the second stage layer of the smoke space detail branch after the convolution operation. The output of the first aggregation expansion layer is sent to the second aggregation expansion layer together with the output of the third stage layer of the smoke space detail branch after the convolution operation. The output of the second aggregation expansion layer is sent to the context embedding block after the convolution operation, and the output of the context embedding block is sent to the feature aggregation module at the decoding end.

[0056] Firstly, the input image of the lightweight smoke segmentation network is up-dimensioned and down-sampled by convolution operation, and then sent to the improved self-correction convolution layer to expand the receptive field of smoke features, and a 1 / 2 smoke feature map is obtained through the improved self-correction convolution module; the output of the improved self-correction convolution layer is sent to the first aggregation extension layer together with the output of the second stage layer of the smoke spatial detail branch after the convolution operation to obtain a 1 / 4 smoke feature map; the output of the first aggregation extension layer is sent to the second aggregation extension layer together with the output of the third stage layer of the smoke spatial detail branch after the convolution operation to obtain a 1 / 8 smoke feature map, so as to achieve the purpose of the smoke spatial detail branch guiding the deep semantic branch of the smoke; finally, the output of the second aggregation extension layer is further up-dimensioned and down-sampled by convolution operation, and then sent to the context embedding block to enhance the global context information.

[0057] 3) The feature aggregation module constructed at the decoding end is as follows:

[0058] Build feature aggregation modules, such as Figure 2 As shown in FIG. 1 , it includes a convolution layer and two parallel branches. The convolution layer has a 1×1 convolution kernel and a Sigmoid activation function. The first branch includes a 1×1 convolution layer, a 3×3 convolution layer, and a Sigmoid activation function layer. The second branch includes a 3×3 convolution layer and a Sigmoid activation function layer. The feature map Y output by the deep semantic branch of the smoke S After the 3×3 convolution layer and Sigmoid activation function layer of the second branch and up-sampled to the original size of the input image, the output segmentation prediction image Y is obtained. S ′; Feature map Y output by the deep semantic branch of smoke S After upsampling to the feature map Y output by the smoke space detail branch D The size of the smoke space detail branch output feature map Y D Perform the concatenation operation, and reduce the dimension of the result of the concatenation operation through the 1×1 convolution layer of the first branch, and then pass it through the 3×3 convolution layer and the Sigmoid activation function layer and upsample it to the original size of the input image to obtain the output segmentation prediction image Y D ′; Output segmentation prediction map Y D ′ and output segmentation prediction map Y S After performing the Element-wise Sum operation, the image is sent to a convolutional layer with a 1×1 convolution kernel and a Sigmoid activation function to obtain the segmentation result map Y O , the size of the segmentation result image is restored to the size of the input image.

[0059] At the decoding end, a feature aggregation module is constructed to integrate deep semantic information into spatial detail information. In the feature aggregation module, the information of the smoke spatial detail branch and the information of the smoke deep semantic branch are reintegrated to enhance the expression of feature information, obtain accurate smoke segmentation results, and generate the final smoke prediction map.

[0060] Among them, when setting the training parameters in the lightweight smoke segmentation network, the loss function uses the cross entropy loss, the activation function uses the Sigmoid, and the network optimization method uses the stochastic gradient descent algorithm. In addition, it also includes:

[0061] The images in the smoke image dataset are used as input images, and the size of the input images is unified, such as 256×256. The weight initialization method in the lightweight smoke segmentation network is Kaiming. The lightweight smoke segmentation network is trained using the stochastic gradient descent algorithm with momentum. The number of iterations is 100, the weight decay is 1e-5, the momentum is 0.9, the batch size is 32, the initial learning rate is 0.01, and the learning rate iteration strategy is "Poly".

[0062] A lightweight smoke segmentation system based on deep learning, including:

[0063] An image acquisition module is used to acquire scene images and pre-process the scene images;

[0064] The smoke segmentation module is equipped with a trained lightweight smoke segmentation network, which is used to perform real-time smoke detection and segmentation on scene images to obtain segmentation result images;

[0065] The training process of the lightweight smoke segmentation network is as follows:

[0066] Construct a smoke image dataset with annotations and labels; build a lightweight smoke segmentation network, including constructing a smoke spatial detail branch on the encoding end, constructing a smoke deep semantic branch on the encoding end, and constructing a feature aggregation module on the decoding end, and then set the training parameters in the lightweight smoke segmentation network. The loss function uses cross entropy loss, the activation function uses Sigmoid, and the network optimization method uses the stochastic gradient descent algorithm. The encoding end extracts features from the input image to obtain a spatial detail feature map and a deep semantic feature map. The decoding end obtains a segmentation result map based on the spatial detail feature map and the deep semantic feature map. Use the smoke images in the smoke image dataset to train the lightweight smoke segmentation network to obtain a trained lightweight smoke segmentation network.

[0067] This embodiment takes the real-time semantic segmentation of video images as an example, installs the trained lightweight smoke segmentation network into the corresponding equipment, calls the camera device to obtain real-time images, then processes the read video frame by frame, and converts the image format, and inputs the pre-processed video image frame by frame into the trained lightweight smoke segmentation network, which performs detection and segmentation, and issues an alarm when smoke is detected. Figure 4 The segmentation results of smoke detection in different scenes are given.

[0068] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A lightweight smoke segmentation method based on deep learning, characterized in that: Including model training and model application, model training is specifically: Construct a smoke image dataset with annotations and labels; build a lightweight smoke segmentation network, including constructing a smoke spatial detail branch at the encoding end, constructing a smoke deep semantic branch at the encoding end, and constructing a feature aggregation module at the decoding end, and then set the training parameters in the lightweight smoke segmentation network, the loss function uses cross entropy loss, the activation function uses Sigmoid, and the network optimization method uses a stochastic gradient descent algorithm. The encoding end extracts features from the input image to obtain a spatial detail feature map and a deep semantic feature map, and the decoding end obtains a segmentation result map based on the spatial detail feature map and the deep semantic feature map; use the smoke images in the smoke image dataset to train the lightweight smoke segmentation network to obtain a trained lightweight smoke segmentation network; The model applications are as follows: Equipped with a trained lightweight smoke segmentation network, smoke detection and segmentation are performed to obtain a segmentation result map; The encoding end constructs the smoke space detail branch as follows: Build a smoke space detail branch, which includes the first stage layer, the second stage layer and the third stage layer. Each stage layer includes two connected convolutional layers. The convolution operation of each convolutional layer is connected to batch normalization and activation function. The input image is sent to the first stage layer, the output of the first stage layer is sent to the second stage layer, the output of the second stage layer is sent to the third stage layer, and the output of the third stage layer is sent to the feature aggregation module at the decoding end; The encoding end constructs the deep semantic branches of smoke as follows: Build a smoke deep semantic branch, including an improved self-correction convolution layer, a first aggregation expansion layer, a second aggregation expansion layer and a context embedding block. The input image is sent to the improved self-correction convolution layer after the convolution operation. The output of the improved self-correction convolution layer is sent to the first aggregation expansion layer together with the output of the second stage layer of the smoke space detail branch after the convolution operation. The output of the first aggregation expansion layer is sent to the second aggregation expansion layer together with the output of the third stage layer of the smoke space detail branch after the convolution operation. The output of the second aggregation expansion layer is sent to the context embedding block after the convolution operation. The output of the context embedding block is sent to the feature aggregation module at the decoding end. The decoding end constructs the feature aggregation module as follows: Build a feature aggregation module, including a convolution layer and two parallel branches. The convolution layer has a 1×1 convolution kernel and a Sigmoid activation function. The first branch includes a 1×1 convolution layer, a 3×3 convolution layer, and a Sigmoid activation function layer. The second branch includes a 3×3 convolution layer and a Sigmoid activation function layer. The feature map Y output by the smoke deep semantic branch S After the 3×3 convolution layer and Sigmoid activation function layer of the second branch and up-sampled to the original size of the input image, the output segmentation prediction image Y is obtained. S ′; Feature map Y output by the deep semantic branch of smoke S After upsampling to the feature map Y output by the smoke space detail branch D The size of the smoke space detail branch output feature map Y D Perform the concatenation operation, and reduce the dimension of the result of the concatenation operation through the 1×1 convolution layer of the first branch, and then pass it through the 3×3 convolution layer and the Sigmoid activation function layer and upsample it to the original size of the input image to obtain the output segmentation prediction image Y D ′; Output segmentation prediction map Y D ′ and output segmentation prediction map Y S After performing the Element-wise Sum operation, the image is sent to a convolutional layer with a 1×1 convolution kernel and a Sigmoid activation function to obtain the segmentation result map Y O .

2. According to the light-weight smoke segmentation method based on deep learning in claim 1, it is characterized in that: The specific steps of building a smoke image database with annotations and labels are as follows: collecting images with smoke caused by fire, adding labels to the images and annotating masks to obtain a smoke image dataset; flipping, cropping, translating and scaling the images in the smoke image dataset to expand the smoke image dataset.

3. The light-weight smoke segmentation method based on deep learning according to claim 2, characterized in that: Also includes: Synthesize multiple smoke images, add labels and annotate masks, and merge them into the smoke image dataset to further expand the smoke image dataset.

4. A lightweight smoke segmentation system based on deep learning, characterized in that: A lightweight smoke segmentation method based on deep learning according to any one of claims 1 to 3, comprising: An image acquisition module is used to acquire scene images and pre-process the scene images; The smoke segmentation module is equipped with a trained lightweight smoke segmentation network, which is used to perform real-time smoke detection and segmentation on scene images to obtain segmentation result images; The training process of the lightweight smoke segmentation network is specifically as follows: Construct a smoke image dataset with annotations and labels; build a lightweight smoke segmentation network, including constructing a smoke spatial detail branch at the encoding end, constructing a smoke deep semantic branch at the encoding end, and constructing a feature aggregation module at the decoding end, and then set the training parameters in the lightweight smoke segmentation network, the loss function uses cross entropy loss, the activation function uses Sigmoid, and the network optimization method uses a stochastic gradient descent algorithm. The encoding end extracts features from the input image to obtain a spatial detail feature map and a deep semantic feature map, and the decoding end obtains a segmentation result map based on the spatial detail feature map and the deep semantic feature map; use the smoke images in the smoke image dataset to train the lightweight smoke segmentation network to obtain a trained lightweight smoke segmentation network.

5. The light-weight smoke segmentation system based on deep learning according to claim 4, characterized in that: The encoding end constructs the smoke space detail branch as follows: Build a smoke space detail branch, which includes the first stage layer, the second stage layer and the third stage layer. Each stage layer includes two connected convolutional layers. The convolution operation of each convolutional layer is connected to batch normalization and activation function. The input image is sent to the first stage layer, the output of the first stage layer is sent to the second stage layer, the output of the second stage layer is sent to the third stage layer, and the output of the third stage layer is sent to the feature aggregation module at the decoding end.

6. The light-weight smoke segmentation system based on deep learning according to claim 5, characterized in that: The encoding end constructs the deep semantic branches of smoke as follows: A deep semantic branch of smoke is built, including an improved self-correction convolution layer, a first aggregation expansion layer, a second aggregation expansion layer and a context embedding block. The input image is sent to the improved self-correction convolution layer after convolution operation. The output of the improved self-correction convolution layer is sent to the first aggregation expansion layer together with the output of the second stage layer of the smoke space detail branch after convolution operation. The output of the first aggregation expansion layer is sent to the second aggregation expansion layer together with the output of the third stage layer of the smoke space detail branch after convolution operation. The output of the second aggregation expansion layer is sent to the context embedding block after convolution operation, and the output of the context embedding block is sent to the feature aggregation module at the decoding end.

7. The light-weight smoke segmentation system based on deep learning according to claim 4, characterized in that: The decoding end constructs the feature aggregation module as follows: Build a feature aggregation module, including a convolution layer and two parallel branches. The convolution layer has a 1×1 convolution kernel and a Sigmoid activation function. The first branch includes a 1×1 convolution layer, a 3×3 convolution layer, and a Sigmoid activation function layer. The second branch includes a 3×3 convolution layer and a Sigmoid activation function layer. The feature map Y output by the smoke deep semantic branch S After the 3×3 convolution layer and Sigmoid activation function layer of the second branch and up-sampled to the original size of the input image, the output segmentation prediction image Y is obtained. S ′; Feature map Y output by the deep semantic branch of smoke S After upsampling to the feature map Y output by the smoke space detail branch D The size of the smoke space detail branch output feature map Y D Perform the concatenation operation, and reduce the dimension of the result of the concatenation operation through the 1×1 convolution layer of the first branch, and then pass it through the 3×3 convolution layer and the Sigmoid activation function layer and upsample it to the original size of the input image to obtain the output segmentation prediction image Y D ′; Output segmentation prediction map Y D ′ and output segmentation prediction map Y S After performing the Element-wise Sum operation, the image is sent to a convolutional layer with a 1×1 convolution kernel and a Sigmoid activation function to obtain the segmentation result map Y O .