A multi-scale lightweight smoke image segmentation method and device

Abstract

The application discloses a multi-scale light-weight smoke image segmentation method and device, and belongs to the technical field of smoke image semantic segmentation. The method comprises the following steps: acquiring a smoke image dataset and performing pretreatment; constructing a multi-scale light-weight convolutional neural network, wherein the network takes a ResNet50 residual structure as a backbone network, further comprises three multi-scale light-weight grouping attention modules, a multi-level context information aggregation module and a cross fusion gating module; training the network and performing evaluation; inputting smoke image data to be segmented into the trained network to obtain a segmentation result. The application introduces a light-weight attention module, uses a channel mixed grouping processing method to fully utilize information between views, and reduces the calculation amount of data; uses an upsampling and gating method to process feature information at different levels, further improves the correlation degree between low-level information and high-level information, and obtains a higher-quality segmentation result.

Description

Technical Field

[0001] This invention relates to the field of semantic segmentation technology for smoke images, and particularly to a multi-scale lightweight smoke image segmentation method and apparatus. Background Technology

[0002] The fundamental purpose of fire early warning for smoke images and videos is a classification problem to determine whether there is sufficient smoke in the video frame to trigger an alarm. Therefore, smoke monitoring tasks can be divided into four granularities, from coarse to fine, corresponding to smoke recognition, smoke detection, smoke segmentation, and smoke concentration estimation. This paper focuses on the smoke segmentation task, which, based on label values, can be divided into hard segmentation and soft segmentation. The former is binary classification, while the latter is regression or fuzzy classification. From the perspective of algorithm classification granularity, it can be divided into coarse smoke region segmentation and pixel-by-pixel fine segmentation. The former aims to separate smoke components to assist in localization, and in videos, it usually relies on motion features; the latter is dense classification, and currently, methods capable of pixel-by-pixel segmentation include traditional smoke segmentation methods and deep learning-based methods.

[0003] In traditional smoke segmentation methods, those that rely more heavily on statistical learning processes are better able to achieve finer-grained tasks. Regarding feature extraction, dynamic features are often used for coarse extraction to minimize false negatives; stable static features such as texture are frequently used to help reduce false alarms in smoke area identification; when video resolution is low, texture features have limited effectiveness, while other features such as color can help exclude non-smoke features. However, without effectively combining these visual features, false alarm rates are still difficult to avoid. Furthermore, the evaluation criteria for smoke detection and segmentation based on traditional methods mostly focus on recognition performance, while the effectiveness of intra-frame localization (i.e., segmentation accuracy) can mostly only be subjectively evaluated through visualization.

[0004] Some of the aforementioned problems can be solved using deep learning methods. Although deep learning has not been widely used as long as traditional methods, it has already achieved higher efficiency and accuracy than traditional methods in finer-grained smoke monitoring tasks. Lin et al. extracted spatial texture features (volume LBP) from smoke sequences for smoke detection based on sampling blocks. Filonenko et al. also used color-based secondary smoke segmentation and added edge roughness features to distinguish non-smoke objects with the same color as the smoke, thereby reducing false alarms. In recent years, with the deepening of algorithm research in the field of deep learning, research on deep learning in the field of smoke image segmentation and detection has gradually received more attention. Liu Zhiying et al. proposed a smoke region segmentation algorithm based on an improved Deeplabv3+ model, innovatively using a dilated convolution-based pyramid configuration (ASPP) structure with heterogeneous receptive field fusion to further expand the receptive field of dilated convolution and reduce the adverse effects of information loss; optimized the backbone network by adding a multi-scale fusion module to reduce the number of network parameters and computational cost; and introduced a channel attention module to enhance the feature learning ability of key channels, thereby improving the model training speed and segmentation accuracy. Yuan et al. constructed basic modules using parallel convolutional layers with the same number of channels but different kernel sizes to extract multi-scale smoke features. Each module aggregated the outputs of each branch, and smoke detection was achieved by stacking multiple basic modules. Jia Yang et al. used a deep neural network with a U-Net structure for automatic segmentation of smoke in early fires. They manually segmented image samples of smoke regions using a semi-automatic algorithm, and then learned the segmented smoke regions based on the deep neural network to obtain a mapping model from the original video frames to the segmentation results. This model was then used for smoke region segmentation. All of the above methods achieve smoke detection by constructing deep convolutional neural networks and automatically learning features from training data. However, due to the semi-transparency of smoke and its irregular shape leading to its diffusion and continuity, existing deep learning algorithms still need improvement in effectively acquiring semantic information and multi-scale contextual information. Given the real-time requirements of smoke detection, the issue of lightweight network computation should also be considered in the algorithm process. Summary of the Invention

[0005] This invention addresses the current state of research in the field of semantic segmentation of smoke images by proposing a multi-scale lightweight smoke image segmentation network model. It fully utilizes contextual feature information at different scales to improve the network's ability to acquire global information, and obtains higher-quality semantic segmentation results for smoke images through lightweight grouping and gating modules.

[0006] To achieve the above objectives, this invention proposes a multi-scale lightweight smoke image segmentation method, comprising the following steps:

[0007] (1) Obtain a smoke image dataset and preprocess the smoke image dataset;

[0008] (2) Construct a multi-scale lightweight convolutional neural network. The multi-scale lightweight convolutional neural network uses the ResNet50 residual structure as the backbone network and also includes three multi-scale lightweight group attention modules (MSA), a multi-level context information aggregation module (MCA), and a cross-fusion gating module (CLGD).

[0009] (3) The multi-scale lightweight convolutional neural network is trained using the preprocessed smoke image dataset, and the semantic segmentation results of the trained network on multiple smoke image datasets are evaluated.

[0010] (4) Input the smoke image data to be segmented into the trained multi-scale lightweight convolutional neural network to obtain the segmentation result.

[0011] According to the aforementioned multi-scale lightweight smoke image segmentation method, the preprocessing includes:

[0012] Enhance contrast and crop along the image center;

[0013] The smoke image dataset is divided into a training set and a test set.

[0014] Furthermore, the multi-scale lightweight convolutional neural network specifically comprises:

[0015] The multi-scale lightweight convolutional neural network uses a ResNet50 residual structure as the backbone network to extract features from the input smoke image.

[0016] The feature maps extracted from Layer 2, Layer 3, and Layer 4 in the residual structure are each processed by a 3x3 convolution to extract features and the number of channels is unified. The output feature maps are then fed into the multi-scale lightweight grouped attention module MSA.

[0017] The output of the multi-scale lightweight group attention module (MSA) at each level is used as the input of the multi-level context information aggregation module (MCA). This module uses a method based on the aggregation of context information at different levels and dense connections to obtain and adapt feature maps at different scales. It achieves accurate semantic segmentation by aggregating information from different stages.

[0018] The output of the multi-level context information aggregation module MCA is used as a high-level feature, and the feature map extracted from Layer 1 in the residual structure is used as a low-level feature. The input is fed into the cross-fusion gating module CLGD. The high-level image features are used as a gating to assign weights to the low-level features, thereby effectively selecting useful low-level features and high-level features to stack and output the result.

[0019] Furthermore, the multi-scale lightweight grouped attention module (MSA) includes n sets of cascaded spatial attention modules and channel attention modules, an aggregation module, and a channel mixing module.

[0020] The input feature map is divided into n groups along the channel dimension, and each group is input into the concatenated channel attention module and spatial attention module.

[0021] The channel attention module aggregates the channel information of the feature map through two parallel average pooling layers and max pooling layers, and then inputs them into a shared 1x1 convolutional layer, a ReLU activation function, and a shared 1x1 convolutional layer connected in sequence. The two outputs are then added element-wise and passed through a sigmoid activation function to obtain the output of the channel attention module.

[0022] The spatial attention module passes the output of the channel attention module through two parallel max pooling layers and average pooling layers. The two outputs are concatenated by concat, then transformed into a 1-channel feature map through a 7x7 convolution. After passing through a sigmoid activation function, the feature map is multiplied by the original input of the spatial attention module and then convolved to obtain the output of the spatial attention module.

[0023] The output of the spatial attention module is multiplied by the input of the channel attention module and then output. The n sets of outputs are concatted by the aggregation module and then passed through the channel mixing module to enable cross-group information flow along the channel dimension, enabling information exchange between different sub-features and outputting the final result.

[0024] Furthermore, the channel mixing module divides the input information into n groups along the channel dimension, totaling... There are 10 channels, where c is the total number of channels, formed by reshaping. The matrix is ​​then transposed to... After flattening, the data is divided into n groups and then connected sequentially for output.

[0025] Furthermore, the multi-level context information aggregation module (MCA) specifically comprises:

[0026] The output results of the multi-scale lightweight grouped attention module MSA at each level are magnified to the same size and then added between branches.

[0027] The output of Layer 2 in the residual structure is sequentially passed through an average pooling layer, a 1x1 convolution, and a sigmoid activation function, and then multiplied with the output of Layer 2 in the residual structure after passing through the multi-scale lightweight grouped attention module MSA. Then, each branch is convolved laterally.

[0028] Finally, all convolutional outputs are enlarged to 1 / 8 the size of the original image and then concatenated for output.

[0029] Furthermore, the cross-fusion gating module CLGD specifically comprises:

[0030] The low-level and high-level features are concatenated after undergoing 3x3 convolution and 2x upsampling, then concatenated, followed by 1x1 convolution and sigmoid activation, and multiplied by the result of the 3x3 convolution of the low-level features. Finally, the result of the 2x upsampling of the high-level features is concatenated, followed by two sets of 3x3 convolutions and one 1x1 convolution, and then upsampled to restore the original feature map size before being output.

[0031] Furthermore, the evaluation uses mIoU and is calculated as follows:

[0032]

[0033] Where: TP represents both the true value and the predicted value as Positive; FN represents the true value as Positive and the predicted value as Negative; FP represents the true value as Negative and the predicted value as Positive.

[0034] The present invention also provides a multi-scale lightweight smoke image segmentation device, comprising:

[0035] Acquisition unit: used to acquire smoke image dataset and preprocess the smoke image dataset;

[0036] Modeling unit: used to construct a multi-scale lightweight convolutional neural network, which uses ResNet50 residual structure as the backbone network and also includes three multi-scale lightweight group attention modules (MSA), a multi-level context information aggregation module (MCA), and a cross-fusion gating module (CLGD).

[0037] Training unit: used to train the multi-scale lightweight convolutional neural network using the preprocessed smoke image dataset, and to evaluate the semantic segmentation results of the trained network on multiple smoke image datasets;

[0038] Segmentation unit: Input the smoke image data to be segmented into the trained multi-scale lightweight convolutional neural network to obtain the segmentation result.

[0039] The beneficial effects of this invention are:

[0040] (1) This invention introduces a lightweight attention module and uses a channel-mixing grouping processing method to make full use of information between views, thereby reducing the amount of data computation.

[0041] (2) This invention processes multi-scale feature information and uses upsampling and gating methods to process feature information at different levels, thereby further improving the correlation between low-level and high-level information.

[0042] (3) By employing a grouped convolutional hybrid module and a multi-level context information aggregation module, this invention avoids the network memory burden and high computational complexity caused by large matrix operations in traditional attention mechanisms, thus improving the speed and accuracy of semantic segmentation of smoke images. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating the multi-scale lightweight smoke image segmentation method according to an embodiment of the present invention.

[0044] Figure 2 This is a schematic diagram of the overall structure of the multi-scale lightweight smoke image segmentation network according to an embodiment of the present invention.

[0045] Figure 3 This is a schematic diagram of the structure of the multi-scale grouped attention hybrid module in an embodiment of the present invention.

[0046] Figure 4 This is a schematic diagram of the structure of the multi-level context information aggregation module in an embodiment of the present invention.

[0047] Figure 5 This is a schematic diagram of the channel shuffle module in an embodiment of the present invention.

[0048] Figure 6 This is a schematic diagram of the cross-fusion gating module in an embodiment of the present invention. Detailed Implementation

[0049] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0050] like Figure 1 As shown, this embodiment of the invention provides a multi-scale lightweight smoke image segmentation method, including the following steps:

[0051] S101. Obtain the smoke image dataset and preprocess the smoke image dataset.

[0052] According to the aforementioned multi-scale lightweight smoke image segmentation method, the preprocessing includes:

[0053] Multiple smoke image datasets were acquired, in which the images were virtually synthesized smoke images. In the preprocessing part, the images were processed by means of increasing contrast, and then the original 256×256 images were cropped to a size of 224×224 along the image center.

[0054] The smoke image dataset was then divided into training and testing sets for network training and performance evaluation.

[0055] S102. Construct a multi-scale lightweight convolutional neural network. The multi-scale lightweight convolutional neural network uses the ResNet50 residual structure as the backbone network and also includes three multi-scale lightweight group attention modules (MSA), a multi-level context information aggregation module (MCA), and a cross-fusion gating module (CLGD).

[0056] like Figure 2 As shown, the multi-scale lightweight convolutional neural network uses a ResNet50 residual structure as the backbone network to extract features from the input RGB smoke image. Then, the feature maps extracted from Layer 2, Layer 3, and Layer 4 of the residual structure are each processed by a 3x3 convolution to extract features and unify the number of channels. The output feature maps are then fed into the multi-scale lightweight grouped attention module (MSA). The output of each layer of the MSA module is used as the input of the multi-level context information aggregation module (MCA). This module uses a method based on the aggregation of context information at different levels and dense connections to obtain and adapt feature maps at different scales, achieving accurate semantic segmentation by aggregating information from different stages. The fusion result is then used as high-level features, and the feature map extracted from Layer 1 of the residual structure is used as low-level features. These are input into the cross-fusion gating module (CLGD), which uses high-level image features as a gating mechanism to assign weights to low-level features, thereby effectively selecting useful low-level features and stacking them with high-level features to output the final result.

[0057] The multi-scale lightweight grouping attention module (MSA) extracts and fuses smoke image features from different scales by grouping feature images at different scales into multiple sub-features along the channel dimension based on the correlation between low-level and high-level feature information. For example... Figure 3 As shown, the multi-scale lightweight grouped attention module (MSA) includes n cascaded spatial and channel attention modules, an aggregation module, and a channel mixing module. In the multi-scale lightweight grouped attention module (MSA), for a given feature map X∈R... C×H×WWhere C, H, and W represent the number of input channels, height, and width, respectively. X is divided into n groups along the channel dimension, i.e., Xn = ... k The size of (k∈[1,n]) is In X k Following this is a cascaded channel attention module and a spatial attention module. The channel attention module focuses on meaningful information in the input image, aggregating the channel information of the feature map through two parallel average pooling and max pooling layers. These are then fed into a shared 1x1 convolutional layer, a ReLU activation function, and another shared 1x1 convolutional layer, respectively. The two outputs are then element-wise summed, and finally passed through a sigmoid activation function to obtain the output of the channel attention module.

[0058]

[0059] The spatial attention module focuses on the target's location information. It takes the output of the channel attention module and performs max pooling and average pooling respectively to obtain two 1×H×W feature maps. After concatenation, these maps are transformed into a 1-channel feature map through a 7x7 convolution. This map is then activated by a sigmoid function, multiplied by the original input of the spatial attention module, and then convolved again to obtain the output of the spatial attention module.

[0060]

[0061] The output of the spatial attention module is multiplied by the input of the channel attention module, and then the n outputs are concatenated and shuffled by the channel mixing module. Figure 5 As shown, the channel mixing module divides the input information into n groups along the channel dimension, with a total of There are 10 channels, where c is the total number of channels, formed by reshaping. The matrix is ​​then transposed to... After flattening, the data is divided into n groups and connected sequentially for output. The channel mixing module enables cross-group information flow along the channel dimension to facilitate information exchange between different sub-features, ultimately outputting the result of the MSA module.

[0062] like Figure 4As shown, the multi-level context information aggregation module MCA takes the outputs of each level MSA module as input. The MSA outputs of each branch are enlarged to the same size and then added between branches. The output of Layer 2 in the residual structure is multiplied by the output of Layer 2 in the residual structure after passing through an average pooling layer, a 1*1 convolution, and a sigmoid activation function. Then, each branch is convolved laterally. Finally, all feature maps are enlarged to 1 / 8 of the original image size and then concatenated for output.

[0063] like Figure 6 As shown, the Cross-Gate fusion module CLGD uses the feature map extracted from Layer 1 of the residual structure as the low-level feature input, and the output of the Multi-Level Context Information Aggregation module MCA as the high-level feature input. The low-level and high-level features are convolved with 3x3 and upsampled by 2x, respectively, then concatenated. After a 1x1 convolution and sigmoid operation, the result is multiplied by the output of the low-level feature after a 3x3 convolution. Finally, it is concatenated with the output of the high-level feature after a 2x upsampled, followed by two sets of 3x3 convolutions and one 1x1 convolution operation, and then upsampled by 4x to restore the original feature map size to obtain the output.

[0064] S103. The multi-scale lightweight convolutional neural network is trained using the preprocessed training smoke image data, and the semantic segmentation results of the trained network on multiple smoke image datasets are evaluated.

[0065] This invention utilizes the cross-entropy loss function to train a multi-scale lightweight convolutional neural network, thereby obtaining a trained network model.

[0066] For multi-scale lightweight convolutional neural network models, this invention uses mIoU for evaluation and compares it with existing deep learning semantic segmentation algorithms in smoke segmentation.

[0067] The IoU value represents the ratio of the intersection to the union of the true and predicted values, and is related to the following values: TP (True Positive) indicates that both the true and predicted values ​​are positive; FN (False Negative) indicates that both the true and predicted values ​​are positive; FP (False Positive) indicates that both the true and predicted values ​​are negative; and TN (True Negative) indicates that both the true and predicted values ​​are negative. The TP part represents the intersection of the true and predicted values, and the FN+FP+TP part represents the union of the true and predicted values. The IoU is the ratio of the intersection to the union.

[0068]

[0069] mIoU (Mean Intersection over Union) is the average ratio of the intersection to the union of all classes, and is a standard metric for semantic segmentation. A higher mIoU value indicates higher accuracy and better segmentation performance on the dataset. mIoU is calculated using the formula:

[0070]

[0071] Where i represents the actual value, j represents the predicted value, and p ij This means predicting i as j. This formula is also equivalent to:

[0072]

[0073] S104. Input the smoke image data to be segmented into the trained multi-scale lightweight convolutional neural network to obtain the segmentation result.

[0074] This invention also provides a multi-scale lightweight smoke image segmentation device, comprising:

[0075] Acquisition unit: used to acquire smoke image dataset and preprocess the smoke image dataset;

[0076] Modeling unit: used to construct a multi-scale lightweight convolutional neural network, which uses ResNet50 residual structure as the backbone network and also includes three multi-scale lightweight group attention modules (MSA), a multi-level context information aggregation module (MCA), and a cross-fusion gating module (CLGD).

[0077] Training unit: used to train the multi-scale lightweight convolutional neural network using the preprocessed smoke image dataset, and to evaluate the semantic segmentation results of the trained network on multiple smoke image datasets;

[0078] Segmentation unit: Input the smoke image data to be segmented into the trained multi-scale lightweight convolutional neural network to obtain the segmentation result.

[0079] In summary, the deep learning method proposed in this invention, which combines multi-scale attention mechanism and gating, can make full use of contextual feature information at different scales to improve the network's ability to acquire global information. By using lightweight group attention and gating modules, higher quality semantic segmentation results for smoke images are obtained, which is of great significance to the field of computer vision.

[0080] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A multi-scale lightweight smoke image segmentation method, characterized in that, Includes the following steps: (1) Obtain the smoke image dataset and preprocess the smoke image dataset; (2) Construct a multi-scale lightweight convolutional neural network. The multi-scale lightweight convolutional neural network uses the ResNet50 residual structure as the backbone network and also includes three multi-scale lightweight group attention modules (MSA), a multi-level context information aggregation module (MCA), and a cross-fusion gating module (CLGD). The multi-scale lightweight convolutional neural network is specifically: The multi-scale lightweight convolutional neural network uses a ResNet50 residual structure as the backbone network to extract features from the input smoke image. The feature maps extracted from Layer 2, Layer 3, and Layer 4 in the residual structure are each processed by a 3x3 convolution to extract features and the number of channels is unified. The output feature maps are then fed into the multi-scale lightweight grouped attention module MSA. The output of the multi-scale lightweight group attention module (MSA) at each level is used as the input of the multi-level context information aggregation module (MCA). This module uses a method based on the aggregation of context information at different levels and dense connections to obtain and adapt feature maps at different scales. It achieves accurate semantic segmentation by aggregating information from different stages. The output of the multi-level context information aggregation module MCA is used as a high-level feature, and the feature map extracted from Layer 1 in the residual structure is used as a low-level feature and input into the cross-fusion gating module CLGD. The high-level feature is used as a gate to assign weights to the low-level feature, thereby effectively selecting useful low-level features and high-level features to stack and output the result. The multi-level context information aggregation module MCA is specifically as follows: The output results of the multi-scale lightweight grouped attention module MSA at each level are magnified to the same size and then added between branches. The output of Layer 2 in the residual structure is sequentially passed through an average pooling layer, a 1x1 convolution, and a sigmoid activation function, and then multiplied with the output of Layer 2 in the residual structure after passing through the multi-scale lightweight grouped attention module MSA. Then, each branch is convolved laterally. Finally, all convolutional outputs are enlarged to 1 / 8 the size of the original image and then concatenated for output. The Cross-Fusion Gating Module (CLGD) is specifically: The low-level features and high-level features are concatenated after 3x3 convolution and 2x upsampling, then concatenated, and then multiplied by the result of the 3x3 convolution of the low-level features after 1x1 convolution and sigmoide activation function operation. Finally, they are concatenated with the result of the 2x upsampling of the high-level features, and then multiplied by two sets of 3x3 convolution and one 1x1 convolution operation. Finally, they are upsampled to restore the original feature map size before being output. (3) The multi-scale lightweight convolutional neural network is trained using the preprocessed smoke image dataset, and the semantic segmentation results of the trained network on multiple smoke image datasets are evaluated. The evaluation uses mIoU and is calculated as follows: Where: TP represents both the true value and the predicted value as Positive; FN represents the true value as Positive and the predicted value as Negative; FP represents the true value as Negative and the predicted value as Positive. (4) Input the smoke image data to be segmented into the trained multi-scale lightweight convolutional neural network to obtain the segmentation result.

2. The multi-scale lightweight smoke image segmentation method according to claim 1, characterized in that, The preprocessing includes: Enhance contrast and crop along the image center; The smoke image dataset is divided into a training set and a test set.

3. The multi-scale lightweight smoke image segmentation method according to claim 1, characterized in that: The aforementioned multi-scale lightweight grouped attention module (MSA) includes A cascaded spatial attention module and a channel attention module, an aggregation module and a channel hybrid module; The input feature map is divided along the channel dimension. The groups are respectively input to the channel attention module and the spatial attention module connected in series; The channel attention module aggregates the channel information of the feature map through two parallel average pooling layers and max pooling layers, and then inputs them into a shared 1x1 convolutional layer, a ReLU activation function, and a shared 1x1 convolutional layer connected in sequence. The two outputs are then added element-wise and passed through a sigmoid activation function to obtain the output of the channel attention module. The spatial attention module passes the output of the channel attention module through two parallel max pooling layers and average pooling layers. The two outputs are concatenated by concat, then transformed into a 1-channel feature map through a 7x7 convolution. After passing through a sigmoid activation function, the feature map is multiplied by the original input of the spatial attention module and then convolved to obtain the output of the spatial attention module. The output of the spatial attention module is multiplied by the input of the channel attention module and then output. After the group outputs are connected by the aggregation module concat, the channel mixing module enables the cross-group information flow to flow along the channel dimension, allowing information exchange between different sub-features, and outputting the final result.

4. The multi-scale lightweight smoke image segmentation method according to claim 3, characterized in that, The channel mixing module divides the input information along the channel dimension. Group, totaling One channel, The total number of channels is formed by reshaping. The matrix is ​​then transposed to... After flattening, it is divided into... The groups are connected sequentially to the outputs.

5. A multi-scale lightweight smoke image segmentation device, characterized in that, include: Acquisition unit: used to acquire smoke image dataset and preprocess the smoke image dataset; Modeling unit: used to construct a multi-scale lightweight convolutional neural network, which uses ResNet50 residual structure as the backbone network and also includes three multi-scale lightweight group attention modules (MSA), a multi-level context information aggregation module (MCA), and a cross-fusion gating module (CLGD). The multi-scale lightweight convolutional neural network is specifically: The multi-scale lightweight convolutional neural network uses a ResNet50 residual structure as the backbone network to extract features from the input smoke image. The feature maps extracted from Layer 2, Layer 3, and Layer 4 in the residual structure are each processed by a 3x3 convolution to extract features and the number of channels is unified. The output feature maps are then fed into the multi-scale lightweight grouped attention module MSA. The output of the multi-scale lightweight group attention module (MSA) at each level is used as the input of the multi-level context information aggregation module (MCA). This module uses a method based on the aggregation of context information at different levels and dense connections to obtain and adapt feature maps at different scales. It achieves accurate semantic segmentation by aggregating information from different stages. The output of the multi-level context information aggregation module MCA is used as a high-level feature, and the feature map extracted from Layer 1 in the residual structure is used as a low-level feature and input into the cross-fusion gating module CLGD. The high-level feature is used as a gate to assign weights to the low-level feature, thereby effectively selecting useful low-level features and high-level features to stack and output the result. The multi-level context information aggregation module MCA is specifically as follows: The output results of the multi-scale lightweight grouped attention module MSA at each level are magnified to the same size and then added between branches. The output of Layer 2 in the residual structure is sequentially passed through an average pooling layer, a 1x1 convolution, and a sigmoid activation function, and then multiplied with the output of Layer 2 in the residual structure after passing through the multi-scale lightweight grouped attention module MSA. Then, each branch is convolved laterally. Finally, all convolutional outputs are enlarged to 1 / 8 the size of the original image and then concatenated for output. The Cross-Fusion Gating Module (CLGD) is specifically: The low-level features and high-level features are concatenated after 3x3 convolution and 2x upsampling, then concatenated, and then multiplied by the result of the 3x3 convolution of the low-level features after 1x1 convolution and sigmoide activation function operation. Finally, they are concatenated with the result of the 2x upsampling of the high-level features, and then multiplied by two sets of 3x3 convolution and one 1x1 convolution operation. Finally, they are upsampled to restore the original feature map size before being output. Training unit: used to train the multi-scale lightweight convolutional neural network using the preprocessed smoke image dataset, and to evaluate the semantic segmentation results of the trained network on multiple smoke image datasets; The evaluation uses mIoU and is calculated as follows: Where: TP represents both the true value and the predicted value as Positive; FN represents the true value as Positive and the predicted value as Negative; FP represents the true value as Negative and the predicted value as Positive. Segmentation Unit: Input the smoke image data to be segmented into the trained multi-scale lightweight convolutional neural network to obtain the segmentation result.

Images