A multi-scale fusion smoke segmentation method based on deep separable convolution

Abstract

The application discloses a multi-scale fusion smoke segmentation method based on a depth separable convolution, and comprises the following steps: acquiring a smoke image to be segmented; inputting the smoke image to be segmented into a pre-trained smoke segmentation network model to obtain an output smoke segmentation result image; wherein the smoke segmentation network model comprises an encoding module and a decoding module, the encoding module comprises a backbone network based on an improved Xception adopting a depth separable convolution and an improved ASPP_SE module; and the decoding module comprises a channel feature enhancement module, a multi-scale feature fusion module, a convolution and an up-sampling.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing and neural network, and particularly relates to a multi-scale fusion smoke segmentation method based on deep separable convolution. BACKGROUND

[0002] Among various disasters, fire is one of the main disasters that most commonly threatens public safety and social development. In the early stage of fire, smoke often appears before flame, so smoke detection is of great significance to prevent fire and ensure people's safety. Traditional smoke detection technology is usually based on sensors to sense smoke, temperature or particle concentration, etc., but there is a delay, which can lead to rapid spread of fire and cannot be controlled, and needs to be installed near the fire point, which limits the application of sensors in small indoor spaces. In addition, sensors cannot provide information such as initial position, propagation direction, scale, growth rate, etc. With the continuous development of computer vision, smoke detection technology based on image processing technology has made great progress. Monitoring and positioning smoke from images is more rapid and accurate than traditional sensors, and is more suitable for smoke detection in open and outdoor scenes. Traditional image processing technology needs to extract features manually with professional knowledge, but the obtained smoke features are limited to color, edge and simple texture.

[0003] In recent years, smoke detection technology based on deep learning has developed rapidly. Instead of manually extracting target features, deep convolutional neural networks are used. According to the output form, it can be divided into two categories: one is through the method of target detection, outputting a rectangular box to realize the recognition and labeling of smoke area; the other is through the method of semantic segmentation, judging the category of all pixel points in the image and outputting a segmentation map. The semantic segmentation method can better extract the essential features of smoke, distinguish different target areas by taking the edge of the detected object as the boundary, and realize fine recognition and segmentation. The fully convolutional network (FCN) is the pioneer in the field of semantic segmentation, which uses convolutional layers instead of fully connected layers and outputs two-dimensional images. After that, the encoder-decoder architecture appeared, and researchers used this architecture for smoke segmentation and achieved ideal results. However, existing smoke segmentation methods still face many challenges: (1) it is difficult to capture enough context information and detailed information of different level feature maps at the same time; (2) smoke has characteristics such as variable scale, blurred edge and semi-transparency, which significantly increases the difficulty of accurate segmentation. The existing smoke segmentation method does not pay special attention to the challenge brought by the texture detail information of variable and blurred edges. SUMMARY

[0004] To solve the problems in the prior art, the present application provides a multi-scale fusion smoke segmentation method based on deep separable convolution, which combines the characteristics of smoke morphology and the needs of engineering application, and builds an efficient network model suitable for smoke segmentation. The improved Xception using deep separable convolution is used as the backbone network, and the atrous spatial pyramid pooling (ASPP) is used to form the segmentation model coding module, and the channel attention is applied to enhance the feature expression ability of the model. The channel feature enhancement module, multi-scale feature fusion module, convolution and upsampling are used to form the decoding module, which can accurately segment the smoke.

[0005] The technical scheme adopted by the present application to solve its technical problems is:

[0006] In the first aspect, the present application provides a multi-scale fusion smoke segmentation method based on deep separable convolution, comprising:

[0007] Obtaining a smoke image to be segmented;

[0008] Inputting the smoke image to be segmented into a pre-trained smoke segmentation network model to obtain an output smoke segmentation result image;

[0009] The smoke segmentation network model comprises an encoding module and a decoding module, the encoding module comprises an improved Xception backbone network based on deep separable convolution and an improved ASPP_SE module; the decoding module comprises a channel feature enhancement module, a multi-scale feature fusion module, a convolution and an upsampling.

[0010] The training method of the smoke segmentation network model comprises:

[0011] Step S1, obtaining a smoke image, segmenting and labeling the smoke image for preprocessing, and constructing an original smoke data set;

[0012] Step S2, using data enhancement technology to perform data enhancement processing on the original smoke data set to obtain an enhanced data set, and dividing the enhanced data set into a training set and a test set;

[0013] Step S3, building a smoke segmentation network, using the improved Xception with deep separable convolution as the backbone network, and using the improved ASPP_SE module to form the encoding module, using the channel feature enhancement module, the multi-scale feature fusion module, the convolution and the upsampling to form the multi-level structure decoding module;

[0014] Step S4, setting the network training environment and parameters, using the training set to train and optimize the built smoke segmentation network, using the test set to test the trained network, obtaining the network model weight file, and obtaining the trained smoke segmentation network model.

[0015] In some embodiments, in step S1, the smoke images under different scenes are obtained by using a website crawler or frame-by-frame interception of smoke video, the smoke images are screened and unified to a specification that can be trained by a network, and the images are segmented and labeled.

[0016] In some embodiments, in step S2, the original smoke data set is processed by using a data enhancement technique, including: performing data enhancement on the original smoke data set by random rotation, cropping, flipping, and scaling, and random brightness, saturation, and contrast change operations, and expanding the smoke data set.

[0017] In some embodiments, in step S3, the encoding module adopts an improved Xception as a backbone network, the initial image is input into the encoding module, and a feature vector with a resolution of 1 / 16 of the initial image is generated through input stream, intermediate stream, and output stream convolution operations; the intermediate stream is a main feature learning module, which is directly connected by 8 repeated separated convolution modules DSCM, each separated convolution module is connected by 3 deep separated convolutions with 728 channels in a residual structure; the intermediate stream structure with residual connection is used, 1x1 convolution is used to reduce the input channel number to 600, and then the 8 separated convolution modules are divided into 4 groups, each group is connected in a residual manner, and 1x1 convolution is used again to restore the channel number to 728, thereby obtaining a first feature tensor with a sampling rate of 2, a second feature tensor with a sampling rate of 4, a third feature tensor with a sampling rate of 8, and a fourth feature tensor with a sampling rate of 16 extracted by the backbone network;

[0018] The fourth feature tensor extracted by the backbone network is input into the improved ASPP_SE module, and the improved ASPP_SE module includes a first branch, a second branch, a third branch, a fourth branch and a fifth branch; the first branch is a 1*1 standard convolution, which is used to maintain the original receptive field; the second to fourth branches are depth separable convolutions with three different dilation rates of r=6, r=12 and r=18, which are used for feature extraction to obtain different receptive fields; the fifth branch is a global average pooling, which is used to obtain global features; the input feature map in the fifth branch is input into two fully connected layers and a Sigmoid function after global average pooling to generate a channel attention weight vector; the channel attention weight vector is used for weighting the feature maps of the first branch, the second branch, the third branch and the fourth branch; the feature maps output by the first branch, the second branch, the third branch, the fourth branch and the fifth branch are stacked in the channel dimension, and the information of different scales is fused through a 1*1 standard convolution; and the output is multiplied by the fourth feature tensor extracted by the backbone network to obtain a feature tensor output by the encoding module.

[0019] In some embodiments, in step S3, the decoding module adopts a channel feature enhancement module, a multi-scale feature fusion module, a convolution and an upsampling to recover the image size through the upsampling and complete the smoke image segmentation;

[0020] The feature tensor from the output of the encoding module is 2 times up-sampled through bilinear interpolation to obtain a first feature map;

[0021] The first feature tensor, the second feature tensor and the second feature tensor from the backbone network are subjected to channel feature enhancement by the channel feature enhancement module to obtain a first channel attention feature map, a second channel attention feature map and a third channel attention feature map, respectively;

[0022] The first channel attention feature map and the first feature map are subjected to feature fusion by the multi-scale feature fusion module to obtain a first fusion feature map, and the first fusion feature map is subjected to 2 times up-sampling and then subjected to feature fusion with the second channel attention feature map to obtain a second fusion feature map; the second fusion feature map is subjected to 2 times up-sampling and then subjected to feature fusion with the third channel attention feature map to obtain a third fusion feature map;

[0023] The third fusion feature map is subjected to a 3*3 convolution and then subjected to 2 times up-sampling to output a smoke segmentation result map.

[0024] In some embodiments, the channel feature enhancement module introduces a channel attention mechanism, and the channel attention mechanism includes a compression part and an activation part;

[0025] The compression part compresses the channels through global average pooling, and for the ith channel, there is:

[0026]

[0027] wherein WxH represents the original feature map resolution, μ represents the feature map with HxWxC, C is the total number of channels, μ i represents the two-dimensional matrix of the i-th channel in the feature map; through global average pooling, the feature mapping of the i-th channel is z i , and the compression process obtains a one-dimensional vector of 1x1xC.

[0028] The activation part obtains the relationship expression between channels through two fully connected layers:

[0029] s = σ (g (z, W) ) = σ (W2 (δ) )

[0030] The first fully connected layer with a weight of W1 is used to reduce the number of channels to 1 / h of the original, and after the ReLU function δ activation, the second fully connected layer with a weight of W2 is used to restore the number of channels, and finally the Sigmoid function σ is used to generate normalized channel weights s, whose scale is 1x1xC.

[0031] The normalized channel weights are multiplied by the corresponding channels of the original feature map to obtain the channel attention feature map.

[0032] In some embodiments, step S4 includes: taking the smoke images of the training set as the input images of the smoke segmentation network, uniformly inputting the size of the images, training the network using a random gradient descent algorithm with momentum, setting the number of cycles to 200,000, the momentum to 0.9, the initial learning rate to 1x10 -4 , the final learning rate to 1x10 -5 , and the learning rate linearly decreases with the number of training times; according to the computer configuration, the batchsize is set to 4, and the ASPP hole convolution rate is set to 6, 12, and 18.

[0033] In a second aspect, the present application provides a multi-scale fusion smoke segmentation device based on deep separable convolution, comprising a processor and a storage medium.

[0034] The storage medium is used to store instructions.

[0035] The processor is used to operate according to the instructions to perform the steps of the method according to the first aspect.

[0036] In a third aspect, the present application provides a storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the method according to the first aspect.

[0037] The beneficial effect of the present application is that, in the face of smoke with variable scale, blurred edges and semi-transparent characteristics, the present application can accurately segment the smoke and finely process the smoke boundary. Compared with the prior art, the present application improves the encoding and decoding modules, designs some modules to fully extract the relevant information between multi-scale features and the internal features, and introduces the attention mechanism to screen out the channel information with better distinguishing force and guide the recovery of the underlying spatial details. The improved network takes into account the segmentation accuracy and detection efficiency, significantly improves the segmentation accuracy of smoke pixels, and provides a new reference for the field of smoke detection. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 is a flow block diagram of a smoke segmentation network of an embodiment of the present application;

[0039] Figure 2 is a general schematic diagram of a smoke segmentation network of an embodiment of the present application;

[0040] Figure 3 is a schematic diagram of an Xception network residual intermediate flow structure of an embodiment of the present application;

[0041] Figure 4 is a schematic diagram of an improved ASPP_SE attention module of an embodiment of the present application;

[0042] Figure 5 is a schematic diagram of a channel feature enhancement module of an embodiment of the present application;

[0043] Figure 6 is a schematic diagram of a multi-scale feature fusion module of an embodiment of the present application;

[0044] Figure 7 is a partial test result diagram of a smoke segmentation network of an embodiment of the present application. DETAILED DESCRIPTION

[0045] The present application will be further described below in conjunction with the drawings and embodiments.

[0046] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "illustrative embodiment", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0047] Embodiment 1:

[0048] A multi-scale fusion smoke segmentation method based on deep separable convolution, comprising:

[0049] Obtaining a smoke image to be segmented;

[0050] Inputting the smoke image to be segmented into a pre-trained smoke segmentation network model to obtain an output smoke segmentation result image;

[0051] The smoke segmentation network model comprises an encoding module and a decoding module, the encoding module comprises a backbone network based on an improved Xception using deep separable convolution and an improved ASPP_SE module; the decoding module comprises a channel feature enhancement module, a multi-scale feature fusion module, convolution and up-sampling.

[0052] The training method of the smoke segmentation network model comprises:

[0053] Step S1, obtaining a smoke image, segmenting and labeling the smoke image for preprocessing, and constructing an original smoke data set;

[0054] Step S2, using data enhancement technology to perform data enhancement processing on the original smoke data set to obtain an enhanced data set, and dividing the enhanced data set into a training set and a test set;

[0055] Step S3, building a smoke segmentation network, using an improved Xception using deep separable convolution as a backbone network, and an improved ASPP_SE module to form an encoding module, and using a channel feature enhancement module, a multi-scale feature fusion module, convolution and up-sampling to form a multi-level structure decoding module;

[0056] Step S4, setting a network training environment and parameters, training and optimizing the built smoke segmentation network using the training set, testing the trained network using the test set, obtaining a network model weight file, and obtaining a trained smoke segmentation network model.

[0057] In some embodiments, a multi-scale fusion smoke segmentation method based on deep separable convolution, the network training process is as shown in Figure 1 The present embodiment uses Python language as the basis, open source Pytorch as the neural network framework, completes the building of the smoke segmentation network model, trains the smoke segmentation network through the smoke image data set, finds out the optimal model parameters, and the specific steps are as follows:

[0058] An improved Xception using deep separable convolution is used as the backbone network, and an all-sky pyramid (ASPP) based on a convolutional pyramid structure is used to form a segmentation model coding module. Channel attention is applied to enhance the feature expression capability of the model. A channel feature enhancement module, a multi-scale feature fusion module, a convolution, and an upsampling are used to form a decoding module. The built smoke segmentation network model is as shown in Figure 2 The trained network model is used to segment the smoke image to obtain a more accurate segmentation result.

[0059] Step S1, obtaining a smoke image, segmenting and labeling the smoke image for preprocessing, and constructing an original smoke data set;

[0060] In some embodiments, the smoke images in different scenes are obtained by using a website crawler or frame interception of a smoke video, and are filtered and unified into the same specification for network training to construct a smoke data set. The smoke target is manually labeled at the pixel level using Photoshop.

[0061] Step S2, using data enhancement technology to perform data enhancement processing on the original smoke data set to obtain an enhanced data set, and dividing the enhanced data set into a training set and a test set;

[0062] In some embodiments, the data enhancement technology is used to perform random rotation, cropping, flipping, and scaling, as well as random brightness, saturation, and contrast changes, and other operations on the original smoke data set for enhancement processing. The data set is divided into a training set and a test set according to a certain proportion.

[0063] Step S3: building a smoke segmentation network;

[0064] The coding module uses an improved Xception as the backbone network. The initial smoke image is input into the coding module, and after convolution operation of the input stream, the intermediate stream, and the output stream, the first feature vector with a resolution of 1 / 16 of the original image is generated. The improved Xception network mainly optimizes the intermediate stream structure. The intermediate stream is a main feature learning module, which is composed of 8 repeated separable convolution modules (DSCM) connected directly. Each separable convolution module is connected by 3 deep separable convolutions with 728 channels in a residual structure. The dense connection causes the network to be too complex, and the large amount of calculation introduced by channel splicing is not conducive to model lightweight. Therefore, a residual connection intermediate stream structure is proposed, as shown in Figure 3As shown. After reducing the number of input channels to 600 with 1x1 convolution, 8 modules are evenly divided into 4 groups, each group is connected with residual connection to restore the number of channels to 728 with 1x1 convolution again, to obtain the first feature tensor with a sampling rate of 2, the second feature tensor with a sampling rate of 4, the third feature tensor with a sampling rate of 8, and the fourth feature tensor with a sampling rate of 16 extracted by the backbone network. This method enhances the correlation between modules, reduces the network operation amount, and is beneficial to the feature learning and model lightweight of the network.

[0065] The fourth feature tensor extracted by the backbone network is input into the improved ASPP_SE module, as shown. Figure 4 The fixed dilation rate is used to integrate the limitations of different receptive field information, and the information is more flexibly fused, which improves the modeling ability and learning efficiency of the network at the cost of a small amount of model complexity, and introduces channel attention to enhance the response ability of specific semantics under the channel. The improved ASPP_SE module includes a first branch, a second branch, a third branch, a fourth branch, and a fifth branch; the first branch is a 1x1 standard convolution, which aims to maintain the original receptive field; the second to fourth branches are depth separable convolutions with r=6, r=12, and r=18, which are used for feature extraction to obtain different receptive fields; the fifth branch is a global average pooling, which is used to obtain global features. At the same time, the input feature map is input into two fully connected layers and a Sigmoid function after global average pooling in the fifth branch to generate a channel attention weight vector. Then, the channel attention weight vector is used to weight the feature maps of the first branch, the second branch, the third branch, and the fourth branch. This spatial size close to the weighting can better play the role of channel weight. Finally, the feature maps of the five branches are stacked in the channel dimension, and the information of different scales is fused through 1x1 standard convolution; and the output is multiplied by the fourth feature tensor extracted by the backbone network, which can provide better spatial attention information for the feature map.

[0066] The encoding module increases the receptive field and extracts features through downsampling, and the decoding module uses a channel feature enhancement module, a multi-scale feature fusion module, a convolution, and upsampling to restore the image size and complete the smoke image segmentation. To improve the segmentation accuracy and reduce the loss of details, lower layer features are introduced into the feature map predicted by the decoding module, and a new feature map is formed after multi-scale feature fusion.

[0067] The first feature tensor, the second feature tensor, and the second feature tensor from the backbone network are input into the channel feature enhancement module for channel feature enhancement to obtain the first channel attention feature map, the second channel attention feature map, and the third channel attention feature map, respectively.

[0068] The first fusion feature map is obtained by fusing the first channel attention feature map and the first feature map through a multi-scale feature fusion module, and the first fusion feature map is fused with the second channel attention feature map after 2 times upsampling to obtain a second fusion feature map; the second fusion feature map is fused with the third channel attention feature map after 2 times upsampling to obtain a third fusion feature map;

[0069] The third fusion feature map is subjected to 3x3 convolution and 2 times upsampling operation, and an output smoke segmentation result map is obtained.

[0070] The channel feature enhancement module is introduced before feature fusion, that is, the channel attention mechanism is introduced, which enhances the feature maps of different sampling rates input into the backbone network, as shown in FIG. 1. Figure 5 The original network does not perform channel weighting on the feature map. If the channels of the feature map are weighted, and the features that contribute more to the target prediction are learned and screened, the burden of processing high-dimensional data can be reduced, and the network can pay more attention to the important part of the input information, better judge the mapping relationship from input to output, and further improve the prediction accuracy and generalization ability of the model. The channel attention mechanism includes compression and activation. The compression part compresses the channels through global average pooling, and the i-th channel has:

[0071]

[0072] wherein WxH represents the resolution of the original feature map, μ represents the feature map of HxWxC, C represents the total number of channels, and μ i represents a two-dimensional matrix of the i-th channel in the feature map. Through global average pooling, the feature map of the i-th channel is z i , and the compression process obtains a one-dimensional vector of 1x1xC. The activation part obtains the relationship expression between channels through two fully connected layers:

[0073] s = σ (g (z, W)) = σ (W2 (δ))

[0074] First, a fully connected layer with a weight W1 is used to reduce the number of channels to 1 / h of the original, and after ReLU function (δ) activation, the second fully connected layer (weight W2) is input to restore the number of channels, and finally the Sigmoid function (σ) is used to generate the normalized channel, and the scale is 1x1xC. The normalized channel weight is multiplied by the corresponding channel of the original feature map, and the channel attention feature map is obtained.

[0075] High-level semantic information preserves the large semantic structure, but suffers significant loss of smaller structures; low-level spatial information segmentation results retain rich details, but semantic category prediction is poor. Therefore, this invention, based on the concept of multi-scale fusion, integrates high-resolution feature maps from the lower layers of the backbone network and employs a multi-scale feature fusion module. When upsampling is required for feature fusion, it establishes the positional relationships between pixels on different levels of feature maps, efficiently and accurately integrating high-level semantic information into the lower-level feature maps, such as... Figure 6 As shown, firstly, two feature maps from different levels are fused and concatenated as input. One is a feature map from the output of the encoding module that has been upsampled by 2x, and the other is a feature map from the backbone network input stream with a downsampling rate of 8x (its feature map has been processed by the channel feature enhancement module). Then, the fused and concatenated feature map is upsampled by 2x again and fused with the feature map from the input stream with a downsampling rate of 4x. It is then upsampled by 2x again and fused with the feature map from the input stream with a downsampling rate of 2x. Therefore, the multi-scale feature fusion module involves three feature fusions. A 1x1 convolution is used to adjust the number of channels to match the previous values, followed by a 3x3 convolution and a 2x upsampling operation to restore the feature map resolution to the original value. Figure 1 The output is the segmented smoke image.

[0076] Step 4: Set up the network training environment and parameters: Use the processed smoke training set as the network input images, standardize the input image size, and use the stochastic gradient descent algorithm with momentum for training. Set the number of iterations to 200,000, the momentum to 0.9, and the initial learning rate to 1×10⁻⁶. -4 The termination learning rate is set to 1×10. -5 The learning rate decreases linearly with the number of training iterations; the batch processing rate is set to 4 based on the computer configuration, and the ASPP dilated convolution rate is set to 6, 12, and 18.

[0077] Specific application example: This embodiment uses a smoke video from a forest fire in a southern province in 2022 as an example, and employs the smoke segmentation model trained in Embodiment 1 to specifically segment the smoke image. Real-time fire footage is obtained from cameras retrieved from the forest monitoring system. The extracted video images are processed frame by frame and converted to the required image format and size. The trained weight file is loaded into the network model to initialize the smoke segmentation model. Then, the smoke image is input into the network model, and the segmentation result image is output through the trained smoke segmentation network. Figure 7 This image shows partial test results for a multi-scale fusion smoke segmentation method based on depthwise separable convolution. The results demonstrate that the method provided in this invention significantly improves the segmentation accuracy of smoke images and has significant application value in the field of smoke detection.

[0078] Example 2

[0079] In a second aspect, the embodiment provides a multi-scale fusion smoke segmentation device based on deep separable convolution, comprising a processor and a storage medium;

[0080] The storage medium is configured to store instructions.

[0081] The processor is configured to operate according to the instructions to perform the steps of the method according to Embodiment 1.

[0082] Embodiment 3

[0083] In a third aspect, the embodiment provides a storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of the method according to Embodiment 1.

[0084] Those skilled in the art will understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage media, etc.) having computer-usable program code contained therein.

[0085] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as a combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks

[0086] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction devices that implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks

[0087] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable data processing devices to generate a computer implemented process, so that the instructions executed on the computer or other programmable data processing devices provide a process for implementing the flowchart Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0088] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present application should be equivalent replacement methods, and are all included in the protection scope of the present application.

Claims

1. A multi-scale fusion smoke segmentation method based on depthwise separable convolution, characterized in that, include: Obtain the smoke image to be segmented; Input the smoke image to be segmented into a pre-trained smoke segmentation network model to obtain the output smoke segmentation result image; The smoke segmentation network model includes an encoding module and a decoding module. The encoding module includes a backbone network based on an improved Xception using depthwise separable convolutions and an improved ASPP_SE module. The decoding module includes a channel feature enhancement module, a multi-scale feature fusion module, convolutions, and upsampling. The encoding module uses an improved Xception as its backbone network. The initial image is input into the encoding module, and after convolution operations on the input stream, intermediate stream, and output stream, a feature vector with a resolution of 1 / 16 of the initial image is generated. The intermediate stream is the main feature learning module, which consists of eight directly connected, repeated separable convolutional modules (DSCM). Each separable convolutional module consists of three 728-channel depthwise separable convolutions connected by a residual structure. Using the residual-connected intermediate stream structure, the number of input channels is reduced to 600 using 1×1 convolutions. Then, the eight separable convolutional modules are divided into four groups, and each group is again connected by residuals and used with 1×1 convolutions to restore the number of channels to 728. This yields a first feature tensor with a sampling rate of 2, a second feature tensor with a sampling rate of 4, a third feature tensor with a sampling rate of 8, and a fourth feature tensor with a sampling rate of 16 extracted by the backbone network. The fourth feature tensor extracted by the backbone network is input into the improved ASPP_SE module, which includes a first branch, a second branch, a third branch, a fourth branch, and a fifth branch. The first branch is a 1×1 standard convolution to preserve the original receptive field; the second to fourth branches are three depthwise separable convolutions with different dilation rates (r=6, r=12, r=18) used for feature extraction to obtain different receptive fields; the fifth branch is global average pooling to obtain global features; in the fifth branch, the input feature map is processed by global average pooling and then two fully connected layers and a sigmoid function are added to generate a channel attention weight vector; the feature maps of the first, second, third, and fourth branches are weighted using the channel attention weight vector; the feature maps output from the first, second, third, fourth, and fifth branches are stacked along the channel dimension and fused with information at different scales through a 1×1 standard convolution; this output is multiplied by the fourth feature tensor extracted by the backbone network to become the feature tensor output by the encoding module; The decoding module employs a channel feature enhancement module, a multi-scale feature fusion module, convolution, and upsampling. Upsampling restores the image size and completes smoke image segmentation. The feature tensor output from the encoding module is upsampled by 2 times through bilinear interpolation to obtain the first feature map. The first feature tensor, second feature tensor, and third feature tensor from the backbone network are enhanced by the channel feature enhancement module to obtain the first channel attention feature map, the second channel attention feature map, and the third channel attention feature map, respectively. A multi-scale feature fusion module is used to fuse the first channel attention feature map and the first feature map to obtain a first fused feature map. The first fused feature map is then upsampled by 2 times and fused with the second channel attention feature map to obtain a second fused feature map. The second fused feature map is upsampled by 2 times and then fused with the third channel attention feature map to obtain the third fused feature map; the third fused feature map is then convolved with 3×3 and upsampled by 2 times to output the smoke segmentation result map.

2. The multi-scale fusion smoke segmentation method based on depthwise separable convolution according to claim 1, characterized in that, The training method for the smoke segmentation network model includes: Step S1: Obtain smoke images, perform segmentation and annotation preprocessing on the smoke images, and construct the original smoke dataset; Step S2: Use data augmentation techniques to augment the original smoke dataset to obtain the augmented dataset, which is then divided into a training set and a test set. Step S3: Construct a smoke segmentation network with an improved Xception network using depthwise separable convolution as the backbone network, and an improved ASPP_SE module to form an encoding module. A multi-level decoding module is composed of a channel feature enhancement module, a multi-scale feature fusion module, convolution and upsampling. Step S4: Set up the network training environment and parameters, use the training set to train and optimize the built smoke segmentation network, use the test set to test the trained network, obtain the network model weight file, and obtain the trained smoke segmentation network model.

3. The multi-scale fusion smoke segmentation method based on depthwise separable convolution according to claim 2, characterized in that, In step S1, smoke images in different scenes are obtained by using website crawlers or frame-by-frame capture of smoke videos. The smoke images are then filtered to be standardized to a size suitable for network training, and the images are segmented and labeled.

4. The multi-scale fusion smoke segmentation method based on depthwise separable convolution according to claim 2, characterized in that, In step S2, data augmentation techniques are used to augment the original smoke dataset, including random rotation, cropping, flipping, scaling, and random changes in brightness, saturation, and contrast to expand the smoke dataset.

5. The multi-scale fusion smoke segmentation method based on depthwise separable convolution according to claim 1, characterized in that, The channel feature enhancement module introduces a channel attention mechanism, which includes two parts: compression and activation. The compression part compresses the channels using global average pooling. For the i-th channel, we have: Where W×H represents the original feature map resolution, μ represents the feature map of size H×W×C, C is the total number of channels, and μ i This represents a two-dimensional matrix with channel i in the feature map; through global average pooling, the feature map of the i-th channel is z. i The compression process yields a one-dimensional vector of size 1×1×C. The activation part uses two fully connected layers to express the relationship between channels: s=σ(g(z,W))=σ(W2(δ)) First, the number of channels is reduced to 1 / h of the original number by a first fully connected layer with weight W1. After activation by the ReLU function δ, the number of channels is restored by inputting a second fully connected layer with weight W2. Finally, the normalized channel weights s are generated by the Sigmoid function σ, with a scale of 1×1×C. Multiply the normalized channel weights by the corresponding channels of the original feature map to obtain the channel attention feature map.

6. The multi-scale fusion smoke segmentation method based on depthwise separable convolution according to claim 1, characterized in that, Step S4 includes: using the smoke images from the training set as input images for the smoke segmentation network, standardizing the size of the input images, training the network using stochastic gradient descent with momentum, setting the number of iterations to 200,000, the momentum to 0.9, and the initial learning rate to 1×10⁻⁶. -4 The termination learning rate is set to 1×10. -5 The learning rate decreases linearly with the number of training iterations; the batch size is set to 4 according to the computer configuration, and the ASPP dilated convolution rate is set to 6, 12, and 18.

7. A multi-scale fusion smoke segmentation device based on depthwise separable convolution, characterized in that, Including processor and storage media; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method according to any one of claims 1 to 6.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images