A fire smoke image segmentation method based on efficient channel attention mechanism

Abstract

This paper discloses a fire smoke image segmentation method based on an efficient channel attention mechanism, comprising the following steps: obtaining a training image and inputting its label information; downsampling and upsampling the label information; obtaining data of the same size as the training image using a skip connection structure; calculating a Dice loss function and determining model convergence based on the Dice loss function; and outputting an image segmentation model if the model converges; and testing the image segmentation model to obtain a segmentation result. This paper proposes a (DCECA‑Unet++) model based on an efficient channel attention mechanism, which not only reduces the number of network model parameters but also achieves good performance in fire smoke segmentation.

Description

Technical Field

[0001] The present invention belongs to the technical field of fire smoke image segmentation, and in particular relates to a fire smoke image segmentation method based on an efficient channel attention mechanism. Background Art

[0002] Traditional segmentation algorithms mainly include threshold-based segmentation, edge segmentation, region segmentation, and graph segmentation. However, these methods can only handle relatively simple scenes and suffer from problems such as insufficient accuracy and unclear segmented images in fire smoke segmentation. Deep learning segmentation algorithm models mainly include FCN, Unet, Vnet, Unet++, and Segnet. Jonathan Long et al. (2015) proposed the FCN model, which introduced pixel-level classification and was a major breakthrough in semantic segmentation. Olaf Ronneberger et al. (2015) proposed the Unet model, which deepened the decoder with convolution based on the FCN model. Milletari F et al. (2016) proposed the Vnet model, which verified that DiceLoss can handle an imbalance in the number of foreground and background pixels. Zongwei Zhou et al. (2018) proposed the Unet++ model, which uses deep supervised learning to superimpose shallow and deep features. Feature values ​​at different depths are restored through different decoding paths, allowing the network to learn features at different depths. Badrinarayanan V et al. (2017) proposed the SegNet model, in which the entire feature map is not indexed by pooling, but is transferred from the encoder to the decoder, and then concatenated to perform convolution. The above method has the following shortcomings: (1) During the downsampling process, due to the insufficient receptive field, some feature information will be lost. (2) Only the overall features of the image are extracted, without paying special attention to important features. Summary of the Invention

[0003] The purpose of the present invention is to provide a fire smoke image segmentation method based on an efficient channel attention mechanism to solve the problems existing in the above-mentioned prior art.

[0004] To achieve the above objectives, the present invention provides a fire smoke image segmentation method based on an efficient channel attention mechanism, comprising:

[0005] Obtain training images and input label information of the training images;

[0006] Constructing an image segmentation model, and performing downsampling and upsampling on the label information based on the image segmentation model;

[0007] Acquire data of the same size as the training image based on a skip connection structure;

[0008] Calculating a Dice loss function and determining whether the image segmentation model has converged based on the Dice loss function, and outputting the image segmentation model if the model has converged;

[0009] The image segmentation model is tested to obtain a segmentation result.

[0010] Optionally, the process of downsampling and upsampling the label information based on the image segmentation model includes:

[0011] Build an efficient channel attention mechanism;

[0012] Downsampling the label information based on the efficient channel attention mechanism and the image segmentation model;

[0013] Upsampling is performed based on the output result of the downsampling.

[0014] Optionally, the process of downsampling and upsampling the label information further includes:

[0015] During downsampling, the convolution pooling unit of the image segmentation model is replaced with a dilated convolution;

[0016] The fourth convolution kernel of the dilated convolution is 5×5, and the fifth convolution kernel becomes 21×21.

[0017] Optionally, the dilation rates of the fourth and fifth layers of the dilated convolution are set to 2 and 4 respectively. When the dilation rate is 2, the calculation formula of the dilated convolution receptive field is:

[0018] RF i+1 =RF i +(K-1)×S i

[0019] Among them, RF i+1 Represents the receptive field of the current layer, RF i represents the receptive field of the previous layer, and K represents the size of the convolution kernel.

[0020] When the dilation rate is 4, the receptive field of the previous layer is changed to the receptive field calculated when the dilation rate is 2 based on the above calculation formula, and the size of the convolution kernel changes.

[0021] Optionally, the efficient channel attention mechanism includes:

[0022] After channel-level global average pooling without dimensionality reduction, local cross-channel interaction information is captured by considering each channel and its K neighbors, where K represents the coverage of local cross-channel interactions.

[0023] Optionally, the process of obtaining data of the same size as the original image based on the skip connection structure includes:

[0024] Based on the skip connection structure, the downsampled image and the upsampled image of the same dimension are added together to connect the bottom-level information and the high-level information to obtain data of the same size as the original image.

[0025] Optionally, the Dice loss function is calculated as follows:

[0026]

[0027] Here, X represents the ground truth label and Y represents the predicted segmentation image.

[0028] Optionally, the process of determining whether the model has converged based on the Dice loss function further includes:

[0029] If the model does not converge, the error is back-propagated to update the network model parameters.

[0030] Optionally, the process of testing the image segmentation model includes:

[0031] Input the test sample image into the trained deep neural network for testing.

[0032] The technical effects of the present invention are:

[0033] The present invention proposes a (DCECA-Unet++) model based on an efficient channel attention mechanism. First, in order to expand the receptive field during downsampling, the model replaces the ordinary convolution in the neural network with a hole convolution to avoid the problem of feature image loss caused by increasing the network depth. Secondly, the efficient channel attention mechanism is used to automatically select important feature information from the features extracted by the deep network to improve segmentation accuracy. Finally, the network is trained to achieve end-to-end image segmentation. The method proposed in the present invention is significantly superior to existing image segmentation methods such as Unet, Unet++, DC-Unet++, and DCSE-Unet++, and combines the idea of ​​expanding the receptive field of hole convolution with the efficient channel attention mechanism to effectively improve the image segmentation accuracy. Experimental results show that compared with other methods, the (DCECA-Unet++) model based on the efficient channel attention mechanism proposed in the present invention not only reduces the number of parameters of the network model, but also has good performance in the segmentation of flame smoke. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0035] Figure 1 is a flowchart of steps in an embodiment of the present invention;

[0036] Figure 2 Schematic diagram of the DCECA-Unet++ structure in an embodiment of the present invention;

[0037] Figure 3 4 is a comparison chart of the segmentation experiment results in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0039] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0040] Example 1

[0041] like Figure 1-3 As shown, this embodiment provides a fire smoke image segmentation method based on an efficient channel attention mechanism.

[0042] Reference Figure 1 , the implementation steps of the present invention are as follows:

[0043] (1) Training phase:

[0044] 1a) Input the label information of the training image;

[0045] 1b) passing information to the downsampling module with an efficient channel attention mechanism;

[0046] 1c) A downsampling module that replaces the convolutional pooling unit with an efficient channel attention mechanism using dilated convolution;

[0047] 1d) The output of each layer of the downsampling module is passed to the upsampling module to upsample the data. Using a skip connection structure, the downsampled image and the upsampled image of the same dimension are added together to connect the low-level information with the high-level information. The resulting data is the same size as the original image.

[0048] The specific implementation of the efficient channel attention mechanism involved in the above process is as follows:

[0049] After channel-level global average pooling without dimensionality reduction, local cross-channel interaction information is captured by considering each channel and its k neighbors, where k represents the coverage of local cross-channel interactions to avoid manual tuning of k using cross-validation;

[0050] The k value of the ECA module is 5, and the local cross-channel interaction coverage of the module is 5. The channel dimension calculation formula is:

[0051] C=φ(k)=2 (γ*k-b) (5)

[0052] The formula for calculating the convolution kernel size is:

[0053]

[0054] where |t odd represents the odd number closest to t, C is the channel dimension, and the convolution kernel size is k.

[0055] The implementation of the dilated convolution involved in the above process is as follows:

[0056] The dilated convolution is combined with the 4th and 5th layers of the downsampling, with the dilation ratio set to 2 and 4 respectively. Using different dilation ratios is beneficial for obtaining images of different scales while expanding the receptive field. The figure below shows the convolution process when the dilation ratio is 2. The formula for calculating the receptive field RF is:

[0057] RF i+1 =RF i +(K-1)×S i (7)

[0058] Among them, RF i+1 Represents the receptive field of the current layer, RF i represents the receptive field of the previous layer, and K represents the size of the convolution kernel.

[0059] When the dilation rate is 4, we only need to change the receptive field of the previous layer to the receptive field calculated when the dilation rate is 2 based on the above calculation formula, and the size of the convolution kernel changes.

[0060] In this model, the convolution kernel of the fourth layer is changed to 5×5, and the convolution kernel of the fifth layer is changed to 21×21. This allows the fourth and fifth layers to maintain larger feature maps while the receptive fields of the convolution layers in the first three stages remain unchanged. This avoids adding too much downsampling and prevents the feature maps from becoming too small, which helps solve problems such as the loss of internal data structure and spatial hierarchical information caused by ordinary convolution.

[0061] 1e) Calculate the Dice loss function

[0062]

[0063] Here, X represents the ground truth label and Y represents the predicted segmented image. This loss function ignores a large number of background pixels, solving the problem of imbalanced positive and negative samples, and making the network converge faster than other loss functions such as cross entropy.

[0064] 1f) Determine whether the model has converged based on the Dice loss function:

[0065] Image features are obtained through downsampling. The network learns the convolutional features and makes predictions and evaluations. The evaluation metric, dice loss, is used to evaluate the quality of the model's predictions. When the model converges, the prediction results are optimal. If the model does not converge, the error is backpropagated to update the network model parameters. When the model converges, the image segmentation model is output.

[0066] (2) Testing phase:

[0067] 2a) Input the image of the test sample into the trained deep neural network;

[0068] (3) Check the segmentation results.

[0069] The present invention uses an efficient channel attention mechanism and a dilated convolution image segmentation method, further comprising:

[0070] Use the labelme tool to label the collected images.

[0071] Randomly divide the training samples and test samples according to the labeled images.

[0072] Add unet++ convolutional network model based on the unet convolutional network model.

[0073] In the downsampling part of the previously constructed model, the fourth and fifth convolution layers are set to have a dilation rate of 2 and 4 respectively.

[0074] An efficient channel attention mechanism is introduced to capture local cross-channel interaction information by considering each channel and its K neighbors after channel-level global average pooling without dimensionality reduction.

[0075] The divided verification samples are put into the trained convolutional network model to obtain the prediction results.

[0076] The prediction results are compared with the actual annotations to calculate the recognition rate.

[0077] Compared with existing technologies and methods, this embodiment proposes a (DCECA-Unet++) model based on an efficient channel attention mechanism. First, in order to expand the receptive field during downsampling, the model replaces the ordinary convolution in the neural network with a hole convolution to avoid the problem of feature image loss caused by increasing the network depth. Secondly, using the efficient channel attention mechanism, important feature information is automatically selected from the features extracted by the deep network to improve segmentation accuracy. Finally, the network is trained to achieve end-to-end image segmentation. The method proposed in the present invention is significantly superior to existing image segmentation methods such as Unet, Unet++, DC-Unet++, and DCSE-Unet++, and combines the idea of ​​expanding the receptive field of hole convolution and the efficient channel attention mechanism to effectively improve the image segmentation results. Experimental results show that compared with other methods, the (DCECA-Unet++) model based on the efficient channel attention mechanism proposed in the present invention not only reduces the number of parameters of the network model, but also has good performance in the segmentation of flame smoke.

[0078] 1. Reasons for introducing dilated convolution: Before the introduction of dilated convolution, reducing image size while expanding the receptive field required increasing the number of downsampling layers and using pooling layers to reduce image size and expand the receptive field. However, downsampling during pooling can lead to a serious problem: image detail is lost, and information about small objects cannot be reconstructed. The introduction of dilated convolution addresses these two issues: incomplete or lost image information and reduced resolution caused by downsampling.

[0079] In this model, the convolution kernel of the fourth layer is changed to 5×5, and the convolution kernel of the fifth layer is changed to 21×21. This allows the fourth and fifth layers to maintain larger feature maps while the receptive fields of the convolution layers in the first three stages remain unchanged. This avoids adding too much downsampling and prevents the feature maps from becoming too small, which is conducive to solving problems such as the loss of internal data structure and spatial hierarchical information caused by ordinary convolution.

[0080] 2. Reason for Introducing ECA Channel Attention: To enhance important features and suppress unimportant ones, this method introduces an attention mechanism to select deep features extracted by the network, enhancing the features most relevant for segmentation. In the Unet++ model, to avoid losing a large amount of precise spatial detail during the encoder, a skip connection method is used to directly concatenate the map extracted from the encoder to the corresponding layer of the decoder. The integration of the attention mechanism effectively suppresses activations in irrelevant areas and reduces skipping of redundant parts.

[0081] The model proposed in this paper captures local cross-channel interaction information by considering each channel and its k neighbors after channel-level global average pooling without reducing the dimensionality. k represents the coverage of local cross-channel interactions in order to avoid manual tuning of k using cross-validation.

[0082] In summary, this embodiment proposes a method based on an efficient channel attention mechanism (DCECA-Unet++). This method significantly outperforms existing image segmentation methods such as Unet, Unet++, DC-Unet++, and DCSE-Unet++. It also combines the idea of ​​expanding the receptive field of view with the efficient channel attention mechanism, effectively improving image segmentation accuracy.

[0083] The above description is merely a preferred embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A fire smoke image segmentation method based on an efficient channel attention mechanism, characterized in that: The following steps are involved: Obtain training images and input label information of the training images; Build an image segmentation model and an efficient channel attention mechanism; Downsampling the label information based on the efficient channel attention mechanism and the image segmentation model; Performing upsampling based on the output result of the downsampling; During downsampling, the convolution pooling unit of the image segmentation model is replaced with a dilated convolution; The fourth convolution kernel of the dilated convolution is 5 5. The fifth convolution kernel becomes 21 twenty one; The dilation rates of the fourth and fifth layers of the dilated convolution are set to 2 and 4 respectively. When the dilation rate is 2, the calculation formula of the dilated convolution receptive field is: Among them, RF i+1 Represents the receptive field of the current layer, RF i represents the receptive field of the previous layer, K Indicates the convolution kernel size; When the dilation rate is 4, the receptive field of the previous layer is changed to the receptive field calculated when the dilation rate is 2 based on the above calculation formula, and the size of the convolution kernel changes; After channel-level global average pooling without dimensionality reduction, local cross-channel interaction information is captured by considering each channel and its K neighbors, where K represents the coverage of local cross-channel interactions; Based on the skip connection structure, the downsampled image and the upsampled image of the same dimension are added together to connect the bottom-level information and the high-level information to obtain data of the same size as the original image; Calculating a Dice loss function and determining whether the image segmentation model has converged based on the Dice loss function, and outputting the image segmentation model if the model has converged; The image segmentation model is tested to obtain a segmentation result.

2. The fire smoke image segmentation method based on efficient channel attention mechanism according to claim 1 is characterized in that: The calculation method of the Dice loss function is: Here, X represents the ground truth label and Y represents the predicted segmentation image.

3. The fire smoke image segmentation method based on efficient channel attention mechanism according to claim 1 is characterized in that: The process of determining whether the model has converged based on the Dice loss function also includes: If the model does not converge, the error is back-propagated to update the network model parameters.

4. The fire smoke image segmentation method based on efficient channel attention mechanism according to claim 1 is characterized in that: The process of testing the image segmentation model includes: Input the test sample image into the trained deep neural network for testing.

Images