An image segmentation method based on improved FastFCN

Abstract

The present invention discloses an image segmentation method based on an improved FastFCN; the present invention innovatively proposes a dual parallel asymmetric convolution module for extracting nonlinear features and a serial dilated spatial pyramid pooling module for capturing multi-scale information. The asymmetric convolution module, the serial dilated spatial pyramid pooling module, the dual attention CBAM module and the bottleneck layer are integrated into sub-modules to construct a semantic text information sub-module. The semantic text information sub-module is then embedded into the FastFCN network after the encoder structure is optimized. The structural optimization is mainly reflected in the encoder part enhancing the information flow by adding feature skip connection paths. No complex preprocessing is required before training. Multi-scale deep supervision is used during training, and flipping test is used during testing. The method of the present invention achieves excellent segmentation performance and extremely low model parameter amount, wherein the Jaccard index is 84.05% and the model parameter amount is only 0.96M.

Description

Technical Field

[0001] The present invention relates to an image segmentation method, in particular to an image segmentation method based on an improved FastFCN, and is applied to the segmentation of skin lesion images in the medical field. Background Art

[0002] The fully convolutional neural network (FCN) is a representative application of deep learning to image segmentation. It is an end-to-end image segmentation method that allows the network to make pixel-level predictions and directly produce segmentation maps. Because all layers in the FCN network are convolutional layers, it is called a fully convolutional network. Fully convolutional neural networks mainly use three technologies:

[0003] 1. Convolutional

[0004] 2. Upsample;

[0005] 3. Skip Layer.

[0006] FCN classifies images at the pixel level, solving the problem of semantic segmentation. Unlike classic CNNs, which use a fully connected layer after the convolutional layer to obtain a fixed-length feature vector for classification (fully connected layer + softmax output), FCN can accept input images of any size and uses a deconvolution layer to upsample the feature map of the last convolutional layer to restore it to the same size as the input image. This generates a prediction for each pixel while preserving the spatial information in the original input image. Finally, pixel-by-pixel classification is performed on the upsampled feature map.

[0007] However, dilated convolutions are often used in FCN backbone networks to obtain high-resolution feature maps, which increases computational complexity and memory usage. Therefore, researchers proposed the FastFCN network, which uses a novel joint upsampling module (JPU) to replace dilated convolutions. This method effectively reduces computational complexity and memory usage.

[0008] However, the traditional FastFCN network has some shortcomings, such as insufficient feature extraction capability of the encoder part, lack of attention to important features, and failure to capture rich multi-scale features. To address these problems, this study attempts to adopt a skin lesion image segmentation method based on an improved FastFCN and conducts experiments on the ISIC2018 skin lesion image dataset. Summary of the Invention

[0009] In response to the shortcomings of the existing technology, the present invention proposes an image segmentation method based on an improved FastFCN. The purpose of the present invention is to reduce the hardware requirements of the convolutional network in image segmentation, reduce network parameters, and shorten training time; secondly, to improve the image segmentation accuracy by using the improved FastFCN.

[0010] According to the technical solution provided by the present invention, a skin lesion image segmentation method based on improved FastFCN is proposed, comprising the following steps:

[0011] Step 1: Select the ISIC2018 dataset as experimental data, preprocess the data, determine the test set and training set, and input the data after data enhancement;

[0012] Step 2: Construct a dual parallel asymmetric convolution module to extract nonlinear features;

[0013] Step 3: Construct a serial dilated spatial pyramid pooling module to obtain multi-scale features;

[0014] Step 4: Integrate the asymmetric convolution module, serial dilated spatial pyramid pooling module, dual attention CBAM module, and bottleneck layer into submodules to construct a semantic text information submodule and a lightweight semantic text information submodule;

[0015] Step 5: Embed the semantic text information submodule and the lightweight semantic text information submodule into the FastFCN network after encoder structure optimization to construct the improved FastFCN network;

[0016] Step 6: Perform multi-scale supervised auxiliary training on the constructed improved FastFCN network to obtain the required segmentation network;

[0017] Step 7: After data enhancement, the test set images are input into the segmentation network obtained in step 5, and the segmentation map is output by flipping the test.

[0018] In step 1, the ISIC2018 Skin Lesion Analysis Melanoma Detection Challenge dataset was used. This dataset contains 2,594 training images and their corresponding labels, 100 validation images and their corresponding labels, and 1,000 unlabeled test images. Image data preprocessing was then performed. The specific steps for preprocessing are as follows:

[0019] 1-1. Unify the size of the training set images;

[0020] 1-2. Divide the training set into new training set and test set;

[0021] 1-3. Process the training set images and labels separately and input them into the network.

[0022] The specific steps of constructing the dual parallel asymmetric convolution module in step 2 are as follows:

[0023] 2-1. The dual parallel asymmetric convolution module consists of two parallel asymmetric convolution modules connected in series. One parallel asymmetric convolution module adopts a dual-branch parallel connection structure, which is divided into vertical branches and horizontal branches.

[0024] 2-2. The outputs of the vertical and horizontal branches are combined by adding feature maps and used as the output of the parallel asymmetric convolution module;

[0025] 2-3. Feature extraction will use two parallel asymmetric convolution modules in series, named dual parallel asymmetric convolution module.

[0026] In step 3, a serial dilated spatial pyramid pooling module is constructed, and the specific steps are as follows:

[0027] 3-1. The Serial Dilated Spatial Pyramid Pooling Module improves upon the existing dilated spatial pyramid pooling module's larger dilation rate branch, featuring a four-branch parallel connection. Finally, the outputs of the four branches are fused via concat to produce the final output of the Serial Dilated Spatial Pyramid Pooling Module.

[0028] In step 4, the asymmetric convolution module, serial atrous spatial pyramid pooling module, dual attention CBAM module, and bottleneck layer are integrated into the submodule to construct the semantic text information submodule. The specific details of the semantic text information submodule are as follows (assuming that the number of input and output channels of this module is m and n respectively):

[0029] 4-1. First, use the bottleneck layer;

[0030] 4-2. Then use a dual parallel asymmetric convolution module;

[0031] 4-3. Then use the serial atrous spatial pyramid pooling module;

[0032] 4-4. Finally, the CBAM module with spatial channel dual attention is introduced.

[0033] The lightweight semantic text information submodule is replaced by a normal dilated spatial pyramid pooling module in step 4-3. The other components are the same as the semantic text information submodule.

[0034] In step 5, the semantic text information submodule and the lightweight semantic text information submodule are embedded into the FastFCN network after the encoder structure is optimized to construct the improved FastFCN network. The specific steps are:

[0035] 5-1. The improved FastFCN network uses 6 sub-modules, including 5 modules in the encoder part and 1 module in the decoder part.

[0036] 5-2. Based on the original FastFCN network, the encoder part adds 6 skip connection paths from shallow feature maps to deep feature maps as one of the inputs of the deep sub-module in the encoder.

[0037] In step 6, the constructed improved FastFCN network is subjected to multi-scale supervised auxiliary training to obtain the required segmentation network. The specific details are as follows:

[0038] 6-1. Perform multi-scale supervised training on the feature maps output by Encoder3, Encoder4, Encoder5, and Dncoder1 in the network. There are four independent supervision paths in total.

[0039] 6-2. The segmentation maps output by the four supervision paths are respectively compared with the training labels to calculate the loss value. Each path has the same weight, and the results are accumulated to obtain the final loss value.

[0040] 6-3. The network is trained twice. The first training uses a single Lovasz Loss function. The second training uses the parameters of the model after the first training and uses a joint loss function for further training to overcome the problem of sample imbalance.

[0041] In step 7, the test set image is input into the segmentation network obtained in step 5 after data augmentation, and the segmentation map is output by flipping the test output. The specific steps are as follows:

[0042] 7-1. After grayscale processing, the test set images are input into the trained network. The feature maps output by the network encoder are normalized using the Sigmoid() function and binarized with a threshold of 0.5. Since the resolution is smaller than the input image resolution, the resize function is used to restore the resolution to 224*224 and re-binarize to obtain the segmentation map, denoted as m1.

[0043] 7-2. After flipping the test set images horizontally, vertically, and horizontally and vertically, repeat the above 7-1. Step to obtain the segmentation maps, denoted as m2, m3, and m4;

[0044] 7-3. Vote for m1, m2, m3, and m4. The pixel with the most votes in the corresponding category is taken as the final segmentation result.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] On the one hand, existing convolutional neural network-based skin lesion segmentation network models generally have a large number of parameters and are computationally expensive. On the other hand, dermatoscope images of diseased skin areas face challenges such as blurred boundaries and hair interference. Based on the lightweight network FastFCN, this paper improves network segmentation performance by enhancing the encoder's ability to extract features, capturing multi-scale information, and introducing an attention mechanism to focus on important features. This paper proposes a lightweight skin lesion segmentation model with excellent segmentation performance. The model has only 0.96M parameters and an average Jarracd index of 84% on the test set. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of the present invention;

[0048] Figure 2 This is the structural diagram of the dual parallel asymmetric convolution module;

[0049] Figure 3 This is the structural diagram of the serial void spatial pyramid pooling module;

[0050] Figure 4(a) is a structural diagram of the semantic text information submodule;

[0051] Figure 4(b) is a diagram showing the structure of the lightweight semantic text information submodule;

[0052] Figure 5 This is the overall structure diagram of the improved FastFCN network;

[0053] Figure 6 A comparison of the segmentation effects of the improved FastFCN network on skin lesion images; (the green line and red line represent the label and segmentation results, respectively). DETAILED DESCRIPTION

[0054] The present invention will be further described below with reference to specific embodiments. The following description is for demonstration and explanation only and does not limit the present invention in any form.

[0055] like Figure 1 As shown, according to the technical solution provided by the present invention, a skin lesion image segmentation method based on improved FastFCN is proposed, comprising the following steps:

[0056] Step 1: Select the ISIC2018 dataset as experimental data, preprocess the data, determine the test set and training set, and input the data after data enhancement;

[0057] Step 2: Construct a dual parallel asymmetric convolution module to extract nonlinear features;

[0058] Step 3: Construct a serial dilated spatial pyramid pooling module to obtain multi-scale features;

[0059] Step 4: Integrate the asymmetric convolution module, serial dilated spatial pyramid pooling module, dual attention CBAM module, and bottleneck layer into submodules to construct a semantic text information submodule and a lightweight semantic text information submodule;

[0060] Step 5: Embed the semantic text information submodule and the lightweight semantic text information submodule into the FastFCN network after encoder structure optimization to construct the improved FastFCN network;

[0061] Step 6: Perform multi-scale supervised auxiliary training on the constructed improved FastFCN network to obtain the required segmentation network;

[0062] Step 7: After data enhancement, the test set images are input into the segmentation network obtained in step 5, and the segmentation map is output by flipping the test.

[0063] In step 1, the ISIC2018 Skin Lesion Analysis Melanoma Detection Challenge dataset was used. This dataset contains 2,594 training images and their corresponding labels, 100 validation images and their corresponding labels, and 1,000 unlabeled test images. Image data preprocessing was then performed. The specific steps for preprocessing are as follows:

[0064] 1-1. Resize the training set images to 224*224 and the training labels to 112*112;

[0065] 1-2. After fully shuffling the order of the 2594 images in the training set, split them into a new training set (2076 images and corresponding labels) and a test set (518 images and corresponding labels) in a 4:1 ratio;

[0066] 1-3. Grayscale the training set images, normalize the labels, and then use data augmentation methods such as random flipping and random brightness changes before inputting them into the network.

[0067] like Figure 2 As shown, the construction of the dual parallel asymmetric convolution module in step 2 is as follows:

[0068] 2-1. The dual parallel asymmetric convolution module consists of two parallel asymmetric convolution modules connected in series, while the single parallel asymmetric convolution module adopts a dual-branch parallel connection structure, divided into vertical branches and horizontal branches. The structure of the vertical branch is as follows: the first layer is a DOConv (Depthwise Over-parameterized Convolutional, 3*1) convolution layer, which extracts vertical features; the second layer is a BN layer, which controls gradient explosion and prevents gradient vanishing; the third layer is a PRelu layer, which uses parameterized ReLU to activate nonlinear features. The structure of the horizontal branch uses a DOConv (Depthwise Over-parameterized Convolutional, 1*3) convolution layer in the first layer, and the rest of the structure is the same as the vertical branch.

[0069] 2-2. The outputs of the vertical branch and the horizontal branch are combined by adding the feature maps and used as the output of the parallel asymmetric convolution module.

[0070] 2-3. Feature extraction uses two parallel asymmetric convolutional modules in series, named dual parallel asymmetric convolutional modules. The convolutional layer in the first parallel asymmetric convolutional module adjusts the number of input channels to the number of output channels, while the second parallel asymmetric convolutional module keeps the number of channels unchanged.

[0071] like Figure 3 As shown, in step 3, a serial dilated spatial pyramid pooling module is constructed, and the specific steps are:

[0072] 3-1. The serial atrous spatial pyramid pooling module improves the branch with a larger expansion rate in the existing atrous spatial pyramid pooling module. It also has a four-branch parallel connection structure. The details of the four branches are as follows (assuming that the number of input channels of each branch is m):

[0073] The first branch adopts direct connection mode without any processing. The number of input and output channels of this branch is m.

[0074] The second branch uses a DO-Conv (3*3, dilation=3) convolutional layer + BN layer + PReLU layer in series. The number of input and output channels of this branch remains unchanged.

[0075] The third branch uses a combination of two DO-Conv (3*3, dilation=3) convolutional layers, a BN layer, and a PReLU layer. After the first DO-Conv (3*3, dilation=3) convolutional layer, the number of output channels increases to 2m. After the second DO-Conv (3*3, dilation=3) convolutional layer, the number of output channels returns to m.

[0076] The fourth branch consists of three DO-Conv (3*3, dilation=3) convolutional layers, a batch normalization layer, and a pre-relu layer. The input channels of this branch are m. After the first DO-Conv (3*3, dilation=3) convolutional layer, the number of output channels increases to 3m. After the second DO-Conv (3*3, dilation=3) convolutional layer, the number of output channels decreases to 2m. After the third DO-Conv (3*3, dilation=3) convolutional layer, the number of output channels decreases further to m.

[0077] Finally, the output results of the four branches are fused through concat to obtain the final output of the serial dilated spatial pyramid pooling module, with the number of output channels being 4m.

[0078] In step 4, the asymmetric convolution module, serial atrous spatial pyramid pooling module, dual attention CBAM module, and bottleneck layer are integrated into the submodule to construct the semantic text information submodule. As shown in Figure 4(a), the specific details of the semantic text information submodule are as follows (assuming that the number of input and output channels of this module is m and n, respectively):

[0079] 4-1. First, use the bottleneck layer and the combination of DO-Conv (1*1) convolution layer, BN layer, and PReLU layer to compress the number of input channels m to n / 4;

[0080] 4-2. Then, a dual parallel asymmetric convolution module is used to extract nonlinear features, while the number of input and output channels remains unchanged at n / 4;

[0081] 4-3. Then, a serial dilated spatial pyramid pooling module is used to capture multi-scale features. After the serial dilated spatial pyramid pooling module, the number of output channels is restored to n.

[0082] 4-4. Finally, the CBAM module with dual attention of spatial channels is introduced. With the help of the combination of DO-Conv (1*1) convolution layer + BN layer + CBAM module + PReLU layer, it focuses on the important features in multi-scale features. During this period, the number of input and output channels remains unchanged.

[0083] As shown in Figure 4(b), the lightweight semantic text information submodule is replaced by a normal atrous spatial pyramid pooling module in step 4-3. Its other components are the same as those of the semantic text information submodule. The four branches in the normal atrous spatial pyramid pooling module are as follows: the first branch uses a DO-Conv (1*1, dilation=1) convolutional layer + BN layer + PReLU layer in series; the second branch uses a DO-Conv (3*3, dilation=1) convolutional layer + BN layer + PReLU layer; the third branch uses a DO-Conv (3*3, dilation=2) convolutional layer + BN layer + PReLU layer; and the fourth branch uses a DO-Conv (3*3, dilation=3) convolutional layer + BN layer + PReLU layer. The number of input and output channels on the four branches remains unchanged.

[0084] In step 5, the semantic text information submodule and the lightweight semantic text information submodule are embedded into the FastFCN network after the encoder structure is optimized to construct the improved FastFCN network. The specific steps are:

[0085] 5-1. The improved FastFCN network consists of six submodules: Encoder1, Encoder2, Encoder3, Encoder4, and Encoder5 in the encoder section, and Dncoder1 in the decoder section. The semantic text information submodule is applied to Encoder1, Encoder2, Encoder3, Encoder4, and Dncoder1, while the Encoder5 submodule uses a lightweight semantic text information submodule.

[0086] 5-2. Based on the original FastFCN network, the encoder adds six skip connection paths from shallow feature maps to deep feature maps, which serve as one of the inputs to the deep sub-modules in the encoder. The six skip connection paths are: Encoder1→Encoder3, Encoder1→Encoder4, Encoder1→Encoder5, Encoder2→Encoder4, Encoder2→Encoder5, and Encoder3→Encoder5. The internal composition of each skip connection path is similar: the first layer is a max pooling layer, which reduces the feature map resolution to the required size. Then, a dual parallel asymmetric convolution module is used to extract nonlinear features. The convolution layer in the first parallel asymmetric convolution module reduces the number of input channels to 16, and the number of channels remains unchanged.

[0087] like Figure 5As shown, in step 6, the constructed improved FastFCN network is subjected to multi-scale supervised auxiliary training to obtain the required segmentation network. The specific details are as follows:

[0088] 6-1. Multi-scale supervised training is performed on the feature maps output by Encoder3, Encoder4, Encoder5, and Dncoder1 in the network. There are four independent supervised paths. The feature maps output by Encoder3, Encoder4, and Encoder5 are upsampled using bilinear interpolation to increase the feature map resolution to 112*112. Each of the four supervised paths is then connected to a combination of a parallel asymmetric convolutional module and a DO-Conv (3*3) convolutional layer to extract nonlinear features. The number of input channels is reduced to 16 by the convolutional layers in the parallel asymmetric convolutional module, and then reduced to 1 by the DO-Conv (3*3) convolutional layer, outputting a segmentation map at the corresponding scale.

[0089] 6-2. The segmentation maps output by the four supervision paths are respectively compared with the training labels to calculate the loss value. Each path has the same weight, and the results are accumulated to obtain the final loss value.

[0090] 6-3. The network is trained twice. The first training uses a single Lovasz Loss loss function. The second training uses the parameters of the model after the first training and uses the Lovasz Loss + Focalloss loss function for further training to overcome the problem of sample imbalance.

[0091] In step 7, the test set image is input into the segmentation network obtained in step 5 after data augmentation, and the segmentation map is output by flipping the test output. The specific steps are as follows:

[0092] 7-1. After grayscale processing, the test set images are input into the trained network. The feature maps output by the network encoder are normalized using the Sigmoid() function and binarized with a threshold of 0.5. Since the resolution is smaller than the input image resolution, the resize function is used to restore the resolution to 224*224 and re-binarize to obtain the segmentation map, denoted as m1.

[0093] 7-2. After flipping the test set images horizontally, vertically, and horizontally and vertically, repeat 7-1 to obtain the segmentation maps, denoted as m2, m3, and m4;

[0094] 7-3. Vote for m1, m2, m3, and m4. The pixel with the most votes in the corresponding category is taken as the final segmentation result.

[0095] Regarding network hyperparameter settings: The network uses the Adam algorithm to optimize the loss function, with an initial learning rate of 0.001. The training batch size is 16, and the epoch is set to 48.

[0096] The image segmentation method based on the improved FastFCN was verified, and the experimental results are shown in Tables 1 and 2. In terms of the number of model parameters, the proposed method requires significantly fewer model parameters than other methods, and the network is very lightweight. In terms of model segmentation performance, the ISIC2018 dataset was used for verification. On the key indicator Jarracd, the proposed method achieved the most advanced segmentation performance. Figure 6 The figure shows the comparison of the segmentation effect of the improved FastFCN network on skin lesion images.

[0097] Table 1The comparison of different architectures' parameters

[0098]

[0099] Table 1

[0100] Table 2Results of different architectures on the ISIC 2018

[0101]

Claims

1. An image segmentation method based on improved FastFCN, characterized in that: The steps include: Step 1: Select the ISIC2018 dataset as experimental data, preprocess the data, fully shuffle the data into test set and training set, and enhance the data before input; Step 2: construct a dual parallel asymmetric convolution module to extract nonlinear features; the dual parallel asymmetric convolution module is composed of two parallel asymmetric convolution modules connected in series; Step 3: Construct a serial dilated spatial pyramid pooling module to obtain multi-scale features; Step 4: Integrate the asymmetric convolution module, serial dilated spatial pyramid pooling module, normal dilated spatial pyramid pooling module, dual attention CBAM module, and bottleneck layer into submodules to construct semantic text information submodule and lightweight semantic text information submodule. The specific steps are as follows: 4-1. First, use the bottleneck layer and a combination of a 1x1 DO-Conv convolution layer, a BN layer, and a PReLU layer to compress the number of input channels m to n / 4. 4-2. Then, a dual parallel asymmetric convolution module is used to extract nonlinear features, while the number of input and output channels remains unchanged at n / 4; 4-3. Then, a serial dilated spatial pyramid pooling module is used to capture multi-scale features. After the serial dilated spatial pyramid pooling module, the number of output channels is restored to n. 4-4. Finally, we introduce the CBAM module for spatial channel dual attention. By combining a 1x1 DO-Conv convolutional layer, a BN layer, a CBAM module, and a PReLU layer, we focus on important features in multi-scale features while keeping the number of input and output channels n unchanged. Step 5: Embed the semantic text information submodule and the lightweight semantic text information submodule into the FastFCN network after encoder structure optimization to build the improved FastFCN network. The specific steps are as follows: 5-1. The improved FastFCN network consists of six submodules: Encoder1, Encoder2, Encoder3, Encoder4, and Encoder5 in the encoder section, and Dncoder1 in the decoder section. The semantic text information submodule is applied to Encoder1, Encoder2, Encoder3, Encoder4, and Dncoder1, while the Encoder5 submodule uses a lightweight semantic text information submodule. 5-2. Based on the original FastFCN network, the encoder part adds six skip connection paths from shallow feature maps to deep feature maps, which serve as one of the inputs of the deep sub-modules in the encoder; the six skip connection paths are: Encoder1→Encoder3, Encoder1→Encoder4, Encoder1→Encoder5, Encoder2→Encoder4, Encoder2→Encoder5, and Encoder3→Encoder5; the internal composition of each skip connection path is similar: the first layer is a maximum pooling layer to reduce the feature map resolution to the required size, followed by a dual parallel asymmetric convolution module to extract nonlinear features. The convolution layer in the first parallel asymmetric convolution module reduces the number of input channels to 16, and the number of channels remains unchanged. Step 6: Perform multi-scale supervised auxiliary training on the constructed improved FastFCN network to obtain the required segmentation network; Step 7: After data enhancement, the test set images are input into the segmentation network obtained in step 5, and the segmentation map is output by flipping the test.

2. The image segmentation method based on the improved FastFCN according to claim 1, characterized in that: In step 2, a dual parallel asymmetric convolution module is constructed to extract nonlinear features. The specific steps are as follows: The dual parallel asymmetric convolution module consists of two parallel asymmetric convolution modules connected in series, while the single parallel asymmetric convolution module adopts a dual-branch parallel connection structure, which is divided into vertical branches and horizontal branches; the structure of the vertical branch is as follows: the first layer is a 3*1 DOConv convolution layer to extract vertical features; the second layer is a BN layer to control gradient explosion and prevent gradient disappearance; the third layer is a PRelu layer to activate nonlinear features; the structure of the horizontal branch is the same as the vertical branch except that the DOConv 1*3 convolution layer is used in the first layer; the outputs of the vertical branch and the horizontal branch are then merged by adding the feature maps as the output of the parallel asymmetric convolution module.

3. The image segmentation method based on the improved FastFCN according to claim 1, characterized in that: In step 3, a serial dilated spatial pyramid pooling module is constructed to obtain multi-scale features; The specific steps are: It has a structure with four branches connected in parallel. Assume that the number of input channels of each branch is m. The specific situations of the four branches are as follows: the first branch adopts a direct connection method without any processing. The number of input and output channels of this branch is m; the second branch adopts a combination of a 3*3, dilation=3 DO-Conv convolution layer + BN layer + PReLU layer in series. The number of input and output channels of this branch remains unchanged at m; the third branch adopts a combination of two 3*3, dilation=3 DO-Conv convolution layers + BN layer + PReLU layers in series. The number of input channels of this branch is m, which passes through the first and second 3*3, dilation=3 layers respectively. After the DO-Conv convolution layer with dilation=3, the number of output channels first increases to 2m and then decreases to m; the fourth branch adopts the combination of three 3*3, dilation=3 DO-Conv convolution layers + BN layer + PReLU layer in series; the number of input channels of this branch is m, and when passing through the first, second, and third 3*3, dilation=3 DO-Conv convolution layers, the number of output channels first increases to 3m, then decreases to 2m, and finally to m; finally, the output results of the four branches are fused by concat to obtain the final output of the serial atrous spatial pyramid pooling module, and the number of output channels is 4m.

4. The image segmentation method based on the improved FastFCN according to claim 1, characterized in that: The specific steps of preprocessing are: 1-1. Unify the size of the training set images; 1-2. Divide the training set into new training set and test set; 1-3. Process the training set images and labels separately and input them into the network.

5. The image segmentation method based on improved FastFCN according to claim 1, characterized in that: The test set images are input into the segmentation network obtained in step 5 after data enhancement, and the segmentation map is output by flipping the test; Specifically: 7-1. After grayscale processing, the test set images are input into the trained network. The feature map output by the network encoder is normalized using the Sigmoid() function and binarized with a threshold of 0.

5. Since the resolution is smaller than that of the input image, the resolution is restored to 224*224 using the resize function and binarized again to obtain the segmentation map, denoted as m1. 7-2. After flipping the test set images horizontally, vertically, and horizontally and vertically, repeat the above 7-1. Step to obtain the segmentation maps, denoted as m2, m3, and m4; 7-3. Vote for m1, m2, m3, and m4. The pixel with the most votes in the corresponding category is taken as the final segmentation result.

Images