An efficient semantic segmentation method based on improved deeplabv3+

Abstract

The application provides an efficient semantic segmentation method based on improved Deeplabv3+, comprising the following steps: after an input picture is cropped to 512*512, the picture is input into a Mobilenetv2 feature extraction network for feature extraction, and four features of different levels of 128*128*24, 64*64*32, 32*32*96 and 32*32*320 are obtained; the features of the first three levels are input into an FPN module for feature fusion to output shallow layer features, and the output features after the shallow layer features are input into a PCSM attention module are input into a decoder; the feature of the last level is input into an EASPP module to obtain deep layer features, and the output features after the deep layer features are input into the PCSM attention module are input into the decoder; the shallow layer features input into the decoder are adjusted in channel number through 1*1 convolution, the deep layer features are adjusted in channel number through 1*1 convolution and are up-sampled, and then the shallow layer features adjusted in channel number through 1*1 convolution are spliced with the deep layer features, and finally, 3*3 convolution and 4 times up-sampling are performed to obtain a final prediction result. The application realizes optimization of a traditional Deeplabv3+.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of semantic segmentation, and particularly relates to an efficient semantic segmentation method based on improved Deeplabv3+. BACKGROUND

[0002] Semantic segmentation is a key problem in the field of computer vision. The role of semantic segmentation is to label each pixel point in an image as different colors according to the category to which the object of the pixel point belongs, thereby realizing the division of categories. Semantic segmentation has been widely applied in the fields of remote sensing images, medical images, autonomous driving, etc., and provides more intelligent applications for various fields.

[0003] Current semantic segmentation methods have high accuracy, but still have many problems. First, there is a limitation of convolutional neural network (CNN). CNN, as the basic framework of semantic segmentation, has some problems, such as limited receptive field, which cannot fully capture the context semantic information, resulting in information loss in the feature extraction process, affecting the final segmentation accuracy, and the segmentation effect in small objects, object edge segmentation and complex scenes with multiple categories is not good. Secondly, the current semantic segmentation model is usually complex and has a large amount of calculation, which requires a large amount of memory, limiting their applicability in resource-constrained mobile devices, embedded devices and industrial production applications that require real-time processing. The existing technology may be difficult to meet the real-time processing requirements. In order to deploy on resource-constrained devices, a lighter and more efficient model is needed while ensuring high accuracy.

[0004] In view of the deficiencies of the prior art, the present application provides an efficient semantic segmentation method based on improved Deeplabv3+. SUMMARY

[0005] The purpose of the present application is to provide an efficient semantic segmentation method based on improved Deeplabv3+, which optimizes the traditional Deeplabv3+ to greatly reduce the parameter amount and calculation amount, significantly improves the inference speed, and can be deployed on resource-limited devices and realize real-time processing. At the same time, through the FPN module and the EASPP module, the multi-level features are fully fused and the PCSM attention module is introduced, realizing the mutual complementation of deep semantic information and shallow semantic information, increasing the receptive field of the model, solving the problem of information loss in the feature extraction process of the ASPP module, and greatly improving the segmentation accuracy of the model in small object segmentation, edge segmentation and complex scenes with multiple categories.

[0006] To achieve the above purpose, the technical scheme of the present application is as follows: an efficient semantic segmentation method based on improved Deeplabv3+, specifically comprising the following steps:

[0007] S1, after the input picture is cropped to 512x512, the input is input into a Mobilenetv2 feature extraction network for feature extraction, and four different levels of features of 128x128x24, 64x64x32, 32x32x96 and 32x32x320 are obtained;

[0008] S2, the first three levels of features of 128x128x24, 64x64x32 and 32x32x96 extracted by the Mobilenetv2 feature extraction network are input into an FPN module for feature fusion, and after fusion, the output of 128x128x152 is shallow layer feature, the shallow layer feature of 128x128x152 is input into the PCSM attention module, and the feature output by the PCSM attention module is input into the decoder;

[0009] S3, the feature of 32x32x320 extracted by the Mobilenetv2 feature extraction network is input into the EASPP module to obtain the deep layer feature of 32x32x1792, and the deep layer feature of 32x32x1792 is input into the PCSM attention module, and the output feature of the PCSM attention module is input into the decoder;

[0010] S4, the feature input into the decoder in step S2 is adjusted in channel number by 1x1 convolution, the feature input into the decoder in step S3 is adjusted in channel number by 1x1 convolution and up-sampling, and then spliced with the feature input in step S2 adjusted in channel number by 1x1 convolution, and then 3x3 convolution and 4 times up-sampling are performed to obtain the final prediction result.

[0011] Preferably, the Mobilenetv2 feature extraction network applies the network structure configuration of the first 8 layers of the traditional Mobilenetv2 network, wherein the step length s of the 7th layer is changed from 2 to 1, the input feature size of the 1st layer of the Mobilenetv2 feature extraction network is 512x512x3, and the output feature sizes of the 3rd layer, the 4th layer, the 6th layer and the 8th layer of the Mobilenetv2 feature extraction network are 128x128x24, 64x64x32, 32x32x96 and 32x32x320 respectively.

[0012] Preferably, the FPN module specifically performs the following operations: the feature of 32x32x96 extracted by the Mobilenetv2 feature extraction network is spliced with the feature of 64x64x32 after being up-sampled by two times to output the feature of 64x64x128, and the feature of 64x64x128 is spliced with 128x128x24 after being up-sampled by two times, and finally the shallow layer feature of 128x128x152 is output.

[0013] Preferably, the EASPP module includes three branches which work in parallel and output three parts of features with sizes of 32x32x256, 32x32x1280 and 32x32x256 respectively, and the output results of the three branches are spliced in the channel dimension to obtain the output features of the EASPP module with a size of 32x32x1792; specifically as follows:

[0014] The first branch is a 1x1 convolutional layer which inputs the deep features with a size of 32x32x320 extracted by the Mobilenetv2 feature extraction network, and directly outputs the features with a size of 32x32x256 after 1x1 convolution and adjustment of the number of channels;

[0015] The second branch includes five 3x3 atrous convolutions, and the 32x32x1280 features spliced from the output results of the five 3x3 atrous convolutions;

[0016] The third branch is an average pooling layer which inputs the deep features with a size of 32x32x320, performs average pooling and adjustment of the number of channels, and outputs the features with a size of 32x32x256; finally, the output results of the three branches of the EASPP module are spliced to obtain the features with a size of 32x32x1792.

[0017] Preferably, the atrous rates of the five 3x3 atrous convolutions are 3, 6, 12, 18 and 24 respectively.

[0018] Preferably, in the five 3x3 atrous convolutions of the EASPP module, each of the first to fourth 3x3 atrous convolutions outputs two identical results, one of which is directly output for splicing with the output results of the other four 3x3 atrous convolutions, and the other is used as the input feature splicing of the next 3x3 atrous convolution; specifically as follows:

[0019] The deep features with a size of 32x32x320 extracted by the Mobilenetv2 feature extraction network are denoted as feature one;

[0020] The feature one is input into the first 3x3 atrous convolution and output as feature two;

[0021] The feature one and the feature two are spliced to obtain feature three, and the feature three is input into the second 3x3 atrous convolution and output as feature four;

[0022] The feature one and the feature three are spliced to obtain feature five, and the feature five and the feature four are spliced to obtain feature six, and the feature six is input into the third 3x3 atrous convolution and output as feature seven;

[0023] The feature one and the feature five are spliced to obtain a feature eight, the feature eight and the feature six are spliced to obtain a feature nine, the feature nine and the feature seven are spliced to obtain a feature ten, and the feature ten is input into a fourth layer 3*3 hollow convolution to output a feature eleven;

[0024] The feature one and the feature eight are spliced to obtain a feature twelve, the feature twelve and the feature nine are spliced to obtain a feature thirteen, the feature thirteen and the feature ten are spliced to obtain a feature fourteen, the feature fourteen and the feature eleven are spliced to obtain a feature fifteen, and the feature fifteen is input into a fifth layer 3*3 hollow convolution to output a feature sixteen;

[0025] The feature two, the feature four, the feature seven, the feature eleven and the feature sixteen are spliced to obtain an output result of a second large branch.

[0026] Preferably, the PCSM module comprises a channel attention module and a spatial attention module connected in parallel; and the PCSM module specifically performs the following operations:

[0027] The H*W*C feature input into the PCSM module generates channel weights and spatial weights through the channel attention module and the spatial attention module, respectively, and then the channel weights and the spatial weights are multiplied by the feature input into the PCSM module to obtain spatial features and channel features with a size of H*W*C, the channel features, the spatial features and the input feature are spliced in the channel dimension to output a feature with a size of H*W*3C, and then a 1*1 convolution is used to adjust the number of channels to output a PCSM module output feature with a size of H*W*C, wherein H, W and C represent the length, width and number of channels of the feature, respectively.

[0028] Preferably, the channel attention module specifically performs the following operations: the input feature of the channel attention module is processed through parallel connection of average pooling and maximum pooling to respectively output mean features and maximum value features, the two features are input into a shared multi-layer perceptron (MLP), the two features processed by the MLP are added, and a channel weight is generated by using a Sigmoid activation function.

[0029] Preferably, the spatial attention module specifically performs the following operations: the input feature of the spatial attention module is processed through maximum pooling and average pooling in the channel dimension, the number of channels is compressed to 1, and then the two 1-channel features obtained after processing are spliced in the channel dimension to obtain a 2-channel feature; a 7*7 convolution operation with a padding rate of 3 is performed on the 2-channel feature obtained by splicing to compress the number of channels to 1, and finally a spatial weight is generated by using a Sigmoid activation function

[0030] Compared with the prior art, the present application has the following beneficial effects:

[0031] The original model is optimized and improved, the parameter quantity and the calculation quantity of the model are greatly reduced, the reasoning speed is greatly improved, the segmentation precision is still maintained, and a good balance between high precision and high efficiency is realized. Through the fusion of multi-level semantic features, the segmentation precision of the object edge of the application is higher than that of the original model, and the overall performance is better than that of the current advanced semantic segmentation method. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 It is a schematic diagram of the overall structure of the application;

[0033] Figure 2 It is a schematic diagram of the FPN structure of the application;

[0034] Figure 3 It is a schematic diagram of the CAM attention module of the application;

[0035] Figure 4 It is a schematic diagram of the SAM attention module of the application;

[0036] Figure 5 It is a schematic diagram of the PCSM attention module of the application;

[0037] Figure 6 It is a comparison chart of the visualization experiment of the application. DETAILED DESCRIPTION

[0038] The technical solutions of the application will be specifically described below. Figures 1-6 The technical solutions of the application will be specifically described below.

[0039] The application provides an efficient semantic segmentation method based on improved Deeplabv3+, which specifically includes the following steps:

[0040] S1, the input picture is cropped to 512*512, and then input into the Mobilenetv2 feature extraction network for feature extraction, and Mobilenetv2 extracts four different levels of features of 128*128*24, 64*64*32, 32*32*96 and 32*32*320;

[0041] S2, the first three are input into the FPN module as shallow features for feature fusion, and the fused shallow features with rich semantic information of 128*128*152 are output, the features are input into the PCSM attention module, and the weighted features of 128*128*152 are output, then the features are input into the decoder;

[0042] S3, the deep layer feature input EASPP is input into three branches of the feature enhancement, and the features output by the three branches are 32x32x256, 32x32x1280 and 32x32x256 respectively. The deep layer feature is obtained by splicing the features in the channel dimension, and then the deep layer feature is input into the PCSM attention module, and the output feature is input into the decoder;

[0043] S4, the features input into the decoder in step S2 are adjusted in channel number by 1x1 convolution, the features input into the decoder in step S3 are adjusted in channel number by 1x1 convolution and up-sampling, and then spliced with the features input into step S2, and then obtained by 3x3 convolution and 4 times up-sampling to obtain the final prediction result.

[0044] The detailed description of each part is as follows:

[0045] 1. Backbone network replacement

[0046] Xception is used for feature extraction in the original network, but the Xception model has large parameters and complex structure, so the calculation amount is also very large, which requires more computing resources. In the scene of mobile devices, embedded devices and industrial production, etc. It is difficult to adapt to the limited computing resources and the need for fast reasoning. The present application uses Mobilenetv2 as the backbone network for feature extraction. Mobilenetv2 is a lightweight feature extraction network, which mainly consists of linear bottleneck, reverse residual and depth separable convolution structures. The main structure and parameters are shown in Table 1.

[0047] Table 1 Mobilenetv2 structure and main parameters

[0048]

[0049] Considering the limited computing resources, Mobilenetv2 is used 4 times in the present application. Therefore, the step s of the 7th layer in Table 1 is changed to 1 (if s is 2, 5 times down-sampling is performed). In the feature extraction process, the input is 513x512x3, and the output is 128x128x24, 64x64x32 and 32x32x96 three different scale features.

[0050] The Mobilenetv2 used in the present application is described in detail: the complete Mobilenetv2 network structure is shown in Table 1, and there are a total of 11 layers, but only the first 8 layers are needed in the actual application of the present application. Since 4 times of down-sampling are adopted in the present application, the step s of the 7th layer is changed to 1 (when the step s is 2, one time of down-sampling is performed, and if the s of the 7th layer is 2, 5 times of down-sampling are performed), and 128x128x24, 64x64x32 and 32x32x96 are respectively the output of the 3rd layer, the 4th layer and the 6th layer, that is, the output features of the layers can be extracted in the running of the Mobilenetv2. And 32x32x320 is the output result of the 8th layer (S=2 is the parameter in the original network, and it is changed to 1 in the present application, so that no down-sampling is performed, and therefore the input features of the 8th layer are 32x32x160, and the output is 32x32x320 after processing by the 8th layer).

[0051] 2. FPN module

[0052] In the original network, the decoder only uses a shallow layer feature of 128x128x24 output by the feature extraction part, and the information contained in the shallow layer feature is less, and the feature information in the image cannot be fully expressed, so the FPN module is introduced in the present application. The FPN module can combine the deep semantic features and the shallow semantic features, so that the output features can have both the semantic information of the deep features and the detail information of the shallow features. In the present application, the first three down-sampling results (corresponding to the output results of the 3rd, 4th and 6th layers in Table 1) output from the Mobilenetv2 are 128x128x24, 64x64x32 and 32x32x96, respectively. After two times of up-sampling, 64x64x32 is spliced with 64x64x128, and 64x64x128 is spliced with 128x128x24 after two times of up-sampling, and the final output feature size of the FPN module is 128x128x152. The specific structure of the FPN module is shown in Figure 2 .

[0053] 3. EASPP module

[0054] The five branches of the ASPP module in the original network work in parallel, and after processing the input features respectively, the five output features are spliced, and there is no information sharing in the processing process, which is easy to cause the loss of part of the information. In the present application, the EASPP module is designed, which mainly consists of three branches, and the three branches work in parallel to output three parts of features and splice them.

[0055] The first branch is a 1x1 convolutional layer, which inputs the deep features of 32x32x320 extracted by Mobilenetv2, and directly outputs the features of 32x32x256 after 1x1 convolution and adjusting the number of channels.

[0056] The second large branch is composed of five 3x3 hollow convolutions, and the hollow rates are 3, 6, 12, 18 and 24 respectively. Each layer of 3x3 convolution outputs two identical results, one of which is directly outputted to splice with the output results of the other four layers of 3x3 hollow convolution, and the other is used as the input feature splicing of the next layer of 3x3 hollow convolution, which is as follows:

[0057] The deep features of 32x32x320 extracted by Mobilenetv2 feature extraction network are denoted as feature one;

[0058] The feature one is inputted into the first layer of 3x3 hollow convolution and the feature two is outputted;

[0059] The feature one and the feature two are spliced to obtain the feature three, and the feature three is inputted into the second layer of 3x3 hollow convolution and the feature four is outputted;

[0060] The feature one and the feature three are spliced to obtain the feature five, the feature five and the feature four are spliced to obtain the feature six, and the feature six is inputted into the third layer of 3x3 hollow convolution and the feature seven is outputted;

[0061] The feature one and the feature five are spliced to obtain the feature eight, the feature eight and the feature six are spliced to obtain the feature nine, the feature nine and the feature seven are spliced to obtain the feature ten, and the feature ten is inputted into the fourth layer of 3x3 hollow convolution and the feature eleven is outputted;

[0062] The feature one and the feature eight are spliced to obtain the feature twelve, the feature twelve and the feature nine are spliced to obtain the feature thirteen, the feature thirteen and the feature ten are spliced to obtain the feature fourteen, the feature fourteen and the feature eleven are spliced to obtain the feature fifteen, and the feature fifteen is inputted into the fifth layer of 3x3 hollow convolution and the feature sixteen is outputted;

[0063] The feature two, the feature four, the feature seven, the feature eleven and the feature sixteen are spliced to obtain the output result of the second branch, that is, the final output of the second branch is the 32x32x1280 feature spliced by the output results of the five layers of 3x3 convolution.

[0064] The third branch is an average pooling layer, which inputs the deep features of 32x32x320 and outputs the features of 32x32x256 after average pooling. Finally, the output results of the three branches of the EASPP module are spliced to obtain the features of 32x32x1792.

[0065] At the same time, multiple hollow convolution superposition uses can greatly increase the receptive field, and the large receptive field can capture semantic information in a larger range, which can help the algorithm to determine the relationship between each pixel, and the calculation formula of the receptive field is: R = (d-1) x (K-1) + K, R = R1 + R2-1, wherein K is the size of the convolution kernel, d is the inflation rate, and R1 and R2 are the receptive fields of the superimposed convolution kernels. The receptive field of the EASPP module reaches 127.

[0066] 4. PCSM attention module

[0067] The channel attention module is as shown in Figure 3 The input feature is first processed by parallel connection of average pooling and maximum pooling, and the mean feature and the maximum value feature are output respectively, and the two features are input into the shared multi-layer perceptron (MLP), and finally the two features processed by the MLP are added, and the channel weight is generated by using the Sigmoid activation function.

[0068] The spatial attention module is as shown in Figure 4 The input feature is first processed by maximum pooling and average pooling in the channel dimension, and the channel number is compressed to 1, which can reduce the calculation amount to a certain extent and can better learn the spatial feature, and the two 1-channel features obtained after processing are spliced in the channel dimension to obtain a 2-channel feature. Then, a 7x7 convolution operation with a padding rate of 3 is performed on the 2-channel feature spliced, and the channel number is also compressed to 1, and finally the spatial weight is generated by using the Sigmoid activation function.

[0069] The PCSM module designed in the application connects the channel attention module and the spatial attention module in parallel as shown in Figure 5 After the input HxWxC feature generates the channel weight and the spatial weight, it is multiplied by the input feature, and the channel feature and the spatial feature with the size of HxWxC are output respectively, and then the channel feature, the spatial feature and the input feature are spliced in the channel dimension to output the feature with the size of HxWx3C. In order to avoid excessive calculation, a 1x1 convolution is used to adjust the channel number, and the feature with the size of HxWxC is output, which contains the channel weight information, the spatial weight information and the original feature information, avoiding the loss of part of important information caused by multiple weighting of the channel attention module and the spatial attention module connected in series in the traditional CBAM attention module.

[0070] 6. Experiment and analysis

[0071] In order to verify the effectiveness of the semantic segmentation method proposed in the present application, a corresponding experiment is set up, and the PASCAL VOC2012Aug data set is selected, which has 20 different categories of objects and a background class, a total of 10582 images, of which 1449 are verification sets and 1456 are test sets.

[0072] The operating system of the experimental equipment is Windows 10, the graphics card is NVIDIA GeForce RTX 2080 (8G), and the CPU is Intel Core i9-10900K CPU. The experimental environment is CUDA11.1, PyTorch1.7, and Python3.8. In order to improve the experimental efficiency and accelerate the convergence speed of the model, transfer learning is used, and the initial weight of Mobilenetv2 is obtained by training on the Imagenet data set. The input image is cropped to 512x512, and the Adam algorithm is used for gradient descent during model training, and the momentum method with a momentum of 0.9 is used for optimization. The initial learning rate is 5x10^-4, and the training is 300 rounds, of which the first 50 rounds are frozen training, and the last 250 rounds are unfrozen training.

[0073] Firstly, the effectiveness of the four modules modified or proposed in the present application is verified, and comparative experiments of models with different modules on the PASCAL VOC2012Aug data set are set up, as shown in Table 2, and the model after replacing the feature extraction part of the original model with Mobilenetv2 is used as the benchmark model, denoted as D.

[0074] Table 2 Comparison results of algorithms with different modules on the PASCAL VOC2012Aug data set

[0075]

[0076] Then, the best number and position of the PCSM attention module designed in the present application in the model are verified, and comparative experiments of models with different numbers and positions of PCSM modules on the PASCAL VOC2012Aug data set are set up, and the results are shown in Table 3. It can be seen that the number and position of the PCSM module deployed in the present application are the best.

[0077] Table 3 Accuracy comparison of algorithms with different numbers and positions of PCSM modules

[0078]

[0079] To verify that the PCSM module designed in this invention is superior to other attention modules, ECA attention module, CBAM attention module, and CA attention module were selected to replace the PCSM module and comparative experiments were conducted on the PASCALVOC2012Aug dataset. The experimental results are shown in Table 4, where B represents a Deeplabv3+ model with Mobilenetv2 as the backbone network, plus FPN and EASPP modules. It can be seen that the PCSM module designed in this invention has a higher average crossover ratio and average pixel accuracy than other attention modules when deployed in this model.

[0080] Table 4 shows the comparison results of algorithms with different attention mechanisms on the PASCAL VOC2012 Aug dataset.

[0081]

[0082] To demonstrate the performance comparison between the semantic segmentation method of this invention and other advanced semantic segmentation methods, a comparative experiment was conducted on the PASCALVOC2012Aug dataset. The experimental results are shown in Table 5. It can be seen that compared with the original model, the present invention reduces the number of parameters by 82.55% and the computation by 62.52%, and improves the inference speed by 71.92%, while the average intersection-union ratio and average pixel accuracy reach 75.44% and 84.99%, respectively.

[0083] Table 5 shows the comparison results with other advanced algorithms on the PASCAL VOC2012Aug dataset.

[0084]

[0085] Finally, to demonstrate the segmentation effect of the semantic segmentation method of this invention in practical applications and to compare it with the original model, five images were randomly selected from the test set of the PASCALVOC2012Aug dataset for segmentation. The segmentation results are as follows: Figure 6 As shown, Figure 6 In the image, (a) is the original image; (b) is the label; (c) is the segmentation result of Deeplabv3+; and (d) is the segmentation result of the present invention. It can be seen that the segmentation accuracy of the present invention in complex scenes and object edges is higher than that of the original model.

[0086] This invention optimizes and improves the original model, significantly reducing the number of parameters and computational load, greatly improving inference speed, while maintaining high segmentation accuracy, achieving a good balance between high precision and high efficiency. By fusing multi-level semantic features, this invention achieves higher segmentation accuracy for object edges than the original model, and its overall performance surpasses commonly used advanced semantic segmentation methods.

[0087] The application can be used in the field of automatic driving, the real-time semantic segmentation algorithm can quickly identify roads, vehicles, pedestrians and the like, and provide accurate environmental perception capability for an automatic driving system; the field of intelligent security, such algorithms can realize fast segmentation and identification of targets in monitored videos, and provide support for intelligent monitoring and early warning; in mobile devices, embedded devices and industrial production with limited computing resources, real-time processing of images and videos; in the medical field, the real-time segmentation algorithm can help to quickly identify and analyze different tissues and structures in medical images; remote sensing image processing, which can automatically identify and classify different types of land cover in remote sensing images, such as farmland, forest, water body and urban area.

[0088] The use mode is as follows:

[0089] Step one, determine the scene to be applied;

[0090] Step two, collect the corresponding data, and the data amount should not be less than 500 (preferably more, not less);

[0091] Step three, label the object to be segmented using Labelme;

[0092] Step four, convert the json file generated by labeling into a png format mask;

[0093] Step five, put the original image and the mask into the corresponding folder and put them into the model for training;

[0094] Step six, save the final parameter file of the training;

[0095] Step seven, the model reads the final parameter file and inputs the image for segmentation, or deploys the model into the software that can collect images to realize real-time image segmentation.

[0096] The above is the preferred embodiment of the application, any changes made according to the technical solutions of the application, as long as the generated function does not exceed the scope of the technical solutions of the application, belong to the protection scope of the application.

Claims

1. An efficient semantic segmentation method based on improved Deeplabv3+, characterized in that, Specifically comprising the following steps: S1, after the input picture is cropped to 512*512, the picture is input into a Mobilenetv2 feature extraction network for feature extraction, and four features of different levels of 128*128*24, 64*64*32, 32*32*96 and 32*32*320 are obtained; S2, the features of the first three levels of 128*128*24, 64*64*32 and 32*32*96 extracted by the Mobilenetv2 feature extraction network are input into an FPN module for feature fusion, and after fusion, a shallow feature of 128*128*152 is output, the shallow feature of 128*128*152 is input into a PCSM attention module, and the feature output by the PCSM attention module is input into a decoder; S3, the feature of 32*32*320 extracted by the Mobilenetv2 feature extraction network is input into an EASPP module to obtain a deep feature of 32*32*1792, the deep feature of 32*32*1792 is input into the PCSM attention module, and the output feature of the PCSM attention module is input into the decoder; S4, the feature input into the decoder in step S2 is adjusted in channel number through 1*1 convolution, the feature input into the decoder in step S3 is adjusted in channel number through 1*1 convolution and up-sampling, and then the feature input into the decoder in step S2 is spliced with the feature adjusted in channel number through 1*1 convolution, and finally a 3*3 convolution and 4 times up-sampling are performed to obtain a final prediction result; The EASPP module comprises three branches, the three branches work in parallel and output three features of 32*32*256, 32*32*1280 and 32*32*256 respectively, and the output results of the three branches are spliced in the channel dimension to obtain the output feature of the EASPP module of 32*32*1792; Specifically as follows: The first branch is a 1*1 convolution layer, the deep feature of 32*32*320 extracted by the Mobilenetv2 feature extraction network is input, and the feature of 32*32*256 is directly output after 1*1 convolution and adjustment of the channel number; The second branch comprises five 3*3 atrous convolutions, and the output results of the five 3*3 atrous convolutions are spliced to obtain the feature of 32*32*1280; The third branch is an average pooling layer, the deep feature of 32*32*320 is input, and the feature of 32*32*256 is output after average pooling and adjustment of the channel number; finally, the output results of the three branches of the EASPP module are spliced to obtain the feature of 32*32*1792; The atrous rates of the five 3*3 atrous convolutions are 3, 6, 12, 18 and 24 respectively; In the five 3*3 atrous convolutions of the EASPP module, each 3*3 atrous convolution of the first to fourth layers outputs two identical results, one of which is directly output for splicing with the output results of the other four 3*3 atrous convolutions, and the other is used as the input feature splicing of the next 3*3 atrous convolution, specifically as follows: Mobilenetv2 feature extraction network extracts deep features with a size of 32*32*320, and the deep features are denoted as feature one; The feature one is input into the first 3*3 atrous convolution to output feature two; The feature one and the feature two are spliced to obtain feature three, and the feature three is input into the second 3*3 atrous convolution to output feature four; The feature one and the feature three are spliced to obtain feature five, the feature five and the feature four are spliced to obtain feature six, and the feature six is input into the third 3*3 atrous convolution to output feature seven; The feature one and the feature five are spliced to obtain feature eight, the feature eight and the feature six are spliced to obtain feature nine, the feature nine and the feature seven are spliced to obtain feature ten, and the feature ten is input into the fourth 3*3 atrous convolution to output feature eleven; The feature one and the feature eight are spliced to obtain feature twelve, the feature twelve and the feature nine are spliced to obtain feature thirteen, the feature thirteen and the feature ten are spliced to obtain feature fourteen, the feature fourteen and the feature eleven are spliced to obtain feature fifteen, and the feature fifteen is input into the fifth 3*3 atrous convolution to output feature sixteen. The feature two, the feature four, the feature seven, the feature eleven and the feature sixteen are spliced to obtain the output result of the second large branch.

2. The efficient semantic segmentation method based on improved Deeplabv3+ according to claim 1, characterized in that, The Mobilenetv2 feature extraction network applies the network structure configuration of the first 8 layers of the traditional Mobilenetv2 network, wherein the step s of the 7th layer is changed from 2 to 1, the input feature size of the first layer of the Mobilenetv2 feature extraction network is 512*512*3, and the output feature sizes of the third layer, the fourth layer, the sixth layer and the eighth layer of the Mobilenetv2 feature extraction network are 128*128*24, 64*64*32, 32*32*96 and 32*32*320 respectively.

3. The efficient semantic segmentation method based on improved Deeplabv3+ according to claim 1, characterized in that, The FPN module specifically performs the following operations: the 32*32*96 features extracted by the Mobilenetv2 feature extraction network are spliced with the 64*64*32 features after being up-sampled by two times to output 64*64*128 features, and the 64*64*128 features are spliced with the 128*128*24 features after being up-sampled by two times to finally output 128*128*152 shallow features.

4. The efficient semantic segmentation method based on improved Deeplabv3+ according to claim 1, characterized in that, The PCSM module includes a channel attention module and a spatial attention module connected in parallel; the PCSM module specifically performs the following operations: The H*W*C features input into the PCSM module generate channel weights and spatial weights through the channel attention module and the spatial attention module respectively, and then the channel weights and the spatial weights are multiplied with the input features of the PCSM module to obtain spatial features and channel features with a size of H*W*C, the channel features, the spatial features and the input features are spliced in the channel dimension to output features with a size of H*W*3C, and then a 1*1 convolution is used to adjust the number of channels to output PCSM module output features with a size of H*W*C, wherein H, W and C respectively represent the length, width and channel number of the features.

5. The efficient semantic segmentation method based on improved Deeplabv3+ according to claim 4, characterized in that, The channel attention module specifically performs the following operations: input features of the channel attention module are processed through parallel connection of average pooling and maximum pooling, and mean features and maximum value features are respectively output; the two features are input into a shared multi-layer perceptron (MLP), and the two features processed by the MLP are added and channel weights are generated by using a Sigmoid activation function.

6. The efficient semantic segmentation method based on improved Deeplabv3+ according to claim 4, characterized in that, The spatial attention module specifically performs the following operations: input features of the spatial attention module are processed through maximum pooling and average pooling in the channel dimension, and the channel number is compressed to 1; two 1-channel features obtained after processing are spliced in the channel dimension to obtain a 2-channel feature; A 7*7 convolution operation with a padding rate of 3 is performed on the spliced 2-channel feature, the channel number is compressed to 1, and finally a spatial weight is generated by using a Sigmoid activation function.

Images