Power line semantic segmentation method for improving a deeplabv3+ model

Abstract

The application discloses a power line semantic segmentation method based on an improved DeepLabv3+ model, wherein a lightweight PP-LCNet is used to replace an original DeepLabv3+ main network Xception, so that the parameter quantity is effectively reduced and the prediction speed is improved, a hollow convolution branch and a cascaded convolution are added in a hollow spatial pyramid pooling module, multi-scale features with a larger receptive field are obtained, and the missing segmentation phenomenon is reduced, the hollow convolution branch is further changed into a bottleneck structure to reduce the parameter quantity, and the decoder fuses three layers of shallow features again to recover the lost detail features and spatial information in the down-sampling process. Finally, a bottleneck attention module is introduced to reduce the missegmentation phenomenon of the power line. The experimental environment is configured, the training parameters are set according to the device performance and the characteristics of the power line, and finally, the model is trained, verified and tested by using the divided data set.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of power line detection and relates to a power line semantic segmentation method based on an improved DeepLabv3+ model. BACKGROUND

[0002] Transmission line inspection is an important part of daily maintenance of power grids and plays an important role in ensuring stable operation of power systems. Due to the complex and changeable environment of transmission line erection, manual inspection is inefficient and dangerous, which cannot meet the inspection requirements. In recent years, with the rapid development of unmanned aerial vehicles and high-resolution camera technologies, intelligent power inspection based on unmanned aerial vehicles has been widely used. However, unmanned aerial vehicles are prone to collisions and entanglements with power lines during inspection, which poses a great security risk to the stable operation of transmission lines. Power line segmentation is a key technology for realizing automatic obstacle avoidance of unmanned aerial vehicles and ensuring the safety of low-altitude flight of unmanned aerial vehicles, so it is of great significance to develop a power line segmentation algorithm with high precision and good real-time performance.

[0003] Existing power line segmentation methods can be divided into traditional image processing methods and semantic segmentation methods based on deep learning. Traditional methods can be divided into two categories: edge detection operator-based extraction algorithms and joint feature-based extraction algorithms. The former usually uses edge detection operators and line detectors combined with prior knowledge to extract power lines. The latter uses line detectors combined with global aids or context information to extract power lines. The extraction accuracy of traditional methods for power lines is greatly affected by prior knowledge and aids, and false detection and missed detection may occur in complex backgrounds, so they are only suitable for some specific scenarios.

[0004] The above information disclosed in the background section is only used to enhance the understanding of the background of the application, and therefore can contain information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0005] In view of the problems in the prior art, the application provides a power line semantic segmentation method based on an improved DeepLabv3+ model, which has high precision and good real-time performance, and provides a reliable basis for realizing automatic obstacle avoidance of unmanned aerial vehicles and ensuring the safety of low-altitude flight of unmanned aerial vehicles.

[0006] The object of the application is achieved by the following technical solutions:

[0007] A power line semantic segmentation method based on an improved DeepLabv3+ model comprises the following steps:

[0008] Step 1: For the encoder of the DeepLabv3+ model, replace the original backbone network Xception with a PP-LCNet network;

[0009] Step 2, the hollow space pyramid pooling module increases the hollow convolution branch and cascaded convolution to obtain multi-scale features;

[0010] Step 3, introduce 3 layers of shallow features in the decoder to recover the details and spatial information lost in the downsampling process;

[0011] Step 4: introduce a bottleneck attention module to reduce the misclassification of power lines.

[0012] Step 5, select power line images from the public power line dataset for dataset making, and divide the dataset into training set, validation set and test set;

[0013] Step 6, configure the experimental environment, and set the training parameters according to the device performance and the characteristics of the power line;

[0014] Step 7, use the training set and test set in the dataset to train and verify the model;

[0015] Step 8, use the test set in the dataset to test the model and view the segmentation effect.

[0016] In the method, in step 1, the PP-LCNet network uses a depth separable convolution instead of a standard convolution, wherein the activation function ReLU in the depth separable convolution is replaced by H-Swish, the PP-LCNet network uses a 5×5 convolution instead of a 3×3 convolution at the tail of the network, and a compression and excitation network module is added in the last two depth separable convolution blocks to weight the network channels.

[0017] In the method, in step 2, the bottleneck cascaded hollow space pyramid pooling module with a combination of 6, 12 and 18 hollow rates extracts multi-scale features by increasing the hollow convolution branch and cascaded convolution.

[0018] In the method, in step 2, in the structure of the bottleneck cascaded hollow space pyramid pooling module, a hollow convolution branch is added, and the combination of hollow rates is modified to 3, 6, 9 and 12, then 2 times of 3×3 convolution cascaded operation is performed on the middle 4 convolution branches to extract multi-scale features, when the hollow rate of the hollow convolution branch is r and the convolution kernel size is k, the receptive field size is:

[0019] R=(r-1)×(k-1)+k,

[0020] And when the two-layer hollow convolution branch is cascaded, the receptive field size is:

[0021] R=R1+R2-1,

[0022] Wherein R1, R2 are receptive fields provided by two-layer hollow convolution respectively, the middle four convolution branches are first reduced to 64 channels by 1x1 convolution, then processed by 3x3 convolution twice, and then increased to 256 channels by 1x1 convolution.

[0023] In the method, in step 3, 3 layers of shallow feature maps with downsampling coefficients of 1 / 4, 1 / 8 and 1 / 16 in the PP-LCNet introduced in the decoder are introduced, and the channel numbers of the 3 layers of shallow feature maps are adjusted to 48, 32 and 16 respectively by using 1x1 convolution.

[0024] In the method, in step 4, the bottleneck attention module is composed of a parallel channel attention network and a spatial attention network, the input feature map F is processed by the two parallel networks to obtain a feature map M(F), then the feature map M(F) and the feature map F are multiplied point by point to highlight important features, and the obtained feature map and the feature map F are added to output an attention feature map F', the expressions of the feature map M(F) and the feature map F are as follows:

[0025] M(F)=σ(M c (F)+M s (F)),

[0026] F′=F+F⊙M(F),

[0027] Wherein sigma is a sigmoid function, is a matrix corresponding element point by point multiplication, M c (F) and M s (F) represent the output feature maps of the channel attention network and the spatial attention network respectively.

[0028] In the method, in step 4, in the channel attention network, first, the features in each channel are aggregated by global average pooling to generate a channel vector F c Then, the attention network from F c is evaluated by using a multilayer perceptron, and a channel attention feature map is obtained through a fully connected layer.

[0029] In the method, in step 4, in the spatial attention network, first, the feature map dimension is compressed by using 1x1 convolution, then 2 3x3 convolutions with a hole rate of 4 are used to obtain context information, and finally the spatial attention network feature map is output after further dimension compression by using 1x1 convolution.

[0030] In the method, in step 5, 415 images containing power lines are selected from the public power line dataset TTPLA and WireDataset respectively, and 852 images are expanded to 5000 images through rotation, left and right flipping, enlargement, reduction, cropping and changing brightness as a dataset, 10% of the dataset is used as a test set, and the rest is divided into a training set and a validation set according to a 9:1 ratio.

[0031] In the method, in step 6, the operating system used in the experiment is Windows 10, the processor is Intel(R) Xeon(R) Gold 6230, the graphics card is NVIDIA Tesla V100-PCIE-16GB, the RAM size is 320G, the network framework used in the experiment is pytorch, the python version is 3.7.11, and the initial learning rate is 5e. The average pixel accuracy and the average intersection over union are used as the evaluation criteria of the segmentation accuracy.

[0032] In the method, in step 7, the image size of the training input is 512*512 pixels, the Batchsize used in the training is 8, the iteration number is 100, the CE Loss and the Dice Loss are combined as the loss function of the training, and the validation set is used to verify the effect of the model after each training to prevent overfitting.

[0033] In the method, in step 8, the test set part in the data set is used to test the model, the average pixel accuracy and the average intersection over union of the test model segmentation are tested, and the average pixel accuracy, the average intersection over union and the actual segmentation effect of the power line image are compared with other test models.

[0034] Beneficial effects

[0035] The method provided by the application provides a power line semantic segmentation method based on an improved DeepLabv3+ model, replaces the original main network Xception with a lightweight PP-LCNet to effectively reduce the parameter quantity and improve the prediction speed, adds a hollow convolution branch and a cascade convolution to the ASPP module to strengthen the extraction of power line detail features, reduces the missed segmentation phenomenon, changes the hollow convolution branch to a bottleneck structure to reduce the parameter quantity, introduces 3 shallow layers in the decoder part to more fully utilize the different level features extracted by the main network to restore the lost detail features and spatial information in the down-sampling process, and introduces a bottleneck attention module to obtain more channel and spatial feature information of interest, effectively reduces the interference of the background and other factors, and is more helpful for power line feature extraction. BRIEF DESCRIPTION OF DRAWINGS

[0036] Various other advantages and benefits of the present application will become apparent to those of ordinary skill in the art, upon reading the following detailed description of the preferred embodiment. The accompanying drawings are included to provide a better understanding of the preferred embodiment and are not to be considered limitations of the present application. It should be readily understood that the drawings are merely included to illustrate the embodiments and that other drawings can be derived from these drawings without paid creative labor. Moreover, the same reference numerals are used throughout the drawings to represent the same components.

[0037] In the drawings:

[0038] Figure 1 is a flow chart of a power line semantic segmentation method based on an improved DeepLabv3+ model according to an embodiment of the present application;

[0039] Figure 2 is a DeepLabv3+ model structure diagram of a power line semantic segmentation method based on an improved DeepLabv3+ model according to an embodiment of the present application;

[0040] Figure 3 is a PP-LCNet structure diagram of a power line semantic segmentation method based on an improved DeepLabv3+ model according to an embodiment of the present application;

[0041] Figure 4 is a BC-ASPP module diagram of a power line semantic segmentation method based on an improved DeepLabv3+ model according to an embodiment of the present application;

[0042] Figure 5 is a bottleneck attention module structure diagram of a power line semantic segmentation method based on an improved DeepLabv3+ model according to an embodiment of the present application;

[0043] Figure 6 is a complete structure diagram of a power line semantic segmentation method based on an improved DeepLabv3+ model according to an embodiment of the present application;

[0044] Figure 7 is an effect diagram of a power line image segmentation by a power line semantic segmentation method based on an improved DeepLabv3+ model according to an embodiment of the present application.

[0045] The present application will be further explained in conjunction with the drawings and embodiments. DETAILED DESCRIPTION

[0046] Reference will now be made to the drawings to describe the preferred embodiment of the present application. Figures 1 to 7Specific embodiments of the present application are described in more detail. While the embodiments of the application are shown in the drawings, it is understood that the application can be practiced in a variety of forms other than those specifically shown and described herein. Rather, the embodiments are provided as examples so that one skilled in the art can understand the principles of the application and can best implement the application in practice.

[0047] It should be noted that certain terms are used throughout the present specification and claims which have particular meanings as set forth below. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an overly literal sense unless expressly so defined herein.

[0048] In order to make the present application more clearly understood, the following will further explain and describe specific embodiments of the present application with reference to the accompanying drawings, and each of the accompanying drawings does not constitute a limitation on the embodiments of the present application.

[0049] In order to better understand, Figures 1 to 6 As shown in the figure, the power line semantic segmentation method based on the improved DeepLabv3+ model includes the following steps:

[0050] Step 1: In order to solve the problem of slow prediction speed of the DeepLabv3+ model, the original backbone network Xception is replaced with a lightweight PP-LCNet in the encoder part, so as to improve the prediction speed and reduce the parameter quantity, and further improve the real-time performance of power line segmentation;

[0051] Step 2: In order to strengthen the feature extraction of slender power lines, the atrous spatial pyramid pooling (ASPP) module is improved, and a hollow convolution branch and a cascade convolution are added to obtain multi-scale features with a larger receptive field, so as to reduce the missed segmentation phenomenon. In addition, the hollow convolution branch is further adopted with a bottleneck structure of dimension reduction and dimension increase to reduce the calculation amount;

[0052] Step 3: In order to further improve the segmentation accuracy, 3 layers of shallow features are introduced in the decoder part, so as to make full use of the different levels of features extracted by the backbone network to restore the lost detail features and spatial information in the down-sampling process;

[0053] Step 4: To solve the problem of power line segmentation being easily disturbed by background objects, a bottleneck attention module (BAM) is introduced in the encoder and decoder to enhance the extraction of power line features and reduce false segmentation of background objects.

[0054] Step 5: Based on the power line dataset, 852 images are selected to expand to 5000 images as the dataset. 10% of the dataset is used as the test set, and the remaining is divided into the training set and the validation set according to the ratio of 9:1.

[0055] Step 6: The operating system used in the experiment is Windows 10, the processor is Intel(R) Xeon(R) Gold 6230, the graphics card is NVIDIA Tesla V100-PCIE-16GB, the RAM size is 320G, the network framework used in the experiment is pytorch, the python version is 3.7.11, and the initial learning rate is 5e -4 The average pixel accuracy and average intersection over union are used as the evaluation criteria for segmentation accuracy.

[0056] Step 7: The image size of the training input is 512x512 pixels, the Batchsize used in the training is 8, the number of training iterations is 100, and the combination of CE Loss and Dice Loss is used as the loss function for training. After each training, the validation set is used to verify the effect of the model to prevent overfitting.

[0057] Step 8: The test set part of the dataset is used to test the model, the average pixel accuracy and average intersection over union of the test model are tested, and the average pixel accuracy, average intersection over union, and actual segmentation effect of the power line image are compared with other models.

[0058] In one embodiment, the method comprises,

[0059] Step 1: For the encoder of the DeepLabv3+ model, replace the original backbone network Xception with PP-LCNet network;

[0060] Step 2: Add a dilated convolution branch and a cascade convolution to the empty space pyramid pooling module to obtain multi-scale features;

[0061] Step 3: Introduce 3 layers of shallow features in the decoder to restore the details and spatial information lost in the down-sampling process;

[0062] Step 4: Introduce a bottleneck attention module to reduce false segmentation of power lines.

[0063] Step 5, select power line images from the public power line dataset for dataset making, and divide the dataset into a training set, a validation set and a test set;

[0064] Step 6, configure the experimental environment, and set the training parameters according to the device performance and the characteristics of the power line;

[0065] Step 7, train and validate the model using the training set and the test set in the dataset;

[0066] Step 8, test the model using the test set in the dataset to view the segmentation effect.

[0067] In a preferred embodiment of the method, in step 1, the PP-LCNet network uses a depth separable convolution instead of a standard convolution, wherein the activation function ReLU in the depth separable convolution is replaced by H-Swish, the PP-LCNet network uses a 5x5 convolution instead of a 3x3 convolution at the tail of the network, and a compression and excitation network module is added in the last two depth separable convolution blocks to weight the network channels.

[0068] In a preferred embodiment of the method, in step 2, the bottleneck cascade dilated spatial pyramid pooling module with a combination of dilated rates of 6, 12 and 18 extracts multi-scale features by increasing the dilated convolution branch and cascading convolution.

[0069] In a preferred embodiment of the method, in step 2, in the structure of the bottleneck cascade dilated spatial pyramid pooling module, a dilated convolution branch is added, and the combination of dilated rates is modified to 3, 6, 9 and 12, then 2 times of 3x3 convolution cascading operation is performed on the middle 4 convolution branches to extract multi-scale features, when the dilated rate of the dilated convolution branch is r and the convolution kernel size is k, the receptive field size is:

[0070] R=(r-1)x(k-1)+k,

[0071] And when two dilated convolution branches are cascaded, the receptive field size is:

[0072] R=R1+R2-1,

[0073] Wherein R1 and R2 are the receptive fields provided by the two dilated convolution branches respectively, the middle 4 convolution branches are first reduced to 64 channels using 1x1 convolution, then processed by 2 times of 3x3 convolution, and then increased to 256 channels using 1x1 convolution.

[0074] In a preferred embodiment of the method, in step 3, 3 layers of shallow feature maps with downsampling coefficients of 1 / 4, 1 / 8 and 1 / 16 are introduced into the PP-LCNet in the decoder, and the channel numbers of the 3 layers of shallow feature maps are adjusted to 48, 32 and 16 respectively using 1x1 convolution.

[0075] In a preferred embodiment of the method, in step 4, the bottleneck attention module is composed of a parallel channel attention network and a spatial attention network, the input feature map F is processed by the two parallel networks to obtain a feature map M(F), then the feature map M(F) and the feature map F are multiplied point by point to highlight important features, and the obtained feature map is added to the feature map F to output an attention feature map F'.

[0076] M(F) = σ(M c (F) + M s (F)),

[0077] F' = F + F o M(F),

[0078] where σ is a sigmoid function, o is a point-by-point multiplication of corresponding elements of a matrix, M c (F) and M s (F) represent the output feature maps of the channel attention network and the spatial attention network, respectively.

[0079] In a preferred embodiment of the method, in step 4, in the channel attention network, first, the features in each channel are aggregated using global average pooling to generate a channel vector F c , then a multilayer perceptron is used to evaluate the attention network from F c , and a channel attention feature map is obtained through a fully connected layer.

[0080] In a preferred embodiment of the method, in step 4, in the spatial attention network, first, a 1x1 convolution is used to compress the feature map dimension, then 2 3x3 convolutions with a hole rate of 4 are used to obtain context information, and finally a 1x1 convolution is used to further compress the dimension to output a spatial attention network feature map.

[0081] In a preferred embodiment of the method, in step 5, 415 images containing power lines are selected from the public power line dataset TTPLA and WireDataset, and 437 images containing power lines are selected from the public power line dataset TTPLA and WireDataset, and these 852 images are expanded to 5000 images through rotation, left and right flipping, enlargement, reduction, cropping and changing brightness as a dataset, 10% of the dataset is used as a test set, and the rest is divided into a training set and a validation set in a ratio of 9:1.

[0082] In a preferred embodiment of the method, in step 6, the operating system used in the experiment is Windows 10, the processor is Intel(R) Xeon(R) Gold 6230, the graphics card is NVIDIA Tesla V100-PCIE-16GB, the RAM size is 320G, the network framework used in the experiment is pytorch, the python version is 3.7.11, and the initial learning rate is 5e. The average pixel accuracy and the average intersection over union are used as the evaluation criteria for the segmentation accuracy.

[0083] In a preferred embodiment of the method, in step 7, the image size of the training input is 512x512 pixels, the Batchsize used in the training is 8, the number of training iterations is 100, the CE Loss and the Dice Loss are combined as the loss function for training, and the validation set is used to verify the effect of the model after each training to prevent overfitting.

[0084] In a preferred embodiment of the method, in step 8, the test set part of the data set is used to test the model, and the average pixel accuracy and the average intersection over union of the model segmentation are 91.9% and 82.07%, respectively. The average pixel accuracy, the average intersection over union, and the actual segmentation effect of the power line image are compared with other models.

[0085] In one embodiment, the method comprises the following steps:

[0086] Step 1: Replace the original DeepLabv3+ backbone network Xception with a lightweight PP-LCNet to reduce the number of parameters and improve the prediction speed;

[0087] The PP-LCNet structure diagram is as shown in Figure 3 The core part of the PP-LCNet is to replace the standard convolution with a depth separable convolution without short connection operations, which effectively reduces the number of parameters and speeds up the running speed. In the PP-LCNet, the activation function ReLU in the depth separable convolution block is replaced with H-Swish, which further improves the performance.

[0088] Under the condition of balancing speed and accuracy, the PP-LCNet replaces the 3x3 convolution with a 5x5 convolution at the tail of the network, which effectively improves the feature extraction capability of the network. In the last two depth separable convolution blocks, a SENet module is added to weight the network channels and strengthen the feature extraction of important information, which improves the performance of the network without affecting the speed.

[0089] Step 2: Add a dilated convolution branch and cascade convolution in the ASPP module to obtain multi-scale features and reduce the missed segmentation phenomenon;

[0090] Dilated convolution: Dilated convolution is also called expansion convolution or inflation convolution. In simple terms, it is the process of adding some spaces (zeros) between the convolution kernel elements to expand the convolution kernel.

[0091] Cascade convolution: In this paper, cascade means replacing the original 1x3 convolution in the dilated spatial pyramid pooling with 2x3 convolution and 2x1x1 bottleneck structure in series.

[0092] The DeepLabv3+ model uses the ASPP module with dilated rates of 6, 12, and 18 to extract multi-scale features. The larger the dilated rate, the larger the receptive field, but it also brings problems such as insufficient detail feature extraction and poor small target extraction. To improve the ASPP module, a bottleneck cascade dilated spatial pyramid pooling (BC-ASPP) module is proposed, which strengthens the extraction of power line detail features by adding a dilated convolution branch and cascading dilated convolution, reducing the missed segmentation phenomenon.

[0093] The structure of the BC-ASPP module is shown in Figure 4 First, a dilated convolution branch is added, and the dilated rate combination is modified to 3, 6, 9, and 12, which is more conducive to small target and detail feature extraction. Then, 2x3 convolution is cascaded for the middle 4 convolution branches to extract multi-scale features with larger receptive fields, effectively reducing the missed segmentation problem. When the dilated rate of dilated convolution is r and the convolution kernel size is k, the receptive field size is:

[0094] R = (r-1)x(k-1)+k (1)

[0095] And when two dilated convolutions are cascaded, the receptive field size is:

[0096] R = R1+R2-1 (2)

[0097] Where R1 and R2 are the receptive fields provided by the two dilated convolutions, respectively. It can be seen that the receptive field increases nearly 1 times when two dilated convolutions are cascaded. Finally, the parameters of the BC-ASPP module are simplified. The middle 4 convolution branches are first reduced to 64 channels using 1x1 convolution, then processed by 2x3 convolution, and then upgraded to 256 channels using 1x1 convolution. This bottleneck structure effectively reduces the parameter amount.

[0098] Step 3: Fuse 3 layers of shallow features in the decoder to recover the details and spatial information lost in the downsampling process;

[0099] Detail feature: The detail feature of image mainly includes color feature, character feature, texture feature and spatial relationship feature.

[0100] Spatial information: The spatial position information can be generally divided into relative spatial position information and absolute spatial position information. The former emphasizes the relative situation between targets, such as up-down and left-right relationship, and the latter emphasizes the distance and direction between targets.

[0101] In the decoder part, partial shallow feature maps with downsampling coefficients of 1 / 4, 1 / 8 and 1 / 16 are introduced in the PP-LCNet. And the channel numbers of the three shallow feature maps are adjusted to 48, 32 and 16 respectively by using 1x1 convolution to prevent excessive shallow semantic information from affecting the expression of deep semantic information of the encoder output. Such multi-scale fusion process only increases 2 times of 1x1 convolution operation, which increases very little parameter amount compared with the feature fusion method of the original DeepLabv3+ network, but the segmentation accuracy is effectively improved.

[0102] Step 4: Introducing bottleneck attention module to reduce the phenomenon of misclassification of power lines;

[0103] The bottleneck attention module is a hybrid attention mechanism network composed of parallel channel attention network and spatial attention network, and its structural diagram is shown in Figure 5 . The input feature map F is processed by two parallel networks to obtain the feature map M(F). Then M(F) and F are multiplied point by point to highlight important features, and the obtained feature map is added to F to output the attention feature map F'. The expressions of the feature maps M(F) and F are as follows:

[0104] M(F) = σ(M c (F) + M s (F)) (3)

[0105] F' = F + F o M(F) (4)

[0106] Where σ is the sigmoid function, o is the point-by-point multiplication of corresponding elements of matrix, M C (F) and M s (F) represent the output feature maps of the channel attention network and the spatial attention network respectively.

[0107] In the channel attention network, the features in each channel are first aggregated by global average pooling to generate a channel vector F c . Then, a multilayer perceptron is used to evaluate F cThe channel attention network is obtained through a full connection layer. In the spatial attention network, the feature map is first compressed by a 1x1 convolution, and then 2 3x3 convolutions with a hole rate of 4 are used to obtain context information. Finally, a 1x1 convolution is used to further compress the dimension to output the spatial attention network feature map.

[0108] The bottleneck attention module is introduced into the encoder and the decoder respectively, so as to pay more attention to the power line features and reduce the false segmentation of the background objects.

[0109] Step 5: Select power line images from the public power line dataset for dataset making, and divide the dataset into training set, validation set and test set;

[0110] From the public power line datasets TTPLA and WireDataset, 415 and 437 images containing power lines are selected respectively, and the 852 images are expanded to 5000 images through rotation, left and right flipping, enlargement, reduction, cropping and brightness change as a dataset. 10% of the dataset is used as the test set, and the rest is divided into the training set and the validation set according to the ratio of 9:1.

[0111] Step 6: Configure the experimental environment and set the training parameters according to the device performance and the characteristics of the power line;

[0112] The operating system used in the experiment is Windows 10, the processor is Intel(R) Xeon(R) Gold 6230, the graphics card is NVIDIA Tesla V100-PCIE-16GB, the RAM size is 320G, the network framework used in the experiment is pytorch, the python version is 3.7.11, and the initial learning rate is 5e. The average pixel accuracy and the average intersection over union are used as the evaluation criteria for the segmentation accuracy.

[0113] Step 7: Use the training set and the test set in the dataset to train and verify the model;

[0114] The size of the training input image is 512x512 pixels, the Batchsize used in the training is 8, the number of iterations is 100, and the CE Loss and Dice Loss loss functions are combined as the training loss function. After each training, the validation set is used to verify the effect of the model to prevent overfitting.

[0115] Step 8: Use the test set in the dataset to test the model and test the segmentation effect.

[0116] The model is tested by using the test set part of the data set division, the average pixel accuracy and the average intersection over union of the model segmentation are 91.9% and 82.07% respectively, and the average pixel accuracy, the average intersection over union and the actual segmentation effect of the power line image are compared with other models.

[0117] Although the embodiments of the present application are described above with reference to the drawings, the present application is not limited to the above-described specific embodiments and application fields, and the above-described specific embodiments are merely illustrative and instructive, but not restrictive. A person of ordinary skill in the art can make many forms under the guidance of the present specification and without departing from the scope protected by the claims of the present application, which all belong to the protection of the present application.

Claims

1. A power line semantic segmentation method based on an improved DeepLabv3+ model, characterized in that, The method comprises the following steps: Step 1, for the encoder of the DeepLabv3+ model, replace the original backbone network Xception with the PP-LCNet network; Step 2, the hollow space pyramid pooling module increases the hollow convolution branch and the cascade convolution to obtain multi-scale features; Step 3, introduce 3 layers of shallow features in the decoder to restore the details and spatial information lost in the down-sampling process; Step 4: introduce a bottleneck attention module to reduce the misclassification of power lines; Step 5, based on the power line dataset, select power line images to make a dataset, and divide the dataset into a training set, a validation set and a test set; Step 6, configure the experimental environment, and set the training parameters according to the device performance and the characteristics of the power line; Step 7, use the training set and the test set in the dataset to train and verify the model; Step 8, use the test set in the dataset to test the model and view the segmentation effect; In step 1, the PP-LCNet network uses depth separable convolution instead of standard convolution, where the activation function ReLU in the depth separable convolution is replaced by H-Swish, and the PP-LCNet network uses 5×5 convolution instead of 3×3 convolution at the tail of the network, and adds a compression and excitation network module to the last two depth separable convolution blocks to weight the network channels; In step 2, the bottleneck cascade hollow space pyramid pooling module with a combination of hollow rates of 6, 12 and 18 extracts multi-scale features by increasing the hollow convolution branch and the cascade convolution; In the structure of the bottleneck cascade hollow space pyramid pooling module in step 2, a hollow convolution branch is added, and the hollow rate combination is modified to 3, 6, 9, 12, then 2 times of 3x3 convolution cascade operation is performed on the middle 4 convolution branches to extract multi-scale features, when the hollow rate of the hollow convolution branch is , and the size of the convolution kernel is , the size of the receptive field is: ， And when the two-layer hollow convolution branch is cascaded, the receptive field size is: ， wherein 、 The receptive field provided for the two-layer hollow convolution is 2, the middle four convolution branches are first reduced to 64 channels by 1*1 convolution, then processed by 3*3 convolution twice, and then increased to 256 channels by 1*1 convolution. In step 4, the bottleneck attention module is composed of a parallel channel attention network and a spatial attention network, and the input feature map is processed by two parallel networks respectively to obtain a feature map Then the feature map is multiplied by the feature map by point-by-point multiplication to highlight important features, and the obtained feature map is added to the feature map to output an attention feature map The expressions of the feature map and the feature map are as follows respectively: ， ， wherein is the input feature map, the feature map is the input feature map is respectively processed by two parallel networks to obtain the feature map, is the sigmoid function, and respectively represent the channel attention network and the spatial attention network output feature map, is the point-by-point multiplication of the corresponding elements of the matrix, is the and the feature map obtained by point-by-point multiplication is added to the output attention feature map.

2. The method of claim 1, wherein, In step 3, 3 layers of shallow feature maps with down-sampling coefficients of 1 / 4, 1 / 8 and 1 / 16 in the PP-LCNet are introduced into the decoder, and the channel numbers of the 3 layers of shallow feature maps are adjusted to 48, 32 and 16 respectively by using 1×1 convolution.

3. The method of claim 1, wherein, In step 4, in the channel attention network, first, the features in each channel are aggregated by global average pooling to generate a channel vector F c Then, the channel attention feature map is obtained through a fully connected layer by evaluating the attention network from F c ​ 4. The method of claim 1, wherein, In step 4, in the spatial attention network, first, use 1×1 convolution to compress the feature map dimension, then use 2 3×3 convolutions with a hollow rate of 4 to obtain context information, and finally use 1×1 convolution to further compress the dimension and output the spatial attention network feature map.

5. The method as claimed in claim 1, wherein, In step 5, based on the power line datasets TTPLA and WireDataset, 415 and 437 images containing power lines are selected respectively, and these 852 images are expanded to 5000 images by rotating, flipping left and right, enlarging, reducing, cropping and changing brightness as a dataset. 10% of the dataset is used as the test set, and the rest is divided into the training set and the validation set according to the ratio of 9:

1.

6. The method as claimed in claim 1, wherein, In step 6, the network framework used in the experiment is pytorch, and the initial learning rate is 5e -4 In the experiment, the mean pixel accuracy MPA and the mean intersection over union MIoU are used as the evaluation criteria for segmentation accuracy.

7. The method as claimed in claim 1, wherein, In step 7, the image size of the training input is 512×512 pixels, the Batchsize used in the training is 8, the number of iterations is 100, and the CE Loss and Dice Loss loss functions are combined as the training loss function. After each training, the validation set is used to verify the effect of the model to prevent overfitting.

8. The method of claim 1, wherein, In step 8, the model is tested by using the test set part of the data set division, the average pixel accuracy MPA and the average intersection over union MIoU of the model segmentation are tested, and the average pixel accuracy MPA, the average intersection over union MIoU and the actual segmentation effect of the power line image are compared with other test models.

Images