Aerial photography power transmission line segmentation method based on attention mechanism and feature aggregation

Abstract

The invention discloses an aerial photography power transmission line segmentation method based on an attention mechanism and feature aggregation, and the method comprises the steps: S1, constructing a power transmission line segmentation data set, carrying out the data enhancement operation, and dividing a training set, a verification set and a test set; s2, images in the data set are input into a neural network model for training, network convergence is constrained by using a weighted mixed loss function, and the S2 comprises the following steps: S21, a fusion attention module is added to a jump connection part of an encoder and a decoder; s22, designing a feature aggregation module, and adding the feature aggregation module behind the fifth layer of the encoder; and S23, adding the above modules into a U-Net network, constructing a neural network model and training the network model, and using a weighted mixed loss function to guide the network to train. By adding the fusion attention module, the perception capability of the network for global context information is enhanced, and meanwhile, the feature aggregation module is provided to obtain higher-level semantic features, so that the accuracy of power transmission line segmentation is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of power transmission line inspection, and relates to an aerial transmission line segmentation method based on attention mechanism and feature aggregation. Background Art

[0002] Power transmission has a vital impact on all walks of life in modern society. Ensuring the safety of power transmission lines is an important basis for ensuring the development of the national economy and improving people's living standards. Regular inspections of transmission lines can timely discover faults in the lines and hidden dangers that endanger line safety, which helps the power department to carry out targeted maintenance and maintenance of transmission lines, thereby ensuring the safety of power transmission lines and extending their service life. In the past, power detection mainly relied on traditional manual inspections, and manual inspections of high-voltage transmission lines were very dangerous and time-consuming. With the advancement of technology, more and more power line inspections are completed by unmanned aerial vehicles (UAVs). In the process of UAV inspections, semantic segmentation of transmission lines is a critical task.

[0003] The existing aerial transmission line segmentation methods have the following two problems:

[0004] 1. The proportion of pixels occupied by power transmission lines in the image is small. Most of the pixels in a picture containing power transmission lines are background pixels that are not power lines. In addition, the pixel width of a single power line is extremely small, making it difficult to detect.

[0005] 2. Serious background interference. Other linear objects that are almost the same shape and color as the power lines, such as ropes, fences, branches of trees, or just the edge of a building wall in the image, are easily misdetected as power lines.

[0006] Therefore, in view of the defects of the prior art, it is necessary to propose a technical solution to solve the technical problems existing in the prior art. Summary of the invention

[0007] To solve the above problems, the technical solution of the present invention is an aerial transmission line segmentation method based on attention mechanism and feature aggregation, comprising the following steps:

[0008] S1, construct a transmission line segmentation dataset, perform data enhancement operations, and divide it into training set, validation set, and test set;

[0009] S2, input the images in the dataset into the neural network model for training, and use the weighted mixed loss function to constrain the network convergence.

[0010] Preferably, S2 comprises the following steps:

[0011] S21, add a fusion attention module to the jump connection between the encoder and decoder;

[0012] S22, design feature aggregation module, added after the fifth layer of encoder;

[0013] S23, add the above modules into the U-Net network, build a neural network model and train the network model, and use a weighted mixed loss function to guide the network training.

[0014] Preferably, the S21 comprises the following steps:

[0015] S211, input the input feature map into the channel attention module;

[0016] S212, multiplying the original input feature map by the obtained channel attention map weight to obtain an enhanced feature map;

[0017] S213, input the feature map obtained in S212 into the spatial attention module;

[0018] S214, multiply the feature map obtained in S212 by the weight of the spatial attention map to obtain the final output feature map.

[0019] Preferably, the S211 specifically includes the following steps:

[0020] Perform global average pooling and global maximum pooling on the input feature map for height and width respectively;

[0021] They are fed into a shared two-layer neural network consisting of two fully connected layers and a Relu activation function;

[0022] Add the two output features;

[0023] The final channel attention map is obtained after Sigmoid activation.

[0024] Preferably, the S213 specifically includes the following steps:

[0025] Perform global average pooling and global maximum pooling on the feature maps obtained by S212, and concatenate the two feature maps in the channel dimension;

[0026] The images are added after 3*3 convolution, 7*7 convolution and 3*3 convolution with a dilation rate of 4 respectively;

[0027] After the Sigmoid activation operation, the spatial attention map is obtained.

[0028] Preferably, the S22 comprises the following steps:

[0029] S221, the input features are respectively passed through four dilated convolution branches with different dilation rates to obtain different receptive fields, where the dilation rates are set to 1, 2, 4, and 8;

[0030] S222, concatenating the obtained four features with the input features to obtain a multi-scale feature map;

[0031] S223, use 1×1 convolution to compress the channel to be consistent with the channel of the input feature.

[0032] Preferably, the S23 further comprises:

[0033] S231, build a network model based on the proposed module, add a fusion attention module to the jump connection part between the encoder and the decoder; add a feature aggregation module after the fifth layer of the encoder;

[0034] S232, use the weighted mixed loss function to guide the network training, including Focal loss, Dice loss and Mcc loss, calculated as follows:

[0035] Loss PL =0.8*Loss Focal -0.2*ln(1-Loss Dice )+0.1Loss Mcc

[0036] Among them, the refined loss function is as follows

[0037]

[0038] Among them, α is the weight factor, γ is the adjustment factor, x represents the true value of the label category, and p represents the model's estimated probability of the class with label x=1;

[0039]

[0040] Among them, y i With q i They represent the label value and predicted value of the i-th pixel respectively, and N is the total number of pixels;

[0041]

[0042] The calculation formula of MCC is as follows:

[0043]

[0044] Among them, TP represents the number of pixels predicted to be transmission lines correctly; FP represents the number of pixels predicted to be transmission lines but are actually background pixels; FN represents the number of pixels predicted to be background pixels but are actually transmission lines; TN represents the number of pixels predicted to be background pixels correctly.

[0045] The present invention has at least the following beneficial effects: the present invention embeds a fusion attention module in the jump connection part of the encoder and decoder to guide the model to obtain global context information so that it can better emphasize the area of ​​the power transmission line; a feature aggregation block is proposed to generate richer semantic features to further improve the segmentation accuracy of the power transmission line. The technical solution of the present invention is trained and tested based on existing public data sets, and the test results are significantly better than the existing algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A structural diagram of a neural network model in an aerial transmission line segmentation method based on attention mechanism and feature aggregation provided by the present invention;

[0047] Figure 2 A structural diagram of the fusion attention module in the aerial transmission line segmentation method based on attention mechanism and feature aggregation provided by the present invention;

[0048] Figure 3 A structural diagram of a feature aggregation module in an aerial transmission line segmentation method based on an attention mechanism and feature aggregation provided by the present invention;

[0049] Figure 4 A schematic diagram showing the comparison of effects of an aerial transmission line segmentation method based on attention mechanism and feature aggregation provided by the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0051] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present invention without the description of these details.

[0052] The present invention comprises the following steps: S1, constructing a transmission line segmentation data set, performing a data enhancement operation, and dividing the data into a training set, a validation set, and a test set;

[0053] S2, input the images in the dataset into the neural network model for training, and use the weighted mixed loss function to constrain the network convergence.

[0054] See also Figure 1 , shown is a neural network model structure diagram in a method for aerial transmission line segmentation based on attention mechanism and feature aggregation.

[0055] S2 includes the following steps:

[0056] S21, add a fusion attention module to the jump connection between the encoder and decoder;

[0057] S22, design feature aggregation module, added after the fifth layer of encoder;

[0058] S23, add the above modules into the U-Net network, build a neural network model and train the network model, and use a weighted mixed loss function to guide the network training.

[0059] S21 includes the following steps:

[0060] S211, input the input feature map into the channel attention module;

[0061] S212, multiplying the original input feature map by the obtained channel attention map weight to obtain an enhanced feature map;

[0062] S213, input the feature map obtained in S212 into the spatial attention module;

[0063] S214, multiply the feature map obtained in S212 by the weight of the spatial attention map to obtain the final output feature map.

[0064] S211 specifically includes the following steps:

[0065] Perform global average pooling and global maximum pooling on the input feature map for height and width respectively;

[0066] They are fed into a shared two-layer neural network consisting of two fully connected layers and a Relu activation function;

[0067] Add the two output features;

[0068] The final channel attention map is obtained after Sigmoid activation.

[0069] S213 specifically includes the following steps:

[0070] Perform global average pooling and global maximum pooling on the feature maps obtained by S212, and concatenate the two feature maps in the channel dimension;

[0071] The images are added after 3*3 convolution, 7*7 convolution and 3*3 convolution with a dilation rate of 4 respectively;

[0072] After the Sigmoid activation operation, the spatial attention map is obtained.

[0073] S22 includes the following steps:

[0074] S221, the input features are respectively passed through four dilated convolution branches with different dilation rates to obtain different receptive fields, where the dilation rates are set to 1, 2, 4, and 8;

[0075] S222, concatenating the obtained four features with the input features to obtain a multi-scale feature map;

[0076] S223, use 1×1 convolution to compress the channel to be consistent with the channel of the input feature.

[0077] S23 further comprises:

[0078] S231, build a network model based on the proposed module, add a fusion attention module to the jump connection part between the encoder and the decoder; add a feature aggregation module after the fifth layer of the encoder;

[0079] S232, use the weighted mixed loss function to guide the network training, including the focal loss function, the dice loss function, and the loss function Mcc loss based on the Matthews correlation coefficient, calculated as follows:

[0080] Loss PL =0.8*Loss Focal -0.2*ln(1-Loss Dice )+0.1Loss Mcc

[0081] Among them, the refined loss function is as follows

[0082]

[0083] Among them, α is the weight factor, γ is the adjustment factor, x represents the true value of the label category, and p represents the model's estimated probability of the class with label x=1;

[0084]

[0085] Among them, y i With q i They represent the label value and predicted value of the i-th pixel respectively, and N is the total number of pixels;

[0086]

[0087] The calculation formula of MCC is as follows:

[0088]

[0089] Among them, TP represents the number of pixels predicted to be transmission lines correctly; FP represents the number of pixels predicted to be transmission lines but are actually background pixels; FN represents the number of pixels predicted to be background pixels but are actually transmission lines; TN represents the number of pixels predicted to be background pixels correctly.

[0090] Figure 2 A structural diagram of the fused attention module in the aerial transmission line segmentation method based on attention mechanism and feature aggregation provided by the present invention; wherein Input Feature is the input feature map, Channel Attention Module is the channel attention module, Spatial Attention Module is the spatial attention module, and Refined Feature is the output feature map.

[0091] Figure 3 A structural diagram of a feature aggregation module in an aerial transmission line segmentation method based on an attention mechanism and feature aggregation provided by the present invention; wherein Input is an input feature, Conv3*3Rate=1 indicates a dilated convolution with a convolution kernel size of 3*3 and a dilation rate of 1, Conv3*3Rate=2 indicates a dilated convolution with a convolution kernel size of 3*3 and a dilation rate of 2, Conv3*3Rate=4 indicates a dilated convolution with a convolution kernel size of 3*3 and a dilation rate of 4, Conv3*3Rate=8 indicates a dilated convolution with a convolution kernel size of 3*3 and a dilation rate of 8, Concat indicates concatenation of five features in the channel dimension, 1*1Conv indicates a convolution kernel size of 1*1, and Output indicates an output feature.

[0092] Figure 4 A schematic diagram showing the effect comparison of an aerial transmission line segmentation method based on attention mechanism and feature aggregation provided by the present invention. In which, Input image is the input image, Ground truth is the true value map, PSPNet, DeepLabv3+, SegFormer, TransUNet and UNet are the compared network models, and AMFA-Net is the network model proposed by the present invention.

[0093] In order to verify the technical effect of the present invention, the effectiveness of the present invention will be verified by algorithm comparison:

[0094] The dataset used in this method is the TTPLA dataset, where 80% of the data is used as the training set, 10% of the data is used as the validation set, and 10% of the data is used as the test set. The evaluation indicators are: intersection over union (IoU), precision, recall, F1-score, and accuracy.

[0095] Table 1 is a comparison of the indicators of some related segmentation networks and the present invention on the TTPLA dataset. Table 1 Comparison of the indicators of the present invention algorithm and other algorithms on the TTPLA dataset

[0096]

[0097] It can be seen from the results in the table that the model effect of the present invention has achieved excellent performance in various indicators and can effectively improve the segmentation accuracy of the transmission line.

[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for segmenting aerial transmission lines based on attention mechanism and feature aggregation, characterized in that: The following steps are involved: S1, construct a transmission line segmentation dataset, perform data enhancement operations, and divide it into training set, validation set, and test set; S2, input the images in the dataset into the neural network model for training, and use the weighted mixed loss function to constrain the network convergence.

2. The method according to claim 1, characterized in that The S2 comprises the following steps: S21, add a fusion attention module to the jump connection between the encoder and decoder; S22, design feature aggregation module, added after the fifth layer of encoder; S23, add the above modules into the U-Net network, build a neural network model and train the network model, and use a weighted mixed loss function to guide the network training.

3. The method according to claim 2, characterized in that The S21 comprises the following steps: S211, input the input feature map into the channel attention module; S212, multiplying the original input feature map by the obtained channel attention map weight to obtain an enhanced feature map; S213, input the feature map obtained in S212 into the spatial attention module; S214, multiply the feature map obtained in S212 by the weight of the spatial attention map to obtain the final output feature map.

4. The method according to claim 3, characterized in that: The S211 specifically includes the following steps: Perform global average pooling and global maximum pooling on the input feature map for height and width respectively; They are fed into a shared two-layer neural network consisting of two fully connected layers and a Relu activation function; Add the two output features; The final channel attention map is obtained after Sigmoid activation.

5. The method according to claim 3, characterized in that: The S213 specifically includes the following steps: Perform global average pooling and global maximum pooling on the feature maps obtained by S212, and concatenate the two feature maps in the channel dimension; The images are added after 3*3 convolution, 7*7 convolution and 3*3 convolution with a dilation rate of 4 respectively; After the Sigmoid activation operation, the spatial attention map is obtained.

6. The method according to claim 2, characterized in that The S22 comprises the following steps: S221, the input features are respectively passed through four dilated convolution branches with different dilation rates to obtain different receptive fields, where the dilation rates are set to 1, 2, 4, and 8; S222, concatenating the obtained four features with the input features to obtain a multi-scale feature map; S223, use 1×1 convolution to compress the channel to be consistent with the channel of the input feature.

7. The method according to claim 2, characterized in that: The S23 further comprises: S231, build a network model based on the proposed module, add a fusion attention module to the jump connection part between the encoder and the decoder; add a feature aggregation module after the fifth layer of the encoder; S232, using a weighted mixed loss function to guide the network training, including Focalloss, Diceloss and Mccloss, calculated as follows: Loss PL =0.8*Loss Focal -0.2*ln(1-Loss Dice )+0.1Loss Mcc Among them, the refined loss function is as follows Among them, α is the weight factor, γ is the adjustment factor, x represents the true value of the label category, and p represents the model's estimated probability of the class with label x=1; Among them, y i With q i They represent the label value and predicted value of the i-th pixel respectively, and N is the total number of pixels; The calculation formula of MCC is as follows: Among them, TP represents the number of pixels predicted to be transmission lines correctly; FP represents the number of pixels predicted to be transmission lines but are actually background pixels; FN represents the number of pixels predicted to be background pixels but are actually transmission lines; TN represents the number of pixels predicted to be background pixels correctly.

Images