Semantic segmentation network model of improved DeepLab V3 +
By using an improved DeepLab V3+ semantic segmentation network model, combined with lightweight MobileNet V2 and multi-scale feature fusion, the problem of low accuracy in power line detection in UAV aerial images is solved, achieving more efficient power line recognition and localization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-10
- Publication Date
- 2026-03-10
AI Technical Summary
Power line detection in drone aerial images suffers from low recognition accuracy and is difficult to apply to real-world scenarios, especially in complex backgrounds where it is difficult to effectively identify power line targets of different categories, with significant morphological differences and large scale variations.
An improved DeepLab V3+ semantic segmentation network model is adopted, combined with a lightweight MobileNet V2 network, a hollow pyramid pooling module, a channel attention mechanism, and multi-scale feature fusion to improve the accuracy of power line detection.
It improves the accuracy of power line detection, reduces the number of parameters in the network model, expands the receptive field, extracts rich spatial location information from the image, obtains more detailed information, and improves detection accuracy.
Smart Images
Figure CN121640066A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of power line detection technology in aerial images, specifically to an improved semantic segmentation network model of DeepLab V3+. Background Technology
[0002] Power lines, as the most crucial component of the power grid, are responsible for the transmission and distribution of electrical energy, and their importance to the safety and stability of power supply is self-evident. Due to the long spans of transmission lines and the long-term exposure of electrical equipment outdoors, aging, damage, and corrosion are inevitable, posing significant hidden dangers to the safe and stable operation of the power grid. Drones inspecting transmission lines offer advantages such as low cost and high efficiency, great flexibility in space and time, and safer and more convenient operation in complex environments. Major power grids utilize drones for transmission line inspections, saving significant amounts of money and time and helping to avoid dangerous operations for inspection personnel. However, during low-altitude drone flight, power lines are among the most threatening hazards and the most difficult obstacles to avoid. Detecting power lines in drone aerial images is crucial for the drone's automatic obstacle avoidance for safe flight and for ensuring the safe and stable operation of the power grid.
[0003] Meanwhile, drone inspections of power transmission lines generate massive amounts of images. Detecting power lines and line faults from these images is a tedious task, and traditional manual inspection is not only time-consuming and labor-intensive. Traditional image processing methods for power line detection are easily affected by complex backgrounds. For the detection of targets with different categories, significant morphological differences, and large scale variations, they are often implemented based on specific features, resulting in low recognition accuracy, lack of scalability, and difficulty in applying them to real-world scenarios. Summary of the Invention
[0004] (a) Technical problems to be solved This invention addresses the problems existing in the detection of power lines in UAV aerial images by providing an improved semantic segmentation network model of DeepLab V3+. The model uses a deep learning-based semantic segmentation network to automatically identify and locate power lines in massive amounts of UAV aerial images, thereby improving the accuracy of power line identification in UAV aerial images.
[0005] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: An improved semantic segmentation network model for DeepLab V3+ includes an encoder and a decoder. The encoder includes a feature extraction network, a dilated pyramid pooling module, and a channel attention mechanism module. The dilated pyramid pooling module and the channel attention mechanism module are respectively connected to the last feature layer of the feature extraction network. The output feature maps of the dilated pyramid pooling module and the channel attention mechanism module are fused to form a deep feature map. The deep feature map is then upsampled and fused with the shallow feature map at the decoder to form an output feature map, which is used for segmentation prediction of the target to be detected.
[0006] Furthermore, the feature extraction network is a lightweight MobileNet V2 network, and the five feature layers of the MobileNet V2 feature extraction network output feature maps of 256*256*64, 128*128*128, 64*64*256, 32*32*512, and 32*32*1024.
[0007] Furthermore, the input image undergoes deep feature fusion via the MobileNet V2 feature extraction network, the dilated pyramid pooling module, and the channel attention mechanism module; the dilated pyramid pooling module and the channel attention mechanism module are integrated with MobileNet... The last feature layer of the V2 feature extraction network is 32*32*1024 connected together. The dilated pyramid pooling module uses 1*1 convolution, 3*3 dilated convolution with dilation rate of 6, 3*3 dilated convolution with dilation rate of 12, 3*3 dilated convolution with dilation rate of 18, and global average pooling layers connected in parallel. The 3*3 dilated convolution with dilation rate of 6, 3*3 dilated convolution with dilation rate of 12, and 3*3 dilated convolution with dilation rate of 18 are connected in a dense connection manner. After fusing the feature maps of each branch, the dilated pyramid pooling module adjusts the number of channels through 1*1 convolution to obtain a 32*32*512 feature map F1. The feature map output by the attention mechanism module is adjusted through 1*1 convolution to obtain a 32*32*512 feature map F2. The 32*32*512 feature map F1 and the 32*32*512 feature map F2 are fused together to obtain a 32*32*512 deep feature map F3.
[0008] Furthermore, one dense connection method of the dilated pyramid pooling module is a parallel connection of 1*1 convolution, 3*3 dilated convolution with an inflation rate of 6, 3*3 dilated convolution with an inflation rate of 12, 3*3 dilated convolution with an inflation rate of 18, and a global average pooling layer; the output feature map of the 3*3 dilated convolution with an inflation rate of 6 is fused with the input feature map of the dilated pyramid pooling module as the input feature map of the 3*3 dilated convolution with an inflation rate of 12, and the output feature map of the 3*3 dilated convolution with an inflation rate of 12 is fused with the input feature map of the dilated pyramid pooling module as the input feature map of the 3*3 dilated convolution with an inflation rate of 18.
[0009] Furthermore, another dense connection method of the dilated pyramid pooling module is a parallel connection of 1*1 convolution, 3*3 dilated convolution with an inflation rate of 6, 3*3 dilated convolution with an inflation rate of 12, 3*3 dilated convolution with an inflation rate of 18, and a global average pooling layer; the output feature map of the 3*3 dilated convolution with an inflation rate of 6 is fused with the input feature map of the dilated pyramid pooling module as the input feature map of the 3*3 dilated convolution with an inflation rate of 12, and the output feature map of the 3*3 dilated convolution with an inflation rate of 12 is fused with the input feature map of the dilated pyramid pooling module and the output feature map of the 3*3 dilated convolution with an inflation rate of 6 as the input feature map of the 3*3 dilated convolution with an inflation rate of 18.
[0010] Furthermore, the channel attention mechanism module first performs global average pooling and global max pooling on the input feature map 32*32*1024, and uses the resulting feature map as input to the fully connected layer with shared parameters, generating a global average pooling feature map and a global max pooling feature map with dimensions of 1*1*1024 respectively; then, the two fully connected layers compress and restore the number of channels respectively, and then add the two feature maps together and normalize them using the sigmoid function to obtain the channel attention weight vector; finally, the generated weight vector is multiplied by the input feature map, and different weights are assigned to the feature maps of each channel, outputting a 32*32*1024 feature map.
[0011] Furthermore, the deep feature map F3 of the encoder part is upsampled by 4 times to obtain a 128*128*512 feature map. The 128*128*512 feature map is fused with the shallow feature map F4 to obtain a 128*128*512 feature map F5. The shallow feature map is obtained by multi-scale feature fusion. The 128*128*512 feature map is then convolved by 3*3 and upsampled by 4 times to restore the image to a size of 512*512.
[0012] Furthermore, the multi-scale feature fusion is performed by fusing the 256*256, 128*128, and 64*64 feature maps output by the MobileNet V2 feature extraction network to output a 128*128*512 feature map.
[0013] Furthermore, the multi-scale feature fusion includes a spatial attention mechanism module. The 256*256*64 feature map output by the MobileNet V2 feature extraction network is first downsampled by a factor of 2, and then output as a 128*128*128 feature map through the first spatial attention mechanism module. The 128*128*128 feature map output by the MobileNet V2 feature extraction network is then output as a 128*128*128 feature map through the second spatial attention mechanism module. The 64*64*256 feature map output by the MobileNet V2 feature extraction network is first upsampled by a factor of 2, and then output as a 128*128*256 feature map through the third spatial attention mechanism module. The three 128*128 feature maps obtained above are fused to obtain a 128*128*512 feature map F4.
[0014] (III) Beneficial Effects The beneficial effects of this invention are as follows: An improved semantic segmentation network model for DeepLab V3+ uses the lightweight MobileNet V2 network as the feature extraction network, effectively reducing the number of network model parameters; the cross-layer dense connection method improves spatial pyramid pooling, further expanding the receptive field and obtaining a denser feature map output; the channel attention mechanism is fused with spatial pyramid pooling features to extract rich image spatial location information and more refined deep features; the combination of multi-scale feature fusion and spatial attention mechanism allows for full utilization of the spatial location information in the shallow features of the feature extraction network, obtaining more detailed information from the original image in the segmented image. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 This is a structural diagram of the semantic segmentation network model of the present invention; Figure 2 A structural diagram of the improved ASPP module of this invention; Figure 3 This is another structural diagram of the improved ASPP module of the present invention; Figure 4 This is a structural diagram of the channel attention mechanism module of the present invention; Figure 5 This is a diagram of the shallow feature fusion structure of the present invention. Detailed Implementation
[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0018] Combination Figure 1 An improved semantic segmentation network model for DeepLab V3+ is disclosed, comprising an encoder and a decoder. The encoder includes a feature extraction network, an Atrous Spatial Pyramid Pooling (ASPP) module, and a channel attention mechanism module. The Atrous Spatial Pyramid Pooling module and the channel attention mechanism module are respectively connected to the last feature layer of the feature extraction network. The output feature maps of the Atrous Spatial Pyramid Pooling module and the channel attention mechanism module are fused to form a deep feature map. The deep feature map is then upsampled and fused with the shallow feature map at the decoder to form an output feature map, which is used for segmentation prediction of the target to be detected.
[0019] The semantic segmentation network model is a lightweight model, and its feature extraction network is a lightweight MobileNet V2 network. The five feature layers of the MobileNet V2 feature extraction network output feature maps of 256*256*64, 128*128*128, 64*64*256, 32*32*512, and 32*32*1024, respectively.
[0020] The semantic segmentation network model consists of an encoder and a decoder. The encoder includes a MobileNetV2 feature extraction network, a dilated pyramid pooling module, a channel attention mechanism module, and deep feature fusion. The dilated pyramid pooling module and the channel attention mechanism module are connected to the last feature layer (32*32*1024) of the MobileNetV2 feature extraction network. The hollow pyramid pooling module employs 1*1 convolutions, 3*3 hollow convolutions with dilation rates of 6, 12, and 18, connected in parallel with a global average pooling layer. The 3*3 hollow convolutions with dilation rates of 6, 12, and 18 are densely connected. After fusing the feature maps from each branch, the hollow pyramid pooling module adjusts the number of channels using 1*1 convolutions to obtain a 32*32*512 feature map F1. The attention mechanism module outputs a feature map that is then adjusted for channel counts using 1*1 convolutions to obtain a 32*32*512 feature map F2. The 32*32*512 feature map F1 and F2 are then fused to obtain a 32*32*512 deep feature map F3.
[0021] The semantic segmentation model using MobileNet V2 as the feature extraction network is 23.5MB in size, while the original DeepLab V3+ model using Xception as the feature extraction network is 215MB in size. The lightweight semantic segmentation network model is only 1 / 9 the size of the original DeepLab V3+ model.
[0022] Combination Figure 2 One dense connection method of the dilated pyramid pooling module is a parallel connection of 1*1 convolution, 3*3 dilated convolution with an inflation rate of 6, 3*3 dilated convolution with an inflation rate of 12, 3*3 dilated convolution with an inflation rate of 18, and a global average pooling layer. The output feature map of the 3*3 dilated convolution with an inflation rate of 6 is fused with the input feature map of the dilated pyramid pooling module as the input feature map of the 3*3 dilated convolution with an inflation rate of 12. The output feature map of the 3*3 dilated convolution with an inflation rate of 12 is fused with the input feature map of the dilated pyramid pooling module as the input feature map of the 3*3 dilated convolution with an inflation rate of 18.
[0023] Combination Figure 3Another dense connection method of the dilated pyramid pooling module is a parallel connection of 1*1 convolution, 3*3 dilated convolution with an inflation rate of 6, 3*3 dilated convolution with an inflation rate of 12, 3*3 dilated convolution with an inflation rate of 18, and a global average pooling layer. The output feature map of the 3*3 dilated convolution with an inflation rate of 6 is fused with the input feature map of the dilated pyramid pooling module as the input feature map of the 3*3 dilated convolution with an inflation rate of 12. The output feature map of the 3*3 dilated convolution with an inflation rate of 12 is fused with the input feature map of the dilated pyramid pooling module and the output feature map of the 3*3 dilated convolution with an inflation rate of 6 as the input feature map of the 3*3 dilated convolution with an inflation rate of 18.
[0024] By improving the ASPP module through dense connections, dilated convolutions with different dilation rates become interdependent, enabling the output of each layer to be connected to the input feature map of the next layer. This results in a larger receptive field and a denser feature map output.
[0025] Combination Figure 4 The channel attention mechanism module first performs global average pooling and global max pooling on the input feature map 32*32*1024, and uses the resulting feature map as input to the fully connected layer with shared parameters, generating a global average pooling feature map and a global max pooling feature map with dimensions of 1*1*1024 respectively; then, the two fully connected layers compress and restore the number of channels respectively, and then the two feature maps are added together, and the sigmoid function is used to normalize them to 0~1 to obtain the channel attention weight vector; finally, the generated weight vector is multiplied with the input feature map, and different weights are assigned to the feature map of each channel, outputting a 32*32*1024 feature map.
[0026] The deep feature map F3 of the decoder is upsampled by 4 times to obtain a 128*128*512 feature map. The 128*128*512 feature map is fused with the shallow feature map F4 to obtain a 128*128*512 feature map F5. The shallow feature map is obtained by multi-scale feature fusion. The multi-scale feature fusion is to fuse the 256*256, 128*128, and 64*64 feature maps output by the MobileNet V2 feature extraction network to output a 128*128*512 feature map. The 128*128*512 feature map is then subjected to a 3*3 convolution and upsampled by 4 times to restore the image to a size of 512*512.
[0027] Combination Figure 5The multi-scale feature fusion includes a spatial attention mechanism module. The 256*256*64 feature map output by the MobileNet V2 feature extraction network is first downsampled by a factor of 2, and then output as a 128*128*128 feature map through the first spatial attention mechanism module. The 128*128*128 feature map output by the MobileNet V2 feature extraction network is then output as a 128*128*128 feature map through the second spatial attention mechanism module. The 64*64*256 feature map output by the MobileNet V2 feature extraction network is first upsampled by a factor of 2, and then output as a 128*128*256 feature map through the third spatial attention mechanism module. The three 128*128 feature maps obtained above are fused to obtain a 128*128*512 feature map F4.
[0028] To verify the effectiveness of the semantic segmentation network model in power line detection, a power line dataset consisting of 1500 power line images was constructed. Training and testing were conducted on a Windows 10 system using the PyTorch 1.8.0 deep learning framework and an NVIDIA GeForce GTX 3080 GPU. The semantic segmentation network model achieved a mean IoU (mean intersection and over union) of 92.62% for power line feature extraction, a 2.63 percentage point improvement over the original DeepLab V3+ network model. The network model achieved a mean pixel accuracy (mPA) of 96.19% for power line feature extraction, a 2 percentage point improvement over the original DeepLab V3+ network model. The network model achieved a power line feature extraction accuracy of 93%, a 3 percentage point improvement over the original DeepLab V3+ network model.
[0029] In summary, this invention provides an improved semantic segmentation network model for DeepLab V3+. It uses the lightweight MobileNet V2 network as the feature extraction network, effectively reducing the number of network model parameters. The cross-layer dense connection method improves spatial pyramid pooling, further expanding the receptive field and resulting in denser feature map outputs. The fusion of channel attention mechanism and spatial pyramid pooling features extracts rich spatial location information from the image and more refined deep features. The combination of multi-scale feature fusion and spatial attention mechanism fully utilizes the spatial location information in the shallow features of the feature extraction network, obtaining more detailed information from the original image during segmentation.
[0030] This invention employs deep learning methods to automatically identify and locate power lines in massive amounts of drone aerial images, solving the long-standing problem of low utilization and weak conversion rate of unstructured power grid data due to reliance on manual processing. It improves the accuracy of power line detection in drone aerial images, laying the foundation and providing technical support for subsequent tasks such as power line icing detection, intrusion detection, mapping and detection of electrical components on transmission lines, disaster monitoring, and navigation and obstacle avoidance for line inspection drones.
[0031] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An improved semantic segmentation network model of DeepLab V3+, characterized in that, The application relates to an encoder and a decoder, wherein the encoder comprises a feature extraction network, a hollow pyramid pooling module and a channel attention mechanism module; the hollow pyramid pooling module and the channel attention mechanism module are connected with the last feature layer of the feature extraction network respectively; the output feature maps of the hollow pyramid pooling module and the channel attention mechanism module are fused to form a deep feature map; the deep feature map is fused with a shallow feature map at the decoder end after being up-sampled to form an output feature map for target segmentation prediction.
2. The improved DeepLab V3+ semantic segmentation network model of claim 1, wherein: The feature extraction network is a light MobileNet V2 network, and the output feature maps of the five feature layers of the MobileNet V2 feature extraction network are 256*256*64, 128*128*128, 64*64*256, 32*32*512 and 32*32*1024.
3. The semantic segmentation network model improved from DeepLab V3+ according to claim 2, wherein, The input image is subjected to deep feature fusion through the MobileNet V2 feature extraction network, the hollow pyramid pooling module and the channel attention mechanism module; the hollow pyramid pooling module and the channel attention mechanism module are connected with the last feature layer 32*32*1024 of the MobileNet V2 feature extraction network; the hollow pyramid pooling module adopts parallel connection of a 1*1 convolution, a 3*3 hollow convolution with an expansion rate of 6, a 3*3 hollow convolution with an expansion rate of 12, a 3*3 hollow convolution with an expansion rate of 18 and a global average pooling layer; the 3*3 hollow convolution with an expansion rate of 6, the 3*3 hollow convolution with an expansion rate of 12 and the 3*3 hollow convolution with an expansion rate of 18 are connected in a dense connection mode; the hollow pyramid pooling module is fused with the feature maps of the branches to obtain a 32*32*512 feature map F1 through 1*1 convolution adjustment of the channel number; the output feature map of the attention mechanism module is adjusted in the channel number through 1*1 convolution to obtain a 32*32*512 feature map F2; the 32*32*512 feature map F1 and the 32*32*512 feature map F2 are subjected to deep feature fusion to obtain a 32*32*512 deep feature map F3.
4. The semantic segmentation network model improved from DeepLab V3+ according to claim 3, wherein, The dense connection mode of the hollow pyramid pooling module is parallel connection of a 1*1 convolution, a 3*3 hollow convolution with an expansion rate of 6, a 3*3 hollow convolution with an expansion rate of 12, a 3*3 hollow convolution with an expansion rate of 18 and a global average pooling layer; the output feature map of the 3*3 hollow convolution with an expansion rate of 6 is fused with the input feature map of the hollow pyramid pooling module as the input feature map of the 3*3 hollow convolution with an expansion rate of 12, and the output feature map of the 3*3 hollow convolution with an expansion rate of 12 is fused with the input feature map of the hollow pyramid pooling module as the input feature map of the 3*3 hollow convolution with an expansion rate of 18.
5. The semantic segmentation network model improved from DeepLab V3+ according to claim 3, wherein, Another dense connection mode of the cavity pyramid pooling module is 1*1 convolution, 3*3 cavity convolution with an expansion rate of 6, 3*3 cavity convolution with an expansion rate of 12, 3*3 cavity convolution with an expansion rate of 18, and global average pooling layer parallel connection; the output feature map of the 3*3 cavity convolution with an expansion rate of 6 is fused with the input feature map of the cavity pyramid pooling module as the input feature map of the 3*3 cavity convolution with an expansion rate of 12, and the output feature map of the 3*3 cavity convolution with an expansion rate of 12 is fused with the input feature map of the cavity pyramid pooling module and the output feature map of the 3*3 cavity convolution with an expansion rate of 6 as the input feature map of the 3*3 cavity convolution with an expansion rate of 18.
6. The semantic segmentation network model improved from DeepLab V3+ according to claim 3, wherein, The channel attention mechanism module first performs global average pooling and global maximum pooling on the input feature map 32*32*1024, and takes the operation result feature map as the input of the fully connected layer with shared parameters to respectively generate a global average pooling feature map and a global maximum pooling feature map with a dimension of 1*1*1024; then the two fully connected layers are used to compress and restore the channel number in sequence, and then the above two feature maps are added and normalized by using a sigmoid function to obtain a channel attention weight vector; finally, the generated weight vector is multiplied by the input feature map to assign different weights to the feature maps of each channel, and a 32*32*1024 feature map is output.
7. The semantic segmentation network model improved DeepLab V3+ of claim 6, wherein, The deep feature map F3 of the encoder part is obtained by 4 times up-sampling to obtain a 128*128*512 feature map, and the 128*128*512 feature map is fused with a shallow feature map F4 to obtain a 128*128*512 feature map F5, wherein the shallow feature map is obtained by using a multi-scale feature fusion manner, and the 128*128*512 feature map is restored to a 512*512 size after 3*3 convolution and 4 times up-sampling.
8. The semantic segmentation network model improved from DeepLab V3+ according to claim 7, wherein, The multi-scale feature fusion is to fuse the 256*256, 128*128 and 64*64 feature maps output by the MobileNet V2 feature extraction network to output a 128*128*512 feature map.
9. The semantic segmentation network model improved DeepLab V3+ of claim 8, wherein, The multi-scale feature fusion includes a spatial attention mechanism module, the 256*256*64 feature map output by the MobileNet V2 feature extraction network is first down-sampled by 2 times, and then output as a 128*128*128 feature map through a first spatial attention mechanism module, the 128*128*128 feature map output by the MobileNet V2 feature extraction network is output as a 128*128*128 feature map through a second spatial attention mechanism module, and the 64*64*256 feature map output by the MobileNet V2 feature extraction network is first up-sampled by 2 times, and then output as a 128*128*256 feature map through a third spatial attention mechanism module, and the three obtained 128*128 feature maps are fused to obtain a 128*128*512 feature map F4.