A power transmission line high-precision semantic segmentation method and system based on a PIDNet

Abstract

The application provides a power transmission line high-precision semantic segmentation method and system based on a PIDNet, which extracts features of a preprocessed image by using a line feature interaction double-branch network structure, including a context feature extraction branch and a detail mining branch; the context feature extraction branch uses an I path in the PIDNet to extract a global semantic relationship feature map of the power transmission line and a background; the detail mining branch combines P and D paths in the PIDNet to extract a power transmission line detail feature map; the context feature extraction branch and the detail mining branch interact through a bidirectional fusion module; a feature map extracted by the context feature extraction branch and the detail mining branch is fused by using a detail recovery decoder, and power transmission line pixels are adaptively aligned; a boundary perception auxiliary training head is used to superimpose the power transmission line detail feature map and the preprocessed image by using boundary perception, and boundary features of the power transmission line are extracted.

Description

Technical Field

[0001] This invention belongs to the field of power system monitoring technology, specifically relating to a high-precision semantic segmentation method and system for transmission lines based on PIDNet. Background Technology

[0002] With the growth of electricity demand and the expansion of the power grid, transmission line galloping has become a significant threat to power grid safety, and frequent galloping accidents can cause huge losses to social power safety and economic development. Therefore, improving the ability to detect transmission line galloping is crucial, and intelligent monitoring technology has become an important direction due to the development of artificial intelligence. Feature pixel-level analysis, as an effective means of obtaining global feature points, can quantitatively measure galloping, but it is necessary to consider the lightweight deployment of the model to meet real-time monitoring requirements.

[0003] Existing pixel-level segmentation techniques for power transmission line features are mainly divided into two categories: traditional image processing and deep learning. While traditional methods have made progress, their reliance on manual threshold selection results in poor robustness in complex backgrounds and scenarios where power transmission lines constitute a small percentage of pixels, making them unsuitable for practical applications. In contrast, while classic deep learning network architectures such as UNet provide a foundation for pixel-level segmentation, their high algorithmic complexity makes them insufficient to meet the stringent real-time requirements of power transmission line galloping detection.

[0004] To balance accuracy and real-time performance, lightweight model design has become a hot topic. Researchers have reduced model complexity through techniques such as depthwise separable convolution and pointwise group convolution, or by combining lightweight backbones with segmentation networks. However, existing pixel-level feature parsing and lightweight design techniques still suffer from poor adaptability and insufficient real-time performance when dealing with targets like power transmission lines that have fixed spatial features and a small pixel ratio. Summary of the Invention

[0005] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention proposes a high-precision semantic segmentation method and system for power transmission lines based on PIDNet.

[0006] To achieve the above objectives, according to one aspect of the present invention, a high-precision semantic segmentation method for power transmission lines based on PIDNet is provided, comprising: Preprocess the original image of the power transmission line; A dual-branch network structure with line feature interaction is used to extract features from the preprocessed image, including a context feature extraction branch and a detail mining branch. The context feature extraction branch adopts the I path in PIDNet to reduce the resolution of the preprocessed image and extract global semantic relationship feature maps between the transmission lines and the background. The detail mining branch merges the P and D paths in PIDNet to maintain the high resolution of the preprocessed image and extract detailed feature maps of the transmission lines. The context feature extraction branch and the detail mining branch interact through a bidirectional fusion module. By using the detail recovery decoder, the feature maps extracted by the context feature extraction branch and the detail mining branch are fused together, and the transmission line pixels are adaptively aligned to obtain the aligned feature map. By using a boundary-aware auxiliary training head, the boundary feature map of the transmission line is superimposed with the preprocessed image to extract the boundary features of the transmission line and obtain the boundary feature map.

[0007] According to the above method, both the context feature extraction branch and the detail mining branch are composed of residual basic modules; the context feature extraction branch obtains low-resolution feature maps by continuously downsampling the preprocessed image; the detail mining branch maintains the resolution of the preprocessed image.

[0008] Following the above method, the bidirectional fusion module includes high-to-low fusion and low-to-high fusion; among which, The high-to-low fusion downsamples the feature map of the detail mining branch and adds it element-wise to the feature map of the context feature extraction branch; The low-to-high fusion upsamples the feature map of the context feature extraction branch and adds it element-wise to the feature map of the detail mining branch.

[0009] Following the above method, the aligned feature map is obtained through the following steps: After unifying the channel dimension of the feature maps extracted by the context feature extraction branch and the detail mining branch, they are added element by element to obtain the primary fusion feature. The initial fusion features are used to generate: filter kernels for low-resolution feature maps to smooth global semantics; filter kernels for high-resolution feature maps to enhance detail boundaries; and pixel-level spatial offset values. The predicted offset corresponding to each pixel coordinate; The cosine similarity of adjacent pixels of the primary fusion feature is calculated, and the size of the feature map is filled by padding pixels to obtain the spatial offset relationship matrix, which reflects the positional correspondence of the transmission line pixels in the feature map. The spatial offset relation matrix is ​​weighted and combined with the feature maps extracted by the original context feature extraction branch and detail mining branch. The feature map of the context feature extraction branch is smoothed with a filter kernel of low-resolution feature map and its position is adjusted according to the spatial offset value. The feature map of the detail mining branch is enhanced with a filter kernel of high-resolution feature map and its position is adjusted according to the spatial offset value. Finally, the aligned feature map is obtained.

[0010] The boundary feature map is obtained through the following steps using the method described above: The detail feature map of the transmission line extracted by the detail mining branch is used to extract the boundary features of the preprocessed image, and then the result is output through the boundary segmentation head.

[0011] According to the above method, the preprocessing includes: adjusting the original image of the power transmission line to a uniform size and performing preliminary convolution processing.

[0012] According to another aspect of the present invention, a high-precision semantic segmentation system for power transmission lines based on PIDNet is provided, comprising: The preprocessing module is used to preprocess the original images of the transmission lines; The network structure employs a dual-branch feature extraction architecture to extract features from the preprocessed image, including a context feature extraction branch and a detail extraction branch. The context feature extraction branch uses the I path from PIDNet to reduce the resolution of the preprocessed image and extract global semantic relationship feature maps between the transmission lines and the background. The detail extraction branch merges the P and D paths from PIDNet to maintain the high resolution of the preprocessed image and extract detailed feature maps of the transmission lines. The context feature extraction branch and the detail extraction branch interact through a bidirectional fusion module. The detail recovery decoder is used to fuse the feature maps extracted by the context feature extraction branch and the detail mining branch, and adaptively align the transmission line pixels to obtain the aligned feature map. The boundary-aware auxiliary training head is used to overlay the detailed feature map of the transmission line with the preprocessed image to extract the boundary features of the transmission line and obtain the boundary feature map.

[0013] In the above system, both the context feature extraction branch and the detail mining branch are composed of residual basic modules; The context feature extraction branch obtains low-resolution feature maps by continuously downsampling the preprocessed image; the detail mining branch maintains the resolution of the preprocessed image.

[0014] According to the above system, the bidirectional fusion module includes a high-to-low fusion module and a low-to-high fusion module; among which... The high-to-low fusion module is used to downsample the feature map of the detail mining branch and add it element-wise to the feature map of the context feature extraction branch; The low-to-high fusion module is used to upsample the feature map of the context feature extraction branch and add it element-wise to the feature map of the detail mining branch.

[0015] According to the above system, the detailed recovery decoder specifically includes: The primary fusion module is used to unify the channel dimension of the feature maps extracted by the context feature extraction branch and the detail mining branch, and then add them element by element to obtain the primary fused feature. The spatial offset relation learning module is used to generate the following from the initial fusion features: a filter kernel for the low-resolution feature map to smooth global semantics; a filter kernel for the high-resolution feature map to enhance detail boundaries; and pixel-level spatial offset values. The predicted offset corresponds to the coordinates of each pixel; at the same time, the cosine similarity of the adjacent pixels of the primary fusion feature is calculated, and the size of the feature map is made up by padding pixels, thereby obtaining the spatial offset relationship matrix, which is used to reflect the positional correspondence of the transmission line pixels in the feature map. The weighted alignment module is used to weight and combine the spatial offset relation matrix with the feature maps extracted from the original context feature extraction branch and detail mining branch. The feature map of the context feature extraction branch is smoothed with a filter kernel of low-resolution feature map to smooth the global semantics and the position is adjusted according to the spatial offset value. The feature map of the detail mining branch is enhanced with a filter kernel of high-resolution feature map to enhance the detail boundary and the position is adjusted according to the spatial offset value. Finally, the aligned feature map is obtained.

[0016] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: By designing a dual-branch network structure for line feature interaction and constructing parallel processing branches, efficient interaction of line features at different levels is promoted, thereby enhancing the ability to extract transmission line features. Furthermore, a detail recovery decoder is used, employing upsampling and feature fusion techniques to restore the fine structure of the transmission line from the low-resolution feature map, thus compensating for the loss of transmission line details during segmentation. A line boundary awareness auxiliary training head is introduced to assist the main network in learning the boundary features of the transmission line, enhancing the model's sensitivity to boundary details. Through these structural improvements, high-precision semantic segmentation of transmission lines is effectively achieved. Attached Figure Description

[0017] Figure 1 This is a system network architecture diagram provided in an embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram of the bidirectional fusion module provided in an embodiment of the present invention.

[0019] Figure 3 This is an architecture diagram of the detail recovery decoder provided in an embodiment of the present invention.

[0020] Figure 4 This is a schematic diagram of the boundary awareness auxiliary training head provided in an embodiment of the present invention.

[0021] Figure 5 This is a visualization result of the activated region of the dual-branch feature map provided in an embodiment of the present invention.

[0022] Figure 6 This is a visualization result of the feature map provided in an embodiment of the present invention.

[0023] Figure 7 This is a comparison chart of the foreground segmentation index of the power transmission line provided in the embodiments of the present invention.

[0024] Figure 8 This is a comparison chart of the overall mIoU and foreground IoU segmentation indices provided in an embodiment of the present invention.

[0025] Figure 9 This is a comparison chart of foreground segmentation metrics provided in an embodiment of the present invention.

[0026] Figure 10 This is a comparison diagram of the segmentation results provided in the embodiments of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0028] According to one aspect of the present invention, this embodiment provides a high-precision semantic segmentation system for power transmission lines based on PIDNet, such as... Figure 1 As shown, it includes: The preprocessing module is used to preprocess the original image of the power transmission line; including: adjusting the original image of the power transmission line to a uniform size and performing preliminary convolution processing.

[0029] A dual-branch network structure with interactive features is used for feature extraction from the preprocessed image, including a context feature extraction branch and a detail mining branch. The context feature extraction branch uses the I path from PIDNet to reduce the resolution of the preprocessed image and extract global semantic relationship feature maps between the transmission lines and the background. The detail mining branch merges the P and D paths from PIDNet to maintain the high resolution of the preprocessed image and extract detailed feature maps of the transmission lines. The context feature extraction branch and the detail mining branch interact through a bidirectional fusion module. Both the context feature extraction branch and the detail mining branch are composed of residual basic modules. The context feature extraction branch obtains low-resolution feature maps by continuously downsampling the preprocessed image; the detail mining branch maintains the resolution of the preprocessed image.

[0030] like Figure 2 As shown, the bidirectional fusion module includes a high-to-low fusion module and a low-to-high fusion module; wherein, the high-to-low fusion module is used to downsample the feature map of the detail mining branch and add it element-wise to the feature map of the context feature extraction branch; the low-to-high fusion module is used to upsample the feature map of the context feature extraction branch and add it element-wise to the feature map of the detail mining branch.

[0031] The detail recovery decoder is used to fuse the feature maps extracted by the context feature extraction branch and the detail mining branch, and adaptively align the transmission line pixels to obtain the aligned feature map.

[0032] like Figure 3 As shown, the detail recovery decoder specifically includes: The primary fusion module is used to unify the channel dimension of the feature maps extracted by the context feature extraction branch and the detail mining branch, and then add them element by element to obtain the primary fused feature. The spatial offset relation learning module is used to generate the following from the initial fusion features: a filter kernel for the low-resolution feature map to smooth global semantics; a filter kernel for the high-resolution feature map to enhance detail boundaries; and pixel-level spatial offset values. The predicted offset corresponds to the coordinates of each pixel; at the same time, the cosine similarity of the adjacent pixels of the primary fusion feature is calculated, and the size of the feature map is made up by padding pixels, thereby obtaining the spatial offset relationship matrix, which is used to reflect the positional correspondence of the transmission line pixels in the feature map. The weighted alignment module is used to weight and combine the spatial offset relation matrix with the feature maps extracted from the original context feature extraction branch and detail mining branch. The feature map of the context feature extraction branch is smoothed with a filter kernel of low-resolution feature map to smooth the global semantics and the position is adjusted according to the spatial offset value. The feature map of the detail mining branch is enhanced with a filter kernel of high-resolution feature map to enhance the detail boundary and the position is adjusted according to the spatial offset value. Finally, the aligned feature map is obtained.

[0033] The boundary-aware auxiliary training head is used to overlay the detailed feature map of the transmission line with the preprocessed image to extract the boundary features of the transmission line and obtain the boundary feature map.

[0034] According to one aspect of the present invention, this embodiment also provides a high-precision semantic segmentation method for power transmission lines based on PIDNet, comprising: S1. Preprocess the original image of the power transmission line.

[0035] S2. The preprocessed image is feature extracted using a dual-branch network structure with line feature interaction, including a context feature extraction branch and a detail mining branch. The context feature extraction branch uses the I path in PIDNet to reduce the resolution of the preprocessed image and extract the global semantic relationship feature map between the transmission line and the background. The detail mining branch merges the P and D paths in PIDNet to maintain the high resolution of the preprocessed image and extract the detail feature map of the transmission line. The context feature extraction branch and the detail mining branch interact through a bidirectional fusion module.

[0036] Both the context feature extraction branch and the detail mining branch are composed of residual basic modules; the context feature extraction branch obtains low-resolution feature maps by continuously downsampling the preprocessed image; the detail mining branch maintains the resolution of the preprocessed image.

[0037] The bidirectional fusion module includes high-to-low fusion and low-to-high fusion; among them, The high-to-low fusion downsamples the feature map of the detail mining branch and adds it element-wise to the feature map of the context feature extraction branch; The low-to-high fusion upsamples the feature map of the context feature extraction branch and adds it element-wise to the feature map of the detail mining branch.

[0038] S3. Using the detail recovery decoder, the feature maps extracted by the context feature extraction branch and the detail mining branch are fused, and the transmission line pixels are adaptively aligned to obtain the aligned feature map. The aligned feature map is obtained through the following steps: 301. After unifying the channel dimension of the feature maps extracted by the context feature extraction branch and the detail mining branch, add them element by element to obtain the primary fusion feature.

[0039] The feature maps extracted by the context feature extraction branch and the detail mining branch are compressed by 1×1 convolution. The purpose is to reduce the number of channels, avoid excessive computation during subsequent fusion, and retain the core features.

[0040] After element-wise addition, the high-resolution "detail information" and the low-resolution "global context" are initially combined, providing a unified feature basis for subsequent learning of spatial offset relationships.

[0041] 302. Generate the following from the initial fusion features: filter kernels for low-resolution feature maps to smooth global semantics; filter kernels for high-resolution feature maps to enhance detail boundaries; and pixel-level spatial offset values, corresponding to the predicted offset of each pixel coordinate.

[0042] The cosine similarity of adjacent pixels in the primary fusion feature is calculated, and the size of the feature map is filled by padding pixels to obtain a spatial offset relationship matrix, which reflects the positional correspondence of transmission line pixels in the feature map, such as "how much a pixel in the low-resolution image needs to be offset to align with the corresponding transmission line pixel in the high-resolution image".

[0043] Cosine similarity measures the semantic similarity between pixels. The initial fusion feature map of the power transmission line-related region contains both high-resolution "power transmission line details" and low-resolution "global context," but the semantic relationships between pixels differ. Power transmission line pixels, due to their elongated shape and consistent grayscale / texture, exhibit higher semantic similarity between adjacent pixels; while background pixels (such as trees and sky) have strong semantic randomness, resulting in low similarity between adjacent pixels. Calculating the "cosine similarity between adjacent pixels" yields a similarity matrix: when the cosine similarity between adjacent pixels is high (close to 1), it indicates that the two pixels likely belong to the same target (such as continuous segments of a power transmission line), with a close semantic relationship; when the cosine similarity is low (close to 0 or negative), it indicates that the pixels may belong to different targets (such as the boundary between the power transmission line and the background), with a weak semantic relationship. This step helps the model quickly "focus" on the power transmission line-related pixel region, avoiding background noise interference with subsequent offset learning.

[0044] Furthermore, cosine similarity provides "semantic weights" for the spatial offset matrix. The core of improving alignment accuracy with the spatial offset matrix is ​​determining "which pixels in the high- and low-resolution feature maps correspond to the same transmission line position," and the result of cosine similarity is incorporated into this matrix as semantic weights: for adjacent pixels with high cosine similarity (transmission line regions), the model assigns higher weights, prioritizing the learning of their spatial offset patterns to ensure the alignment accuracy of transmission line pixels; for pixels with low cosine similarity (background or boundary regions), the model reduces their weights to minimize interference with offset learning and avoid alignment errors caused by cluttered backgrounds. Ultimately, the spatial offset matrix combined with cosine similarity allows the model to more accurately calculate the bias values ​​of transmission line pixels, ensuring a complete match between transmission line positions in the high- and low-resolution feature maps, providing reliable alignment features for subsequent detail recovery. In short, cosine similarity is the model's "semantic filter"—it helps the model locate transmission line regions and provides a priority basis for offset learning, fundamentally improving the accuracy of transmission line feature fusion.

[0045] 303. The spatial offset relation matrix is ​​weighted and combined with the feature maps extracted by the original context feature extraction branch and detail mining branch. The feature map of the context feature extraction branch is smoothed with a filter kernel of low-resolution feature map to smooth the global semantics and the position is adjusted according to the spatial offset value. The feature map of the detail mining branch is enhanced with a filter kernel of high-resolution feature map to enhance the detail boundary and the position is adjusted according to the spatial offset value. Finally, the aligned feature map is obtained, which not only retains the high-resolution transmission line details, but also incorporates the low-resolution global context, and there is no positional misalignment.

[0046] S4. Using a boundary-aware auxiliary training head, the boundary feature map of the transmission line is superimposed with the preprocessed image to extract the boundary features of the transmission line and obtain the boundary feature map.

[0047] The boundary feature map is obtained through the following steps: The detail feature map of the transmission line extracted by the detail mining branch is used to extract the boundary features of the preprocessed image, and then the result is output through the boundary segmentation head.

[0048] The invention will be further described below with reference to specific examples and accompanying drawings.

[0049] This embodiment provides a high-precision semantic segmentation method for power transmission lines based on PIDNet, including: S1. Preprocess the original image of the power transmission line.

[0050] In this implementation, the original power transmission line monitoring images are uniformly resized to 640×640 pixels. Preliminary convolutional processing is performed using the Stem module, including: a 3×3 convolutional layer (stride 2, padding pixel 1), a batch normalization layer (BatchNorm), and a ReLU activation function. The output feature map size is reduced to the original size. Figure 1 / 8 (80×80).

[0051] S2. The preprocessed image is feature extracted using a dual-branch network structure with line feature interaction, including a context feature extraction branch and a detail mining branch. The context feature extraction branch uses the I path in PIDNet to reduce the resolution of the preprocessed image and extract the global semantic relationship feature map between the transmission line and the background. The detail mining branch merges the P and D paths in PIDNet to maintain the high resolution of the preprocessed image and extract the detail feature map of the transmission line. The context feature extraction branch and the detail mining branch interact through a bidirectional fusion module.

[0052] Observation of the foreground and background of the power transmission line reveals that the semantics contained in the power transmission line features are divided into low-level features and high-level semantics. Low-level features describe the inherent characteristics of the power transmission line without excessive downsampling to avoid loss of detailed information. Low-level features are described at the semantic level and are extracted by maintaining a high-resolution feature map. High-level semantics represent the contextual information between the power transmission line and its environment, and are characterized by low-resolution features after layers of convolution. These features focus on global information, thereby learning the semantic category information of the foreground and background. Based on this, this patent designs a Line-featuresInteractive Two-Branch network structure (LITB) for these two types of features, including a Context-extraction Branch (CB) and a Detail-digging Branch (DB).

[0053] In the dual-path method, one path extracts contextual semantics, while the other path preserves high resolution and extracts details. First, both paths of PIDNet are retained, and then the P and D paths are merged. The overall structure is as follows: Figure 1 As shown, each branch in the network consists of residual basic blocks, containing 3×3 convolutions, batch normalization (BN) layers, and ReLU activation functions. The improved dual-branch network comprises a main segmentation head and a boundary-aware auxiliary training head.

[0054] The feature maps in the detail mining branch are not downsampled, maintaining an 8x downsampling rate compared to the Stem output, thus preserving a high-resolution 80×80 feature map. This allows for deep feature representation complementarity between the low-resolution and high-resolution feature maps. The context feature extraction branch (CB) obtains a low-resolution feature map (20×20) through continuous downsampling and uses a 3×3 convolution with stride for downsampling, focusing on extracting the global semantic relationship between the transmission line and the background.

[0055] like Figure 2 As shown, the interaction between the detail mining branch and the context feature extraction branch is achieved through a bidirectional fusion module. This mechanism can be represented as follows:

[0056] in, R Represents the ReLU activation function. and These represent the context feature extraction branch (CB) at the [number]th [year]. Layer and first The output feature map; and These represent the details mining (DB) branch at the [number]th [year]. Layer and first The output feature map, and These represent the residual reference blocks for the corresponding branches. and These represent the feature scale changes from high to low and from low to high, respectively, achieved by convolutional downsampling and bilinear interpolation downsampling.

[0057] The specific operation of the bidirectional fusion module is as follows: High-to-low fusion: The high-resolution feature map is first processed through two 3×3 convolutions and a ReLU activation function to extract preliminary information. Then, it is downsampled through convolution to adapt to the scale of the low-resolution feature map. After downsampling, the high and low feature maps are added element-wise to improve the richness of global semantic information.

[0058] Low-to-high fusion: In contrast to high-to-low fusion, low-to-high fusion is used to supplement detailed information with contextual information. In this process, the low-resolution feature map is first compressed using a 1x1 convolution, then upsampled using bilinear interpolation to generate a feature map of the same scale as the high-resolution branch. Next, the low-resolution feature map and the high-resolution feature map are added element-wise. This process effectively combines the global information extracted from the low-resolution branch with the detailed information from the high-resolution branch.

[0059] PAPPM is connected to the back end of the CB branch, using 5 parallel pooling branches for scale processing: 1×1 (global context), 3×3 (local context), 6×6 (medium-range context), 12×12 (large-range context), and global average pooling. The number of output channels for each branch is uniformly 96. The feature maps at each scale are concatenated and fused through 1×1 convolution, and the output is connected to the original feature residual.

[0060] S3. Using the detail recovery decoder, the feature maps extracted by the context feature extraction branch and the detail mining branch are fused together, and the transmission line pixels are adaptively aligned to obtain the aligned feature map.

[0061] In power transmission line segmentation, detailed features of the power transmission lines are often lost during downsampling during convolutional feature extraction. To address the issue of small power transmission line targets, this section proposes a Detail Recovery Decoder (DRD) for feature amplification and reconstruction.

[0062] The detail restoration decoder consists of two main steps: cross-scale connectivity and spatial offset alignment. In the cross-scale connectivity part, the 4x downsampled feature maps and 8x downsampled feature maps are extracted separately, and the 4x downsampled feature map from the encoder is used to supplement details.

[0063] To fully extract the fine features of transmission lines, high-resolution feature maps need to be fused. However, there are positional offsets between different feature maps. Therefore, a Spatial Offset Alignment (SOA) module is proposed for adaptive alignment of transmission line pixels. This combines low-resolution and high-resolution features to capture spatial semantic relationships. Figure 3 As shown, the spatial offset alignment module consists of two basic components: a filter and an offset perceptron. The fusion process is two-stage; before final fusion, low-level features need to be processed. and advanced features Compress to obtain the output and This process can be represented as

[0064] in, For the low-resolution branch (context branch) in "the Layer, coordinates The final fusion characteristic; For the low-resolution branch (context branch) in "the The characteristics of the "layer" after offset In coordinates The value; For high-resolution branches (detail branches) in "the Layer, coordinates The characteristics of "". This represents upsampling.

[0065] For the low-resolution branch "the first The features of the "layer" after "downsampling → upsampling" enhancement; This is for "downsampling / low-pass filtering" operation; This is an "upsampling" operation; For the low-resolution branch "the first The original characteristics of the "layer".

[0066] For high-resolution branches (detail branches) in "the The characteristics of the "layer" after "Qualcomm enhancement"; For high-resolution branch "the first The original characteristics of the "layer".

[0067] Primary characteristics of fusion Z It can be derived from the following formula

[0068]

[0069]

[0070] in, For low-resolution feature maps Intermediate features obtained after compression and correlation transformation; For high-resolution feature maps Intermediate features obtained after compression and correlation transformation.

[0071] The first step is a 1×1 convolution operation. Then, the cosine similarity of adjacent pixels in the feature maps is calculated, and the feature map size is padded to obtain a learned spatial offset matrix between pixels. This matrix is ​​then weighted and added to the original high-resolution and low-resolution features to finally obtain the aligned feature map. This method allows the network to learn the spatial offset relationships between high and low feature maps and adaptively align features through network training.

[0072] S4. Using a boundary-aware auxiliary training head, the boundary feature map of the transmission line is superimposed with the preprocessed image to extract the boundary features of the transmission line and obtain the boundary feature map.

[0073] To effectively extract edge information, a boundary-aware auxiliary training head utilizes a boundary-aware module to extract boundary features from a 2x downsampled feature map, thereby uncovering edge details of the power transmission line. For example... Figure 4As shown, the edge-aware auxiliary training head consists of an Edge Perception Module (EPM) and a raw segmentation head. The EPM adopts a ResNet-like residual structure design, consisting of a wavelet convolution (WTConv), a batch normalization layer, and a ReLU activation function in the main path. Residual connections are used in the branch paths.

[0074] WTConv is the core of the boundary-aware module, utilizing two-dimensional Haar wavelet transform to perform multi-level decomposition of the input image. The multi-level wavelet transform is expressed as follows:

[0075]

[0076] The Haar wavelet transform uses four filters to decompose the image into four sub-bands. Low-frequency components... Capture low-frequency information from the image. Horizontal high-frequency components. Capture horizontal edge information in the image. Vertical high-frequency components. Capture vertical edge information in the image. Diagonal high-frequency components. Capture the diagonal details of the image. In each level of wavelet transform, the image is downsampled while the frequency information is decomposed into finer values, and the wavelet transform is performed recursively to finally obtain frequency components at different scales.

[0077] Then, small-kernel depthwise convolutions are performed on different frequency maps, combined with inverse wavelet transform (IWT) to obtain the output.

[0078]

[0079] in, W These are the weights of the depthwise convolutional kernel, and the input channels are... Four times that of the previous output. After convolution, the inverse wavelet transform (IWT) is used to combine the convolution results of each subband into a single complete output.

[0080] This process fuses features from different frequency levels. Since the IWT operation is linear, the convolution result can be reconstructed back to the original space without loss. The following formula yields the concatenated form of wavelet convolution.

[0081]

[0082] in, It is the input of the (i-1)th layer. It is the low-frequency component after wavelet transform. These are the remaining three high-frequency components, which are then subjected to convolution to obtain the new intermediate layer output. A two-layer cascade approach is used to enhance the ability to extract high-frequency information from the boundaries. It is the output of the convolutional components.

[0083] The boundary-aware module upsamples the extracted feature map and adds it to the high-resolution feature map, thus making the network focus more on the boundary and assigning greater weight to the boundary regions. Simultaneously, performing boundary-aware calculations on the 8x downsampled feature map effectively avoids massive computational overhead. Then, semantic alignment is performed using the edge detection results from the label mask, and a boundary-weighted loss function Lb is designed for supervision. The expression for Lb is:

[0084] in, N , H and W These represent the number of images, height, and width, respectively. To address the class imbalance problem, the loss function calculates a weighted loss for each pixel. .

[0085] First, calculate the number of positive and negative samples.

[0086]

[0087] in, I This is a judgment function. This means the predicted class and the actual pixel class are equal, at which point the number of positive samples is [number missing]. Add 1. This indicates that the predicted class and the actual pixel class are not equal, in which case the number of negative samples is... Add 1. In the weight matrix, the weight values ​​are obtained by weighting the positive and negative samples, and the expression is:

[0088] The core idea of ​​this boundary loss function is to calculate the binary cross-entropy loss between the predicted boundary and the true boundary, and to weight it according to the number of positive and negative samples, thereby effectively focusing on the boundary region of the transmission line and adaptively allocating more weight to the foreground transmission line.

[0089] The other parts of the boundary awareness-assisted training head are all conventional methods, and will not be repeated here.

[0090] The effectiveness of the improved strategy of this invention will be analyzed through experimental experiments below: To verify the improvement effect of the line feature interaction dual-branch network structure, detail recovery decoder and line boundary awareness auxiliary training head on the semantic segmentation of transmission lines, the effectiveness of the proposed improvement strategy is verified based on the PIDNet model.

[0091] 1) Validation of the effectiveness of the dual-branch network structure with characteristic interaction of transmission lines This experiment verifies the effectiveness of the LITB (Limited Institutional Branch Network) structure, which features interaction between power transmission lines, in the power transmission line segmentation scenario. The experimental results are shown in Table 1.

[0092] Table 1 Experimental Results of the LITB Improvement Strategy

[0093] As shown in Table 1, the model's accuracy metrics improved after modifying the three-branch architecture to LITB. Specifically, mIoU and mAcc increased by 2.25% and 2.04%, respectively. This indicates greater overlap between the model's predicted regions and the ground truth labels, reducing mis-segmented regions and improving overall accuracy. mDice increased by 1.64%, demonstrating the model's ability to more accurately segment power line boundaries. Furthermore, the dual-branch design makes the model more lightweight, reducing the number of parameters by 17.01M, computational FLOPS by 25.79G, and real-time inference speed by 42.06 FPS. Experiments demonstrate that the proposed power line feature interaction dual-branch network structure LITB improves segmentation accuracy while optimizing computational efficiency, resulting in enhanced real-time performance in power line segmentation tasks.

[0094] To verify the effectiveness of the detail mining branch and context feature extraction branch in LITB, the intermediate feature maps of the two branches are visualized below. For example... Figure 5 As shown, rows 1, 2, and 3 are the feature maps of the original image, DB, and CB branches, respectively, and row 4 is the semantic segmentation result of the transmission line.

[0095] By performing max pooling on the output feature maps of the CB and DB branches along the channel dimension, the activation regions of the feature maps are visualized. The intensity of the color reflects the degree of activation of these regions by the model; darker colors indicate higher model attention to these pixels. Figure 5 It can be seen that after introducing the LITB structure, the activation region of the DB branch mainly focuses on extracting the details and contours of line features, while the CB branch focuses on the abstract semantics of the background and foreground. This indicates that after adopting the LITB strategy, the two branches can focus on the details of the transmission line and the global contextual information between the foreground and background, respectively. Furthermore, simplifying the three-branch structure of PIDNet into a two-branch structure can reduce model redundancy.

[0096] 2) Verification of the effectiveness of the transmission line detail recovery decoder To fully verify the effectiveness of the Transmission Line Detail Recovery Decoder (DRD), the output 8x downsampled feature map and the encoder 4x downsampled result were fused based on PIDNet, and the feature map was adjusted to 4x size. The experimental results are shown in Table 2.

[0097] Table 2 Experimental Results of DRD Improvement Strategy

[0098] As shown in Table 2, after introducing DRD, the model's mIoU and mAcc improved by 3.28% and 1.75%, respectively. This indicates that DRD effectively enhances the model's ability to identify detailed regions of transmission lines. mDice increased from 87.93% to 90.30%, a 2.37% improvement, indicating that the overlap between the predicted and actual values ​​of the transmission line boundaries increased after adding the DRD strategy, resulting in segmentation results closer to reality. However, the addition of DRD also increased the model parameters from 37.31M to 40.02M, and the number of floating-point operations increased to 134G. This shows that while the DRD strategy improves performance, it also increases the model's complexity and computational burden.

[0099] To more clearly demonstrate the role of the detail recovery decoder in the network, this invention compares the power line feature extraction results under different weather and background conditions before and after the DRD is applied, and presents heatmaps to reflect the activation contribution of this improved module to the foreground.

[0100] like Figure 6 As shown, the first row is the original example image, the second and third rows are the output feature maps with and without DRD, respectively, and the last row represents the foreground activation regions of the transmission line that the model focuses on. The yellow area represents the outline of the transmission line in the feature map without DRD, while the red area shows the outline of the transmission line in the feature map after adding DRD. The marked areas represent the degree of attention the model pays to the foreground of the transmission line. Through comparative analysis, it can be seen that compared with the original PIDNet algorithm, the details of the transmission line are more clearly presented in the feature map after introducing DRD. The heatmap shows that the model's attention to the foreground transmission line is significantly improved. Experimental results show that DRD can effectively capture and enhance the details of the transmission line, improving the accuracy of transmission line segmentation.

[0101] 3) Validation of the effectiveness of the transmission line boundary perception auxiliary training head The evaluation metrics of the model before and after introducing EPM were compared. As shown in Table 3, the model after introducing EPM improved by 3.48%, 2.25%, and 2.51% in the three key metrics of mIoU, mAcc, and mDice, respectively. This indicates that the boundary-aware auxiliary training head strategy proposed in this invention, by integrating wavelet convolution and residual structure, significantly enhances the model's ability to extract features of power transmission line edges.

[0102] Table 3 Experimental Results of EPM Improvement Strategies

[0103] To intuitively analyze the role of EPM in sensing the line boundaries of transmission line foregrounds, this invention extracts and analyzes the segmentation metrics of transmission line foregrounds. As shown in Table 4, after introducing the transmission line boundary sensing auxiliary training head EPM, the model exhibits a significant performance improvement in transmission line foreground segmentation. This indicates that EPM accurately captures high-frequency component information through multi-level wavelet transform, enabling the model to more accurately identify and locate the edges of transmission lines.

[0104] Table 4 Foreground Segmentation Results of Transmission Lines

[0105] like Figure 7 As shown, after introducing EPM, improvements of 6.67%, 4.32%, and 4.88% were achieved in IoU, Acc, and Dice, respectively, and improvements of 5.35% and 4.32% in P and R, respectively. Experimental results show that adding the boundary-aware auxiliary training head significantly improves the accuracy of the model in classifying transmission line pixels, effectively extracting the foreground of transmission lines and improving segmentation accuracy.

[0106] To verify the effectiveness of the three strategies proposed in this invention working together on the benchmark algorithm, an ablation experiment was set up in this section, and the experimental results are shown in Table 5.

[0107] Table 5. Results of ablation experiments using the improved strategy

[0108] Table 5 shows that (a) the model achieved varying degrees of improvement when the three improvements were applied separately. Among them, the model's mIoU, mAcc, and mDice were significantly improved by 3.48%, 2.25%, and 2.51% respectively when only the EPM strategy was used, representing the largest improvement. This indicates that in the power transmission line segmentation task, the boundary-aware auxiliary training head can more effectively help the model capture these key edge features. After introducing the LITB strategy, the FPS increased significantly by 42.06. This result shows that the dual-branch structure not only optimizes the model architecture but also significantly improves the speed of real-time inference. (b) When both DRD and EPM strategies were applied simultaneously, the model's mIoU and mAcc reached 87.01% and 93.34% respectively, further improving accuracy. Compared with using only the DRD strategy, these metrics improved by 0.43%, 0.1%, and 0.27% respectively. However, compared with using only EPM, the improvement in mIoU was relatively small, only 0.06%. The results show that both DRD and EPM strategies enhance the details and edge information of the transmission line. (c) When the three improvements are applied simultaneously, the model's mIoU and mAcc are improved to 87.21% and 93.66%, respectively, achieving optimal performance. This indicates that the improvements proposed in this invention can effectively synergistically improve the model's accuracy.

[0109] The algorithm combining the three improved strategies is called TLNet (Transmission Line Net), and the accuracy changes during model training before and after the improvements are recorded. The results are as follows: Figure 8 As shown.

[0110] Depend on Figure 8 It can be seen that the TLNet algorithm improves foreground IoU and overall mIoU by 13.24% and 6.9% respectively compared to the original PIDNet, with a significant improvement in accuracy. Furthermore, the curve converges faster, indicating that the improved algorithm has higher training efficiency. Figure 9 It can be seen that the improved algorithm achieves a double improvement in speed and accuracy.

[0111] Comparative experiment: The proposed TLNet is compared and analyzed with existing mainstream semantic segmentation algorithms such as APCNet, BiseNet series, DANet, DeepLabv3 series, and STDC, with the input resolution unified to 640. The experimental results are shown in Table 6. The proposed algorithm TLNet outperforms other mainstream algorithms in terms of mIoU, mAcc, and mDice. Compared to the poor DeepLabv3+ algorithm, its mIoU is 0.08 percentage points higher. Compared to other dual-branch real-time semantic segmentation algorithms BiseNetv1 and BiseNetv2, its mIoU is 6.94% and 4.65% higher, respectively. This indicates that the detail recovery decoder and boundary-aware auxiliary training head proposed in this invention can significantly improve the segmentation performance of the dual-branch model for line targets.

[0112] Table 6 Comparison of experimental results of the improved algorithm

[0113] like Figure 10 As shown, the inference results of the improved TLNet algorithm, the original PIDNet, and the second most accurate DeepLabv3+ under different weather conditions are visualized. Experimental results show that TLNet's segmentation results are more detailed and accurate, achieving the best performance, followed by DeepLabv3+. TLNet can better focus on the spatially fixed foreground of power transmission lines, uncover pixel details of power transmission lines, and enhance the segmentation ability of power transmission line edges.

[0114] In summary, for the semantic segmentation task of power transmission lines, the TLNet algorithm proposed in this invention outperforms other mainstream algorithms in terms of accuracy, while keeping computational resource consumption within a reasonable range. Compared with real-time semantic segmentation algorithms such as BiseNetv2 and PIDNet, TLNet has a significant advantage in accuracy.

[0115] First, the characteristics of power transmission lines are analyzed, and PIDNet is selected as the optimal algorithm. To address its incompatibility in power transmission line segmentation tasks, this invention proposes an improved semantic segmentation algorithm for power transmission lines, TLNet, based on PIDNet. For the spatial morphological features of power transmission lines, a parallel network structure for line feature interaction is proposed. Introducing this structure improves mIoU, mAcc, and mDice by 2.25%, 2.04%, and 1.64%, respectively. To address the issue of small pixel proportions and detail loss in power transmission lines, a power transmission line detail recovery decoder is proposed. Introducing this structure improves the model's mIoU, mAcc, and mDice by 3.28%, 1.75%, and 2.37%, respectively. To address the model's insufficient ability to perceive line edges, a power transmission line boundary perception auxiliary training head is proposed. Introducing this structure improves the algorithm's ability to perceive line feature boundaries, improving mIoU, mAcc, and mDice by 3.48%, 2.25%, and 2.51%, respectively. By comparing the current mainstream semantic segmentation algorithms with power transmission line segmentation cases under different weather conditions, the superiority of the improved algorithm of this invention in the semantic segmentation task of power transmission lines is demonstrated.

[0116] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0117] It should be noted that, depending on the implementation needs, the various steps / components described in this application can be broken down into more steps / components, or two or more steps / components or parts of the operation of steps / components can be combined into new steps / components to achieve the purpose of this invention.

[0118] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A high-precision semantic segmentation method for power transmission lines based on PIDNet, characterized in that: include: Preprocess the original image of the power transmission line; The preprocessed image is used to extract features using a line feature interaction dual-branch network structure, including a context feature extraction branch and a detail mining branch; The context feature extraction branch uses the I path in PIDNet to reduce the resolution of the preprocessed image and extract the global semantic relationship feature map between the transmission line and the background. The detail mining branch merges the P and D paths in PIDNet to preserve the high resolution of the preprocessed image and extract the detail feature map of the transmission line. The context feature extraction branch and the detail mining branch interact through a bidirectional fusion module; Using a detail recovery decoder, the feature maps extracted by the context feature extraction branch and the detail mining branch are fused, and the transmission line pixels are adaptively aligned to obtain the aligned feature map, specifically including: After unifying the channel dimension of the feature maps extracted by the context feature extraction branch and the detail mining branch, they are added element by element to obtain the primary fusion feature. The primary fusion features are used to generate: filter kernels for low-resolution feature maps to smooth global semantics; filter kernels for high-resolution feature maps to enhance detail boundaries; and pixel-level spatial offset values, corresponding to the predicted offset of each pixel coordinate. The cosine similarity of adjacent pixels of the primary fusion feature is calculated, and the size of the feature map is filled by padding pixels to obtain the spatial offset relationship matrix, which reflects the positional correspondence of the transmission line pixels in the feature map. The spatial offset relation matrix is ​​weighted and combined with the feature maps extracted from the original context feature extraction branch and detail mining branch. The feature map of the context feature extraction branch is smoothed with a filter kernel of low-resolution feature map and its position is adjusted according to the spatial offset value. The feature map of the detail mining branch is enhanced with a filter kernel of high-resolution feature map and its position is adjusted according to the spatial offset value. Finally, the aligned feature map is obtained. The training head, which includes a boundary-aware module and an original segmentation head, extracts boundary features from the feature map of the original image after it has been downsampled by a factor of 2. Then, it upsamples the feature map and overlays it with the detail feature map of the transmission line. Finally, it passes through the original segmentation head to obtain the boundary feature map. The boundary-aware module adopts a ResNet-like residual structure design. In the main path, it consists of a wavelet convolution, a batch normalization layer, and a ReLU activation function. In the branch path, it uses residual connections.

2. The high-precision semantic segmentation method for power transmission lines based on PIDNet according to claim 1, characterized in that: Both the context feature extraction branch and the detail mining branch are composed of residual basic modules; The context feature extraction branch obtains low-resolution feature maps by continuously downsampling the preprocessed image; the detail mining branch maintains the resolution of the preprocessed image.

3. The high-precision semantic segmentation method for power transmission lines based on PIDNet according to claim 1, characterized in that: The bidirectional fusion module includes high-to-low fusion and low-to-high fusion; among them, The high-to-low fusion downsamples the feature map of the detail mining branch and adds it element-wise to the feature map of the context feature extraction branch; The low-to-high fusion upsamples the feature map of the context feature extraction branch and adds it element-wise to the feature map of the detail mining branch.

4. The high-precision semantic segmentation method for power transmission lines based on PIDNet according to claim 1, characterized in that: Preprocessing includes: The original image of the power transmission line is adjusted to a uniform size and then subjected to preliminary convolution processing through the Stem module to output a feature map at 1 / 8 the size of the original image.

5. A high-precision semantic segmentation system for power transmission lines based on PIDNet, characterized in that: include: The preprocessing module is used to preprocess the original images of the transmission lines; A line feature interaction dual-branch network structure is used for feature extraction from preprocessed images, including a context feature extraction branch and a detail mining branch; The context feature extraction branch uses the I path in PIDNet to reduce the resolution of the preprocessed image and extract the global semantic relationship feature map between the transmission line and the background. The detail mining branch merges the P and D paths in PIDNet to preserve the high resolution of the preprocessed image and extract the detail feature map of the transmission line. The context feature extraction branch and the detail mining branch interact through a bidirectional fusion module; The detail recovery decoder is used to fuse the feature maps extracted by the context feature extraction branch and the detail mining branch, and adaptively align the transmission line pixels to obtain the aligned feature map. The boundary-aware auxiliary training head includes a boundary-aware module and an original segmentation head. The boundary-aware module is used to extract boundary features from the feature map after downsampling the original image by a factor of 2, then upsamples it and superimposes it with the transmission line detail feature map. The original segmentation head is used to process the superimposed feature map to obtain the boundary feature map. The boundary-aware module adopts a ResNet-like residual structure, designed to consist of a wavelet convolution, a batch normalization layer, and a ReLU activation function in the main path, and residual connections in the branch paths. The detail recovery decoder specifically includes: The primary fusion module is used to unify the channel dimension of the feature maps extracted by the context feature extraction branch and the detail mining branch, and then add them element by element to obtain the primary fused feature. The spatial offset relation learning module is used to generate the following from the initial fusion features: a filter kernel for the low-resolution feature map to smooth global semantics; a filter kernel for the high-resolution feature map to enhance detail boundaries; and pixel-level spatial offset values. The predicted offset corresponds to the coordinates of each pixel; at the same time, the cosine similarity of the adjacent pixels of the primary fusion feature is calculated, and the size of the feature map is made up by padding pixels, thereby obtaining the spatial offset relationship matrix, which is used to reflect the positional correspondence of the transmission line pixels in the feature map. The weighted alignment module is used to weight and combine the spatial offset relation matrix with the feature maps extracted from the original context feature extraction branch and detail mining branch. The feature map of the context feature extraction branch is smoothed with a filter kernel of low-resolution feature map to smooth the global semantics and the position is adjusted according to the spatial offset value. The feature map of the detail mining branch is enhanced with a filter kernel of high-resolution feature map to enhance the detail boundary and the position is adjusted according to the spatial offset value. Finally, the aligned feature map is obtained.

6. The high-precision semantic segmentation system for power transmission lines based on PIDNet according to claim 5, characterized in that: Both the context feature extraction branch and the detail mining branch are composed of residual basic modules; The context feature extraction branch obtains low-resolution feature maps by continuously downsampling the preprocessed image; the detail mining branch maintains the resolution of the preprocessed image.

7. The high-precision semantic segmentation system for power transmission lines based on PIDNet according to claim 5, characterized in that: The bidirectional fusion module includes a high-to-low fusion module and a low-to-high fusion module; among which, The high-to-low fusion module is used to downsample the feature map of the detail mining branch and add it element-wise to the feature map of the context feature extraction branch; The low-to-high fusion module is used to upsample the feature map of the context feature extraction branch and add it element-wise to the feature map of the detail mining branch.

Images