A method and system for remote sensing detection of tree obstacles in power transmission channels based on self-attention

Abstract

This invention relates to the field of power system image processing technology and discloses a self-attention-based remote sensing method for detecting tree obstacles in power transmission channels. The method first acquires extremely high-resolution remote sensing image data, followed by preprocessing such as image mosaicking and sliding window cropping. A sample library is then constructed, and 5% of the image blocks are randomly selected for manual annotation for training. The annotated categories include tree obstacle areas, power facilities, and other background data. A power facility detection network and a tree obstacle detection network are constructed and trained. Tree obstacle detection within the transmission channel is performed, and the detected power lines and other facilities are superimposed and spliced ​​back to the original image size to obtain the overall tree obstacle detection result. This method extracts data using two single-task networks and then overlays and analyzes the results, avoiding the interference of overlapping regions when using a single network to perform multiple classification tasks, as well as the impact of class imbalance on accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of power system image processing, and in particular to a method and system for remote sensing detection of tree obstacles in power transmission channels based on self-attention. Background Art

[0002] Tree barriers are a common safety hazard in the operation of power transmission lines. Contact with branches or trees may cause line short circuits, tripping and other faults, thereby affecting the normal operation of the power system. Through tree barrier inspections, these potential safety hazards can be discovered and dealt with in a timely manner to ensure the safe operation of the transmission lines. In severe weather conditions, trees may fall or break, causing serious damage to the transmission lines. Through tree barrier inspections, these potential threats can be discovered and dealt with in advance to prevent natural disasters from damaging the transmission lines.

[0003] Although tree barrier detection in transmission channels is crucial, the most commonly used technology currently still relies on manual field inspections, which is time-consuming and labor-intensive, and has poor scalability. To overcome these limitations, methods combining deep learning image analysis algorithms with extremely high-resolution remote sensing images have gradually gained attention. Remote sensing imaging has a wide range, which facilitates tree barrier detection. Although tree barrier monitoring technology in transmission channels based on drone lidar has been developed, lidar sensors are expensive and difficult to inspect long lines due to power limitations. Laser point cloud data is easily affected by weather and has poor quality in rain, snow, fog and other weather conditions. In addition, it is limited by Point cloud data is large in volume, and its real-time online processing also has bottlenecks. However, for optical imagery, there is currently a lack of large-scale, high-resolution remote sensing-based image datasets that can be used to train neural networks. This is especially true for the semantic segmentation task of tree barriers in infrastructure. In its application practice in some areas, it also faces the following challenges: First, there is a pixel-level class imbalance problem between the background, vegetation, and power facility categories. Second, in some cases, the boundaries of power facilities and vegetation may overlap with each other in two-dimensional modeling, resulting in inaccurate semantic segmentation. Current technologies do not consider the impact of the cross-overlap of power facilities and fail to identify semantic information such as power lines. Summary of the Invention

[0004] (1) Technical problems solved

[0005] In response to the shortcomings of the existing technology, the present invention provides a method and system for remote sensing detection of tree obstacles in power transmission channels based on self-attention, which has the advantages of avoiding interference in overlapping areas when using one network to implement multiple classification tasks, and solves the above-mentioned technical problems.

[0006] (2) Technical solution

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for remote sensing detection of tree obstacles in power transmission channels based on self-attention, comprising the following steps:

[0008] S1. Acquire remote sensing image data: Collect sub-meter resolution remote sensing images that include the power transmission channel, geo-reference, stitch, and crop different remote sensing images to obtain a remote sensing image that can completely cover the power transmission channel to be inspected;

[0009] S2, remote sensing image preprocessing to obtain image blocks of the same pixel size;

[0010] S3. Based on the image blocks processed in step S2, 5% of the image blocks are randomly selected and labeled by manual visual interpretation of pixel-level annotation.

[0011] S4. Constructing a power facility detection network based on boundary information enhancement to extract the boundaries of power facilities and obtain the area of ​​television facilities;

[0012] S5. Build a tree obstacle detection network based on the global self-attention mechanism to extract pixel-level semantics of tree obstacles;

[0013] S6. The data from step S3 are combined into a data set, and divided into a training set, a test set, and a validation set in a set ratio, and the power facility detection network constructed in step S4 and the tree obstacle detection network constructed in step S5 are trained respectively;

[0014] S7. Input each image block of step S2 into the power facility detection network and tree barrier detection network trained in step S6, input the detection results of the power facility and tree barrier areas respectively, and superimpose and splice them according to the geographic coordinates to obtain the final result.

[0015] As a preferred technical solution of the present invention, the remote sensing image preprocessing in step S2 is specifically as follows: performing uniform color mosaic processing on the remote sensing image in the transmission channel, using a sliding window of 256×256 pixels and an overlap rate of 10% in the horizontal and vertical directions, and finally obtaining image blocks of the same pixel size.

[0016] As a preferred technical solution of the present invention, the categories marked in step S3 include three categories, namely tree barrier areas, power facilities and other backgrounds.

[0017] As a preferred technical solution of the present invention, the power facility detection network based on boundary information enhancement constructed in step S4 includes 13 3×3 convolutional layers, and a combination of 4 maximum pooling layers and 1×1 convolutions are embedded in the convolutional layers to output feature maps of different depths.

[0018] As a preferred technical solution of the present invention, the tree obstacle detection network based on the global self-attention mechanism constructed in step S5 includes an encoder and a decoder;

[0019] The encoder consists of convolutional layers and attention mechanisms;

[0020] The convolutional layer uses two residual modules, each of which includes two 5×5 two-dimensional convolutions;

[0021] The attention mechanism specifically includes: cropping the output feature map of the convolution part into blocks, adding position encoding and category encoding to it, then converting the block vector into a mapping vector of a specified dimension, and finally encoding the features of the mapping vector through 6 Block modules. Each Block module consists of LayerNorm, Multi-HeadAttention, Dropout and MLP Block.

[0022] The specific workflow of the decoder is as follows:

[0023] S5.1. Project the block embedding into a new low-dimensional feature space through a linear layer.

[0024] S5.2, the synthesized embedded feature vector is used to calculate the attention score through two Block modules;

[0025] S5.3. Normalize the attention scores from step S5.2 and then decompose them into two components: block embedding and category embedding.

[0026] S5.4. Calculate the dot product between the block embedding and the category embedding to generate the mask embedding, and normalize the mask embedding.

[0027] S5.5, by adopting bilinear upsampling technology and combining it with the strategy of selecting the category with the maximum probability.

[0028] As a preferred technical solution of the present invention, the ratio set in step S6 is specifically to divide the data set into a training set, a test set and a validation set in a ratio of 7:2:1;

[0029] The training in step S6 includes training of the learning rate, batch size and loss function. The learning rate adopts the Adam decay strategy, the initial learning rate is 1e-3, the batch size is set to 64, and the loss function adopts the Dice loss function.

[0030] The present invention also provides a transmission channel tree obstacle remote sensing detection system based on self-attention, which is based on the above-mentioned transmission channel tree obstacle remote sensing detection method based on self-attention, including a data preprocessing module, a sample library generation module, a neural network training module and a transmission channel tree obstacle detection module;

[0031] The data preprocessing module realizes the loading of transmission channel area vectors, loading of sub-meter resolution remote sensing images, and preprocessing functions of image registration, cropping and mosaicking;

[0032] The sample library generation module is used to generate image blocks of 256×256 pixels, sample annotations and generate label files;

[0033] The neural network training module is used to train the power facility detection network based on boundary information enhancement and the tree obstacle detection network based on the global self-attention mechanism;

[0034] The tree obstacle detection module in the transmission channel is used to input the 256×256 pixel image blocks generated by the sample library into the power facility detection network based on boundary information enhancement and the tree obstacle detection network based on the global self-attention mechanism for prediction, and then stitch the predicted image blocks back into the original image according to the geographic coordinates.

[0035] Compared with the existing technology, the present invention provides a method and system for remote sensing detection of tree obstacles in power transmission channels based on self-attention, which has the following beneficial effects:

[0036] The present invention designs two single-task networks for extraction based on the different characteristics of power facilities such as transmission lines and tree barrier areas, and then overlays and analyzes the results. The network in step S4 focuses on enhancing boundary information. Through the cross-fusion of five layers of deep and shallow features, it avoids excessive blurring of the convolutional neural network for the slender target of the power line, thereby improving the recognition accuracy. The network in step S5 introduces a global attention algorithm to add global features containing information associations of neighborhood image blocks, avoiding excessive attention to local noise details of the tree crown. The method proposed in the present invention involves the extraction of two single-task networks, avoiding the interference of interleaved areas when using one network to implement multiple classification tasks, and the impact of category imbalance on accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Flowchart of the present invention for remote sensing detection of tree obstacles in power transmission channels based on self-attention

[0038] Figure 2 This is the network structure diagram of the power facility detection based on boundary information enhancement of the present invention

[0039] Figure 3 This is the tree obstacle detection network structure diagram based on the global self-attention mechanism of the present invention

[0040] Figure 4 This is a rendering of an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0042] See also Figure 1-Figure 3 A self-attention-based remote sensing detection method for tree obstacles in power transmission channels includes the following steps:

[0043] S1. Acquire remote sensing image data: Collect sub-meter resolution remote sensing images that include the power transmission channel, geo-reference, stitch, and crop different remote sensing images to obtain a remote sensing image that can completely cover the power transmission channel to be inspected;

[0044] S2, remote sensing image preprocessing to obtain image blocks of the same pixel size;

[0045] The remote sensing image preprocessing in step S2 is specifically as follows: performing uniform color mosaic processing on the remote sensing image in the transmission channel, using a sliding window of 256×256 pixels and an overlap rate of 10% in the horizontal and vertical directions, and finally obtaining image blocks of the same pixel size.

[0046] S3. Based on the image blocks processed in step S2, 5% of the image blocks are randomly selected and labeled by manual visual interpretation of pixel-level annotation. The labeled categories include three categories: tree barrier area, power facilities, and other background.

[0047] S4. Constructing a power facility detection network based on boundary information enhancement to extract the boundaries of power facilities and obtain the area of ​​television facilities;

[0048] The constructed power facility detection network based on boundary information enhancement consists of 13 3×3 convolutional layers, and a combination of 4 maximum pooling layers and 1×1 convolutions are embedded in the convolutional layers to output feature maps of different depths;

[0049] like Figure 2As shown, the network generates edge images of five different scales through a multi-scale, multi-level convolutional structure. These edge images continue to pass through convolution and pooling operators at different scales and directions to generate high-order semantic features. The semantic features of the five different levels of edge enhancement are fused together. After passing through the last convolution layer, the final semantic information map is obtained. The semantic information map has the same size as the input image block. The predicted value of each pixel on the information map represents the probability that the pixel is located at the boundary of power facilities such as transmission lines. Its value is between 0 and 1. Using the discriminant function and giving a threshold T, the boundary of facilities such as power lines can be extracted and the area of ​​power facilities can be obtained. In this embodiment, the threshold T is 0.6.

[0050] S5. Build a tree obstacle detection network based on the global self-attention mechanism to extract pixel-level semantics of tree obstacles;

[0051] The tree obstacle detection network based on the global self-attention mechanism constructed in step S5 includes an encoder and a decoder;

[0052] The encoder consists of convolutional layers and attention mechanisms;

[0053] In the embodiment, the convolutional layer uses two residual modules, each of which includes two 5×5 two-dimensional convolutions. In the attention mechanism, the output feature map of the convolution part is first cropped into blocks. Based on this, the feature map is "flattened" into a block vector and added with a position code (1-1024) and a category code (cls). Then, a linear mapping layer is used to convert the block vector into a mapping vector of a specified dimension. Next, the mapping vector is encoded through six block modules. Each block module consists of LayerNorm, Multi-HeadAttention, Dropout, and MLP Block.

[0054] The specific workflow of the decoder is as follows:

[0055] S5.1. Project the block embedding into a new low-dimensional feature space through a linear layer.

[0056] S5.2, the synthesized embedded feature vector is used to calculate the attention score through two Block modules;

[0057] S5.3. Normalize the attention scores from step S5.2 and then decompose them into two components: block embedding and category embedding.

[0058] S5.4. Calculate the dot product between the block embedding and the category embedding to generate the mask embedding, and normalize the mask embedding.

[0059] S5.5, by using bilinear upsampling technology and combining it with the strategy of selecting the maximum probability category

[0060] S6. The data from step S3 are combined into a data set, and divided into a training set, a test set, and a validation set in a set ratio, and the power facility detection network constructed in step S4 and the tree obstacle detection network constructed in step S5 are trained respectively;

[0061] Set hyperparameters including learning rate, batch size, and loss function. Split the dataset into training, test, and validation sets in a ratio of 7:2:1. Train the networks constructed in steps S4 and S5 separately. In this example, the learning rate uses the Adam decay strategy, with an initial learning rate of 1e-3; the batch size is set to 64, and the loss function uses the Dice loss function.

[0062] S7. Input each image block of step S2 into the power facility detection network and tree barrier detection network trained in step S6, input the detection results of the power facility and tree barrier areas respectively, and superimpose and splice them according to the geographic coordinates to obtain the final result.

[0063] For the high-resolution remote sensing image to be extracted, each image block obtained in step S2 is input into the two network models trained in step S6, and the detection results of the power facilities and tree barrier areas are respectively input. Then, the detected power lines and other facilities results and the tree barrier areas are superimposed to facilitate comprehensive analysis. Finally, the cropped image blocks are spliced ​​back to the original image according to the geographic coordinates to obtain the power transmission lines and other settings in the transmission channel and the tree barrier areas. Since the missed detection of power facilities usually has a greater impact and tree barriers often have false detections, in this embodiment, when splicing back to the original image, the detection results of the power facilities in the overlapping areas are taken as the union, and the detection results of the tree barriers are taken as the intersection;

[0064] Figure 4 This is a rendering of an embodiment of the present invention. On a dataset of UAV images of power transmission corridors (VEPL: https: / / doi.org / 10.5281 / zenodo.7800234), this embodiment achieves optimal accuracy.

[0065]

[0066]

[0067] like Figure 4 As shown, the present invention also provides a transmission channel tree obstacle remote sensing detection system based on self-attention, based on the above-mentioned transmission channel tree obstacle remote sensing detection method based on self-attention, including a data preprocessing module, a sample library generation module, a neural network training module and a transmission channel tree obstacle detection module;

[0068] The data preprocessing module realizes the loading of transmission channel area vectors, loading of sub-meter resolution remote sensing images, and preprocessing functions of image registration, cropping and mosaicking;

[0069] The sample library generation module is used to generate image blocks of 256×256 pixels, sample annotations, and generate label files;

[0070] The neural network training module is used to train the power facility detection network based on boundary information enhancement and the tree obstacle detection network based on the global self-attention mechanism;

[0071] The tree obstacle detection module within the transmission channel is used to input the 256×256 pixel image blocks generated by the sample library into the power facility detection network based on boundary information enhancement and the tree obstacle detection network based on the global self-attention mechanism for prediction, and then stitch the predicted image blocks back into the original image according to the geographic coordinates.

[0072] Compared with the existing technology, the present invention provides a method and system for remote sensing detection of tree obstacles in power transmission channels based on self-attention, which has the following beneficial effects:

[0073] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A self-attention-based remote sensing method for detecting tree obstacles in power transmission channels, characterized by: The following steps are involved: S1. Acquire remote sensing image data: Collect sub-meter resolution remote sensing images that include the power transmission channel, geo-reference, stitch, and crop different remote sensing images to obtain a remote sensing image that can completely cover the power transmission channel to be inspected; S2, remote sensing image preprocessing to obtain image blocks of the same pixel size; S3. Based on the image blocks processed in step S2, 5% of the image blocks are randomly selected and labeled by manual visual interpretation of pixel-level annotation. S4. Constructing a power facility detection network based on boundary information enhancement to extract the boundaries of power facilities and obtain the area of ​​television facilities; S5. Build a tree obstacle detection network based on the global self-attention mechanism to extract pixel-level semantics of tree obstacles; The tree obstacle detection network based on the global self-attention mechanism constructed in step S5 includes an encoder and a decoder; The encoder consists of convolutional layers and attention mechanisms; The convolutional layer uses two residual modules, each of which includes two 5×5 two-dimensional convolutions; The attention mechanism specifically includes: cropping the output feature map of the convolution part into blocks, adding position encoding and category encoding to it, then converting the block vector into a mapping vector of a specified dimension, and finally encoding the features of the mapping vector through 6 Block modules. Each Block module consists of LayerNorm, Multi-HeadAttention, Dropout and MLP Block. The specific workflow of the decoder is as follows: S5.1, project the block embedding into a new low-dimensional feature space through a linear layer; S5.2, the synthesized embedded feature vector is used to calculate the attention score through two Block modules; S5.

3. Normalize the attention scores from step S5.2 and then decompose them into two components: block embedding and category embedding. S5.

4. Calculate the dot product between the block embedding and the category embedding to generate the mask embedding, and normalize the mask embedding. S5.5, by adopting bilinear upsampling technology and combining it with the strategy of selecting the maximum probability category; S6. The data from step S3 are combined into a data set, and divided into a training set, a test set, and a validation set in a set ratio, and the power facility detection network constructed in step S4 and the tree obstacle detection network constructed in step S5 are trained respectively; S7. Input each image block of step S2 into the power facility detection network and tree barrier detection network trained in step S6, input the detection results of the power facility and tree barrier areas respectively, and superimpose and splice them according to the geographic coordinates to obtain the final result.

2. The method for remote sensing detection of tree obstacles in power transmission channels based on self-attention according to claim 1, characterized in that: The remote sensing image preprocessing in step S2 is specifically as follows: performing uniform color mosaic processing on the remote sensing image in the transmission channel, using a sliding window of 256×256 pixels and an overlap rate of 10% in the horizontal and vertical directions, and finally obtaining image blocks of the same pixel size.

3. The method for remote sensing detection of tree obstacles in power transmission channels based on self-attention according to claim 1, characterized in that: The categories marked in step S3 include three categories, namely tree barrier area, power facilities and other backgrounds.

4. The method for remote sensing detection of tree obstacles in power transmission channels based on self-attention according to claim 1, characterized in that: The electric power facility detection network based on boundary information enhancement constructed in step S4 includes 13 3×3 convolutional layers, and a combination of 4 maximum pooling layers and 1×1 convolutions is embedded in the convolutional layers to output feature maps of different depths.

5. The method for remote sensing detection of tree obstacles in power transmission channels based on self-attention according to claim 1, characterized in that: The ratio set in step S6 is specifically to divide the data set into a training set, a test set and a validation set in a ratio of 7:2:1; The training in step S6 includes training of the learning rate, batch size and loss function. The learning rate adopts the Adam decay strategy, the initial learning rate is 1e-3, the batch size is set to 64, and the loss function adopts the Dice loss function.

6. A system for remote sensing detection of tree obstacles in power transmission channels based on self-attention, based on the method for remote sensing detection of tree obstacles in power transmission channels based on self-attention according to any one of claims 1 to 5, characterized in that: It includes data preprocessing module, sample library generation module, neural network training module and tree obstacle detection module in transmission channel; The data preprocessing module realizes the loading of transmission channel area vectors, loading of sub-meter resolution remote sensing images, and preprocessing functions of image registration, cropping and mosaicking; The sample library generation module is used to generate image blocks of 256×256 pixels, sample annotations and generate label files; The neural network training module is used to train the power facility detection network based on boundary information enhancement and the tree obstacle detection network based on the global self-attention mechanism; The tree obstacle detection module in the transmission channel is used to input the 256×256 pixel image blocks generated by the sample library into the power facility detection network based on boundary information enhancement and the tree obstacle detection network based on the global self-attention mechanism for prediction, and then stitch the predicted image blocks back into the original image according to the geographic coordinates.

Images