Power transmission channel tree obstacle remote sensing detection method and system based on self-attention

Abstract

The invention relates to the technical field of power system image processing, and discloses a self-attention-based power transmission channel tree obstacle remote sensing detection method, which comprises the following steps of: firstly, acquiring extremely high resolution remote sensing image data, then preprocessing image mosaic, sliding window cutting and the like, and constructing a sample library at the same time; 5% of image blocks are randomly selected to be manually marked for training, the marked categories comprise tree obstacle areas, electric power facilities and other backgrounds, an electric power facility detection network and a tree obstacle detection network are constructed and trained, tree obstacle detection is carried out in a power transmission channel, and tree obstacle detection is carried out in the power transmission channel. And overlapping and splicing the detected facility results such as the power line and the tree obstacle area to the original image size to obtain an overall tree obstacle detection result. According to the method, extraction is carried out through the two single-task networks, and then results are subjected to overlay analysis, so that the interference of an interlaced region when a multi-classification task is realized by using one network and the influence of class imbalance on precision are avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of power system image processing, and in particular to a method and system for remote sensing detection of tree obstacles in a power transmission channel based on self-attention. Background Art

[0002] Tree barriers are a common safety hazard in the operation of power transmission lines. The contact of branches or trees may cause line short circuits, tripping and other faults, which in turn affect the normal operation of the power system. Through tree barrier inspections, these potential safety hazards can be discovered and dealt with in a timely manner to ensure the safe operation of the transmission lines. Under severe weather conditions, trees may fall or break, causing serious damage to the transmission lines. Through tree barrier inspections, these potential threats can be discovered and dealt with in advance to prevent natural disasters from damaging the transmission lines.

[0003] Although tree obstacle detection in transmission channels is crucial, the most commonly used technology currently still relies on manual field inspections, which is time-consuming and labor-intensive, and has poor scalability. In order to overcome these limitations, the method of combining deep learning image analysis algorithms with extremely high-resolution remote sensing images has gradually gained attention. Remote sensing imaging has a wide range, which facilitates tree obstacle detection. Although tree obstacle monitoring technology in transmission channels based on drone lidar has been developed, lidar sensors are expensive and difficult to inspect long lines due to power limitations. In addition, laser point cloud data is easily affected by weather and has poor quality under weather conditions such as rain, snow and fog. In addition, it is limited by Point cloud data is large in volume, and its real-time online processing also has bottlenecks. However, for optical images, there is currently a lack of large-scale high-resolution remote sensing-based image datasets that can be used to train neural networks, especially for the semantic segmentation task of tree barriers in infrastructure. In the application practice in some areas, it is also subject to the following challenges: First, there is a pixel-level class imbalance problem between the background, vegetation, and power facility categories. Second, in some cases, the boundaries of power facilities and vegetation may overlap with each other in two-dimensional modeling, resulting in inaccurate semantic segmentation. The current technology does not consider the impact of the cross-overlap of power facilities, and does not identify semantic information such as power lines. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the shortcomings of the prior art, the present invention provides a method and system for remote sensing detection of tree obstacles in power transmission channels based on self-attention, which has the advantages of avoiding interference from interleaved areas when using one network to implement multiple classification tasks, and solves the above-mentioned technical problems.

[0006] (II) Technical solution

[0007] To achieve the above object, the present invention provides the following technical solution: a method for remote sensing detection of tree obstacles in power transmission channels based on self-attention, comprising the following steps:

[0008] S1. Acquiring remote sensing image data: Collecting sub-meter resolution remote sensing images containing the power transmission channel, geo-referencing, splicing and cropping different remote sensing images to obtain remote sensing images that can completely cover the power transmission channel to be detected;

[0009] S2, remote sensing image preprocessing to obtain image blocks of the same pixel size;

[0010] S3, based on the image blocks processed in step S2, randomly select 5% of the image blocks, and make label samples by manually visually interpreting pixel-level annotation;

[0011] S4. Constructing a power facility detection network based on boundary information enhancement to extract the boundaries of power facilities and obtain the area of ​​television facilities;

[0012] S5. Build a tree obstacle detection network based on the global self-attention mechanism to extract pixel-level semantics of tree obstacles;

[0013] S6, forming a data set from the data in step S3, and dividing it into a training set, a test set, and a validation set in a set ratio, and respectively training the power facility detection network constructed in step S4 and the tree obstacle detection network constructed in step S5;

[0014] S7, input each image block of step S2 into the power facility detection network and tree barrier detection network trained in step S6, input the detection results of the power facility and tree barrier areas respectively, and superimpose and splice them according to the geographic coordinates to obtain the final result.

[0015] As a preferred technical solution of the present invention, the remote sensing image preprocessing in step S2 is specifically: performing uniform color mosaic processing on the remote sensing image in the transmission channel, with a sliding window of 256×256 pixels and an overlap rate of 10% in the horizontal and vertical directions, and finally obtaining image blocks of the same pixel size.

[0016] As a preferred technical solution of the present invention, the categories marked in step S3 include three categories, namely tree barrier areas, power facilities and other backgrounds.

[0017] As a preferred technical solution of the present invention, the power facility detection network based on boundary information enhancement constructed in step S4 includes 13 3×3 convolutional layers, and a combination of 4 maximum pooling layers and 1×1 convolutions are embedded in the convolutional layers to output feature maps of different depths.

[0018] As a preferred technical solution of the present invention, the tree obstacle detection network based on the global self-attention mechanism constructed in step S5 includes an encoder and a decoder;

[0019] The encoder consists of a convolutional layer and an attention mechanism;

[0020] The convolution layer uses two residual modules, each of which includes two 5×5 two-dimensional convolutions;

[0021] The attention mechanism specifically includes: cutting the output feature map of the convolution part into blocks, adding position encoding and category encoding to it, then converting the block vector into a mapping vector of a specified dimension, and finally encoding the feature of the mapping vector through 6 Block modules, each of which is composed of LayerNorm, Multi-HeadAttention, Dropout and MLP Block;

[0022] The specific working process of the decoder is as follows:

[0023] S5.1, project the block embedding into a new low-dimensional feature space through a linear layer;

[0024] S5.2, the synthetic embedded feature vector is used to calculate the attention score through 2 Block modules;

[0025] S5.3. Normalize the attention scores in step S5.2 and then decompose them into two components: block embedding and category embedding.

[0026] S5.4, calculating the dot product between the block embedding and the category embedding, generating the mask embedding, and normalizing the mask embedding;

[0027] S5.5, by adopting bilinear upsampling technology and combining it with the strategy of selecting the maximum probability category.

[0028] As a preferred technical solution of the present invention, the ratio set in step S6 is specifically to divide the data set into a training set, a test set and a validation set in a ratio of 7:2:1;

[0029] The training in step S6 includes training of the learning rate, batch size and loss function. The learning rate adopts the Adam decay strategy, the initial learning rate is 1e-3, the batch size is set to 64, and the loss function adopts the Dice loss function.

[0030] The present invention also provides a transmission channel tree obstacle remote sensing detection system based on self-attention, based on the above-mentioned transmission channel tree obstacle remote sensing detection method based on self-attention, including a data preprocessing module, a sample library generation module, a neural network training module and a transmission channel tree obstacle detection module;

[0031] The data preprocessing module realizes the loading of transmission channel area vectors, the loading of sub-meter resolution remote sensing images, and the preprocessing functions of image registration, cropping and mosaicking;

[0032] The sample library generation module is used to generate image blocks of 256×256 pixels, sample annotations, and generate label files;

[0033] The neural network training module is used to train the power facility detection network based on boundary information enhancement and the tree obstacle detection network based on the global self-attention mechanism;

[0034] The tree obstacle detection module in the transmission channel is used to input the 256×256 pixel image blocks generated by the sample library into the power facility detection network based on boundary information enhancement and the tree obstacle detection network based on the global self-attention mechanism for prediction, and splice the predicted image blocks back to the original image according to the geographic coordinates.

[0035] Compared with the prior art, the present invention provides a method and system for remote sensing detection of tree obstacles in power transmission channels based on self-attention, which has the following beneficial effects:

[0036] The present invention designs two single-task networks for extraction according to the different characteristics of power facilities such as transmission lines and tree barrier areas, and then overlays and analyzes the results. The network in step S4 focuses on the enhancement of boundary information. Through the cross-fusion of 5 layers of deep and shallow features, the convolutional neural network avoids excessive blurring of the slender target of the power line, thereby improving the recognition accuracy. The network in step S5 introduces a global attention algorithm to add global features related to neighborhood image block information, thereby avoiding excessive attention to local noise details of the tree crown. The method proposed in the present invention involves the extraction of two single-task networks, thereby avoiding the interference of interlaced areas when using one network to realize multiple classification tasks, and the influence of category imbalance on accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Flow chart of the remote sensing detection of tree obstacles in power transmission channels based on self-attention of the present invention

[0038] Figure 2 This is a diagram of the electric power facility detection network structure based on boundary information enhancement of the present invention

[0039] Figure 3 This is the tree obstacle detection network structure diagram based on the global self-attention mechanism of the present invention

[0040] Figure 4 This is a rendering of an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] See also Figure 1-Figure 3 , a transmission channel tree obstacle remote sensing detection method based on self-attention, comprising the following steps:

[0043] S1. Acquiring remote sensing image data: Collecting sub-meter resolution remote sensing images containing the power transmission channel, geo-referencing, splicing and cropping different remote sensing images to obtain remote sensing images that can completely cover the power transmission channel to be detected;

[0044] S2, remote sensing image preprocessing to obtain image blocks of the same pixel size;

[0045] The remote sensing image preprocessing in step S2 is specifically as follows: performing uniform color mosaic processing on the remote sensing image in the transmission channel, using a sliding window of 256×256 pixels and an overlap rate of 10% in the horizontal and vertical directions, and finally obtaining image blocks of the same pixel size.

[0046] S3, based on the image blocks processed in step S2, randomly select 5% of the image blocks, and make label samples by manually visually interpreting pixel-level annotation. The annotated categories include three categories, namely, tree barrier area, power facilities and other backgrounds;

[0047] S4. Constructing a power facility detection network based on boundary information enhancement to extract the boundaries of power facilities and obtain the area of ​​television facilities;

[0048] The constructed power facility detection network based on boundary information enhancement includes 13 3×3 convolutional layers, and a combination of 4 maximum pooling layers and 1×1 convolutions are embedded in the convolutional layers to output feature maps of different depths;

[0049] like Figure 2As shown, the network generates edge images of 5 different scales through a multi-scale, multi-level convolution structure. These edge images continue to generate high-order semantic features through convolution and pooling operators at different scales and directions, and the semantic features of the edges of 5 different levels are fused together. After passing through the last convolution layer, the final semantic information map is obtained. The semantic information map has the same size as the input image block. The predicted value of each pixel on the information map represents the probability that the pixel is located at the boundary of power facilities such as transmission lines, and its value is between 0 and 1. By using the discriminant function and giving a threshold T, the extraction of the boundary of facilities such as power lines can be completed, and the area of ​​power facilities can be obtained; in this embodiment, the threshold T is 0.6;

[0050] S5. Build a tree obstacle detection network based on the global self-attention mechanism to extract pixel-level semantics of tree obstacles;

[0051] The tree obstacle detection network based on the global self-attention mechanism constructed in step S5 includes an encoder and a decoder;

[0052] The encoder consists of convolutional layers and attention mechanisms;

[0053] In the embodiment, the convolution layer uses two residual modules, each of which includes two 5×5 two-dimensional convolutions; attention mechanism part: first, the output feature map of the convolution part is cropped into blocks, based on which, the feature map is "flattened" into a block vector, and the position code (1-1024) and category code (cls) are added, and then the block vector is converted into a mapping vector of a specified dimension through a linear mapping layer, and then the mapping vector is encoded through 6 Block modules, each of which is composed of LayerNorm, Multi-HeadAttention, Dropout and MLP Block.

[0054] The specific workflow of the decoder is as follows:

[0055] S5.1, project the block embedding into a new low-dimensional feature space through a linear layer;

[0056] S5.2, the synthetic embedded feature vector is used to calculate the attention score through 2 Block modules;

[0057] S5.3. Normalize the attention scores in step S5.2 and then decompose them into two components: block embedding and category embedding.

[0058] S5.4, calculating the dot product between the block embedding and the category embedding, generating the mask embedding, and normalizing the mask embedding;

[0059] S5.5. By using bilinear upsampling technology and combining it with the strategy of selecting the maximum probability category

[0060] S6, forming a data set from the data in step S3, and dividing it into a training set, a test set, and a validation set in a set ratio, and respectively training the power facility detection network constructed in step S4 and the tree obstacle detection network constructed in step S5;

[0061] Set hyperparameters including learning rate, batch size, loss function, etc., divide the data set into training set, test set and validation set in a ratio of 7:2:1, and train the networks constructed in step S4 and step S5 respectively. In this embodiment, the learning rate adopts the Adam decay strategy, and the initial learning rate is 1e-3; the batch size is set to 64, and the loss function adopts the Dice loss function;

[0062] S7, input each image block of step S2 into the power facility detection network and tree barrier detection network trained in step S6, input the detection results of the power facility and tree barrier areas respectively, and superimpose and splice them according to the geographic coordinates to obtain the final result.

[0063] For the high-resolution remote sensing image to be extracted, each image block obtained in step S2 is input into the two network models trained in step S6, and the detection results of the power facilities and the tree barrier area are respectively input, and then the detected power lines and other facilities results and the tree barrier area are superimposed for comprehensive analysis. Finally, the cropped image blocks are spliced ​​back to the original image according to the geographic coordinates to obtain the settings of the power lines and the tree barrier area in the transmission channel. In view of the fact that the missed detection of power facilities usually has a greater impact, and the tree barrier often has false detection, in this embodiment, when splicing back to the original image, the detection results of the power facilities in the overlapping area are taken as the union, and the detection results of the tree barrier are taken as the intersection;

[0064] Figure 4 This is a rendering of an embodiment of the present invention. On the UAV image dataset of power transmission channel areas (VEPL: https: / / doi.org / 10.5281 / zenodo.7800234), this embodiment achieves the best accuracy.

[0065]

[0066]

[0067] like Figure 4 As shown, the present invention also provides a transmission channel tree obstacle remote sensing detection system based on self-attention, based on the above-mentioned transmission channel tree obstacle remote sensing detection method based on self-attention, including a data preprocessing module, a sample library generation module, a neural network training module and a transmission channel tree obstacle detection module;

[0068] The data preprocessing module realizes the loading of transmission channel area vectors, loading of sub-meter resolution remote sensing images, and preprocessing functions of image registration, cropping and mosaicking;

[0069] The sample library generation module is used to generate image blocks of 256×256 pixels, sample annotations, and generate label files;

[0070] The neural network training module is used to train the power facility detection network based on boundary information enhancement and the tree obstacle detection network based on the global self-attention mechanism;

[0071] The tree obstacle detection module in the transmission channel is used to input the 256×256 pixel image blocks generated by the sample library into the power facility detection network based on boundary information enhancement and the tree obstacle detection network based on the global self-attention mechanism for prediction, and then splice the predicted image blocks back to the original image according to the geographic coordinates.

[0072] Compared with the prior art, the present invention provides a method and system for remote sensing detection of tree obstacles in power transmission channels based on self-attention, which has the following beneficial effects:

[0073] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for remote sensing detection of tree obstacles in power transmission channels based on self-attention, characterized in that: The following steps are involved: S1. Acquiring remote sensing image data: Collecting sub-meter resolution remote sensing images containing the power transmission channel, geo-referencing, splicing and cropping different remote sensing images to obtain remote sensing images that can completely cover the power transmission channel to be detected; S2, remote sensing image preprocessing to obtain image blocks of the same pixel size; S3, based on the image blocks processed in step S2, randomly select 5% of the image blocks, and make label samples by manually visually interpreting pixel-level annotation; S4. Constructing a power facility detection network based on boundary information enhancement to extract the boundaries of power facilities and obtain the area of ​​television facilities; S5. Build a tree obstacle detection network based on the global self-attention mechanism to extract pixel-level semantics of tree obstacles; S6, forming a data set from the data in step S3, and dividing it into a training set, a test set, and a validation set in a set ratio, and respectively training the power facility detection network constructed in step S4 and the tree obstacle detection network constructed in step S5; S7, input each image block of step S2 into the power facility detection network and tree barrier detection network trained in step S6, input the detection results of the power facility and tree barrier areas respectively, and superimpose and splice them according to the geographic coordinates to obtain the final result.

2. The method for remote sensing detection of tree obstacles in power transmission channels based on self-attention according to claim 1, characterized in that: The remote sensing image preprocessing in step S2 is specifically as follows: performing uniform color mosaic processing on the remote sensing image in the transmission channel, using a sliding window of 256×256 pixels and an overlap rate of 10% in the horizontal and vertical directions, and finally obtaining image blocks of the same pixel size.

3. The method for remote sensing detection of tree obstacles in power transmission channels based on self-attention according to claim 1, characterized in that: The categories marked in step S3 include three categories, namely tree barrier area, power facilities and other backgrounds.

4. The method for remote sensing detection of tree obstacles in power transmission channels based on self-attention according to claim 1, characterized in that: The electric power facility detection network based on boundary information enhancement constructed in the step S4 includes 13 3×3 convolutional layers, and a combination of 4 maximum pooling layers and 1×1 convolutions are embedded in the convolutional layers to output feature maps of different depths.

5. The method for remote sensing detection of tree obstacles in power transmission channels based on self-attention according to claim 4, characterized in that: The tree obstacle detection network based on the global self-attention mechanism constructed in step S5 includes an encoder and a decoder; The encoder consists of a convolutional layer and an attention mechanism; The convolution layer uses two residual modules, each of which includes two 5×5 two-dimensional convolutions; The attention mechanism specifically includes: cutting the output feature map of the convolution part into blocks, adding position encoding and category encoding to it, then converting the block vector into a mapping vector of a specified dimension, and finally encoding the feature of the mapping vector through 6 Block modules, each of which is composed of LayerNorm, Multi-HeadAttention, Dropout and MLP Block; The specific working process of the decoder is as follows: S5.1, project the block embedding into a new low-dimensional feature space through a linear layer; S5.2, the synthetic embedded feature vector is used to calculate the attention score through 2 Block modules; S5.

3. Normalize the attention scores in step S5.2 and then decompose them into two components: block embedding and category embedding. S5.4, calculating the dot product between the block embedding and the category embedding, generating the mask embedding, and normalizing the mask embedding; S5.5, by adopting bilinear upsampling technology and combining it with the strategy of selecting the maximum probability category.

6. The method for remote sensing detection of tree obstacles in power transmission channels based on self-attention according to claim 1, characterized in that: The ratio set in step S6 is specifically to divide the data set into a training set, a test set and a validation set in a ratio of 7:2:1; The training in step S6 includes training of the learning rate, batch size and loss function. The learning rate adopts the Adam decay strategy, the initial learning rate is 1e-3, the batch size is set to 64, and the loss function adopts the Dice loss function.

7. A transmission channel tree obstacle remote sensing detection system based on self-attention, based on a transmission channel tree obstacle remote sensing detection method based on self-attention according to any one of claims 1 to 6, characterized in that: It includes data preprocessing module, sample library generation module, neural network training module and tree obstacle detection module in power transmission channel; The data preprocessing module realizes the loading of transmission channel area vectors, the loading of sub-meter resolution remote sensing images, and the preprocessing functions of image registration, cropping and mosaicking; The sample library generation module is used to generate image blocks of 256×256 pixels, sample annotations, and generate label files; The neural network training module is used to train the power facility detection network based on boundary information enhancement and the tree obstacle detection network based on the global self-attention mechanism; The tree obstacle detection module in the transmission channel is used to input the 256×256 pixel image blocks generated by the sample library into the power facility detection network based on boundary information enhancement and the tree obstacle detection network based on the global self-attention mechanism for prediction, and splice the predicted image blocks back to the original image according to the geographic coordinates.

Images