Power transmission line hidden danger automatic extraction method and device

Abstract

The invention discloses a method and a device for automatically extracting hidden dangers of a power transmission line. The method comprises the following steps: acquiring a remote sensing image; preprocessing the remote sensing image, and constructing a sample data set based on the preprocessed remote sensing image; training and verifying the remote sensing interpretation model by using the sample data set to obtain a trained remote sensing interpretation model; the method comprises the following steps: performing feature extraction on a remote sensing image by adopting a Swin Transform encoder module to generate an encoding feature map; a CBAM attention mechanism module and a PPM module are adopted to carry out feature weighting and fusion on the fourth coded feature map, and a fifth coded feature map is generated; and performing feature extraction and feature fusion on the first coded feature map, the second coded feature map, the third coded feature map and the fifth coded feature map by adopting an FPN module, and outputting a target extraction image. The method is used for extracting the hidden dangers of the power transmission line, and is high in recognition precision and efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of power transmission line hidden danger extraction, and in particular to a method and device for automatically extracting hidden dangers of power transmission lines. Background Art

[0002] The construction of transmission lines is an important task to ensure the normal supply of electricity. Due to the particularity of transmission projects, transmission lines are often built in agricultural areas. There are large areas of agricultural covering materials in agricultural areas, such as ground films, greenhouses, etc. These agricultural covering materials can easily damage transmission lines when blown by wind. It can be seen that agricultural covering materials have become one of the important hidden dangers of transmission lines. Therefore, it is essential to identify and detect the hidden dangers of agricultural covering materials in transmission lines.

[0003] At present, image processing methods and recognition models are commonly used to identify and detect agricultural covering material hazards in transmission lines. However, different types of agricultural covering materials show different characteristics in images. These characteristics are affected by many factors, including the type of material, the covered terrain, seasonal changes, and lighting conditions. For example, greenhouses usually appear in regular geometric shapes, while ground films may show different spectral characteristics depending on the covering method and material type. These differences limit the recognition effect based on traditional image processing methods and make it difficult to meet the requirements of high accuracy and high efficiency.

[0004] Moreover, the recognition model uses remote sensing images. In large-scale remote sensing images, agricultural cover materials are relatively small and often mixed with the surrounding natural environment and other man-made structures, making them difficult to identify. This not only increases the difficulty of the recognition model, but also requires the recognition model to have stronger feature extraction capabilities.

[0005] In addition, the sample imbalance problem is particularly prominent in the recognition of agricultural covering materials in remote sensing images. In practical applications, other landforms are more common than agricultural covering materials, resulting in a serious imbalance in the number of samples of agricultural covering materials in the training data set of the recognition model. This imbalance will affect the training effect of the recognition model, making the recognition model ineffective in identifying the types of agricultural covering materials. Summary of the invention

[0006] Based on this, it is necessary to propose a method and device for automatically extracting hidden dangers of transmission lines in response to the above problems.

[0007] First aspect

[0008] A method for automatically extracting hidden dangers of a power transmission line, the method comprising:

[0009] Obtain satellite remote sensing images of power transmission lines;

[0010] Preprocessing the satellite remote sensing image to obtain a preprocessed satellite remote sensing image;

[0011] Based on the preprocessed satellite remote sensing images, constructing a sample data set for model training;

[0012] The remote sensing interpretation model is trained and verified using the sample data set to obtain a trained remote sensing interpretation model; wherein the remote sensing interpretation model includes a Swin Transformer encoder module, a CBAM attention mechanism module and an UperNet decoder module, the UperNet decoder module includes a PPM module and an FPN module, and the loss function used by the remote sensing interpretation model is a focal binary cross entropy loss function;

[0013] Inputting the satellite remote sensing image of the power transmission line to be extracted into the trained remote sensing interpretation model;

[0014] The Swin Transformer encoder module is used to perform multi-level feature extraction on the satellite remote sensing image of the power transmission line to be extracted, and a coding feature map is generated; wherein the coding feature map includes a first coding feature map, a second coding feature map, a third coding feature map and a fourth coding feature map;

[0015] Using the CBAM attention mechanism module and the PPM module to perform feature weighting and feature fusion on the fourth encoding feature map to generate a fifth encoding feature map;

[0016] The FPN module is used to perform further feature extraction and feature fusion on the first coding feature map, the second coding feature map, the third coding feature map and the fifth coding feature map, and output a target extracted image.

[0017] Preferably, the focal binary cross entropy loss function is specifically:

[0018]

[0019] Where N is the total number of samples, t i is the proportion of correctly detected true value pixels in the i-th sample, is the adjusted predicted probability for the correct category, γ is the adjustment parameter, is the weight for each category.

[0020] Preferably, the Swin Transformer encoder module is used to perform multi-level feature extraction on the satellite remote sensing image of the power transmission line to be extracted to generate a coding feature map, specifically including:

[0021] The Patch Partition layer is used to segment the satellite remote sensing image of the power transmission line to be extracted to obtain the segmented data;

[0022] Using a Liner Embedding layer to perform feature mapping on the segmented data to obtain feature mapped data;

[0023] Inputting the feature mapped data into a first Swin Transformer block to generate a first encoded feature map;

[0024] Down-sampling the first coding feature map using a first Patch Merging layer to obtain down-sampled data corresponding to the first Patch Merging layer;

[0025] Input the downsampled data corresponding to the first Patch Merging layer into a second Swin Transformer block to generate a second encoded feature map;

[0026] Down-sampling the second encoding feature map using a second Patch Merging layer to obtain down-sampled data corresponding to the second Patch Merging layer;

[0027] Input the downsampled data corresponding to the second Patch Merging layer into a third Swin Transformer block to generate a third encoded feature map;

[0028] Down-sampling the third encoding feature map using a third Patch Merging layer to obtain down-sampled data corresponding to the third Patch Merging layer;

[0029] The downsampled data corresponding to the third Patch Merging layer is input into the fourth Swin Transformer block to generate a fourth encoded feature map.

[0030] Preferably, the adopting the CBAM attention mechanism module and the PPM module to perform feature weighting and feature fusion on the fourth coding feature map to generate a fifth coding feature map specifically includes:

[0031] Input the fourth encoded feature map into the CBAM attention mechanism module, and output the feature map corresponding to the CBAM attention mechanism module;

[0032] Input the feature map corresponding to the CBAM attention mechanism module into the PPM module;

[0033] Using pooling layers of different sizes to pool the feature map corresponding to the CBAM attention mechanism module to obtain a pooled feature map;

[0034] Using an upsampling layer to upsample the pooled feature map to obtain an upsampled feature map;

[0035] The upsampled feature map is fused with the feature map corresponding to the CBAM attention mechanism module to generate a fifth encoded feature map.

[0036] Preferably, the CBAM attention mechanism module includes a channel attention module and a spatial attention module, and the inputting of the fourth encoded feature map into the CBAM attention mechanism module and the outputting of the feature map corresponding to the CBAM attention mechanism module specifically include:

[0037] Obtaining the dimensions of the fourth encoded feature map; wherein the dimensions include the number of channels, height, and width;

[0038] Inputting the fourth encoded feature map into a channel attention module;

[0039] Using a pooling layer to perform global average pooling on the fourth encoded feature map to obtain a feature map after global average pooling;

[0040] Using a first fully connected layer to process the feature map after global average pooling, to obtain an output feature map of the first fully connected layer;

[0041] Using a second fully connected layer to process the output feature map of the first fully connected layer to obtain an output feature map of the second fully connected layer;

[0042] Normalizing the output feature map of the second fully connected layer using an activation function to obtain a channel attention weight;

[0043] Multiplying the channel attention weight by the fourth encoded feature map, and outputting a channel attention weighted feature map;

[0044] Inputting the channel attention weighted feature map into the spatial attention module;

[0045] Using the first convolutional layer to process the feature map after the channel attention weighting, to obtain an output feature map of the first convolutional layer;

[0046] Using a second convolutional layer to process the output feature map of the first convolutional layer to obtain an output feature map of the second convolutional layer;

[0047] Normalizing the output feature map of the second convolutional layer using an activation function to obtain a spatial attention weight;

[0048] Multiply the spatial attention weight by the channel attention weighted feature map, and output the feature map corresponding to the CBAM attention mechanism module.

[0049] Preferably, the adopting of the FPN module to further extract and fuse the first coding feature map, the second coding feature map, the third coding feature map and the fifth coding feature map, and outputting the target extracted image specifically includes:

[0050] Processing the fifth encoding feature map using a 1x1 convolutional layer to generate a fifth decoding feature map;

[0051] The third encoding feature map is processed by a 1x1 convolution layer to obtain a convolution feature map corresponding to the third encoding feature map;

[0052] Adding the convolution feature map corresponding to the third encoding feature map and the up-sampled feature map corresponding to the third encoding feature map to obtain a third decoding feature map;

[0053] The second encoded feature map is processed using a 1x1 convolution layer to obtain a convolution feature map corresponding to the second encoded feature map;

[0054] Adding the convolution feature map corresponding to the second encoding feature map and the up-sampled feature map corresponding to the second encoding feature map to obtain a second decoding feature map;

[0055] The first encoding feature map is processed using a 1x1 convolution layer to obtain a convolution feature map corresponding to the first encoding feature map;

[0056] Adding the convolution feature map corresponding to the first encoding feature map and the up-sampled feature map corresponding to the first encoding feature map to obtain a first decoding feature map;

[0057] The first decoding feature map, the second decoding feature map, the third decoding feature map and the fifth decoding feature map are superimposed to output a target extraction image.

[0058] Preferably, the preprocessing of the satellite remote sensing image to obtain the preprocessed satellite remote sensing image specifically includes:

[0059] Perform geometric correction, radiation correction and atmospheric correction on the satellite remote sensing image in sequence to obtain a corrected satellite remote sensing image;

[0060] The corrected satellite remote sensing image is divided into blocks and standardized to obtain a standardized satellite remote sensing image; wherein the standardized processing includes unifying the size, format and naming of the divided image files;

[0061] The standardized satellite remote sensing images are selected to obtain preprocessed satellite remote sensing images.

[0062] Preferably, constructing a sample data set for model training based on the preprocessed satellite remote sensing image specifically includes:

[0063] Screening the pre-processed satellite remote sensing images to obtain satellite remote sensing images containing hidden dangers of power transmission lines;

[0064] Annotating the satellite remote sensing images containing hidden dangers of the power transmission lines to obtain samples;

[0065] Checking whether the sample complies with the specification;

[0066] When the sample meets the specification, the sample is trimmed to obtain a sample library;

[0067] Samples that meet the set conditions are selected from the sample library as sample data sets for model training.

[0068] Preferably, the method further comprises:

[0069] Post-processing the target extraction image to obtain a post-processed target extraction image;

[0070] vectorizing the post-processed target extraction image to obtain a vectorized target extraction image;

[0071] Check whether the vectorized target extraction image meets the index requirements;

[0072] When the vectorized target extraction image does not meet the index requirements, the sample data set is expanded;

[0073] The remote sensing interpretation model is retrained and verified using the amplified sample data set to obtain a trained remote sensing interpretation model;

[0074] The trained remote sensing interpretation model is used to extract targets from the satellite remote sensing images of the transmission lines to be extracted, and the target extraction images are output.

[0075] Second aspect

[0076] A device for automatically extracting hidden dangers of a power transmission line, the device comprising:

[0077] Satellite remote sensing image acquisition module, used to acquire satellite remote sensing images of power transmission lines;

[0078] A satellite remote sensing image preprocessing module is used to preprocess the satellite remote sensing image to obtain a preprocessed satellite remote sensing image;

[0079] A sample data set construction module, used to construct a sample data set for model training based on the preprocessed satellite remote sensing images;

[0080] A remote sensing interpretation model training module, used to train and verify the remote sensing interpretation model using the sample data set to obtain a trained remote sensing interpretation model; wherein the remote sensing interpretation model includes a Swin Transformer encoder module, a CBAM attention mechanism module and an UperNet decoder module, the UperNet decoder module includes a PPM module and an FPN module, and the loss function used by the remote sensing interpretation model is a focal binary cross entropy loss function;

[0081] A satellite remote sensing image input module, used for inputting the satellite remote sensing image of the power transmission line to be extracted into the trained remote sensing interpretation model;

[0082] A coding feature map generation module, used to use the Swin Transformer encoder module to perform multi-level feature extraction on the satellite remote sensing image of the power transmission line to be extracted, and generate a coding feature map; wherein the coding feature map includes a first coding feature map, a second coding feature map, a third coding feature map and a fourth coding feature map;

[0083] A fifth coding feature map generating module, used for performing feature weighting and feature fusion on the fourth coding feature map by using the CBAM attention mechanism module and the PPM module to generate a fifth coding feature map;

[0084] The target extraction image output module is used to use the FPN module to further extract and fuse the first coding feature map, the second coding feature map, the third coding feature map and the fifth coding feature map, and output the target extraction image.

[0085] The embodiments of the present invention have the following beneficial effects:

[0086] The present invention adopts a remote sensing interpretation model composed of a Swin Transformer encoder module, a CBAM attention mechanism module and an UperNet decoder module to extract transmission line hidden dangers in remote sensing satellite images, which can improve the accuracy and efficiency of identifying transmission line hidden dangers in complex scenes, especially complex scenes involving agricultural covering materials. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0088] in:

[0089] Figure 1 A flowchart of a method for automatically extracting hidden dangers of a power transmission line in one embodiment;

[0090] Figure 2 A structural diagram of a remote sensing interpretation model in one embodiment;

[0091] Figure 3 Schematic diagram of the structure of a CBAM attention mechanism module in one embodiment;

[0092] Figure 4 A flowchart of automatic extraction of hidden dangers of power transmission lines in another embodiment;

[0093] Figure 5 The figure is a structural block diagram of a device for automatically extracting hidden dangers of power transmission lines in one embodiment. DETAILED DESCRIPTION

[0094] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0095] like Figure 1 As shown, in one embodiment, a method for automatically extracting hidden dangers of a power transmission line is provided. The method for automatically extracting hidden dangers of a power transmission line specifically includes steps 1100 to 1800:

[0096] Step 1100, obtaining satellite remote sensing images of the power transmission line.

[0097] In this embodiment, satellite remote sensing images are used to prepare sample data sets and extract hidden dangers of power transmission lines.

[0098] Step 1200, preprocessing the satellite remote sensing image to obtain a preprocessed satellite remote sensing image.

[0099] In this embodiment, image preprocessing can eliminate image distortion and noise, and improve image quality and usability.

[0100] In some examples, image preprocessing includes image correction, image segmentation and normalization, manual selection of high-quality images, etc.

[0101] In step 1200, the satellite remote sensing image is preprocessed to obtain the preprocessed satellite remote sensing image, which may further include steps 1201 to 1203:

[0102] Step 1201, sequentially perform geometric correction, radiation correction and atmospheric correction on the satellite remote sensing image to obtain a corrected satellite remote sensing image.

[0103] In some examples, radiation correction specifically includes: band fusion, bit reduction, color grading, mosaic framing, etc.

[0104] Step 1202, dividing and standardizing the corrected satellite remote sensing image to obtain a standardized satellite remote sensing image.

[0105] In this embodiment, a large-scale image is divided into blocks suitable for sample production, and the size, format and naming of the block image files are unified to facilitate storage and management.

[0106] In some examples, the standardization process includes unifying the size, format, and naming of tiled image files.

[0107] Step 1203, selecting the standardized satellite remote sensing images to obtain preprocessed satellite remote sensing images.

[0108] In this embodiment, professional interpreters select images with few clouds, which will not affect information interpretation and will not cause loss of geometric accuracy. Unclear texture, noise, blur, distortion, misalignment and holes will cause image defects.

[0109] Step 1300: construct a sample data set for model training based on the preprocessed satellite remote sensing images.

[0110] In this embodiment, after the image preprocessing, it is used to obtain a sample data set for model training.

[0111] In step 1300, based on the preprocessed satellite remote sensing image, a sample data set for model training is constructed, which may further include steps 1301 to 1305:

[0112] Step 1301, screening the pre-processed satellite remote sensing images to obtain satellite remote sensing images containing transmission line hidden dangers.

[0113] In this embodiment, professional interpreters further screen image blocks containing hidden dangers of power transmission lines for making samples.

[0114] Step 1302: annotate the satellite remote sensing images containing the hidden dangers of the power transmission lines to obtain samples.

[0115] In this embodiment, during the sample labeling process, it is necessary to fully consider the characteristics of hidden dangers of transmission lines, such as buildings, ground covers, tree barriers, etc., and to carefully divide and mark different types of hidden dangers.

[0116] The sample annotation methods include: the method of annotating shp using ArcGIS, the method of annotating Mask using Photoshop, the method of converting and annotating vector data such as existing geographical conditions and the third national survey, and the method of existing model reasoning.

[0117] Step 1303, check whether the sample meets the specification.

[0118] In this embodiment, professional interpreters check whether the produced samples meet the specifications.

[0119] Step 1304: if the sample meets the specification, the sample is trimmed to obtain a sample library.

[0120] In this embodiment, samples that do not meet the specifications are corrected. For samples that meet the specifications, the block images and the sample annotations are combined to create sample pairs, including corresponding image patches and sample patches. Each patch is stored in tif format with a size of 512*512 pixels, thereby obtaining a sample library.

[0121] Step 1305: select samples that meet the set conditions from the sample library as sample data sets for model training.

[0122] In this embodiment, according to the remote sensing interpretation model to be trained, samples of different types, regions, and periods are screened from the sample library according to the set conditions. The set conditions vary with the model and are not limited here.

[0123] The sample data set is divided into training set, validation set and test set. The training set is used for model training and parameter update, the validation set is used for model hyperparameter tuning and performance evaluation, and the test set is used for the final evaluation and application of the model.

[0124] Step 1400: Use the sample data set to train and verify the remote sensing interpretation model to obtain a trained remote sensing interpretation model.

[0125] In this embodiment, before model training, the sample data set is enhanced and training parameters are set. The sample data set is enhanced, including contrast perturbation, image rotation, overexposure processing, color perturbation, tone separation, contrast enhancement, brightness adjustment, image sharpening, image shearing, weather filter, haze processing, random cutting and other image processing algorithms, to simulate the imaging effects under different time and seasonal conditions, enrich the diversity of the sample data set, and thus improve the generalization ability and robustness of the model. Set hyperparameters such as batch size, learning rate, and regularization to balance the convergence speed and generalization ability of the model.

[0126] Figure 2 FIG. 4 is a structural diagram of a remote sensing interpretation model in an embodiment. Figure 2As shown in the figure, the remote sensing interpretation model includes a SwinTransformer encoder module, a CBAM attention mechanism module and an UperNet decoder module. The UperNet decoder module includes a PPM module and an FPN module.

[0127] In a dataset with imbalanced categories, cross entropy loss may cause the model to over-focus on the majority category, resulting in a decrease in the model's recognition performance for the minority category. The remote sensing interpretation model uses the focal binary cross entropy loss function ("FocalBinary Cross-Entropy Loss, FocalBCE"), which is based on the binary cross entropy loss ("Binary Cross-EntropyLoss Function, BCELoss") and applies the weight adjustment of the focal loss function ("Focal Loss, FL").

[0128] Binary cross entropy loss function:

[0129]

[0130] Where N is the total number of samples, y i is the true label of the ith sample, which takes a value of 0 or 1, and pi is the probability that the model predicts that the ith sample is category 1.

[0131] Focal loss function:

[0132] FLoss(p t )=-α t (1-p t ) γ log(p t )

[0133] In the formula, p t is the probability predicted by the model for the correct category. γ is a tuning parameter used to reduce the weight of easy-to-classify samples. α t is the weight for each class, which is usually used to further solve the class imbalance problem.

[0134] Combining the binary cross entropy loss and the focal loss, we can get the overall formula of the focal binary cross entropy loss function:

[0135]

[0136] Where N is the total number of samples, t i is the proportion of correctly detected true value pixels in the i-th sample, is the adjusted predicted probability for the correct category, and γ is a tuning parameter used to reduce the weight of easy-to-classify samples. is the weight for each category, which is used to solve the category imbalance problem.

[0137] When combining the binary cross entropy loss function and the focal loss function, the weight adjustment mechanism of the focal loss function is used to modify the basic binary cross entropy loss of each sample. First, the basic binary cross entropy loss is calculated for each sample. Second, for each sample, the weight factor (1-p t ) γ and α t , this weight factor reduces the easy classification (i.e., p t The loss of samples close to 0 or 1) is reduced, while the loss of samples that are difficult to classify is increased. Finally, the calculated weight factor is applied to the binary cross entropy loss of each sample. In this way, the model is guided to pay more attention to samples that are difficult to classify during training, while paying less attention to samples that are easy to classify. Such a loss function is particularly suitable for datasets with imbalanced categories and can improve the model's recognition ability on minority categories.

[0138] Step 1500: input the satellite remote sensing image of the power transmission line to be extracted into the trained remote sensing interpretation model.

[0139] In this embodiment, the trained remote sensing interpretation model is used to extract the hidden dangers of the transmission line. The performance and robustness of the model are further optimized by using techniques such as early stopping, dynamic learning rate adjustment, and model integration. According to the training indicator parameters, the batch size, learning rate, regularization and other hyperparameters are adjusted to improve the model training effect.

[0140] Step 1600, using a Swin Transformer encoder module to perform multi-level feature extraction on the satellite remote sensing image of the power transmission line to be extracted, and generating a coded feature map.

[0141] In this embodiment, Swin Transformer is used as an encoder to gradually extract and refine features through a hierarchical method. At different network levels, features are downsampled and transformed to form feature representations of different scales. Swin Transformer provides powerful feature extraction capabilities as an encoder. Swin Transformer includes several stacked Swin Transformer Blocks.

[0142] Swin Transformer is a deep learning network structure based on the Transformer architecture, designed for computer vision tasks. It combines the advantages of convolutional neural networks (CNN) and Transformers, and has achieved significant performance improvements in the field of computer vision by introducing innovative designs such as window shift mechanisms and hierarchical feature maps.

[0143] In some examples, the encoding characteristic map includes a first encoding characteristic map, a second encoding characteristic map, a third encoding characteristic map, and a fourth encoding characteristic map.

[0144] In step 1600, a Swin Transformer encoder module is used to perform multi-level feature extraction on the satellite remote sensing image of the power transmission line to be extracted to generate a coded feature map, which may further include steps 1601 to 1609:

[0145] Step 1601, using the Patch Partition layer to perform image segmentation on the satellite remote sensing image of the power transmission line to be extracted, to obtain segmented data.

[0146] The Patch Partition layer is an important component of the Swin Transformer encoder module, which is mainly responsible for dividing the input image into a series of small blocks (patches) for subsequent processing.

[0147] The Patch Partition layer divides the input image into a series of small patches, each of which consists of adjacent pixels. This step is a key step in converting image data into a format that the model can process. While dividing into small patches, the Patch Partition layer also processes the pixel data of each small patch through a linear transformation (usually implemented through a convolutional layer) to extract preliminary feature information.

[0148] Step 1602, using the Liner Embedding layer to perform feature mapping on the segmented data to obtain feature mapped data.

[0149] In this embodiment, the Liner Embedding layer is used for feature mapping.

[0150] Step 1603, input the feature mapped data into the first Swin Transformer block to generate a first encoded feature map.

[0151] Swin Transformer Block is the basic building block in the Swin Transformer model. It uses a layered approach to process large-size images and significantly optimizes computational efficiency and model performance.

[0152] The workflow of Swin Transformer Block can be summarized in the following steps:

[0153] Input feature processing: The input features are first processed by layer normalization (LN) and then enter the windowed self-attention module (W-MSA).

[0154] Windowed self-attention calculation: In the W-MSA module, the input feature map is split into multiple non-overlapping windows, and self-attention calculation is performed independently in each window. After the calculation is completed, the self-attention features are added to the input features (residual connection) and layer normalization (LN) is performed.

[0155] Moving window self-attention calculation (optional): If information interaction between windows is required, the moving window self-attention module (SW-MSA) is entered. In this module, the window position is moved and self-attention calculation is performed in the new window. Similarly, after the calculation is completed, the self-attention feature is added to the input feature (residual connection) and layer normalization (LN) is performed.

[0156] Multi-layer perceptron processing: After the self-attention calculation, the feature map enters the multi-layer perceptron module (MLP). In the MLP module, the feature map is processed by multiple fully connected layers and non-linear activation functions to extract higher-level feature information.

[0157] Output features: Finally, the output of the MLP module is used as the final output features of the Swin Transformer Block. These features will be passed to subsequent blocks or used for other tasks (such as classification, detection, etc.).

[0158] Exemplarily, the first coding feature map is denoted by C1.

[0159] Step 1604, downsample the first coding feature map using the first Patch Merging layer to obtain downsampled data corresponding to the first Patch Merging layer.

[0160] The Patch Merging layer is a downsampling technique that achieves downsampling by splicing adjacent patches (blocks) while adjusting the number of channels to form a hierarchical design and save computation while maintaining information integrity. Unlike traditional pooling operations, Patch Merging does not lose information during the downsampling process.

[0161] Step 1605, input the downsampled data corresponding to the first Patch Merging layer into the second SwinTransformer block to generate a second encoded feature map.

[0162] Exemplarily, the second coding feature map is denoted by C2.

[0163] Step 1606, downsample the second coding feature map using the second Patch Merging layer to obtain downsampled data corresponding to the second Patch Merging layer.

[0164] Step 1607, input the downsampled data corresponding to the second Patch Merging layer into the third SwinTransformer block to generate a third encoded feature map.

[0165] Exemplarily, the third coding feature map is denoted by C3.

[0166] Step 1608, using the third Patch Merging layer to downsample the third encoded feature map to obtain downsampled data corresponding to the third Patch Merging layer.

[0167] Step 1609, input the downsampled data corresponding to the third Patch Merging layer into the fourth SwinTransformer block to generate a fourth encoded feature map.

[0168] Exemplarily, the fourth coding feature map is denoted by C4.

[0169] Step 1700, using the CBAM attention mechanism module and the PPM module to perform feature weighting and feature fusion on the fourth encoding feature map to generate a fifth encoding feature map.

[0170] Figure 3 FIG. 1 is a schematic diagram of the structure of the CBAM attention mechanism module in one embodiment. The CBAM attention mechanism module between the Swin Transformer encoder and the UperNet decoder is shown in FIG. Figure 3 As shown in the figure, the CBAM attention mechanism module is mainly used to improve the model's attention to different areas in the image to improve the representation ability of image features. The CBAM attention mechanism module mainly consists of two parts: channel attention module and spatial attention module.

[0171] In step 1700, the CBAM attention mechanism module and the PPM module are used to perform feature weighting and feature fusion on the fourth coding feature map to generate a fifth coding feature map, which may further include steps 1701 to 1705:

[0172] Step 1701, input the fourth encoded feature map into the CBAM attention mechanism module, and output the feature map corresponding to the CBAM attention mechanism module.

[0173] In step 1701, the fourth encoded feature map is input into the CBAM attention mechanism module, and the feature map corresponding to the CBAM attention mechanism module is output, which specifically includes steps 101 to 112:

[0174] Step 101, obtaining the dimension of the fourth encoding feature map.

[0175] In some examples, the dimensions include number of channels, height, and width.

[0176] Exemplarily, the dimension of the input feature map is c×H×W, where c is the number of channels, and H and W are the height and width.

[0177] Step 102, input the fourth encoded feature map into the channel attention module.

[0178] The channel attention module is used to learn the importance of each channel of the feature map and weight the feature map according to the importance of the channel.

[0179] Step 103: Use a pooling layer to perform global average pooling on the fourth encoded feature map to obtain a feature map after global average pooling.

[0180] Exemplarily, global average pooling is performed on the input feature map to obtain a feature map of size 1×1×c.

[0181] Step 104: Use the first fully connected layer to process the feature map after global average pooling to obtain an output feature map of the first fully connected layer.

[0182] Step 105: Use the second fully connected layer to process the output feature map of the first fully connected layer to obtain the output feature map of the second fully connected layer.

[0183] Exemplarily, the feature maps after global average pooling are respectively passed into two fully connected layers (FC) for processing, and the two fully connected layers are used to learn the weights and biases of the channels respectively.

[0184] Step 106, use the activation function to normalize the output feature map of the second fully connected layer to obtain the channel attention weight.

[0185] Exemplarily, the outputs of the two fully connected layers are normalized using an activation function to obtain channel attention weights.

[0186] Step 107, multiply the channel attention weight by the fourth encoded feature map, and output the feature map after the channel attention weighting.

[0187] Exemplarily, the channel attention weights are multiplied with the input feature map to weight the features of each channel.

[0188] Step 108, input the feature map after channel attention weighting into the spatial attention module.

[0189] The spatial attention module is used to learn the importance of each spatial position in the feature map and weight the feature map according to the importance of the position.

[0190] Step 109, using the first convolutional layer to process the feature map after the channel attention weighting, to obtain the output feature map of the first convolutional layer.

[0191] Step 110: Use the second convolutional layer to process the output feature map of the first convolutional layer to obtain the output feature map of the second convolutional layer.

[0192] Exemplarily, the input feature map is passed into two convolutional layers respectively, which are used to learn the weights of each position.

[0193] Step 111, use the activation function to normalize the output feature map of the second convolutional layer to obtain the spatial attention weight.

[0194] Exemplarily, the outputs of the two convolutional layers are normalized using an activation function to obtain the spatial attention weights.

[0195] Step 112, multiply the spatial attention weight by the feature map weighted by the channel attention, and output the feature map corresponding to the CBAM attention mechanism module.

[0196] Exemplarily, the spatial attention weights are multiplied with the input feature map to weight the features at each position.

[0197] After channel attention calculation and spatial attention calculation, the output feature map of the CBAM attention mechanism module is obtained. The CBAM attention mechanism module improves the network's attention and representation ability to image features, thereby improving model performance. This attention mechanism helps the model better capture important information in the image.

[0198] Step 1702, input the feature map corresponding to the CBAM attention mechanism module into the PPM module.

[0199] PPM (Pyramid Pooling Module) refers to the pyramid pooling module, which is used to aggregate contextual information in different regions to improve the network's ability to obtain global information. The PPM module uses pooling operations of different scales on the original feature map to obtain multiple feature maps of different sizes, and then splices these feature maps (including the original feature map) in the channel dimension, and finally outputs a composite feature map that combines multiple scales. This method can take into account both global semantic information and local detail information, thereby improving the performance of the network.

[0200] Exemplarily, the PPM module receives the feature map from the CBAM attention mechanism module as input.

[0201] Step 1703, using pooling layers of different sizes to pool the feature map corresponding to the CBAM attention mechanism module to obtain a pooled feature map.

[0202] Exemplarily, the input feature map is fed into a series of pooling layers of different sizes. These pooling layers usually include global average pooling and pooling kernels of different sizes. Global average pooling captures the overall scene information, while pooling kernels of other sizes capture local features. With this setting, the PPM module is able to capture image features at different spatial scales, thereby obtaining rich contextual information.

[0203] Step 1704, use an upsampling layer to upsample the pooled feature map to obtain an upsampled feature map.

[0204] Step 1705, fuse the upsampled feature map with the feature map corresponding to the CBAM attention mechanism module to generate a fifth encoded feature map.

[0205] Exemplarily, the pooled feature map will be upsampled (usually using bilinear interpolation) to the same size as the original input feature map. After upsampling, it may be necessary to adjust the number of channels of the feature map using 1x1 convolution or other methods to keep it consistent with the original feature map. All upsampled feature maps will be fused with the original feature map, usually by a simple addition or concatenation operation. This step enables the network to exploit both global and local contextual information. The fused feature map is used as the output of the PPM module, and this output feature map is enriched with contextual information captured from different scales.

[0206] Exemplarily, the fifth coding feature map is denoted by C5.

[0207] Step 1800, using the FPN module to further extract and fuse features of the first coding feature map, the second coding feature map, the third coding feature map and the fifth coding feature map, and output a target extracted image.

[0208] The FPN module, full name Feature Pyramid Network, is a commonly used feature extraction structure in the field of computer vision and deep learning. It is mainly designed for the multi-scale characteristics of objects in images, and improves the performance of tasks such as object detection and semantic segmentation by extracting and fusing features at different scales.

[0209] In step 1800, the FPN module is used to further extract and fuse the first coding feature map, the second coding feature map, the third coding feature map, and the fifth coding feature map, and the target extraction image is output, which may further include steps 1801 to 1808:

[0210] Step 1801, use a 1x1 convolutional layer to process the fifth encoding feature map to generate a fifth decoding feature map.

[0211] Exemplarily, the fifth decoding feature map is denoted by P5.

[0212] Step 1802: Use a 1x1 convolutional layer to process the third encoded feature map to obtain a convolutional feature map corresponding to the third encoded feature map.

[0213] Step 1803, adding the convolution feature map corresponding to the third encoding feature map and the up-sampled feature map corresponding to the third encoding feature map to obtain a third decoding feature map.

[0214] Exemplarily, the third decoding feature map is denoted by P3.

[0215] Step 1804: Use a 1x1 convolutional layer to process the second encoded feature map to obtain a convolutional feature map corresponding to the second encoded feature map.

[0216] Step 1805, adding the convolution feature map corresponding to the second encoding feature map and the up-sampled feature map corresponding to the second encoding feature map to obtain a second decoding feature map.

[0217] Exemplarily, the second decoding feature map is denoted by P2.

[0218] Step 1806: Use a 1x1 convolutional layer to process the first encoding feature map to obtain a convolutional feature map corresponding to the first encoding feature map.

[0219] Step 1807, adding the convolution feature map corresponding to the first encoding feature map and the up-sampled feature map corresponding to the first encoding feature map to obtain a first decoding feature map.

[0220] Exemplarily, the first decoding feature map is denoted by P1.

[0221] Step 1808, superimpose the first decoding feature map, the second decoding feature map, the third decoding feature map and the fifth decoding feature map, and output the target extraction image.

[0222] Exemplarily, the FPN module accepts feature maps generated by the Swin Transformer encoder module and the PPM module. Specifically, the Swin Transformer module generates four sets of feature maps of different scales, C1, C2, C3, and C4, and C4 is passed through the CBAM attention mechanism module and the PPM module to generate the C5 feature map. The scales of these feature maps gradually decrease from C1 to C5 (i.e., the resolution decreases, but the semantic information is richer). Secondly, the calculation logic of the FPN module is a top-down path and a lateral connection. Starting from the deepest feature map C5, upsampling is performed to increase its spatial resolution to form the P5 feature map; for the feature maps of each layer of C3, C2, and C1, first use 1x1 convolution to reduce the number of channels to match the number of channels of the corresponding upsampled feature map; finally, the 1x1 convolution output of each layer is added to the upsampled feature map of the corresponding layer to form the feature maps of P3, P2, and P1. Through the above steps, we obtain a series of feature maps P1, P2, P3 and P5, which combine spatial information and semantic information at different levels. These feature maps are superimposed together to output the target extracted image.

[0223] The lateral connection formula is described as:

[0224] P q = Upsampling (P q+1 )+1x1 convolution(C q ), where q=3, 2, 1.

[0225] Where P is the feature map finally generated by the lateral connection, C is the feature map generated by the Swin Transformer module, and q is the numbering order of the feature maps generated by the Swin Transformer module.

[0226] For P5, C5 is used directly or some processing is performed on C5, and the number of channels is adjusted using 1x1 convolution to match the number of channels of other layers in the FPN module. Then, these generated feature maps (P1, P2, P3, and P5) are used in the UPerNet decoder part, where further feature fusion and upsampling are performed to generate the final pixel-level prediction results.

[0227] In some examples, the method for automatically extracting hidden dangers of power transmission lines further includes steps 1901 to 1906:

[0228] Step 1901, post-processing the target extraction image to obtain a post-processed target extraction image.

[0229] For the hidden danger areas obtained by interpretation, post-processing and refinement are carried out, such as removing noise, smoothing boundaries, merging small areas, etc., to improve the integrity and aesthetics of the hidden danger targets.

[0230] Step 1902, vectorize the post-processed target extraction image to obtain a vectorized target extraction image.

[0231] Convert pixel-level hidden danger segmentation results into vector formats, such as points, lines, and surfaces, to facilitate subsequent spatial analysis and data management.

[0232] Step 1903, checking whether the vectorized target extracted image meets the index requirements.

[0233] Step 1904: when the vectorized target extracted image does not meet the index requirement, the sample data set is expanded.

[0234] Step 1905: Retrain and verify the remote sensing interpretation model using the amplified sample data set to obtain a trained remote sensing interpretation model.

[0235] Step 1906, using the trained remote sensing interpretation model to perform target extraction on the satellite remote sensing image of the power transmission line to be extracted, and outputting the target extraction image.

[0236] The extraction results are quality checked and manually verified, and areas with identification errors or uncertainties are marked and corrected. The sample library is continuously improved and updated to form an iterative optimization closed loop.

[0237] Save the interpretation results to the specified storage location for subsequent use.

[0238] The embodiment of the present invention adopts a remote sensing interpretation model composed of a Swin Transformer encoder module, a CBAM attention mechanism module and an UperNet decoder module to extract transmission line hidden dangers in remote sensing satellite images, which can improve the accuracy and efficiency of identifying transmission line hidden dangers in complex scenes, especially complex scenes involving agricultural covering materials.

[0239] In order to better understand the method for automatic extraction of hidden dangers in transmission lines disclosed in the present application, the embodiment of the present application also discloses a process for automatic extraction of hidden dangers in transmission lines, which continuously improves the accuracy and efficiency of hidden danger identification through iterative optimization. Throughout the entire process, the concept of human-machine collaboration and continuous improvement runs through it. On the one hand, through manual participation in key links such as sample selection, labeling, and quality inspection, professional knowledge and experience are introduced to ensure the reliability of samples and results; on the other hand, through automated model training and prediction, the efficiency and coverage of hidden danger identification are greatly improved, and the intensity of manual labor is reduced. At the same time, the iterative optimization method is adopted to continuously expand and update the sample library, optimize the model structure and parameters, so that the performance of hidden danger identification is continuously improved to adapt to the ever-changing external environment and needs. Figure 4 As shown in the figure, the process includes three stages: construction of remote sensing image interpretation sample library, training of remote sensing image interpretation model, and remote sensing image interpretation and post-processing.

[0240] 1. Construction of remote sensing image interpretation sample library

[0241] (1) Image preprocessing

[0242] Image preprocessing includes geometric correction, radiation correction, atmospheric correction, etc., which can eliminate image distortion and noise and improve image quality and usability.

[0243] (2) Image segmentation and standardization

[0244] Divide large-scale images into blocks suitable for sample production, and unify the size, format and naming of the block image files.

[0245] (3) Manual selection of high-quality images

[0246] Professional interpreters select the images, which have few clouds and no image defects such as unclear texture, noise, blur, distortion, misalignment, holes, etc. that affect the interpretation of image information and cause loss of geometric accuracy.

[0247] (4) Screening samples

[0248] Professional interpreters will further screen image blocks containing hidden dangers of power transmission lines for the purpose of making samples.

[0249] (5) Remote sensing interpretation sample annotation

[0250] Sample annotation methods include using ArcGIS to annotate shp, using Photoshop to annotate masks, converting and annotating vector data such as existing geographic conditions and the third survey, and existing model reasoning methods. In the sample annotation process, it is necessary to fully consider the characteristics of transmission line hazards, such as buildings, surface covers, tree barriers, etc., and carefully divide and mark different types of hazards.

[0251] (6) Manual inspection of samples

[0252] Professional interpreters will check whether the samples produced meet the specifications, and make corrections if they do not meet the standards.

[0253] (7) Sample cutting

[0254] Combine the segmented images and the sample annotations to create sample pairs, including the corresponding image patches and sample patches. Each patch is 512*512 pixels in size and stored in tif format.

[0255] 2. Remote sensing image interpretation model training

[0256] The specific steps for training the corresponding professional remote sensing interpretation model for the hidden dangers of transmission line such as ground cover and tree barriers are as follows:

[0257] (1) Sample screening

[0258] According to the professional model that needs to be trained, samples of different types, regions, and periods can be filtered from the sample library according to the set conditions.

[0259] (2) Constructing a training sample set

[0260] The sample set is divided into training set, validation set and test set. The training set is used for model training and parameter update, the validation set is used for model hyperparameter tuning and performance evaluation, and the test set is used for final evaluation and application of the model.

[0261] (3) Sample data enhancement

[0262] The data set of hidden dangers of transmission lines is enhanced, including contrast disturbance, image rotation, overexposure processing, color disturbance, tone separation, contrast enhancement, brightness adjustment, image sharpening, image shearing, weather filter, haze processing and random cutting, etc. The image processing algorithms simulate the imaging effects under different time and seasonal conditions, enrich the diversity of the data set, and thus improve the generalization ability and robustness of the model.

[0263] (4) Training parameter setting

[0264] Set hyperparameters such as batch size, learning rate, regularization, etc. to balance the convergence speed and generalization ability of the model.

[0265] (5) Model training

[0266] Techniques such as early stopping, dynamic learning rate adjustment, and model integration are used to further optimize the performance and robustness of the model.

[0267] (6) Parameter Tuning

[0268] According to the training indicator parameters, adjust the batch size, learning rate, regularization and other hyperparameters to improve the model training effect.

[0269] (7) Analysis of interpretation results

[0270] After training is completed, a comprehensive performance evaluation of the model is conducted, including indicators such as precision, recall rate, and F1 score. The model's prediction results and error samples are analyzed through visualization tools to identify the advantages and disadvantages of the model, and sample amplification is performed when necessary.

[0271] (8) Model deployment

[0272] The trained model is encapsulated as a service to implement RESTful interface access.

[0273] 3. Remote sensing image interpretation and post-processing

[0274] The trained model is used to automatically interpret and extract hidden dangers from new transmission line remote sensing images. The specific process is as follows:

[0275] (1) Model Reasoning

[0276] In order to adapt to large-scale, high-resolution image data, strategies such as sliding windows and image pyramids are used to process images in blocks to ensure the accuracy of extraction and improve processing efficiency.

[0277] (2) Post-processing of interpretation results

[0278] For the hidden danger areas obtained by interpretation, post-processing and refinement are carried out, such as removing noise, smoothing boundaries, merging small areas, etc., to improve the integrity and aesthetics of the hidden danger targets.

[0279] (3) Vectorization of Raster Data

[0280] Convert pixel-level hidden danger segmentation results into vector formats, such as points, lines, and surfaces, to facilitate subsequent spatial analysis and data management.

[0281] (4) Quality check of interpretation results

[0282] The extraction results are quality checked and manually verified, and areas with identification errors or uncertainties are marked and corrected. The sample library is continuously improved and updated to form an iterative optimization closed loop.

[0283] (5) Output result set

[0284] Save the interpretation results to the specified storage location for subsequent use.

[0285] This embodiment proposes a continuously improved automatic extraction process of hidden dangers in power transmission lines, which continuously improves the performance of hidden danger identification through human-machine collaboration and iterative optimization.

[0286] The embodiment of the present application also discloses an integrated workstation for automatic extraction of hidden dangers in transmission lines, which integrates functions such as sample library construction and management, model training and optimization, hidden danger extraction and results output, so as to realize the full process automation of hidden danger extraction in transmission lines.

[0287] In terms of sample library construction and management, the workstation provides comprehensive data management and annotation tools. Through the visual interface and human-computer interaction function, users can easily browse and filter data, zoom in, zoom out, and translate the area of ​​interest, and use polygons, rectangles, circles and other tools to annotate and edit hidden danger targets. The system also has some commonly used data preprocessing algorithms built in to ensure data quality and consistency. At the same time, the system supports multi-person collaborative annotation and quality control, and improves the efficiency and accuracy of annotation through task allocation, progress tracking, and audit mechanisms.

[0288] In terms of model training and optimization, the workstation integrates the mainstream PyTorch deep learning framework, and users can choose the appropriate model structure and training strategy according to their needs. The system provides a wealth of hyperparameter tuning and model evaluation tools, such as grid search and cross-validation, to help users quickly find the optimal model configuration. At the same time, the system also supports multi-GPU collaborative training, which significantly accelerates the model training process. In view of the characteristics of transmission line hidden danger identification, the system has built-in optimization technologies, such as difficult example mining and sample resampling, to further improve the performance and generalization ability of the model.

[0289] In terms of hidden danger extraction and output, the workstation provides an efficient reasoning engine and post-processing module, which can quickly identify and extract hidden dangers from a large range of transmission line images. The system adopts a multi-scale, multi-window sliding reasoning strategy to adaptively adjust the window size and step size to improve calculation efficiency while ensuring accuracy. For the extracted hidden danger areas, the system performs post-processing operations such as boundary simplification and attribute calculation to generate standardized vector data, and supports a variety of commonly used spatial data formats, such as Shapefile, GeoJSON, etc. Users can intuitively browse and edit the extraction results through an interactive visual interface, perform error checking and manual corrections. The system also provides flexible results export and report generation functions, supports customized templates and styles, and automatically generates hidden danger distribution maps, statistical charts, analysis reports, etc. to meet the needs of different users.

[0290] The workstation also provides a Web-based access interface that supports remote access and on-site operations, allowing users to perform data management, model training, hidden danger identification and other operations anytime and anywhere.

[0291] The present application also discloses a device for automatically extracting hidden dangers of power transmission lines. Figure 5As shown, the device comprises:

[0292] The satellite remote sensing image acquisition module 5100 is used to acquire satellite remote sensing images of the transmission lines.

[0293] The satellite remote sensing image preprocessing module 5200 is used to preprocess the satellite remote sensing image to obtain the preprocessed satellite remote sensing image.

[0294] The sample data set construction module 5300 is used to construct a sample data set for model training based on the preprocessed satellite remote sensing images.

[0295] The remote sensing interpretation model training module 5400 is used to train and verify the remote sensing interpretation model using the sample data set to obtain a trained remote sensing interpretation model.

[0296] In some examples, the remote sensing interpretation model includes a Swin Transformer encoder module, a CBAM attention mechanism module and an UperNet decoder module, the UperNet decoder module includes a PPM module and an FPN module, and the loss function used by the remote sensing interpretation model is a focal binary cross entropy loss function.

[0297] The satellite remote sensing image input module 5500 is used to input the satellite remote sensing image of the power transmission line to be extracted into the trained remote sensing interpretation model.

[0298] The coding feature map generation module 5600 is used to use the Swin Transformer encoder module to perform multi-level feature extraction on the satellite remote sensing image of the transmission line to be extracted, and generate a coding feature map; wherein the coding feature map includes a first coding feature map, a second coding feature map, a third coding feature map and a fourth coding feature map.

[0299] The fifth coding feature map generation module 5700 is used to use the CBAM attention mechanism module and the PPM module to perform feature weighting and feature fusion on the fourth coding feature map to generate the fifth coding feature map.

[0300] The target extraction image output module 5800 is used to use the FPN module to further extract and fuse the first coding feature map, the second coding feature map, the third coding feature map and the fifth coding feature map, and output the target extraction image.

[0301] In one embodiment, the coding feature map generation module 5600 is specifically used to: use the Patch Partition layer to perform image segmentation on the satellite remote sensing image of the transmission line to be extracted to obtain segmented data; use the LinerEmbedding layer to perform feature mapping on the segmented data to obtain feature mapped data; input the feature mapped data into the first Swin Transformer block to generate a first coding feature map; use the first Patch Merging layer to downsample the first coding feature map to obtain the downsampled data corresponding to the first Patch Merging layer; input the downsampled data corresponding to the first Patch Merging layer into the second Swin Transformer block to generate a second coding feature map; use the second Patch Merging layer to downsample the second coding feature map to obtain the downsampled data corresponding to the second Patch Merging layer; input the downsampled data corresponding to the second Patch Merging layer into the third Swin Transformer block to generate a third coding feature map; use the third Patch Merging layer to downsample the third coding feature map to obtain the downsampled data corresponding to the third Patch Merging layer; input the downsampled data corresponding to the third Patch Merging layer into the fourth Swin Transformer block, generates the fourth encoded feature map.

[0302] In one embodiment, the fifth encoding feature map generation module 5700 is specifically used to: input the fourth encoding feature map into the CBAM attention mechanism module, and output the feature map corresponding to the CBAM attention mechanism module; input the feature map corresponding to the CBAM attention mechanism module into the PPM module; use pooling layers of different sizes to pool the feature map corresponding to the CBAM attention mechanism module to obtain a pooled feature map; use an upsampling layer to upsample the pooled feature map to obtain an upsampled feature map; fuse the upsampled feature map with the feature map corresponding to the CBAM attention mechanism module to generate the fifth encoding feature map.

[0303] In one embodiment, the CBAM attention mechanism module includes a channel attention module and a spatial attention module. The fifth encoding feature map generation module 5700 inputs the fourth encoding feature map into the CBAM attention mechanism module, and outputs the feature map corresponding to the CBAM attention mechanism module, which is specifically used to: obtain the dimension of the fourth encoding feature map; wherein the dimension includes the number of channels, height and width; input the fourth encoding feature map into the channel attention module; use the pooling layer to perform global average pooling on the fourth encoding feature map to obtain the feature map after global average pooling; use the first fully connected layer to process the feature map after global average pooling to obtain the output feature map of the first fully connected layer; use the second fully connected layer to process the output feature map of the first fully connected layer to obtain the output feature map of the second fully connected layer output feature map; use the activation function to normalize the output feature map of the second fully connected layer to obtain the channel attention weight; multiply the channel attention weight by the fourth encoding feature map, and output the channel attention weighted feature map; input the channel attention weighted feature map into the spatial attention module; use the first convolutional layer to process the channel attention weighted feature map to obtain the output feature map of the first convolutional layer; use the second convolutional layer to process the output feature map of the first convolutional layer to obtain the output feature map of the second convolutional layer; use the activation function to normalize the output feature map of the second convolutional layer to obtain the spatial attention weight; multiply the spatial attention weight by the channel attention weighted feature map, and output the feature map corresponding to the CBAM attention mechanism module.

[0304] In one embodiment, the target extraction image output module 5800 is specifically used to: use a 1x1 convolution layer to process the fifth encoding feature map to generate a fifth decoding feature map; use a 1x1 convolution layer to process the third encoding feature map to obtain a convolution feature map corresponding to the third encoding feature map; add the convolution feature map corresponding to the third encoding feature map and the upsampled feature map corresponding to the third encoding feature map to obtain a third decoding feature map; use a 1x1 convolution layer to process the second encoding feature map to obtain a convolution feature map corresponding to the second encoding feature map; add the convolution feature map corresponding to the second encoding feature map and the upsampled feature map corresponding to the second encoding feature map to obtain a second decoding feature map; use a 1x1 convolution layer to process the first encoding feature map to obtain a convolution feature map corresponding to the first encoding feature map; add the convolution feature map corresponding to the first encoding feature map and the upsampled feature map corresponding to the first encoding feature map to obtain a first decoding feature map; superimpose the first decoding feature map, the second decoding feature map, the third decoding feature map and the fifth decoding feature map to output a target extraction image.

[0305] In one embodiment, the satellite remote sensing image preprocessing module 5200 is specifically used to: perform geometric correction, radiation correction and atmospheric correction on the satellite remote sensing image in sequence to obtain a corrected satellite remote sensing image; divide the corrected satellite remote sensing image into blocks and standardize the image to obtain a standardized satellite remote sensing image; wherein the standardization process includes unifying the size, format and naming of the block image files; and select the standardized satellite remote sensing image to obtain a preprocessed satellite remote sensing image.

[0306] In one embodiment, the sample data set construction module 5300 is specifically used to: screen the preprocessed satellite remote sensing images to obtain satellite remote sensing images containing transmission line hidden dangers; mark the satellite remote sensing images containing transmission line hidden dangers to obtain samples; check whether the samples meet the specifications; if the samples meet the specifications, crop the samples to obtain a sample library; and screen samples that meet the set conditions from the sample library as sample data sets for model training.

[0307] In one embodiment, the apparatus further comprises:

[0308] The target extraction image post-processing module is used to post-process the target extraction image to obtain a post-processed target extraction image.

[0309] The target extraction image vectorization module is used to vectorize the post-processed target extraction image to obtain a vectorized target extraction image.

[0310] The checking module is used to check whether the target extracted image after vectorization meets the index requirements.

[0311] The augmentation module is used to augment the sample data set when the vectorized target extraction image does not meet the index requirements.

[0312] The retraining module is used to retrain and verify the remote sensing interpretation model using the amplified sample data set to obtain a trained remote sensing interpretation model.

[0313] The re-extraction module is used to extract the target from the satellite remote sensing image of the power transmission line to be extracted by using the trained remote sensing interpretation model, and output the target extraction image.

[0314] The above description of the embodiment of the automatic extraction device for transmission line hidden dangers is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiment of the automatic extraction device for transmission line hidden dangers of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0315] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0316] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0317] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for automatically extracting hidden dangers of power transmission lines, characterized in that: The method comprises: Obtain satellite remote sensing images of power transmission lines; Preprocessing the satellite remote sensing image to obtain a preprocessed satellite remote sensing image; Based on the preprocessed satellite remote sensing images, constructing a sample data set for model training; The remote sensing interpretation model is trained and verified using the sample data set to obtain a trained remote sensing interpretation model; wherein the remote sensing interpretation model includes a Swin Transformer encoder module, a CBAM attention mechanism module and an UperNet decoder module, the UperNet decoder module includes a PPM module and an FPN module, and the loss function used by the remote sensing interpretation model is a focal binary cross entropy loss function; Inputting the satellite remote sensing image of the power transmission line to be extracted into the trained remote sensing interpretation model; The Swin Transformer encoder module is used to perform multi-level feature extraction on the satellite remote sensing image of the power transmission line to be extracted, and a coding feature map is generated; wherein the coding feature map includes a first coding feature map, a second coding feature map, a third coding feature map and a fourth coding feature map; Using the CBAM attention mechanism module and the PPM module to perform feature weighting and feature fusion on the fourth encoding feature map to generate a fifth encoding feature map; The FPN module is used to perform further feature extraction and feature fusion on the first coding feature map, the second coding feature map, the third coding feature map and the fifth coding feature map, and output a target extracted image.

2. The method for automatically extracting hidden dangers of power transmission lines according to claim 1, characterized in that: The focal binary cross entropy loss function is specifically: Where N is the total number of samples, t i is the proportion of correctly detected true value pixels in the i-th sample, is the adjusted predicted probability for the correct category, γ is the adjustment parameter, is the weight for each category.

3. The method for automatically extracting hidden dangers of power transmission lines according to claim 1, characterized in that: The SwinTransformer encoder module is used to perform multi-level feature extraction on the satellite remote sensing image of the power transmission line to be extracted to generate a coding feature map, specifically including: The Patch Partition layer is used to segment the satellite remote sensing image of the power transmission line to be extracted to obtain the segmented data; Using a Liner Embedding layer to perform feature mapping on the segmented data to obtain feature mapped data; Inputting the feature mapped data into a first Swin Transformer block to generate a first encoded feature map; Down-sampling the first coding feature map using a first Patch Merging layer to obtain down-sampled data corresponding to the first Patch Merging layer; Input the downsampled data corresponding to the first Patch Merging layer into a second Swin Transformer block to generate a second encoded feature map; Down-sampling the second encoding feature map using a second Patch Merging layer to obtain down-sampled data corresponding to the second Patch Merging layer; Input the downsampled data corresponding to the second Patch Merging layer into a third Swin Transformer block to generate a third encoded feature map; Down-sampling the third encoding feature map using a third Patch Merging layer to obtain down-sampled data corresponding to the third Patch Merging layer; The downsampled data corresponding to the third Patch Merging layer is input into the fourth Swin Transformer block to generate a fourth encoded feature map.

4. The method for automatically extracting hidden dangers of power transmission lines according to claim 1, characterized in that: The adopting the CBAM attention mechanism module and the PPM module to perform feature weighting and feature fusion on the fourth coding feature map to generate a fifth coding feature map specifically includes: Input the fourth encoded feature map into the CBAM attention mechanism module, and output the feature map corresponding to the CBAM attention mechanism module; Input the feature map corresponding to the CBAM attention mechanism module into the PPM module; Using pooling layers of different sizes to pool the feature map corresponding to the CBAM attention mechanism module to obtain a pooled feature map; Using an upsampling layer to upsample the pooled feature map to obtain an upsampled feature map; The upsampled feature map is fused with the feature map corresponding to the CBAM attention mechanism module to generate a fifth encoded feature map.

5. The method for automatically extracting hidden dangers of power transmission lines according to claim 4, characterized in that: The CBAM attention mechanism module includes a channel attention module and a spatial attention module, and the fourth encoded feature map is input into the CBAM attention mechanism module, and the feature map corresponding to the CBAM attention mechanism module is output, specifically including: Obtaining the dimensions of the fourth encoded feature map; wherein the dimensions include the number of channels, height, and width; Inputting the fourth encoded feature map into a channel attention module; Using a pooling layer to perform global average pooling on the fourth encoded feature map to obtain a feature map after global average pooling; Using a first fully connected layer to process the feature map after global average pooling, to obtain an output feature map of the first fully connected layer; Using a second fully connected layer to process the output feature map of the first fully connected layer to obtain an output feature map of the second fully connected layer; Normalizing the output feature map of the second fully connected layer using an activation function to obtain a channel attention weight; Multiplying the channel attention weight by the fourth encoded feature map, and outputting a channel attention weighted feature map; Inputting the channel attention weighted feature map into the spatial attention module; Using the first convolutional layer to process the feature map after the channel attention weighting, to obtain an output feature map of the first convolutional layer; Using a second convolutional layer to process the output feature map of the first convolutional layer to obtain an output feature map of the second convolutional layer; Normalizing the output feature map of the second convolutional layer using an activation function to obtain a spatial attention weight; Multiply the spatial attention weight by the channel attention weighted feature map, and output the feature map corresponding to the CBAM attention mechanism module.

6. The method for automatically extracting hidden dangers of power transmission lines according to claim 1, characterized in that: The FPN module is used to further extract and fuse the first coding feature map, the second coding feature map, the third coding feature map and the fifth coding feature map, and output a target extraction image, specifically including: Processing the fifth encoding feature map using a 1x1 convolutional layer to generate a fifth decoding feature map; The third encoding feature map is processed by a 1x1 convolution layer to obtain a convolution feature map corresponding to the third encoding feature map; Adding the convolution feature map corresponding to the third encoding feature map and the up-sampled feature map corresponding to the third encoding feature map to obtain a third decoding feature map; The second encoded feature map is processed using a 1x1 convolution layer to obtain a convolution feature map corresponding to the second encoded feature map; Adding the convolution feature map corresponding to the second encoding feature map and the up-sampled feature map corresponding to the second encoding feature map to obtain a second decoding feature map; The first encoding feature map is processed using a 1x1 convolution layer to obtain a convolution feature map corresponding to the first encoding feature map; Adding the convolution feature map corresponding to the first encoding feature map and the up-sampled feature map corresponding to the first encoding feature map to obtain a first decoding feature map; The first decoding feature map, the second decoding feature map, the third decoding feature map and the fifth decoding feature map are superimposed to output a target extraction image.

7. The method for automatically extracting hidden dangers of power transmission lines according to claim 1, characterized in that: The preprocessing of the satellite remote sensing image to obtain the preprocessed satellite remote sensing image specifically includes: Perform geometric correction, radiation correction and atmospheric correction on the satellite remote sensing image in sequence to obtain a corrected satellite remote sensing image; The corrected satellite remote sensing image is divided into blocks and standardized to obtain a standardized satellite remote sensing image; wherein the standardized processing includes unifying the size, format and naming of the divided image files; The standardized satellite remote sensing images are selected to obtain preprocessed satellite remote sensing images.

8. The method for automatically extracting hidden dangers of power transmission lines according to claim 1, characterized in that: The step of constructing a sample data set for model training based on the preprocessed satellite remote sensing image specifically includes: Screening the pre-processed satellite remote sensing images to obtain satellite remote sensing images containing hidden dangers of power transmission lines; Annotating the satellite remote sensing images containing hidden dangers of the power transmission lines to obtain samples; Checking whether the sample complies with the specification; When the sample meets the specification, the sample is trimmed to obtain a sample library; Samples that meet the set conditions are selected from the sample library as sample data sets for model training.

9. The method for automatically extracting hidden dangers of power transmission lines according to claim 1, characterized in that: The method further comprises: Post-processing the target extraction image to obtain a post-processed target extraction image; vectorizing the post-processed target extraction image to obtain a vectorized target extraction image; Check whether the vectorized target extraction image meets the index requirements; When the vectorized target extraction image does not meet the index requirements, the sample data set is expanded; The remote sensing interpretation model is retrained and verified using the amplified sample data set to obtain a trained remote sensing interpretation model; The trained remote sensing interpretation model is used to extract targets from the satellite remote sensing images of the transmission lines to be extracted, and the target extraction images are output.

10. A device for automatically extracting hidden dangers of power transmission lines, characterized in that: The device comprises: Satellite remote sensing image acquisition module, used to acquire satellite remote sensing images of power transmission lines; A satellite remote sensing image preprocessing module is used to preprocess the satellite remote sensing image to obtain a preprocessed satellite remote sensing image; A sample data set construction module, used to construct a sample data set for model training based on the preprocessed satellite remote sensing images; A remote sensing interpretation model training module, used to train and verify the remote sensing interpretation model using the sample data set to obtain a trained remote sensing interpretation model; wherein the remote sensing interpretation model includes a Swin Transformer encoder module, a CBAM attention mechanism module and an UperNet decoder module, the UperNet decoder module includes a PPM module and an FPN module, and the loss function used by the remote sensing interpretation model is a focal binary cross entropy loss function; A satellite remote sensing image input module, used for inputting the satellite remote sensing image of the power transmission line to be extracted into the trained remote sensing interpretation model; A coding feature map generation module, used to use the Swin Transformer encoder module to perform multi-level feature extraction on the satellite remote sensing image of the power transmission line to be extracted, and generate a coding feature map; wherein the coding feature map includes a first coding feature map, a second coding feature map, a third coding feature map and a fourth coding feature map; A fifth coding feature map generating module, used for performing feature weighting and feature fusion on the fourth coding feature map by using the CBAM attention mechanism module and the PPM module to generate a fifth coding feature map; The target extraction image output module is used to use the FPN module to further extract and fuse the first coding feature map, the second coding feature map, the third coding feature map and the fifth coding feature map, and output the target extraction image.

Images