Methods, devices, systems, and storage media for identifying defects in transmission line tower foundations

Abstract

This invention discloses a method, apparatus, system, and storage medium for identifying defects in transmission line tower foundations, comprising: Step S1, acquiring remote sensing images of transmission line tower foundations; Step S2, preprocessing the remote sensing images of transmission line tower foundations and dividing the preprocessed remote sensing image dataset to obtain a training set and a test set; Step S3, constructing a transmission line tower foundation defect identification network TBDI-Net based on the preprocessed remote sensing image dataset; Step S4, training the transmission line tower foundation defect identification network TBDI-Net based on the training set; Step S5, inputting the test set into the trained transmission line tower foundation defect identification network TBDI-Net to identify tower foundation defects in transmission line towers. The technical solution of this invention can significantly improve the inspection of transmission line tower foundations.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and particularly relates to a method, device, system, and storage medium for identifying defects in power transmission tower foundations. Background Technology

[0002] As a crucial component of power transmission lines, the transmission tower foundation serves as the basic structure supporting the tower, bearing the load and stabilizing it. It is a vital load-bearing element for long-distance power transmission. Transmission tower foundations are typically constructed using concrete, steel, or other durable materials to ensure long-term stability and corrosion resistance.

[0003] Currently, the inspection of tower foundation defects along power transmission lines requires manual screening of image data and inspection, or on-site inspection, which requires a significant amount of time and professional manpower. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method, device, system and storage medium for identifying defects in transmission line tower foundations, which can significantly improve the inspection of transmission line tower foundations.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for identifying defects in transmission line tower foundations includes:

[0007] Step S1: Acquire remote sensing images of the transmission line including the tower bases;

[0008] Step S2: Preprocess the remote sensing images of transmission lines containing tower foundations, and divide the preprocessed remote sensing image dataset of transmission line tower foundations to obtain training set and test set;

[0009] Step S3: Based on the preprocessed remote sensing image dataset of transmission tower foundations, construct the transmission tower foundation defect identification network TBDI-Net; wherein, the transmission tower foundation defect identification network TBDI-Net includes: feature pyramid shared convolution, edge information feature capture module, and feature adaptive bidirectional fusion module based on adaptive feature fusion unit AFFU;

[0010] Step S4: Train the TBDI-Net network for identifying defects in transmission tower foundations based on the training set;

[0011] Step S5: Input the test set into the trained transmission line tower foundation defect identification network TBDI-Net to identify the tower foundation defect targets existing in the transmission line towers.

[0012] As a preferred approach, the feature pyramid shared convolution adopts a structure combining serial and multiple residual connections, constructing a feature pyramid structure through dilated convolutions with different dilation rates.

[0013] As a preferred option, the edge information feature capture module extracts different types of features in parallel through a dual-branch structure to capture tower base damage information in remote sensing images.

[0014] As a preferred option, the feature adaptive bidirectional fusion module designs an adaptive bidirectional fusion structure between deep and shallow features to aggregate boundary information and semantic information.

[0015] The present invention also provides a defect identification device for transmission line tower foundations, comprising:

[0016] The acquisition module is used to acquire remote sensing images of transmission lines, including tower foundations.

[0017] The partitioning module is used to preprocess remote sensing images of transmission lines containing tower foundations and to partition the preprocessed remote sensing image dataset of transmission line tower foundations to obtain training and test sets.

[0018] The construction module is used to construct the TBDI-Net network for identifying defects in power transmission tower foundations based on the preprocessed remote sensing image dataset of power transmission tower foundations. The TBDI-Net network for identifying defects in power transmission tower foundations includes: a feature pyramid shared convolution, an edge information feature capture module, and a feature adaptive bidirectional fusion module based on the adaptive feature fusion unit AFFU.

[0019] The training module is used to train the TBDI-Net network for identifying defects in transmission tower foundations based on the training set;

[0020] The identification module is used to input the test set into the trained transmission line tower foundation defect identification network TBDI-Net to identify the tower foundation defect targets existing in the transmission line towers.

[0021] This invention also provides a transmission line tower foundation defect identification system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a transmission line tower foundation defect identification method when run by the processor.

[0022] This invention also provides a storage medium storing a computer program that executes a method for identifying defects in transmission line tower foundations during runtime.

[0023] This invention combines deep learning computer vision algorithms with remote sensing imagery to achieve real-time monitoring of defects in power transmission tower foundations. First, a feature pyramid shared convolution is designed, and based on this convolution, an edge information feature capture method is designed, which has higher computational efficiency and generalization ability compared to conventional serial convolution feature extraction methods. Then, the invention designs an edge information feature capture module (EIC), which extracts different types of features in parallel through a dual-branch structure, effectively capturing tower foundation damage information (such as concrete cracking) in remote sensing images while improving the algorithm's ability to identify complex, fine-grained features such as multi-shaped cracks. Finally, the invention designs an adaptive feature fusion unit, which solves the loss problem in the fine-grained feature extraction process compared to conventional direct feature fusion methods. Simultaneously, an adaptive bidirectional feature fusion module is designed based on the fusion unit to improve the algorithm's ability to identify fine-grained features such as concrete cracking in the tower foundation. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0025] Figure 1 This is a flowchart of the method for identifying defects in the foundation of a power transmission tower according to an embodiment of the present invention;

[0026] Figure 2 A schematic diagram of the TBDI-Net structure for identifying defects in power transmission tower foundations;

[0027] Figure 3 This is a structural diagram of the FPSC module;

[0028] Figure 4 Here is a structural diagram of the EIC module;

[0029] Figure 5 Here is a structural diagram of the AFFU module;

[0030] Figure 6 This is a structural diagram of the ABF module. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] Example 1:

[0034] like Figure 1 As shown, this embodiment of the invention provides a method for identifying defects in transmission line tower foundations, including:

[0035] Step S1: Acquire remote sensing images of the transmission line including the tower bases.

[0036] Step S2: Preprocess the remote sensing images of transmission lines containing tower bases, and divide the preprocessed remote sensing image dataset of transmission line tower bases into training set, validation set and test set.

[0037] Step S3: Based on the preprocessed remote sensing image dataset of transmission tower foundations, construct the TBDI-Net network for identifying defects in transmission tower foundations;

[0038] Step S4: Train the TBDI-Net network for identifying defects in transmission tower foundations based on the training set;

[0039] Step S5: Input the test set into the trained transmission line tower foundation defect identification network TBDI-Net to identify the tower foundation defect targets existing in the transmission line towers.

[0040] In one embodiment of the present invention, step S1 involves acquiring remote sensing images of transmission line tower foundations via satellite, drone, or aerial photography, and filtering data containing foundation defects. After filtering the remote sensing images of transmission line tower foundations, image annotation tools are used to annotate the categories and coordinate frames of the tower foundation targets in all the remote sensing images. The categories are divided into tower foundation concrete cracking, tower foundation water accumulation, tower foundation vegetation growth, tower foundation debris, missing tower foundation components, and other categories. Finally, the initial remote sensing dataset D1 of transmission line tower foundation defects after filtering and annotation is obtained.

[0041] In one embodiment of the present invention, in step S2, based on the initial remote sensing dataset D1 of transmission tower foundation defects after screening and labeling, data augmentation techniques such as data rotation, flipping, cropping, random occlusion, contrast adjustment, random scaling, and noise addition are used to augment the remote sensing image dataset of transmission tower foundations, resulting in a dataset D2 containing dataset D1 and augmented using all methods. Subsequently, dataset D2 is randomly divided into training data, validation data, and test data according to a certain proportion, resulting in D3, D4, and D5 respectively.

[0042] In one embodiment of the present invention, step S3 involves using the Transmission-Line Base Defect Identification Net (TBDI-Net) to identify base defects in transmission line towers. The TBDI-Net network includes: a feature pyramid shared convolution, an edge information feature capture module, and a feature adaptive bidirectional fusion module based on the Adaptive Feature Fusion Unit (AFFU), such as... Figure 2 As shown.

[0043] Step S3 specifically includes:

[0044] Step 3.1: Design Feature Pyramid Shared Convolution (FPSC). FPSC enhances the defect feature extraction capability of power transmission tower bases by constructing a feature pyramid structure through dilated convolutions with different dilation rates. It captures finer-grained local features while ensuring global context feature capture. Simultaneously, it reduces parameter redundancy and computational overhead by sharing the internal parameters of the dilated convolution kernels. The structure of the Feature Pyramid Shared Convolution designed in this invention is as follows: Figure 3 As shown. In the disclosed pyramid convolution structure, the input is processed in parallel in the pyramid convolution (parallel and side-by-side, resembling a pyramid), and then the corresponding multiple outputs are added together. This is structurally completely different from the pyramid convolution proposed in this invention: the disclosed pyramid convolution structure is a multi-convolution parallel structure in which the input enters, while the pyramid convolution structure of this invention is a structure that combines serial processing with multiple residual connections.

[0045] Step 3.1.1: The spatial dimensionality of the input feature map is reduced by compressing the number of channels using 1×1 convolutions, resulting in the first feature pyramid feature map F1. Subsequently, F1 is batch normalized and subjected to ReLU operations to obtain the second feature pyramid feature map F2. The phrase "compressing the number of channels using 1×1 convolutions" means that the 1×1 convolution essentially performs feature recombination or mixing along the channel dimension, preserving key information from the input features. Reducing the dimensionality of the input channels helps reduce the number of network parameters and avoids excessive model complexity. Multiple convolution operations with different dilation rates are available. Directly processing the original high-dimensional feature map without reducing the number of channels would lead to high computational costs.

[0046] Step 3.1.2: Input F2 into a combination of three sequentially arranged dilated convolutions (dilation rates of 1, 3, and 5), where all three dilated convolutions use the same shared kernel during computation. Finally, the output features are obtained sequentially: the third feature pyramid feature map F3, the fourth feature pyramid feature map F4, and the fifth feature pyramid feature map F5.

[0047] Step 3.1.3: Concatenate F2, F3, F4 and F5 along the channel dimension to obtain the sixth feature pyramid feature map F6, which has the sum of the number of channels of F2, F3, F4 and F5.

[0048] Step 3.1.4: Input F6 into a 1×1 convolution, restore and adjust the number of channels, and then perform batch normalization and ReLU function operation to obtain the seventh feature pyramid feature map F7.

[0049] Among them, (1) dilated convolution is an existing technology, while other operations such as convolution, batch normalization, ReLU activation function and concatenation are common operations in the deep learning industry.

[0050] (2) Residual connection: This is a commonly used technique in deep neural networks. By directly adding the output of the previous layer to the input of the current layer, the network can learn the residual (i.e. the change of the current layer relative to the previous layer), making it easier to train deep neural networks.

[0051] (3) Normalized Batch Normalization (BN): This refers to the technique of normalizing the input data of each mini-batch during the training process of a neural network. It reduces the internal covariate shift by normalizing the input data to a distribution with a mean of 0 and a variance of 1 (or close to this distribution), thereby accelerating the training of the neural network and improving its performance.

[0052] (4) The ReLU (Rectified Linear Unit) activation function is a commonly used non-linear activation function in deep learning.

[0053] Step 3.2: Design the Edge Information Capture (EIC) module. Within this module, a dual-branch structure is designed to extract different types of features in parallel. In the first branch, Sobel 3D convolution is used to extract pixel intensity variations in the target image, effectively capturing tower base edge damage information in the remote sensing image. In the second branch, the FPSC designed in Step 3.1 is used to accelerate the fitting of fine-grained features such as cracks of multiple shapes and scales, improving the generalization ability for identifying defects in tower base remote sensing images. Finally, the input features are enhanced through feature stitching and residual connections. The structure of the edge information capture module designed in this invention is as follows: Figure 4 As shown.

[0054] Step 3.2.1: Input the defect feature map of the transmission tower base into a 1×1 convolution. Reduce the spatial dimension of the input by compressing the number of channels to obtain the first edge information feature map T1. Then, perform batch normalization and ReLU function operation on T1 in sequence to obtain the second edge information feature map T2 after channel compression.

[0055] Step 3.2.2: Input T2 into the Sobel convolution to extract edge features from the image. First, for the normal operation of the subsequent Sobel convolution, the four-dimensional tensor T2 is expanded into a five-dimensional tensor. That is, it is expanded from a four-dimensional tensor containing batch size, number of channels, height, and width to a five-dimensional tensor containing batch size, number of channels, expansion dimension placeholder, height, and width. The third dimension of the five-dimensional tensor is a newly inserted expansion dimension placeholder with a value of 0 or 1, so that T2 can perform convolution operations in both the horizontal and vertical directions. Then, the expanded T2 is input into two 3×3 convolution kernels in the Sobel convolution for processing. One convolution kernel is used for edge filtering in the horizontal direction, and the other convolution kernel is used for edge filtering in the vertical direction. Finally, the two convolution results after the horizontal and vertical operations are added together to obtain the output of the Sobel convolution, which is the third edge information feature map T3.

[0056] Step 3.2.3: Input T2 into the FPSC designed in Step 3.1 for feature extraction to obtain the fourth edge information feature map T4.

[0057] Step 3.2.4: Concatenate T3 and T4 along the channel dimension to obtain the fifth edge information feature map T5.

[0058] Step 3.2.5: Input T5 into the ELA (Efficient Local Attention) attention mechanism module. In the ELA module, T5 is first subjected to adaptive pooling in both the horizontal and vertical directions to obtain T... h and T w , will T h and T w The horizontal and vertical positional information of the one-dimensional convolutional enhanced feature map is input separately; then, group normalization and sigmoid activation functions are input separately for computation to obtain T. h and T w .

[0059]

[0060] T h =σ(G n (F h (T h ))) (3)

[0061] T w =σ(G n (F w (T w (4)

[0062] In equation (1), T h The result of horizontal adaptive pooling for T5 is given in equation (2), where T w The result of vertical adaptive pooling for T5, where H represents height and W represents width, is given in equations (3) and (4). h (·) and F w (·) denotes one-dimensional convolution, G n (·) denotes group normalization, and σ(·) denotes the sigmoid function.

[0063] Then, T5, T h and T w Multiplying these together yields the sixth edge information feature map T6 of the ELA module.

[0064] T6 = T5 × T h ×T w

[0065] Step 3.2.6: Input T6 into a 1×1 convolution to restore the number of channels, and then perform batch normalization and ReLU function operations in sequence to obtain the seventh edge information feature map T7.

[0066] Step 3.2.7: Perform a residual connection between T7 and T2 to obtain the eighth edge information feature map T8.

[0067] Step 3.2.8: Input T8 into a 1×1 convolution, adjust the number of channels, and then perform batch normalization and ReLU function operations in sequence to obtain the ninth edge information feature map T9.

[0068] Step 3.3: Design the Adaptive Feature Fusion Unit (AFFU), such as... Figure 5 As shown, an adaptive bidirectional fusion module ABF (Adaptive Bidirectional Fusion) is designed based on AFFU, as follows: Figure 6 As shown, a bidirectional fusion structure between deep and shallow features was designed as an important component of the feature fusion structure of TBDI-Net. This effectively aggregates boundary information and semantic information, reduces the loss of fine-grained targets caused by direct fusion of high and low level features, and further improves the effect of multi-scale feature fusion.

[0069] Step 3.3.1: Input the first shallow feature map L1 from the shallow layer and the first deep feature map H1 from the deep layer into AFFU: In the AFFU module, let the inputs be the first fused feature map X1 and the second fused feature map X2, respectively. First, X1 and X2 are both input into a 1×1 convolution to reduce the number of channels. Then, batch normalization and ReLU function operations are performed sequentially to obtain the third fused feature map X3 and the fourth fused feature map X4, respectively. X3 and X4 are then input into the bidirectional fusion structure designed in this scheme to obtain the output of AFFU, the fifth fused feature map X5. The operation process function expression corresponding to the designed adaptive feature fusion unit is as follows:

[0070] X3=ReLU(BN(C 1×1 (X1))) (5)

[0071] X4=ReLU(BN(C 1×1 (X2))) (6)

[0072] X5=UpSample(X4)·X2+UpSample(X3)·X1·(UpSample(Reverse(X4)))+X2(7)

[0073] In equations (5) and (6), ReLU(·) represents the ReLU activation function, BN(·) represents the batch normalization operation, and C 1×1 (·) represents a 1×1 convolution operation. In equation (7), UpSample(·) represents an upsampling (deconvolution) operation, and Reverse(·) represents a reversal operation.

[0074] Step 3.3.2: From Step 3.3.1 and Figure 5 It can be seen that after inputting L1 and H1 into AFFU, the second shallow feature map L2 and the second deep feature map H2 can be obtained respectively. L2 and H2 are concatenated along the channel dimension to obtain the first bidirectional fused feature map M1. Subsequently, M1 is input into Partial convolution for computation, followed by batch normalization and ReLU function operations to obtain the output of the ABF module, which is the second bidirectional fused feature map M2. The computational process function expression corresponding to the designed adaptive bidirectional fusion structure is as follows:

[0075] L2=AFFU(L1,H1) (8)

[0076] H2=AFFU(H1,L1) (9)

[0077] M1 = Concat(L2, H2) (10)

[0078] M2=ReLU(BN(PartialConv(M1))) (11)

[0079] In equations (8) and (9), AFFU(·) represents the operation of the input using the AFFU module. In equation (10), Concat(·) represents the concatenation operation along the channel dimension. In equation (11), ReLU(·) represents the ReLU activation function, BN(·) represents the batch normalization operation, and PartialConv(·) represents the operation of the input feature map using Partial convolution.

[0080] Step 3.4: Based on the FPSC convolutional module designed in Step 3.1, the EIC module designed in Step 3.2, and the ABF module designed in Step 3.3, a TBDI-Net for transmission tower foundation defect identification is proposed to realize transmission tower foundation defect identification based on remote sensing imagery. Figure 2 As shown.

[0081] Step 3.4.1: In the TBDI-Net for defect identification of transmission tower foundations, a three-layer backbone network for feature extraction was designed. This three-layer backbone network consists of a combination of two convolutional layers, batch normalization, ReLU function, and EIC module, as well as a combination of convolutional layers, batch normalization, ReLU function, EIC module, and spatial pyramid pooling module. The first remote sensing feature map Y1 of the transmission tower foundation is input into the backbone network of the model. After passing through the three layers of feature extraction, the second remote sensing feature map Y2, the third remote sensing feature map Y3, and the fourth remote sensing feature map Y4 of the transmission tower foundation are obtained respectively.

[0082] Step 3.4.2: Input Y3 and Y4 into the ABF module for feature fusion to obtain the remote sensing feature map Y5 of the fifth transmission tower base; input Y5 into the EIC module for feature extraction to obtain the remote sensing feature map Y6 of the sixth transmission tower base; input Y2 and Y6 into the ABF module for feature fusion to obtain the remote sensing feature map Y7 of the seventh transmission tower base; input Y7 into the EIC module for feature extraction to obtain the remote sensing feature map Y8 of the eighth transmission tower base; input Y6 and Y8 into the ABF module for feature extraction to obtain the remote sensing feature map Y9 of the ninth transmission tower base; input Y9 into the EIC module for feature fusion to obtain the remote sensing feature map Y1 of the tenth transmission tower base. 10 ;Y4 and Y 10 The input to the ABF module is used for feature fusion to obtain the remote sensing feature map Y of the eleventh transmission tower foundation. 11 ;Y 11 The EIC module was used for feature extraction to obtain the remote sensing feature map Y of the twelfth transmission tower foundation. 12 .

[0083] Step 3.4.3: Place Y8, Y 10 and Y 12Inputting the identification head will yield the corresponding remote sensing feature map Y of the fifteenth transmission tower foundation. 15 Remote sensing feature map of the fourteenth transmission tower foundation Y 14 Remote sensing feature map of the 13th transmission tower foundation Y 13 In this recognition head, the input is used to perform location regression and defect classification of the tower base defect target using regression convolution and classification convolution, respectively. The regression convolution maps the input to four parameters of the prediction box: the x-coordinate and y-coordinate of the prediction box center, the width w and the height h of the prediction box, and the classification convolution maps the input to the number of target categories n.

[0084] Step 3.4.4: Y 13 Y 14 and Y 15 The images are merged, and non-maximum suppression is used to remove overlapping prediction boxes to obtain the final output, which is the remote sensing feature map Y of the sixteenth transmission tower base. 16 .

[0085] As one embodiment of the present invention, step S4 specifically involves initializing various parameters and hyperparameters of TBDI-Net, such as the number of training iterations, batch size, optimizer, learning rate type, and initial learning rate. After completing the above preparations, training data and validation data are input into TBDI-Net for training until the loss function converges. In each round of training, defect identification data of transmission tower foundations from the training set are captured according to the preset batch size for training, and the loss value and various parameters are updated. After each round of training, defect identification data of transmission tower foundations from the validation set are captured according to the preset batch size for validation to check the effectiveness of each round of training. When the loss value tends to converge, the TBDI-Net training ends.

[0086] Example application:

[0087] Remote sensing images containing the bases of power transmission line towers were acquired using satellite platforms such as Gaofen-1, UAVs, or aerial photography. To ensure sufficient data volume, image data with tower base defects were filtered out, and 300 images containing power transmission line tower base defects {i1,i2,…,i...} were cropped to a size of 640×640 pixels. 300 Subsequently, the LabelImg image annotation tool was used to annotate the categories and coordinate bounding boxes of all transmission tower foundation defect images. A pre-defined VOC annotation format was used to select and mark the tower foundation area, creating bounding boxes and category labels. For the category labels, this scheme categorizes tower foundation defects into {tower foundation concrete cracking, tower foundation water accumulation, tower foundation vegetation growth, tower foundation debris, tower foundation component missing, and other categories}, corresponding to category indices {0, 1, 2, 3, 4}. After annotation, the annotated image data {i1, i2, ..., i...} is generated. 300The annotation files {l1,l2,…,l} correspond one-to-one. 300 The data format is .xml. The labeled dataset {i1,i2,…,i...} is... 300},{l1,l2,…,l 300 Perform data augmentation operations on {i1,i2,…,i}, including data rotation, flipping, cropping, random occlusion, contrast adjustment, random scaling, and noise addition. Each operation affects {i1,i2,…,i}. 300} and {l1,l2,…,l 300 Each data augmentation operation will generate twice the amount of augmented data. Finally, the total dataset {i1, i2, ..., i} will be obtained, including the original data and all the augmented data. 2400} and {l1,l2,…,l 2400}

[0088] After the remote sensing image dataset of the power transmission tower base is preprocessed, the dataset is divided. In this scheme, the training set, validation set and test set are randomly divided in a ratio of 7:2:1, resulting in 1680 training data sets, 480 validation data sets and 240 test data sets.

[0089] The identification algorithm designed according to the present invention trains the dataset after data partitioning: First, the various parameters and hyperparameters of the transmission tower base defect identification training are initialized. This embodiment describes several important parameters in the algorithm training: the corresponding training batch size is initialized according to the hardware environment, the optimizer is initialized to optimizer, the batchsize is set to 4, the number of training rounds is initialized to 200, the learning rate type is cosine learning rate, and the initial learning rate is 0.001. These parameters need to be adjusted according to the effect of multiple training sessions of the algorithm until the algorithm converges.

[0090] After initializing the basic parameters for algorithm training, model training begins. The preprocessed data is input into the TBDI-Net designed in this invention. First, the input training image is scaled to 640×640 pixels to obtain the first remote sensing feature map Y1 of the transmission tower base. In the algorithm network, Y1 is input into the backbone network of the model: Y1 is input into a combination of the first layer convolution, batch normalization, ReLU function, and EIC module to obtain the second remote sensing feature map Y2 of the transmission tower base; Y2 is input into a combination of the second layer convolution, batch normalization, ReLU function, and EIC module to obtain the third remote sensing feature map Y3 of the transmission tower base; Y3 is input into a combination of the third layer convolution, batch normalization, ReLU function, EIC module, and spatial pyramid pooling module to obtain the fourth remote sensing feature map Y4 of the transmission tower base. At this point, the backbone network feature extraction is complete. Inputting Y3 and Y4 into module C yields remote sensing feature image Y5 for the fifth transmission tower foundation; inputting Y5 into module EIC yields remote sensing feature image Y6 for the sixth transmission tower foundation; inputting Y2 and Y6 into module ABF yields remote sensing feature image Y7 for the seventh transmission tower foundation; inputting Y7 into module EIC yields remote sensing feature image Y8 for the eighth transmission tower foundation; inputting Y6 and Y8 into module ABF yields remote sensing feature image Y9 for the ninth transmission tower foundation; inputting Y9 into module EIC yields remote sensing feature image Y1 for the tenth transmission tower foundation. 10 ;Y4 and Y 10 Inputting into the ABF module yields the remote sensing feature map Y of the eleventh transmission tower foundation. 11 At this point, the feature fusion structure processing for defect identification of transmission line tower foundations is complete. The Y8 and Y8 values ​​output from the feature fusion structure are then processed. 10 and Y 12 The corresponding inputs to the algorithm's recognition headers yield the output remote sensing feature map Y of the fifteenth transmission tower foundation. 15 Remote sensing feature map of the fourteenth transmission tower foundation Y 14 Remote sensing feature map of the 13th transmission tower foundation Y 13 The output format is (x,y,w,h,conf1,…,conf) i ,…,conf n ), where x represents the x-coordinate of the predicted bounding box center, y represents the y-coordinate of the predicted bounding box center, w represents the width of the predicted bounding box, h represents the height of the predicted bounding box, and conf i represents the confidence level of the category corresponding to the i-th index, and n is the number of defect categories in the transmission tower foundation. At this point, the head recognition part of the transmission tower foundation defect identification process is complete. Finally, the output Y of the head recognition part is... 13 Y 14 and Y 15 The bounding boxes are merged, and non-maximum suppression is applied to remove overlapping prediction boxes, resulting in the final output Y of the algorithm. 16The output format is the same: (x,y,w,h,conf1,…,conf) i ,…,conf n ).

[0091] TBDI-Net training updates internal parameters through backpropagation of the loss function. In the loss function described in this invention, the bounding box coordinate loss for tower base defects uses DFL loss, the bounding box confidence loss uses WIoU loss, and the classification loss for determining whether a target is a tower base defect uses cross-entropy loss. These loss functions are updated epoch-by-epoch during gradient backpropagation during algorithm training, allowing the algorithm to gradually converge and improve detection accuracy with each epoch. After each training epoch, the algorithm uses the partitioned validation data to verify accuracy (P), recall (R), and mean precision (mAP), serving as evaluation criteria for training effectiveness. After TBDI-Net training is completed, 240 partitioned remote sensing image datasets of transmission tower bases are used as test data. This data is input into the optimally trained TBDI-Net network for performance testing.

[0092] In practical applications, pilot applications are deployed by inputting remote sensing images of power transmission tower foundation defects into TBDI-Net, and the output results are obtained for the extraction of power transmission tower foundation defects. The output results contain tensors of prediction information, with each row corresponding to a prediction, including bounding box coordinates, category labels and confidence scores.

[0093] This invention, Feature Pyramid Shared Convolution (FPSC), constructs a feature pyramid structure through dilated convolutions with different dilation rates. This ensures the capture of finer-grained local features while maintaining global contextual features. Simultaneously, by sharing the internal parameters of the dilated convolution kernels, it reduces algorithm parameter redundancy and computational overhead, thereby improving computational speed in real-world scenarios.

[0094] The Edge Information Feature Capture Module (EIC) of this invention extracts different types of features in parallel through a dual-branch structure, effectively capturing tower foundation damage information (such as tower foundation concrete cracks) in remote sensing images while improving the algorithm's ability to identify complex fine-grained feature information such as multi-shaped cracks.

[0095] This invention designs an adaptive feature fusion unit (AFFU) and an adaptive bidirectional feature fusion module (ABF) based on AFFU. The adaptive bidirectional feature fusion module effectively aggregates high and low layer feature information by designing an adaptive bidirectional fusion structure between deep and shallow features, reducing the loss of fine-grained targets caused by direct fusion of high and low layer features, and improving the multi-scale feature fusion effect of the algorithm on images of transmission tower foundation defects.

[0096] This invention establishes a highly generalizable, accurate, and automated network for identifying defects in transmission tower foundations. It can identify fine-grained defects and potential hazards. After training, the algorithm designed in this solution can be deployed on towers along various transmission lines to achieve real-time observation and maintenance of transmission tower foundations.

[0097] Example 2:

[0098] This invention also provides a device for identifying defects in transmission line tower foundations, comprising:

[0099] The acquisition module is used to acquire remote sensing images of transmission lines, including tower foundations.

[0100] The partitioning module is used to preprocess remote sensing images of transmission lines containing tower foundations and to partition the preprocessed remote sensing image dataset of transmission line tower foundations to obtain training and test sets.

[0101] The construction module is used to construct the TBDI-Net network for identifying defects in power transmission tower foundations based on the preprocessed remote sensing image dataset of power transmission tower foundations. The TBDI-Net network for identifying defects in power transmission tower foundations includes: a feature pyramid shared convolution, an edge information feature capture module, and a feature adaptive bidirectional fusion module based on the adaptive feature fusion unit AFFU.

[0102] The training module is used to train the TBDI-Net network for identifying defects in transmission tower foundations based on the training set;

[0103] The identification module is used to input the test set into the trained transmission line tower foundation defect identification network TBDI-Net to identify the tower foundation defect targets existing in the transmission line towers.

[0104] Example 3:

[0105] This invention also provides a transmission line tower foundation defect identification system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a transmission line tower foundation defect identification method when run by the processor.

[0106] Example 4:

[0107] This invention also provides a storage medium storing a computer program that executes a method for identifying defects in transmission line tower foundations during runtime.

[0108] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for identifying defects in transmission line tower foundations, characterized in that, include: Step S1: Acquire remote sensing images of the transmission line including the tower bases; Step S2: Preprocess the remote sensing images of transmission lines containing tower foundations, and divide the preprocessed remote sensing image dataset of transmission line tower foundations to obtain training set and test set; Step S3: Based on the preprocessed remote sensing image dataset of transmission tower foundations, construct the transmission tower foundation defect identification network TBDI-Net; wherein, the transmission tower foundation defect identification network TBDI-Net includes: feature pyramid shared convolution, edge information feature capture module, and feature adaptive bidirectional fusion module based on adaptive feature fusion unit AFFU; Step S4: Train the TBDI-Net network for identifying defects in transmission tower foundations based on the training set; Step S5: Input the test set into the trained transmission line tower foundation defect identification network TBDI-Net to identify the tower foundation defect targets existing in the transmission line towers; In step S3, the structure of the feature pyramid shared convolution is a combination of dilated convolution serial and multiple residual connections. A dual-branch structure is designed in the edge information feature capture module to extract different types of features in parallel. In the first branch, the pixel intensity changes in the target image are extracted by Sobel 3D convolution. In the second branch, the feature pyramid shared convolution is used to fit the fine-grained feature information of cracks of multiple shapes and scales. The design feature adaptive bidirectional fusion module specifically includes: Step 3.3.1: Input the first shallow feature map L1 from the shallow layer and the first deep feature map H1 from the deep layer into AFFU. In AFFU, let the inputs be the first fused feature map X1 and the second fused feature map X2, respectively. First, X1 and X2 are both input into a 1×1 convolution to reduce the number of channels. Then, batch normalization and ReLU function operations are performed sequentially to obtain the third fused feature map X3 and the fourth fused feature map X4, respectively. X3 and X4 are then input into the bidirectional fusion structure to obtain the output of AFFU, the fifth fused feature map X5. The operation process function expression of the feature adaptive bidirectional fusion module is designed as follows: X3=ReLU(BN(C 1×1 (X1))) (5) X4=ReLU(BN(C 1×1 (X2))) (6) X5=UpSample(X4)·X2+UpSample(X3)·X1·(UpSample(Reverse(X4))) +X2 (7) In equations (5) and (6), ReLU(·) represents the ReLU activation function, BN(·) represents the batch normalization operation, and C 1×1 (·) represents a 1×1 convolution operation. In equation (7), UpSample(·) represents an upsampling operation, and Reverse(·) represents a reversal operation. Step 3.3.2: After inputting L1 and H1 into AFFU, the second shallow feature map L2 and the second deep feature map H2 are obtained respectively. L2 and H2 are concatenated along the channel dimension to obtain the first bidirectional fusion feature map M1. Then, M1 is input into Partial convolution for operation. After the operation, batch normalization and ReLU function operation are performed to obtain the output of the ABF module, which is the second bidirectional fusion feature map M2. The operation process function expression of the feature adaptive bidirectional fusion module is designed as follows: L2=AFFU(L1,H1) (8) H2=AFFU(H1,L1) (9) M1 = Concat(L2, H2) (10) M2=ReLU(BN(PartialConv(M1))) (11) In equations (8) and (9), AFFU(·) means using the AFFU module to perform operations on the input; in equation (10), Concat(·) means performing concatenation operations along the channel dimension; in equation (11), ReLU(·) means the ReLU activation function; BN(·) means batch normalization operation; and PartialConv(·) means using Partial convolution to perform operations on the input feature map. Step 3.4: Based on the feature pyramid shared convolution, edge information feature capture module and feature adaptive bidirectional fusion module, the TBDI-Net for power transmission tower foundation defect identification is obtained, which is used to realize power transmission tower foundation defect identification based on remote sensing images.

2. A transmission tower foundation defect identification device for implementing the transmission tower foundation defect identification method of claim 1, characterized in that, include: The acquisition module is used to acquire remote sensing images of transmission lines, including tower foundations. The partitioning module is used to preprocess remote sensing images of transmission lines containing tower foundations and to partition the preprocessed remote sensing image dataset of transmission line tower foundations to obtain training and test sets. The construction module is used to construct the TBDI-Net network for identifying defects in power transmission tower foundations based on the preprocessed remote sensing image dataset of power transmission tower foundations. The TBDI-Net network for identifying defects in power transmission tower foundations includes: a feature pyramid shared convolution, an edge information feature capture module, and a feature adaptive bidirectional fusion module based on the adaptive feature fusion unit AFFU. The training module is used to train the TBDI-Net network for identifying defects in transmission tower foundations based on the training set; The identification module is used to input the test set into the trained transmission line tower foundation defect identification network TBDI-Net to identify the tower foundation defect targets existing in the transmission line towers.

3. A defect identification system for transmission line tower foundations, characterized in that, include: The system includes a memory and a processor, wherein the memory stores a computer program that is executed by the processor, and the computer program, when executed by the processor, performs the method for identifying defects in the base of a transmission tower as described in claim 1.

4. A storage medium, characterized in that, The storage medium stores a computer program, which executes the method for identifying defects in transmission tower foundations as described in claim 1 when it runs.

Images