Power transmission line tower footing defect identification method, device and system, and storage medium

Abstract

The invention discloses a power transmission line tower footing defect identification method, device and system, and a storage medium. The method comprises the following steps: S1, obtaining a remote sensing image containing a tower footing on a power transmission line; s2, preprocessing the remote sensing image containing the tower footing of the tower on the power transmission line, and dividing the preprocessed remote sensing image data set of the tower footing of the power transmission line to obtain a training set and a test set; s3, constructing a power transmission line tower footing defect identification network TBDI-Net according to the preprocessed power transmission line tower footing remote sensing image data set; s4, training a power transmission line tower footing defect identification network TBDI-Net according to the training set; and S5, inputting the test set into the trained power transmission line tower footing defect identification network TBDI-Net, and identifying a tower footing defect target existing in the power transmission line tower. By the adoption of the technical scheme, troubleshooting of the power transmission line tower footing can be remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a method and device, a system, and a storage medium for identifying defects in a transmission line tower foundation. Background Art

[0002] As an important part of the power transmission line in the power system, the transmission line tower foundation supports the basic structure of the tower, plays the role of bearing and stabilizing the tower, and is an important load for the power system responsible for long-distance power transmission. The transmission line tower foundation is usually built with concrete, steel or other durable materials to ensure long-term stability and corrosion resistance.

[0003] At present, the inspection of tower foundation defects along the transmission lines in the power system requires manual screening of image data and inspection, or on-site inspection, which requires a lot of time and professional manpower. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method and device, system and storage medium for identifying defects in a transmission line tower foundation, which can significantly improve the inspection of transmission line tower foundations.

[0005] To achieve the above object, the present invention adopts the following technical solution:

[0006] A method for identifying defects in a transmission line tower foundation, comprising:

[0007] Step S1, obtaining a remote sensing image including a tower base on a transmission line;

[0008] Step S2, preprocessing the remote sensing images of the transmission line including the tower base, and dividing the preprocessed remote sensing image data set of the transmission line tower base to obtain a training set and a test set;

[0009] Step S3, constructing a transmission line tower base defect recognition network TBDI-Net according to the preprocessed transmission line tower base remote sensing image data set; wherein the transmission line tower base defect recognition network TBDI-Net includes: a feature pyramid shared convolution, an edge information feature capture module, and a feature adaptive bidirectional fusion module based on an adaptive feature fusion unit AFFU;

[0010] Step S4, training the transmission line tower base defect identification network TBDI-Net according to the training set;

[0011] Step S5: input the test set into the trained transmission line tower base defect identification network TBDI-Net to identify tower base defect targets existing in the transmission line towers.

[0012] Preferably, the feature pyramid shared convolution adopts a structure combining serial and multi-residual connections, and constructs a feature pyramid structure through dilated convolutions with different dilation rates.

[0013] Preferably, the edge information feature capture module captures tower base damage information in remote sensing images by designing a dual-branch structure to extract different types of features in parallel.

[0014] Preferably, the feature adaptive bidirectional fusion module aggregates boundary information and semantic information by designing an adaptive bidirectional fusion structure between deep and shallow features.

[0015] The present invention also provides a transmission line tower foundation defect identification device, comprising:

[0016] Acquisition module, used to acquire remote sensing images of transmission lines including tower foundations

[0017] A partitioning module is used to pre-process the remote sensing images of the transmission line including the tower base, and to partition the pre-processed remote sensing image data set of the transmission line tower base to obtain a training set and a test set;

[0018] A construction module is used to construct a transmission line tower base defect recognition network TBDI-Net according to a preprocessed transmission line tower base remote sensing image dataset; wherein the transmission line tower base defect recognition network TBDI-Net includes: a feature pyramid shared convolution, an edge information feature capture module, and a feature adaptive bidirectional fusion module based on an adaptive feature fusion unit AFFU;

[0019] A training module, used to train the transmission line tower base defect identification network TBDI-Net according to the training set;

[0020] The recognition module is used to input the test set into the trained transmission line tower base defect recognition network TBDI-Net to identify the tower base defect targets existing in the transmission line towers.

[0021] An embodiment of the present invention further provides a transmission line tower foundation defect identification system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a transmission line tower foundation defect identification method when executed by the processor.

[0022] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and the computer program executes a method for identifying defects in a transmission line tower foundation when the computer program is run.

[0023] The present invention combines deep learning computer vision algorithms with remote sensing images to achieve real-time monitoring of transmission line tower base defects. First, a feature pyramid shared convolution is designed, and an edge information feature capture is designed based on the convolution, which has higher computational efficiency and generalization ability than the conventional serial convolution feature extraction method; then, the present invention designs an edge information feature capture module EIC, which extracts different types of features in parallel by designing a dual-branch structure, effectively captures tower base damage information in remote sensing images (such as tower base concrete cracks), and improves the algorithm's recognition of more complex fine-grained feature information such as multi-shape cracks; finally, the present invention designs an adaptive feature fusion unit, which solves the problem of loss in the fine-grained feature extraction process compared to the conventional feature direct fusion method; at the same time, a feature adaptive bidirectional fusion module is designed based on the fusion unit to improve the algorithm's recognition ability of fine-grained features such as tower base concrete cracks. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0025] Figure 1 This is a flow chart of a method for identifying defects in a transmission line tower foundation according to an embodiment of the present invention;

[0026] Figure 2 Schematic diagram of the TBDI-Net structure for transmission line tower foundation defect identification;

[0027] Figure 3 This is the FPSC module structure diagram;

[0028] Figure 4 This is the EIC module structure diagram;

[0029] Figure 5 This is the AFFU module structure diagram;

[0030] Figure 6 This is the ABF module structure diagram. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] Embodiment 1:

[0034] like Figure 1 As shown, an embodiment of the present invention provides a method for identifying defects in a transmission line tower foundation, comprising:

[0035] Step S1: Obtain remote sensing images of the transmission line including the tower base

[0036] Step S2, preprocessing the remote sensing images of the transmission line including the tower base, and dividing the preprocessed remote sensing image data set of the transmission line tower base to obtain a training set, a verification set and a test set;

[0037] Step S3, constructing a transmission line tower base defect identification network TBDI-Net according to the pre-processed transmission line tower base remote sensing image dataset;

[0038] Step S4, training the transmission line tower base defect identification network TBDI-Net according to the training set;

[0039] Step S5: input the test set into the trained transmission line tower base defect identification network TBDI-Net to identify tower base defect targets existing in the transmission line towers.

[0040] As an implementation method of an embodiment of the present invention, in step S1, remote sensing images of the tower foundation on the transmission line are obtained by satellite, drone or aerial photography, and data with tower foundation defects are screened. After the remote sensing images of the transmission line tower foundation are screened, the tower foundation targets in all the transmission line remote sensing images are annotated with categories and coordinate frame information using an image annotation tool, where the categories are divided into tower foundation concrete cracks, tower foundation water accumulation, tower foundation plant growth, tower foundation debris, tower foundation component missing and other categories. Finally, the initial transmission line tower foundation defect remote sensing dataset D is obtained after screening and annotation. 1 .

[0041] As an implementation method of an embodiment of the present invention, in step S2, based on the initial transmission line tower foundation defect remote sensing dataset D after screening and labeling, 1 The data rotation, flipping, cropping, random occlusion, contrast adjustment, random scaling and noise addition techniques are used to enhance the remote sensing image dataset of the transmission line tower base, and the dataset D 1 With all the enhanced datasets D 2 , then, for the dataset D 2 Perform random data division of training data, validation data, and test data in proportion to obtain D 3 , D4 and D 5

[0042] As an implementation of an embodiment of the present invention, in step S3, the transmission line tower base defect identification TBDI-Net (Transmission-Line Base Defect Identification Net) is used to identify tower base defect targets existing in transmission line towers. The transmission line tower base defect identification network TBDI-Net includes: a feature pyramid shared convolution, an edge information feature capture module, and a feature adaptive bidirectional fusion module based on an adaptive feature fusion unit AFFU, such as Figure 2 shown.

[0043] Step S3 specifically includes:

[0044] Step 3.1: Design Feature Pyramid Shared Conv (FPSC). FPSC uses a feature pyramid structure constructed by dilated convolutions with different dilation rates to enhance the defect feature extraction capability of the transmission line tower base. It captures finer-grained local features while ensuring the capture of global context features. At the same time, it reduces parameter redundancy and computational overhead by sharing the internal parameters of the dilated convolution kernel. The structure of the feature pyramid shared convolution designed by the present invention is as follows: Figure 3 In the disclosed pyramid convolution structure, the input is parallelized in the pyramid convolution (parallel and side-by-side like a pyramid), and then the corresponding multiple outputs are added, which is completely different from the pyramid convolution proposed in the present invention in structure: the disclosed pyramid convolution structure is that the input enters a multi-convolution parallel structure, while the pyramid convolution structure of the present invention is a structure combining serial and multi-residual connections.

[0045] Step 3.1.1: Use 1×1 convolution to compress the number of channels of the transmission line tower base defect feature map to reduce the input spatial dimension and obtain the first feature pyramid feature map F 1 , then, F 1 Perform batch normalization and ReLU function operations to obtain the second feature pyramid feature map F 2 . Among them, "using 1×1 convolution to compress the number of channels": The role of 1×1 convolution is equivalent to feature reorganization or mixing in the channel dimension, retaining the key information in the input features. Reducing the number of input channels helps reduce the number of network parameters and avoid excessive model complexity. There are multiple convolution operations with different expansion rates. If the number of channels is not reduced, directly processing the original high-dimensional feature map will result in a high computational cost.

[0046] Step 3.1.2: F 2Input three serially arranged dilated convolutions (dilation rates are 1, 3, and 5 respectively), where the three dilated convolutions all use the same shared convolution kernel during operation. Finally, the output third feature pyramid feature map F is obtained in turn. 3 , the fourth feature pyramid feature map F 4 And the fifth feature pyramid feature map F 5 .

[0047] Step 3.1.3: F 2 、F 3 、F 4 and F 5 Splicing along the channel dimension, the number of channels is F 2 、F 3 、F 4 and F 5 The sixth feature pyramid feature map F of the sum of the number of channels 6 .

[0048] Step 3.1.4: F 6 Input 1×1 convolution, restore and adjust the number of channels, and then perform batch normalization and ReLU function operation to obtain the seventh feature pyramid feature map F 7 .

[0049] Among them, (1) dilated convolution is an existing technology, and others such as convolution, batch normalization, ReLU activation function and splicing operations are common operations in the deep learning industry.

[0050] (2) Residual connection: This is a commonly used technique in deep neural networks. By adding the output of the previous layer directly to the input of the current layer, the network can learn the residual (i.e., the change of the current layer relative to the previous layer), making it easier to train deep neural networks.

[0051] (3) Normalization BN: refers to the technology of normalizing the input data of each mini-batch during the training process of the neural network. It reduces the internal covariate shift by normalizing the input data to a distribution with a mean of 0 and a variance of 1 (or close to this distribution), thereby accelerating the training of the neural network and improving its performance.

[0052] (4) ReLU (Rectified Linear Unit) activation function is a nonlinear activation function commonly used in deep learning.

[0053] Step 3.2: Design an edge information feature capture module EIC (Edge Information Capture), and design a dual-branch structure in this module to extract different types of features in parallel: in the first branch, the Sobel three-dimensional convolution is used to extract the pixel intensity changes in the target image, and the damage information of the tower foundation edge in the remote sensing image is effectively captured; in the second branch, the FPSC designed in Step 3.1 is used to speed up the fitting of fine-grained feature information such as cracks of multiple shapes and scales, and improve the generalization ability of defect recognition in tower foundation remote sensing images. Finally, the input features are enhanced by feature splicing and residual connection. The structure of the edge information feature capture module designed by the present invention is as follows: Figure 4 shown.

[0054] Step 3.2.1: Input the transmission line tower base defect feature map into the 1×1 convolution, and reduce the input spatial dimension by compressing the number of channels to obtain the first edge information feature map T 1 , then, T 1 Perform batch normalization and ReLU function operations in sequence to obtain the second edge information feature map T after channel number compression 2 .

[0055] Step 3.2.2: T 2 Input Sobel convolution to extract edge features of the image. First, in order to operate the Sobel convolution normally, the four-dimensional tensor T 2 Expanded to a five-dimensional tensor, that is, from a four-dimensional tensor containing batch size, number of channels, height, and width to a five-dimensional tensor containing batch size, number of channels, extended dimension placeholder, height, and width, where the third dimension of the five-dimensional tensor is a newly inserted extended dimension placeholder with a value of 0 or 1, so that T 2 Convolution operations can be performed in both horizontal and vertical directions. 2 Two 3×3 convolution kernels are input into the Sobel convolution for processing. One convolution kernel is used for edge filtering in the horizontal direction, and the other convolution kernel is used for edge filtering in the vertical direction. Then, the two convolution results after the operations in the horizontal and vertical directions are added together to obtain the output of the Sobel convolution, that is, the third edge information feature map T 3 .

[0056] Step 3.2.3: T 2 Input the FPSC designed in Step 3.1 for feature extraction to obtain the fourth edge information feature map T 4 .

[0057] Step 3.2.4: T 3 and T 4Splicing along the channel dimension to obtain the fifth edge information feature map T 5 .

[0058] Step 3.2.5: T 5 Input the ELA (Efficient Local Attention) attention mechanism module. In the ELA module, first, T 5 Perform adaptive pooling horizontally and vertically respectively to obtain T h and T w , T h and T w The horizontal and vertical position information of the one-dimensional convolution enhancement feature map is input respectively; then, the group normalization and sigmoid activation function are input respectively for operation to obtain T h and T w .

[0059]

[0060] T h =σ(G n (F h (T h ))) (3)

[0061] T w =σ(G n (F w (T w ))) (4)

[0062] In formula (1), T h T 5 The result of horizontal adaptive pooling is: In formula (2), T w T 5 The result of vertical adaptive pooling, H represents the height, W represents the width, in formula (3) and (4), F h (·) and F w (·) represents one-dimensional convolution, G n (·) represents group normalization, and σ(·) represents the sigmoid function.

[0063] Then, T 5 , T h and T w Multiply them together to get the sixth edge information feature map T of the ELA module 6 .

[0064] T 6 =T 5 ×T h ×T w

[0065] Step 3.2.6: T 6 Input 1×1 convolution, restore the number of channels, and then perform batch normalization and ReLU function operations in sequence to obtain the seventh edge information feature map T 7 .

[0066] Step 3.2.7: T 7 With T 2 Perform residual connection to obtain the eighth edge information feature map T 8 .

[0067] Step 3.2.8: T 8 Input 1×1 convolution, adjust the number of channels, and then perform batch normalization and ReLU function operations in sequence to obtain the ninth edge information feature map T 9 .

[0068] Step 3.3: Design the adaptive feature fusion unit AFFU (Adaptive Feature Fusion Unit), such as Figure 5 As shown in FIG, a feature adaptive bidirectional fusion module ABF (Adaptive Bidirectional Fusion) is designed based on AFFU, as shown in FIG. Figure 6 As shown in the figure, as an important part of the feature fusion structure of TBDI-Net, a bidirectional fusion structure between deep and shallow features is designed to effectively aggregate boundary information and semantic information, reduce the loss of fine-grained targets caused by direct fusion of high-level and low-level features, and further improve the effect of multi-scale feature fusion.

[0069] Step 3.3.1: The first shallow feature map L from the shallow layer 1 and the first deep feature map H from the deep layer 1 Input AFFU: In the AFFU module, the inputs are the first fusion feature map X 1 and the second fused feature map X 2 First, X 1 and X 2 The 1×1 convolution is input to reduce the number of channels, followed by batch normalization and ReLU function operations, and the third fusion feature map X is obtained respectively. 3 And the fourth fusion feature map X 4 , X 3 and X 4 Continue to input the bidirectional fusion structure designed in this scheme to obtain the output fifth fusion feature map X of AFFU 5 The calculation process function expression corresponding to the designed adaptive feature fusion unit is as follows:

[0070] X 3 =ReLU(BN(C1×1 (X 1 ))) (5)

[0071] X 4 =ReLU(BN(C 1×1 (X 2 ))) (6)

[0072] X 5 =UpSample(X 4 )·X 2 +UpSample(X 3 )·X 1 (UpSample (Reverse (X 4 )))+X 2 (7)

[0073] In equations (5) and (6), ReLU(·) represents the ReLU activation function, BN(·) represents the batch normalization operation, and C 1×1 (·) represents a 1×1 convolution operation. In formula (7), UpSample(·) represents an upsampling (deconvolution) operation, and Reverse(·) represents a reversal operation.

[0074] Step3.3.2: From Step3.3.1 and Figure 5 It can be seen that when L 1 and H 1 After inputting AFFU, the second shallow feature map L can be obtained respectively. 2 and the second deep feature map H 2 , L 2 and H 2 Splicing along the channel dimension to obtain the first bidirectional fusion feature map M 1 , then, M 1 The partial convolution is input for operation, and then batch normalization and ReLU function operation are performed to obtain the output of the ABF module, that is, the second bidirectional fusion feature map M. 2 The operation process function expression corresponding to the designed adaptive bidirectional fusion structure is as follows:

[0075] L 2 =AFFU(L 1 ,H 1 ) (8)

[0076] H 2 =AFFU(H 1 ,L 1 ) (9)

[0077] M 1 =Concat(L 2 ,H2 ) (10)

[0078] M 2 =ReLU(BN(PartialConv(M 1 ))) (11)

[0079] In formulas (8) and (9), AFFU(·) indicates that the AFFU module is used to operate on the input. In formula (10), Concat(·) indicates the concatenation operation along the channel dimension. In formula (11), ReLU(·) indicates the ReLU activation function, BN(·) indicates the batch normalization operation, and PartialConv(·) indicates the operation on the input feature map using partial convolution.

[0080] Step 3.4: Based on the FPSC convolution designed in Step 3.1, the EIC module designed in Step 3.2, and the ABF module designed in Step 3.3, a transmission line tower foundation defect recognition method TBDI-Net is proposed to realize the transmission line tower foundation defect recognition based on remote sensing images, such as Figure 2 shown.

[0081] Step 3.4.1: A three-layer backbone network (Backbone) for extracting features is designed in the transmission line tower base defect recognition TBDI-Net. The three-layer backbone network consists of a combination of two layers of convolution, batch normalization, ReLU function and EIC module, and a combination of convolution, batch normalization, ReLU function, EIC module and spatial pyramid pooling module. The remote sensing feature map Y of the first transmission line tower base is taken as 1 Input the backbone network of the model, and after three layers of feature extraction, the remote sensing feature map Y of the second transmission line tower base is obtained. 2 , remote sensing characteristic map of the third transmission line tower base Y 3 , Remote sensing characteristic map of the fourth transmission line tower base Y 4 .

[0082] Step 3.4.2: Y 3 and Y 4 Input the ABF module for feature fusion to obtain the remote sensing feature map Y of the fifth transmission line tower base 5 ; Y 5 Input the EIC module for feature extraction to obtain the remote sensing feature map Y of the sixth transmission line tower base 6 ; Y 2 and Y 6 Input the ABF module for feature fusion to obtain the remote sensing feature map Y of the seventh transmission line tower base 7 ; Y 7 Input the EIC module for feature extraction to obtain the remote sensing feature map Y of the eighth transmission line tower base 8 ; Y6 and Y 8 Input the ABF module for feature extraction to obtain the remote sensing feature map Y of the ninth transmission line tower base 9 ; Y 9 Input the EIC module for feature fusion to obtain the remote sensing feature map Y of the tenth transmission line tower base 10 ; Y 4 and Y 10 Input the ABF module for feature fusion to obtain the remote sensing feature map Y of the eleventh transmission line tower base 11 ; Y 11 Input the EIC module for feature extraction to obtain the remote sensing feature map Y of the twelfth transmission line tower base 12 .

[0083] Step 3.4.3: Y 8 , Y 10 and Y 12 Input the identification head (Head) respectively, and get the corresponding remote sensing feature map Y of the fifteenth transmission line tower base 15 , Remote sensing characteristic map of the 14th transmission line tower base Y 14 And the remote sensing characteristic map of the 13th transmission line tower base Y 13 In the recognition head, the input is used for position regression and defect classification of the tower base defect target using regression convolution and classification convolution respectively. The regression convolution maps the input to four parameters of the prediction box: the horizontal coordinate x and vertical coordinate y of the prediction box center, the prediction box width w and the prediction box height h, and the classification convolution maps the input to the number of target categories n.

[0084] Step 3.4.4: Y 13 , Y 14 and Y 15 Merge and perform non-maximum suppression to remove overlapping prediction boxes to obtain the final output, which is the remote sensing feature map Y of the 16th transmission line tower base 16 .

[0085] As an implementation method of an embodiment of the present invention, step S4 is specifically: initialize various parameters and hyperparameters of TBDI-Net, such as: number of training times, batch size, optimizer, learning rate type and initial learning rate, etc. After completing the above preparations, input the training data and verification data into TBDI-Net for training until the loss function converges. In each round of training, the transmission line tower base defect recognition data in the training set is captured according to the batch preset value for training, and the loss value and various parameters are updated. After each round of training is completed, the transmission line tower base defect recognition in the verification set is also captured according to the batch preset value for verification to check the effectiveness of each round of training. When the training is completed until the loss value tends to converge, the TBDI-Net training ends.

[0086] Sample Application:

[0087] Use satellite platforms such as Gaofen-1, drones or aerial photography to obtain remote sensing images containing the foundation of transmission line towers. To ensure sufficient data volume, screen image data with tower foundation defects and crop them to 640×640 pixels to obtain 300 images containing transmission line tower foundation defects. 1 ,i 2 ,…,i 300}, then, the tower base defect targets of all transmission line tower base defect images are labeled with categories and coordinate frame information through the LabelImg image annotation tool: pre-set VOC annotation format, select and mark the tower base area, create bounding boxes and category labels, and for category labels, this solution divides tower base defect categories into {tower base concrete cracking, tower base water accumulation, tower base plant growth, tower base debris, tower base component missing, other categories}, corresponding to category indexes {0, 1, 2, 3, 4} respectively. After the annotation is completed, the annotated image data {i 1 ,i 2 ,…,i 300}One-to-one corresponding annotation files {l 1 ,l 2 ,…,l 300}, the data format is .xml. 1 ,i 2 ,…,i 300},{l 1 ,l 2 ,…,l 300} performs data augmentation operations, including data rotation, flipping, cropping, random occlusion, contrast adjustment, random scaling, and noise addition. Each item has an effect on {i 1 ,i 2 ,…,i 300} and {l 1 ,l 2 ,…,l 300} data enhancement operations will generate double the amount of enhanced data. Finally, the total data set {i 1 ,i 2 ,…,i 2400} and {l 1 ,l 2 ,…,l 2400}.

[0088] After the preprocessing of the transmission line tower base remote sensing image dataset is completed, the dataset is divided. This scheme randomly divides the data into training set, validation set and test set in a ratio of 7:2:1. After the division, 1680 training data, 480 validation data and 240 test data can be obtained.

[0089] The recognition algorithm designed according to the present invention is used to train the data set after data division: first, various parameters and hyperparameters of the transmission line tower foundation defect recognition training are initialized. This embodiment describes several important parameters in the algorithm training: the corresponding training batch size is initialized according to the hardware environment, the optimizer is initialized as the optimizer, the Batchsize is set to 4, the training rounds are initialized to 200, the learning rate type is the cosine learning rate, and the initial learning rate is 0.001. These parameters need to be adjusted according to the results of multiple training of the algorithm until the algorithm converges.

[0090] After completing the initialization of the basic parameters of the algorithm training, the model training begins. The preprocessed data is input into the TBDI-Net designed by the present invention. First, the input image to be trained is scaled to 640×640 pixels to obtain the first transmission line tower base remote sensing feature map Y 1 ; In this algorithm network, Y 1 Input the backbone network of the model: 1 Input the combination of the first layer of convolution, batch normalization, ReLU function and EIC module to obtain the remote sensing feature map Y of the second transmission line tower base 2 ; Y 2 Input the combination of the second layer of convolution, batch normalization, ReLU function and EIC module to obtain the remote sensing feature map Y of the third transmission line tower base 3 ; Y 3 Input the combination of the third layer of convolution, batch normalization, ReLU function, EIC module and spatial pyramid pooling module to obtain the remote sensing feature map Y of the fourth transmission line tower base 4 At this point, the backbone network feature extraction is completed. 3 and Y 4 Input into module C to obtain the remote sensing characteristic map Y of the fifth transmission line tower base 5 ; Y 5 Input into the EIC module to obtain the remote sensing characteristic map Y of the sixth transmission line tower base 6 ; Y 2 and Y 6 Input into the ABF module to obtain the remote sensing characteristic map Y of the seventh transmission line tower base 7 ; Y 7 Input into the EIC module to obtain the remote sensing characteristic map Y of the eighth transmission line tower base 8 ; Y 6 and Y 8 Input into the ABF module to obtain the remote sensing characteristic map Y of the ninth transmission line tower base 9 ; Y 9 Input into the EIC module to obtain the remote sensing characteristic map Y of the tenth transmission line tower base 10 ; Y 4 and Y 10Input into the ABF module to obtain the remote sensing characteristic map Y of the eleventh transmission line tower base 11 At this point, the feature fusion structure part of the transmission line tower foundation defect recognition is completed. 8 , Y 10 and Y 12 Input the recognition head of the algorithm respectively, and the corresponding output is the remote sensing feature map Y of the fifteenth transmission line tower base 15 , Remote sensing characteristic map of the 14th transmission line tower base Y 14 And the remote sensing characteristic map of the 13th transmission line tower base Y 13 The output format is (x,y,w,h,conf 1 ,…,conf i ,…,conf n ), where x represents the horizontal coordinate of the center of the prediction box, y represents the vertical coordinate of the center of the prediction box, w represents the width of the prediction box, h represents the height of the prediction box, and conf i represents the confidence of the category corresponding to the i-th index, and n is the number of transmission line tower foundation defect categories. So far, the head recognition part of the transmission line tower foundation defect recognition is completed. Finally, the output Y of the head recognition part is 13 , Y 14 and Y 15 Merge and perform non-maximum suppression to remove overlapping prediction boxes to obtain the final output Y of the algorithm 16 The output format is the same as (x, y, w, h, conf 1 ,…,conf i ,…,conf n ).

[0091] TBDI-Net training updates the parameters inside the algorithm through the back propagation of the loss function. In the loss function described in the present invention, the loss of the bounding box coordinates of the tower base defect target adopts DFL loss, the confidence loss of the bounding box adopts WIoU loss, and the classification loss of whether it is a tower base defect adopts cross entropy loss. The above loss function is updated round by round during the gradient back propagation process of algorithm training. The algorithm gradually converges in round by round training, and the detection accuracy gradually improves. After each round of training, the algorithm will call the divided verification data to verify the accuracy P, recall rate R and average mean precision mAP of the algorithm as an evaluation standard for whether the training effect is effective. After the TBDI-Net training of the present invention is completed, the 240 divided transmission line tower base remote sensing image data are used as test data, and the data is input into the optimal network parameter TBDI-Net after training for effect testing.

[0092] In actual applications, the remote sensing image of the transmission line tower foundation defects is input into TBDI-Net, and the output is the extraction result of the transmission line tower foundation defects. The output result contains a tensor of prediction information, and each row corresponds to a prediction, including bounding box coordinates, category labels, and confidence scores.

[0093] The feature pyramid shared convolution FPSC of the present invention constructs a feature pyramid structure through dilated convolutions with different dilation rates, thereby capturing finer-grained local features under the premise of capturing the global context transmission line tower base features. At the same time, the internal parameters of the dilated convolution kernel are shared to reduce the algorithm parameter redundancy and computational overhead, thereby improving the computing speed in actual scenarios.

[0094] The edge information feature capture module EIC of the present invention extracts different types of features in parallel by designing a dual-branch structure, effectively capturing tower foundation damage information (such as tower foundation concrete cracks) in remote sensing images while improving the algorithm's recognition of more complex fine-grained feature information such as multi-shape cracks.

[0095] The present invention designs an adaptive feature fusion unit AFFU, and designs a feature adaptive bidirectional fusion module ABF based on AFFU. The feature adaptive bidirectional fusion module effectively aggregates high- and low-level feature information by designing an adaptive bidirectional fusion structure between deep and shallow features, reduces the loss of fine-grained targets caused by direct fusion of high- and low-level features, and improves the multi-scale feature fusion effect of the algorithm on defect images of transmission line tower bases.

[0096] The embodiment of the present invention establishes a transmission line tower base defect target recognition network with strong generalization, high accuracy and automation, which can identify high-granularity defect hazards. After training, the algorithm designed in this scheme can be deployed on the towers along each transmission line to achieve real-time observation and operation and maintenance of the transmission line tower base.

[0097] Embodiment 2:

[0098] The embodiment of the present invention further provides a transmission line tower foundation defect identification device, comprising:

[0099] Acquisition module, used to acquire remote sensing images of transmission lines including tower foundations

[0100] A partitioning module is used to pre-process the remote sensing images of the transmission line including the tower base, and to partition the pre-processed remote sensing image data set of the transmission line tower base to obtain a training set and a test set;

[0101] A construction module is used to construct a transmission line tower base defect recognition network TBDI-Net according to a preprocessed transmission line tower base remote sensing image dataset; wherein the transmission line tower base defect recognition network TBDI-Net includes: a feature pyramid shared convolution, an edge information feature capture module, and a feature adaptive bidirectional fusion module based on an adaptive feature fusion unit AFFU;

[0102] A training module, used to train the transmission line tower base defect identification network TBDI-Net according to the training set;

[0103] The recognition module is used to input the test set into the trained transmission line tower base defect recognition network TBDI-Net to identify the tower base defect targets existing in the transmission line towers.

[0104] Embodiment 3:

[0105] An embodiment of the present invention further provides a transmission line tower foundation defect identification system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a transmission line tower foundation defect identification method when executed by the processor.

[0106] Embodiment 4:

[0107] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and the computer program executes a method for identifying defects in a transmission line tower foundation when the computer program is run.

[0108] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for identifying defects in a transmission line tower foundation, characterized in that: include: Step S1, obtaining a remote sensing image including a tower base on a transmission line; Step S2, preprocessing the remote sensing images of the transmission line including the tower base, and dividing the preprocessed remote sensing image data set of the transmission line tower base to obtain a training set and a test set; Step S3, constructing a transmission line tower base defect recognition network TBDI-Net according to the preprocessed transmission line tower base remote sensing image data set; wherein the transmission line tower base defect recognition network TBDI-Net includes: a feature pyramid shared convolution, an edge information feature capture module, and a feature adaptive bidirectional fusion module based on an adaptive feature fusion unit AFFU; Step S4, training the transmission line tower base defect identification network TBDI-Net according to the training set; Step S5: input the test set into the trained transmission line tower base defect identification network TBDI-Net to identify tower base defect targets existing in the transmission line towers.

2. The method for identifying defects in a transmission line tower foundation according to claim 1, characterized in that: The feature pyramid shared convolution adopts a structure that combines serial and multi-residual connections, and constructs a feature pyramid structure through dilated convolutions with different dilation rates.

3. The method for identifying defects in a transmission line tower foundation according to claim 2, characterized in that: The edge information feature capture module extracts different types of features in parallel by designing a dual-branch structure to capture the tower base damage information in remote sensing images.

4. The method for identifying defects in a transmission line tower foundation according to claim 3, characterized in that: The feature adaptive bidirectional fusion module designs an adaptive bidirectional fusion structure between deep and shallow features to aggregate boundary information and semantic information.

5. A transmission line tower foundation defect identification device, characterized in that: include: Acquisition module, used to acquire remote sensing images of transmission lines including tower foundations A partitioning module is used to pre-process the remote sensing images of the transmission line including the tower base, and to partition the pre-processed remote sensing image data set of the transmission line tower base to obtain a training set and a test set; A construction module is used to construct a transmission line tower base defect recognition network TBDI-Net according to a preprocessed transmission line tower base remote sensing image dataset; wherein the transmission line tower base defect recognition network TBDI-Net includes: a feature pyramid shared convolution, an edge information feature capture module, and a feature adaptive bidirectional fusion module based on an adaptive feature fusion unit AFFU; A training module, used to train the transmission line tower base defect identification network TBDI-Net according to the training set; The recognition module is used to input the test set into the trained transmission line tower base defect recognition network TBDI-Net to identify the tower base defect targets existing in the transmission line towers.

6. A transmission line tower foundation defect identification system, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the method for identifying defects in a transmission line tower foundation as described in any one of claims 1 to 4 is executed.

7. A storage medium, characterized in that: The storage medium stores a computer program, which, when running, executes the method for identifying transmission line tower foundation defects as described in any one of claims 1 to 4.

Images