A method for identifying defects of key components of towers based on local texture enhancement network

Abstract

The present invention discloses a method for identifying defects in key components of pole towers based on a local texture enhancement network. The method can extract discriminative features with texture information through a feature extraction and texture enhancement module. Features of different resolutions are then fused through a feature fusion module, allowing the output features to contain both semantic and pixel-level information. Finally, the pole tower component defect recognition module identifies defects in key components of the pole tower and outputs the location and confidence level of the defects. This network can be directly applied to the power inspection process to identify key components and defects of the pole tower. It has the characteristics of high recognition accuracy and strong generalization capability. By combining the traditional power inspection process with cutting-edge deep learning methods and neural network technology, it can effectively solve the problems of low efficiency of traditional manual inspections and poor results of existing detection methods, realize the intelligence and autonomy of the pole tower inspection process, and to a certain extent, ensure the normal operation of power lines.

Description

Technical Field

[0001] The present invention belongs to the technical field of key components and defect detection of towers in power inspection scenarios, and in particular relates to a method for identifying defects of key components of towers based on a local texture enhancement network. Background Art

[0002] With the rapid development of science and technology in my country and the improvement of people's living standards, society's demand for electricity is increasing. The smooth and safe transmission, transformation, and distribution of electricity is crucial for ensuring people's normal lives. Therefore, inspection of key power equipment in the transmission, transformation, and distribution process is crucial. Power towers, as core equipment in transmission lines, are crucial for ensuring the normal and stable transmission of electricity. Inspection of key tower components, such as insulators and anti-vibration hammers, is a crucial part of power inspection. Traditional power tower inspections are primarily manual, a method characterized by high risk, low efficiency, and a high risk of missed inspections. With the rapid development of robotics technology, drone-assisted power inspections are gradually replacing fully manual inspections. Drone-assisted inspections primarily involve drones capturing images of power towers, which are then uploaded to inspection systems. Defects in the images are detected through manual inspection or algorithmic detection. Typically, this defect detection method requires manually designed feature extractors and can only detect and identify defects in specific scenarios. In recent years, deep learning methods have been widely applied in various fields, and their powerful feature representation capabilities have provided strong support for existing target detection tasks. Although some domestic research has used deep learning for tower defect detection, existing methods generally use anomaly detection networks, which cannot accurately locate defects. Moreover, most of these methods do not utilize texture features, resulting in poor tower inspection results. Summary of the Invention

[0003] In response to the above technical problems, the present invention provides a method for identifying defects in key components of pole towers based on a local texture enhancement network, which avoids the shortcomings of existing technologies that are difficult to locate defects and difficult to effectively utilize texture information. It can not only realize the detection of key components of pole towers, but also effectively utilize texture detail information to locate the position of defects in key components, thereby replacing manual pole tower inspections and providing a new solution for autonomous and intelligent pole tower inspections.

[0004] The technical solution adopted by the present invention to solve the technical problem is:

[0005] A method for identifying defects of key components of a tower based on a local texture enhancement network comprises the following steps:

[0006] S100: Use drones to obtain tower image data, use LabelMe to annotate the components and defects to be inspected in the tower image data to generate a dataset, and preprocess the dataset;

[0007] S200: Constructing a network model for defect recognition of key components of a tower. The network model for defect recognition of key components of a tower includes a feature extraction and local texture enhancement module, a feature fusion module, and a tower component defect recognition module connected in sequence. The feature extraction and local texture enhancement module is used to extract multi-scale discriminative features of tower components and enhance the texture features of key components in the multi-scale discriminative features to obtain texture-enhanced multi-scale discriminative features. The feature fusion module is used to fuse the texture-enhanced multi-scale discriminative features and send the fused multi-scale features to the tower component defect recognition module for defect recognition to obtain classification results and regression results.

[0008] S300: Training a network model for identifying defects in key tower components based on the preprocessed data set, calculating a loss value for the trained network model using a preset loss function, classification results, and regression results, and updating network weights based on the loss value using a gradient descent method to backpropagate the loss gradient, thereby obtaining a trained network model for identifying defects in key tower components;

[0009] S400: Acquire an actual tower image, detect and identify key tower components and their defects in the actual tower image based on the trained tower key component defect recognition network model, and obtain a detection result.

[0010] Preferably, the feature extraction and local texture enhancement module includes a first residual block, a second residual block, a third residual block, a local texture enhancement module, and a fourth residual block. The feature extraction and local texture enhancement module in S200 is used to extract multi-scale discriminative features of tower components and enhance the texture features of key components in the multi-scale discriminative features to obtain multi-scale discriminative features after texture enhancement, including:

[0011] S210: The image size of the preprocessed data set data is uniformly cropped to a preset size, and three discriminative feature maps C1, C2 and C3 of different scales are obtained after passing through the first residual block, the second residual block and the third residual block respectively;

[0012] S220: C2 and C3 are used as inputs of the local texture enhancement module to obtain a feature map C4 with texture detail information;

[0013] S230: C4 is passed through the fourth residual block to obtain a feature map C5 having semantic information and texture detail information, thereby obtaining texture-enhanced multi-scale discriminative features C1, C2, C4 and C5 with different receptive fields, where the resolution difference between adjacent scales is 2 times.

[0014] Preferably, the local texture enhancement module includes a channel texture enhancement module and a neighborhood texture enhancement module, and S220 includes:

[0015] S221: The two feature maps C2 and C3 with different resolutions are concatenated and dimensionally reduced to obtain the fusion feature F;

[0016] S222: Inputting the fusion feature F into the channel texture enhancement module to perform channel texture enhancement to obtain a fusion feature of channel texture enhancement;

[0017] S223: Inputting the fused feature F into the neighborhood texture enhancement module to perform neighborhood texture enhancement, thereby obtaining a neighborhood texture enhanced fused feature;

[0018] S224: splicing the fusion features of channel texture enhancement and the fusion features of neighborhood texture enhancement to obtain a feature map with local texture enhancement.

[0019] Preferably, S222 includes:

[0020] S2221: The fused feature F is passed through a global average pooling layer to obtain a global feature g of size C×1×1. The cosine similarity between the global feature g and the fused feature F is calculated to obtain a similarity matrix S. The cosine similarity calculation formula is as follows:

[0021]

[0022] Among them, g is the global feature, F ij is the eigenvalue of the fusion feature map F at the (i, j) position, S ij Represents the similarity value at any position (i, j) on the similarity matrix graph S;

[0023] S2222: Quantize the similarity matrix S into N levels L, where each level L has a certain number of points. The specific allocation and calculation method is as follows:

[0024]

[0025] Among them, max(.) and min(.) represent the maximum and minimum functions respectively, L n is the quantized n-th similarity level feature vector, 0≤n≤N;

[0026] S2223: According to the above quantization level, encode the Sij of each pixel position in the similarity matrix into a statistical coding vector E ij , the similarity matrix S is quantized and statistically encoded into a matrix E(N×HW), and the specific calculation formula is as follows:

[0027]

[0028] Among them, S i Represents the value of the similarity matrix at position i (0≤i≤HW), E i,n Represents the quantized code value of the nth level at the i-th position;

[0029] S2224: Concatenate the statistical quantization feature E and the quantization level feature L to obtain a quantization count feature map C with texture statistical information, perform dimension upgrading on the feature map C with texture statistical information through a multi-layer perceptron, and concatenate it with the global feature g to obtain a statistical feature map D containing global information and local information;

[0030] S2225: Perform two 1×1 convolutions on the statistical feature map D and calculate the adjacency matrix of the statistical feature map D. Normalize the adjacency matrix using the Softmax function to obtain the matrix X. Then, multiply the normalized matrix by the statistical feature map D after the 1×1 convolution to obtain the reconstructed quantized level feature L'. The calculation formula is as follows:

[0031] X = Softmax(φ1(D)T·φ2(D))

[0032] L′=φ3(D)·X

[0033] Among them, φ1, φ2, and φ3 represent 1×1 convolution operations;

[0034] S2226: Multiply the obtained reconstructed feature L' and the quantization matrix E to obtain the feature map F after channel texture statistics enhancement c .

[0035] Preferably, S223 includes:

[0036] S2231: The input feature F is subjected to a 3x3 convolution and a 1x1 convolution with a stride of 1 and a padding of 1, and the feature dimension is reduced while keeping the size of the feature map unchanged to obtain a new feature F1;

[0037] S2232: Use a sliding window operation to obtain feature blocks with neighborhoods in the new feature F1, whose dimensions are (B, 9C, P), where B is the input batch size, C is the number of channels, and P is the number of feature blocks;

[0038] S2233: New features (B, P, C, 3, 3) are obtained by expanding and channel transforming the feature blocks with neighborhoods, and the local average value F_avg of the neighborhood new features is obtained by operating the neighborhood new features using the three-dimensional global average pooling function;

[0039] S2234: Calculate the cosine similarity between the local average value F_avg and the new feature F1 to obtain the preliminary similarity matrix S0 of the neighborhood. Use the sliding window operation to obtain the feature blocks of the neighborhood of the similarity matrix S0. Then, use the adaptive maximum pooling operation to extract the local maximum value of the neighborhood features of the similarity matrix S0 to obtain the final neighborhood similarity matrix S'.

[0040] S2235: Quantize the neighborhood similarity matrix S' into N levels L', where each level L' has a certain number of points. The specific allocation and calculation method is as follows:

[0041]

[0042] Among them, L' n is the quantized n-th similarity level feature vector, 0≤n≤N.

[0043] S2236: Neighborhood similarity matrix for each pixel position S' ij Encoded into statistical encoding vector E' ij , the neighborhood similarity matrix S' is quantized and statistically encoded into a matrix E'(N×HW), and the specific calculation formula is as follows:

[0044]

[0045] Among them, S' i Represents the value of the similarity matrix at position i (0≤i≤HW), E' i,n Represents the quantized coded value of the nth level at the i-th position.

[0046] S2237: Concatenate the statistical quantization feature E′ and the quantization level feature L′ to obtain a quantization count feature map C′ with texture statistical information, perform dimension upgrading on the feature map C′ with texture statistical information through a multi-layer perceptron, and concatenate it with the global feature g to obtain a statistical feature map D′ containing global and local information;

[0047] S2238: Perform two 1×1 convolutions on the statistical feature map D′ and calculate the adjacency matrix of the statistical feature map D′. Normalize the adjacency matrix using the Soffmax function to obtain a matrix X′. Then, multiply the normalized matrix by the statistical feature map D′ after the 1×1 convolution to obtain the reconstructed quantized level feature L″. The calculation formula is as follows:

[0048] X′=Softmax(φ1(D′) T ·φ2(D′))

[0049] L″=φ3(D′)·X′

[0050] Among them, φ1, φ2, and φ3 represent 1×1 convolution operations;

[0051] S2239: Multiply the obtained reconstructed feature L″ and the quantization matrix E′ to obtain the feature map F after neighborhood texture statistics enhancement s .

[0052] Preferably, the feature fusion module in S200 is used to fuse the multi-scale discriminative features after texture enhancement, including:

[0053] S225: Input the texture-enhanced multi-scale discriminative features C2, C4, and C5 with different receptive fields into a 1×1 convolution for dimensionality reduction to obtain a feature map with 256 channels.

[0054] S226: The multi-scale discriminant features C2′, C4′ and C5′ after dimensionality reduction are fused from top to bottom, and the features of the previous layer are bilinearly interpolated to obtain a feature map of the same size as the features of the next layer. The two feature maps are then added element by element to achieve semantic information fusion, and the fused multi-scale features C2″, C4″ and C5″ are obtained. Finally, C5″ is subjected to 3×3 convolution to obtain feature C6, and feature C6 is subjected to 3×3 convolution to obtain feature C7, and the final output results C2″, C4″, C5″, C6 and C7 are obtained.

[0055] Preferably, the tower component defect recognition module includes four 3×3 convolutional layers, a classification decoding layer, and a regression decoding layer. In S200, the fused multi-scale features are sent to the tower component defect recognition module for defect recognition to obtain classification results and regression results, including:

[0056] The final output results C2″, C4″, C5″, C6 and C7 are passed through four 3×3 convolutional layers, and then through the classification decoding layer and regression decoding layer respectively to obtain the classification results and regression results.

[0057] Preferably, the preset loss function includes classification and regression losses, and the classification and regression losses are FL loss and GIoU loss respectively, and the calculation formulas are as follows:

[0058] FL(p,y)=-y(1-p) γ log(p)-(1-y)p γ log(1-p)

[0059]

[0060] Among them, y represents the classification label, p represents the classification prediction value, γ is a hyperparameter, IoU represents the intersection-over-union ratio between the label and the prediction box, M represents the minimum closed shape, and A MRepresents the area of ​​M, U represents the union area of ​​G and P, G represents the real box, and M represents the predicted box.

[0061] The present invention proposes a method for identifying defects in key tower components based on a local texture enhancement network. Through texture enhancement, the network model can extract discriminative features containing texture information. A feature fusion module fuses features of different resolutions, allowing the output features to contain both semantic and pixel-level information. Finally, a tower component defect recognition module identifies defects in key tower components and outputs the location and confidence level of the defects. This network can be directly applied to power inspection processes to identify key tower components and defects, with high recognition accuracy and strong generalization capabilities. By combining traditional power inspection processes with cutting-edge deep learning methods and neural network technology, it can effectively address the low efficiency of traditional manual inspections and the poor performance of existing detection methods, making the tower inspection process intelligent and autonomous, and, to a certain extent, ensuring the normal operation of power lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flow chart of a method for identifying defects of key components of a tower based on a local texture enhancement network in one embodiment of the present invention;

[0063] Figure 2 This is a flowchart of a method for identifying defects of key components of a tower based on a local texture enhancement network in one embodiment of the present invention;

[0064] Figure 3 Schematic diagram of the structure of a network model for identifying defects in key tower components in one embodiment of the present invention;

[0065] Figure 4 This is a schematic diagram of the results of a local texture enhancement module of the present invention;

[0066] Figure 5 A schematic diagram of a statistical texture quantization module of the present invention;

[0067] Figure 6 Schematic diagram of detection and recognition results in different scenarios according to the present invention. DETAILED DESCRIPTION

[0068] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below with reference to the accompanying drawings.

[0069] In one embodiment, Figure 1 、 2 As shown, a method for identifying defects of key components of a tower based on a local texture enhancement network comprises the following steps:

[0070] S100: Use drone aerial photography to obtain tower image data, and use Labelme (a data annotation tool) to annotate the components and defects to be inspected in the tower image data to generate a data set, and preprocess the data set.

[0071] Specifically, after obtaining the dataset, it was randomly divided into training, validation, and test datasets in a ratio of 8:1:1. Data augmentation was performed on the dataset through preprocessing operations such as rotation, translation, and cropping to enhance the characteristic representation of tower defects.

[0072] S200: Construct a network model for defect recognition of key components of a tower. The network model for defect recognition of key components of a tower includes a feature extraction and local texture enhancement module, a feature fusion module and a tower component defect recognition module connected in sequence. The feature extraction and local texture enhancement module is used to extract multi-scale discriminative features of tower components and enhance the texture features of key components in the multi-scale discriminative features to obtain multi-scale discriminative features after texture enhancement. The feature fusion module is used to fuse the multi-scale discriminative features after texture enhancement and send the fused multi-scale features to the tower component defect recognition module for defect recognition to obtain classification results and regression results.

[0073] Specifically, the structural diagram of the tower key component defect recognition network model is as follows: Figure 3 As shown in the figure, a residual neural network is used to extract multi-scale discriminative features of tower components. Feature maps of different resolutions are used to ensure that the features of multi-scale components are fully extracted and utilized. This ensures that the network model can effectively utilize texture information such as the tower's edges and contours when detecting tower component defects, thereby ensuring the network's detection accuracy. A local texture enhancement network is designed to enhance the texture features of key tower components such as insulators and shock absorbers. This ensures that the neural network can effectively utilize effective texture information such as edges and shapes, improving the network's ability to identify defects in key tower components. A feature fusion module is constructed, using a multi-scale feature pyramid structure to fuse feature maps of different resolutions. This ensures that the tower component features can effectively integrate semantic information and pixel information, further improving the expressiveness of tower component features.

[0074] In one embodiment, the feature extraction and local texture enhancement module includes a first residual block, a second residual block, a third residual block, a local texture enhancement module, and a fourth residual block. The feature extraction and local texture enhancement module in S200 is used to extract multi-scale discriminative features of tower components and enhance the texture features of key components in the multi-scale discriminative features to obtain multi-scale discriminative features after texture enhancement, including:

[0075] S210: The image size of the preprocessed data set data is uniformly cropped to a preset size, and three discriminative feature maps C1, C2 and C3 of different scales are obtained after passing through the first residual block, the second residual block and the third residual block respectively;

[0076] S220: C2 and C3 are used as inputs of the local texture enhancement module to obtain a feature map C4 with texture detail information;

[0077] S230: C4 is passed through the fourth residual block to obtain a feature map C5 having semantic information and texture detail information, thereby obtaining texture-enhanced multi-scale discriminative features C1, C2, C4 and C5 with different receptive fields, where the resolution difference between adjacent scales is 2 times.

[0078] Specifically, in this embodiment, the preprocessed images are uniformly cropped to a size of 1333×800 and input into the residual network.

[0079] In one embodiment, Figure 4 As shown, the local texture enhancement module includes a channel texture enhancement module and a neighborhood texture enhancement module, and S220 includes:

[0080] S221: The two feature maps C2 and C3 with different resolutions are concatenated and dimensionally reduced to obtain the fusion feature F;

[0081] S222: Inputting the fusion feature F into the channel texture enhancement module to perform channel texture enhancement to obtain a fusion feature of channel texture enhancement;

[0082] S223: Inputting the fused feature F into the neighborhood texture enhancement module to perform neighborhood texture enhancement, thereby obtaining a neighborhood texture enhanced fused feature;

[0083] S224: splicing the fusion features of channel texture enhancement and the fusion features of neighborhood texture enhancement to obtain a feature map with local texture enhancement.

[0084] Specifically, the feature extraction and local texture enhancement module includes a residual network for feature extraction and a local texture enhancement module, which consists of a channel texture enhancement module and a neighborhood texture enhancement module. Unlike directly extracting features using a residual network, this method can extract texture features from both the channel and spatial dimensions, significantly improving the inspection of components with distinct texture features, such as insulators.

[0085] In one embodiment, S222 includes:

[0086] S2221: The fused feature F is passed through a global average pooling layer to obtain a global feature g of size C×1×1. The cosine similarity between the global feature g and the fused feature F is calculated to obtain a similarity matrix S. The cosine similarity calculation formula is as follows:

[0087]

[0088] Among them, g is the global feature, F ij is the eigenvalue of the fusion feature map F at the (i, j) position, S ij Represents the similarity value at any position (i, j) on the similarity matrix graph S;

[0089] S2222: Quantize the similarity matrix S into N levels L, where each level L has a certain number of points. The specific allocation and calculation method is as follows:

[0090]

[0091] Among them, max(.) and min(.) represent the maximum and minimum functions respectively, L n is the quantized n-th similarity level feature vector, 0≤n≤N;

[0092] S2223: According to the above quantization level, the S of each pixel position in the similarity matrix is ij The encoding becomes a statistical encoding vector E ij , the similarity matrix S is quantized and statistically encoded into a matrix E(N×HW), and the specific calculation formula is as follows:

[0093]

[0094] Among them, S i Represents the value of the similarity matrix at position i (0≤i≤HW), E i,n Represents the quantized code value of the nth level at the i-th position;

[0095] S2224: Concatenate the statistical quantization feature E and the quantization level feature L to obtain a quantization count feature map C with texture statistical information, perform dimension upgrading on the feature map C with texture statistical information through a multi-layer perceptron, and concatenate it with the global feature g to obtain a statistical feature map D containing global information and local information;

[0096] S2225: Perform two 1×1 convolutions on the statistical feature map D and calculate the adjacency matrix of the statistical feature map D. Normalize the adjacency matrix using the Softmax function to obtain the matrix X. Then, multiply the normalized matrix by the statistical feature map D after the 1×1 convolution to obtain the reconstructed quantized level feature L'. The calculation formula is as follows:

[0097] X=Softmax(φ1(D) T ·φ2(D)

[0098] L′=φ3(D)·X

[0099] Among them, φ1, φ2, and φ3 represent 1×1 convolution operations;

[0100] S2226: Multiply the obtained reconstructed feature L' and the quantization matrix E to obtain the feature map F after channel texture statistics enhancement c .

[0101] Specifically, the schematic diagram of the statistical texture quantization process is as follows: Figure 5 shown.

[0102] In one embodiment, S223 includes:

[0103] S2231: The input feature F is subjected to a 3x3 convolution and a 1x1 convolution with a stride of 1 and a padding of 1, and the feature dimension is reduced while keeping the size of the feature map unchanged to obtain a new feature F1;

[0104] S2232: Use a sliding window operation to obtain feature blocks with neighborhoods in the new feature F1, whose dimensions are (B, 9C, P), where B is the input batch size, C is the number of channels, and P is the number of feature blocks;

[0105] S2233: New features (B, P, C, 3, 3) are obtained by expanding and channel transforming the feature block with the neighborhood, and the local average value F_avg of the neighborhood new features is obtained by operating the neighborhood new features using the three-dimensional global average pooling function;

[0106] S2234: Calculate the cosine similarity between the local average value F_avg and the new feature F1 to obtain the preliminary similarity matrix S0 of the neighborhood. Use the sliding window operation to obtain the feature blocks of the neighborhood of the similarity matrix S0. Then, use the adaptive maximum pooling operation to extract the local maximum value of the neighborhood features of the similarity matrix S0 to obtain the final neighborhood similarity matrix S′.

[0107] S2235: Quantize the neighborhood similarity matrix S′ into N levels L′. Each level L′ has a certain number of points. The specific allocation and calculation method is as follows:

[0108]

[0109] Among them, L′ n is the quantized n-th similarity level feature vector, 0≤n≤N.

[0110] S2236: Neighborhood similarity matrix for each pixel position S' ijEncoded into statistical coding vector E′ ij , the neighborhood similarity matrix S′ is quantized and statistically encoded into a matrix E′(N×HW), and the specific calculation formula is as follows:

[0111]

[0112] Among them, S′ i Represents the value of the similarity matrix at position i (0≤i≤HW), E′ i，n Represents the quantized coded value of the nth level at the i-th position.

[0113] S2237: Concatenate the statistical quantization feature E′ and the quantization level feature L′ to obtain a quantization count feature map C′ with texture statistical information, perform dimension upgrading on the feature map C′ with texture statistical information through a multi-layer perceptron, and concatenate it with the global feature g to obtain a statistical feature map D′ containing global and local information;

[0114] S2238: Perform two 1×1 convolutions on the statistical feature map D′ and calculate the adjacency matrix of the statistical feature map D′. Normalize the adjacency matrix using the Soffmax function to obtain a matrix X′. Then, multiply the normalized matrix by the statistical feature map D′ after the 1×1 convolution to obtain the reconstructed quantized level feature L″. The calculation formula is as follows:

[0115] X′=Soffmax(φ1(D′) T ·φ2(D′))

[0116] L″=φ3(D′)·X′

[0117] Among them, φ1, φ2, and φ3 represent 1×1 convolution operations;

[0118] S2239: Multiply the obtained reconstructed feature L' and the quantization matrix E' to obtain the feature map F after neighborhood texture statistics enhancement s .

[0119] Specifically, texture enhancement is performed on the neighborhood position of each pixel point, and the cosine similarity map between the pixel point and the average value in the 3×3 neighborhood is calculated. The neighborhood texture enhancement is then performed on the cosine similarity map through statistical texture feature extraction and texture feature enhancement.

[0120] Furthermore, the feature extraction and texture enhancement module can effectively extract the texture features of the tower, thereby ensuring that the network model can effectively utilize the texture information such as the tower edge and contour when detecting defects in tower components, thereby ensuring the detection accuracy of the network.

[0121] In one embodiment, the feature fusion module in S200 is used to fuse the multi-scale discriminative features after texture enhancement, including:

[0122] S225: Input the texture-enhanced multi-scale discriminative features C2, C4, and C5 with different receptive fields into a 1×1 convolution for dimensionality reduction to obtain a feature map with 256 channels.

[0123] S226: The multi-scale discriminant features C2', C4' and C5' after dimensionality reduction are fused from top to bottom, and the features of the previous layer are bilinearly interpolated to obtain a feature map of the same size as the features of the next layer. The two feature maps are then added element by element to achieve semantic information fusion, and the fused multi-scale features C2", C4" and C5" are obtained. Finally, C5" is subjected to 3×3 convolution to obtain feature C6, and feature C6 is subjected to 3×3 convolution to obtain feature C7, to obtain the final output results C2", C4", C5", C6 and C7.

[0124] Specifically, multi-scale features with texture information are fused from top to bottom to ensure that the features extracted by the network can have both semantic-level texture and pixel-level texture, effectively integrating the discriminative features of the tower.

[0125] In one embodiment, the tower component defect recognition module includes four 3×3 convolutional layers, a classification decoding layer, and a regression decoding layer. In S200, the fused multi-scale features are sent to the tower component defect recognition module for defect recognition to obtain classification results and regression results, including:

[0126] The final output results C2", C4", C5", C6 and C7 are passed through four 3×3 convolutional layers, and then through the classification decoding layer and regression decoding layer respectively to obtain the classification results and regression results.

[0127] Specifically, the classification decoding layer and regression decoding layer are 1×1 convolutional layers, respectively. They use 1×1 convolutions to generate the network's classification and regression results. The classification result is the classification score, which indicates whether the target is a certain component type. The regression output is the offset of the predicted box, which is the positioning coordinate. The dimensions of the classification and regression results are H×W×K and H×W×4K, respectively, where K is the total number of tower component types.

[0128] S300: The network model for identifying defects in key components of the tower is trained based on the preprocessed data set. The preset loss function, classification results, and regression results are used to calculate the loss value of the trained network model for identifying defects in key components of the tower. The gradient descent method is used to back-propagate the loss gradient based on the loss value to update the network weights, thereby obtaining a trained network model for identifying defects in key components of the tower.

[0129] Specifically, the training environment settings are as follows: CPU: i7-11700K, GPU: Nvidia GeForce RTX 3090 with 24GB of video memory, PyTorch version 1.8.0, CUDA version 11.1. Key parameters for model training are as follows: Stochastic Gradient Descent (SGD) optimizer, initial learning rate: 0.01, number of data loading threads: 4, batch size: 4, training epochs: 24, with the learning rate reduced by a factor of 0.1 at epochs 16 and 22.

[0130] In one embodiment, the preset loss function includes classification and regression losses, where FL loss and GIoU loss are selected for classification and regression respectively, and the calculation formulas are as follows:

[0131] FL(p,y)=-y(1-p) γ log(p)-(1-y)p γ log(1-p)

[0132]

[0133] Among them, y represents the classification label, p represents the classification prediction value, γ is a hyperparameter, IoU represents the intersection-over-union ratio between the label and the prediction box, M represents the minimum closed shape, and A M Represents the area of ​​M, U represents the union area of ​​G and P, G represents the true box, and M represents the predicted box.

[0134] Specifically, classification and regression losses are used to constrain the overall defect recognition network to ensure that the network can learn a set of optimal weights, thereby achieving effective identification of defects in key components of the tower.

[0135] S400: Acquire an actual tower image, detect and identify key tower components and their defects in the actual tower image based on the trained tower key component defect recognition network model, and obtain a detection result.

[0136] Specifically, the detection and identification of defects in key components of the tower and the visualization of the detection results can realize the intelligentization of the power inspection process. Figure 6 As shown, (a) (b) (c) are schematic diagrams of recognition detection in different scenarios, Figure 6In the example, pinsug represents a glass insulator, psusp represents a suspension clamp, pinsumiss represents a missing insulator, pvib represents a vibration damper, pvibmiss represents a missing vibration damper, pinsub represents a ceramic insulator, plinkgrp represents a connection group, and pnest represents a bird's nest. Finally, during the testing phase, the trained network weights are loaded to detect and recognize tower components, and the results are visualized. The visualization outputs confidence scores and location boxes: the location box is a rectangular box drawn from the detection coordinates output by the network, and the confidence score is the confidence score that the object within the rectangular box belongs to a certain component category.

[0137] The present invention is based on the recognition results of defects in key tower components of the local texture enhancement network. It can mainly visualize the network's detection and recognition results of defects in key tower components such as insulator contours and shock-absorbing hammer shapes, and can intuitively show the location and confidence of tower component defects, thereby promoting the intelligence of the tower inspection process.

[0138] The present invention can achieve the following beneficial effects: The proposed method for identifying defects in key tower components based on a local texture enhancement network, through the actions of a channel texture enhancement module and a neighborhood texture enhancement module, enables the network model to extract discriminative features containing texture information. Simultaneously, a multi-scale feature fusion module fuses features of different resolutions, resulting in output features that contain both semantic and pixel-level information. Finally, the defect recognition network identifies defects in key tower components and outputs the defect location and confidence level. This network can be directly applied to power inspections to identify key tower components and defects, improving the inefficiency of manual inspections and, to a certain extent, ensuring the normal operation of power lines.

[0139] The method for identifying defects in key tower components based on a local texture enhancement network proposed in this invention has the characteristics of high recognition accuracy and strong generalization ability. It combines the traditional power inspection process with cutting-edge deep learning methods and neural network technology, which can effectively solve the problems of low efficiency of traditional manual inspections and poor effects of existing detection methods, and realize the intelligence and autonomy of the tower inspection process.

[0140] The above describes in detail the method for identifying defects in key tower components based on a local texture enhancement network provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above examples is only intended to help understand the core concept of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from the principles of the present invention, and such improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A method for identifying defects in key components of a tower based on a local texture enhancement network, characterized in that: The method comprises the following steps: S100: Using drone aerial photography to obtain tower image data, annotating components and defects to be inspected in the tower image data using Labelme to generate a data set, and preprocessing the data set; S200: Constructing a network model for defect recognition of key components of a tower, the network model for defect recognition of key components of a tower includes a feature extraction and local texture enhancement module, a feature fusion module and a tower component defect recognition module connected in sequence, the feature extraction and local texture enhancement module is used to extract multi-scale discriminative features of tower components and perform texture feature enhancement on key components in the multi-scale discriminative features to obtain texture-enhanced multi-scale discriminative features, the feature fusion module is used to fuse the texture-enhanced multi-scale discriminative features and send the fused multi-scale features to the tower component defect recognition module for defect recognition to obtain classification results and regression results; the feature extraction and local texture enhancement module includes a first residual block, a second residual block, a third residual block, a local texture enhancement module and a fourth residual block, the feature extraction and local texture enhancement module in S200 is used to extract multi-scale discriminative features of tower components and perform texture feature enhancement on key components in the multi-scale discriminative features to obtain texture-enhanced multi-scale discriminative features, including: S210: uniformly cropping the image size of the preprocessed data set data to a preset size, and sequentially passing through the first residual block, the second residual block, and the third residual block to obtain three discriminative feature maps C1, C2, and C3 of different scales respectively; S220: The C2 and C3 are used as inputs of a local texture enhancement module to obtain a feature map C4 having texture detail information; S230: Passing the C4 through the fourth residual block to obtain a feature map C5 having semantic information and texture detail information, thereby obtaining texture-enhanced multi-scale discriminative features C1, C2, C4, and C5 with different receptive fields, wherein the resolution difference between adjacent scales is 2 times; The local texture enhancement module includes a channel texture enhancement module and a neighborhood texture enhancement module, and S220 includes: S221: The two feature maps C2 and C3 with different resolutions are concatenated and dimensionally reduced to obtain the fusion feature F; S222: Inputting the fusion feature F into the channel texture enhancement module to perform channel texture enhancement to obtain a fusion feature of channel texture enhancement; S223: Inputting the fusion feature F into the neighborhood texture enhancement module to perform neighborhood texture enhancement to obtain a neighborhood texture enhanced fusion feature; S224: combining the channel texture enhanced fusion features and the neighborhood texture enhanced fusion features to obtain a feature map with local texture enhancement; S300: training the tower key component defect recognition network model according to the preprocessed data set, calculating the loss value of the trained tower key component defect recognition network model using a preset loss function, the classification result, and the regression result, and updating the network weights according to the loss value by back-propagating the loss gradient using the gradient descent method to obtain a trained tower key component defect recognition network model; S400: Acquire an actual tower image, and detect and identify key tower components and their defects in the actual tower image based on the trained tower key component defect recognition network model to obtain a detection result.

2. The method according to claim 1, characterized in that S222 includes: S2221: The fused feature F is passed through a global average pooling layer to obtain a global feature g of size C×1×1, and the cosine similarity between the global feature g and the fused feature F is calculated to obtain a similarity matrix S; wherein the cosine similarity calculation formula is as follows: Among them, g is the global feature, F ij is the eigenvalue of the fusion feature map F at the (i, j) position, S ij Represents the similarity value at any position (i, j) on the similarity matrix graph S; S2222: quantize the similarity matrix S into N levels L, where each level L has a certain number of points. The specific allocation and calculation method is as follows: Among them, max(.) and min(.) represent the maximum and minimum functions respectively, L n is the quantized n-th similarity level feature vector, 0≤n≤N; S2223: According to the above quantization level, the S of each pixel position in the similarity matrix is ij The encoding becomes a statistical encoding vector E ij , the similarity matrix S is quantized and statistically encoded into a matrix E(N×HW), and the specific calculation formula is as follows: Among them, S i Represents the value of the similarity matrix at position i (0≤i≤HW), E i,n Represents the quantized code value of the nth level at the i-th position; S2224: Concatenate the statistical quantization feature E and the quantization level feature L to obtain a quantization count feature map C having texture statistical information, perform dimension upgrading on the feature map C having texture statistical information through a multi-layer perceptron, and concatenate the feature map C with the global feature g to obtain a statistical feature map D containing global information and local information; S2225: Perform two 1×1 convolutions on the statistical feature map D and calculate the adjacency matrix of the statistical feature map D. Normalize the adjacency matrix using the Softmax function to obtain a matrix X. Then, multiply the normalized matrix by the statistical feature map D after the 1×1 convolution to obtain a reconstructed quantized level feature L'. The calculation formula is as follows: X=Softmax(φ1(D) T ·φ2(D)) L'=φ3(D)·X Among them, φ1, φ2, and φ3 represent 1×1 convolution operations; S2226: Multiply the obtained reconstructed feature L' and the quantization matrix E to obtain a feature map F after channel texture statistics enhancement c .

3. The method according to claim 1, characterized in that S223 includes: S2231: The input feature F is subjected to a 3x3 convolution and a 1x1 convolution with a stride of 1 and a padding of 1, and the feature dimension is reduced while keeping the size of the feature map unchanged to obtain a new feature F1; S2232: Using a sliding window operation, obtain a feature block with a neighborhood in the new feature F1, whose dimension is (B, 9C, P), where B is the input batch size, C is the number of channels, and P is the number of feature blocks; S2233: New features (B, P, C, 3, 3) are obtained by expanding and channel transforming the feature block with the neighborhood, and the local average value F_avg of the neighborhood new features is obtained by operating the neighborhood new features using a three-dimensional global average pooling function; S2234: Calculate the cosine similarity between the local average value F_avg and the new feature F1 to obtain a preliminary neighborhood similarity matrix S0, obtain feature blocks of the neighborhood of the similarity matrix S0 using a sliding window operation, and then extract the local maximum value of the neighborhood features of the similarity matrix S0 through an adaptive maximum pooling operation to obtain a final neighborhood similarity matrix S'; S2235: quantize the neighborhood similarity matrix S' into N levels L', each level L' having a certain number of points. The specific allocation and calculation method is as follows: Among them, L' n is the quantized n-th similarity level feature vector, 0≤n≤N; S2236: Neighborhood similarity matrix for each pixel position S' ij Encoded into statistical encoding vector E' ij , the neighborhood similarity matrix S' is quantized and statistically encoded into a matrix E'(N×HW), and the specific calculation formula is as follows: Among them, S' i Represents the value of the similarity matrix at position i (0≤i≤HW), E' i,n Represents the quantized code value of the nth level at the i-th position; S2237: Concatenate the statistical quantization feature E' and the quantization level feature L' to obtain a quantization count feature map C' having texture statistical information, perform dimension upgrading on the feature map C' having texture statistical information through a multi-layer perceptron, and concatenate the feature map C' with the global feature g to obtain a statistical feature map D' containing global information and local information; S2238: Perform two 1×1 convolutions on the statistical feature map D' and calculate the adjacency matrix of the statistical feature map D'. Normalize the adjacency matrix using the Softmax function to obtain a matrix X'. Then, multiply the normalized matrix by the statistical feature map D' after the 1×1 convolution to obtain a reconstructed quantization level feature L'. The calculation formula is as follows: X'=Softmax(φ1(D') T ·φ2(D')) L"=φ3(D')·X' Among them, φ1, φ2, and φ3 represent 1×1 convolution operations; S2239: Multiply the obtained reconstructed feature L' and the quantization matrix E' to obtain a feature map F after neighborhood texture statistics enhancement. s .

4. The method according to claim 3, characterized in that The feature fusion module in S200 is used to fuse the multi-scale discriminative features after texture enhancement, including: S225: Input the texture-enhanced multi-scale discriminative features C2, C4, and C5 with different receptive fields into a 1×1 convolution for dimensionality reduction to obtain a feature map with 256 channels. S226: The multi-scale discriminant features C2', C4' and C5' after dimensionality reduction are fused from top to bottom, and the features of the upper layer are bilinearly interpolated to obtain a feature map of the same size as the features of the lower layer. The two feature maps are then added element by element to achieve semantic information fusion, and the fused multi-scale features C2", C4", and C5" are obtained. Finally, C5" is subjected to 3×3 convolution to obtain feature C6, and the feature C6 is subjected to 3×3 convolution to obtain feature C7, and the final output results C2", C4", C5", C6 and C7 are obtained.

5. The method according to claim 4, characterized in that The tower component defect recognition module includes four 3×3 convolutional layers, a classification decoding layer, and a regression decoding layer. In S200, the fused multi-scale features are sent to the tower component defect recognition module for defect recognition to obtain classification results and regression results, including: The final output results C2", C4", C5", C6 and C7 are passed through four 3×3 convolutional layers, and then passed through a classification decoding layer and a regression decoding layer respectively to obtain classification results and regression results.

6. The method according to claim 5, characterized in that The preset loss functions include classification and regression loss. The classification and regression losses are FLloss and GIoU loss respectively. The calculation formulas are as follows: FL(p,y)=-y(1-p) γ log(p)-(1-y)p γ log(1-p) Among them, y represents the classification label, p represents the classification prediction value, γ is a hyperparameter, IoU represents the intersection-over-union ratio between the label and the prediction box, M represents the minimum closed shape, and A M Represents the area of ​​M, U represents the union area of ​​G and P, G represents the real box, and M represents the predicted box.

Images