Defect detection method, device and equipment for electric power tower and storage medium

Abstract

The invention provides a defect detection method and device for an electric power tower, equipment and a storage medium. The method comprises the following steps: acquiring an image shot by an unmanned aerial vehicle; performing multi-scale feature extraction on the image through an improved DenseNet network to obtain a target feature map; in the improved network, introducing a multi-scale feature fusion module into an output layer of each dense block, performing feature fusion on low-scale features from a shallow layer and high-scale features from a deep layer to obtain a fused feature map, and adaptively adjusting the weight of each channel in the fused feature map through a channel attention mechanism; determining a defect boundary from the target feature map through an improved CornerNet algorithm; in the improved algorithm, the standard deviation of the thermodynamic diagram is adjusted according to the defect size in the target feature map, a self-adaptive key point thermodynamic diagram is generated, the defect boundary is determined according to the key point thermodynamic diagram, and then defect information is determined. Through the method, the precision and accuracy of defect detection can be improved.

Description

Technical Field

[0001] This application relates to the field of pole and tower defect identification, and particularly to a method, device, equipment and storage medium for detecting defects of power poles and towers. Background Art

[0002] The power transmission system is a key infrastructure to ensure the normal operation of society. Power poles and their insulation equipment are easily affected by external factors such as climate, pollution, and mechanical stress during long-term exposure to the outdoor environment, resulting in defects such as corrosion, cracks, and insulator damage.

[0003] Currently, power line inspections mainly rely on manual labor. Inspectors need to climb tall power poles to check whether there are problems with the equipment one by one. This method is not only inefficient, consuming a large amount of manpower and time, but also has great safety hazards due to the complex working environment, especially high-altitude operations.

[0004] With the development of unmanned aerial vehicle (UAV) technology, more and more power companies have started to use UAVs to inspect power poles and towers. UAVs can fly flexibly at high altitudes, quickly cover large areas, and obtain high-resolution images, greatly improving the efficiency of inspections. However, existing UAV inspection systems still mainly rely on manual analysis and judgment of the captured images, making it difficult to achieve automated detection, which affects the inspection speed and accuracy. Summary of the Invention

[0005] The method, device, equipment and storage medium for detecting defects of power poles and towers provided by this application are used to improve the efficiency and accuracy of power pole and tower defect detection.

[0006] In a first aspect, this application provides a method for detecting defects of power poles and towers, the method including:

[0007] Obtain images captured by a UAV;

[0008] Perform multi-scale feature extraction on the images through a pre-trained improved DenseNet network to obtain target feature maps; wherein, in the improved DenseNet network, a multi-scale feature fusion module is introduced in the output layer of each dense block, and the multi-scale feature fusion module fuses low-scale features from a preset shallow layer and high-scale features from a preset deep layer to obtain a fused feature map, and adaptively adjusts the weights of each channel in the fused feature map through a channel attention mechanism;

[0009] Determine the defect boundary from the target feature map by improving the CornerNet algorithm; wherein, in the improved CornerNet algorithm, according to the defect size in the target feature map, adjust the standard deviation of the heat map to generate an adaptive key-point heat map, and determine the defect boundary according to the key-point heat map, wherein the defect size is positively correlated with the standard deviation;

[0010] According to the defect boundary, determine and output the defect type and defect location.

[0011] Optionally, in the improved DenseNet network, reduce the number of channels of the fused feature map through 1×1 convolution operation.

[0012] Optionally, the adjusting the standard deviation of the heat map according to the defect size in the target feature map to generate an adaptive key-point heat map includes:

[0013] Determine the standard deviation according to the defect size;

[0014] Generate the key-point heat map according to the standard deviation through the following formula:

[0015]

[0016] wherein, (x c , y c ) is the center position of the key point, K(x, y) is the value of the heat map, representing the heat intensity at the position (x, y), σ t is the standard deviation, and the standard deviation is used to control the expansion range of the heat map.

[0017] Optionally, the determining the standard deviation according to the defect size includes:

[0018] Determine the defect scale according to the width and height of the defect size, and the defect scale is equal to the average value of the width and the height;

[0019] Multiply the defect scale by a preset adjustment coefficient to obtain the standard deviation.

[0020] Optionally, the determining the defect boundary according to the key-point heat map includes:

[0021] Determine the position of each key point through the local maximum value of the key-point heat map;

[0022] Determine the initial bounding box of the defect according to the position of each key point;

[0023] Optimize the initial bounding box through the pre-trained bounding box regression parameters to obtain the defect boundary.

[0024] Optionally, the bounding box regression parameters are obtained by training with the following loss function:

[0025]

[0026] where (x i , y i ) are the coordinates of the key points predicted by the model, (x i * , y i * ) are the coordinates of the true key points, and N is the number of detected key points.

[0027] Optionally, determining the initial bounding box of the defect according to the position of each key point includes:

[0028] Connecting each of the key points in a preset order to determine the initial bounding box.

[0029] In a second aspect, the present application provides a defect detection device for a power transmission tower, the device includes:

[0030] An acquisition module, configured to acquire an image captured by a drone;

[0031] An improved DenseNet network module, configured to perform multi-scale feature extraction on the image through a pre-trained improved DenseNet network to obtain a target feature map; wherein, in the improved DenseNet network, a multi-scale feature fusion module is introduced in the output layer of each dense block, and the multi-scale feature fusion module fuses low-scale features from a preset shallow layer and high-scale features from a preset deep layer to obtain a fused feature map, and adaptively adjusts the weights of each channel in the fused feature map through a channel attention mechanism;

[0032] An improved CornerNet algorithm module, configured to determine a defect boundary from the target feature map through an improved CornerNet algorithm; wherein, in the improved CornerNet algorithm, according to the size of the defect in the target feature map, the standard deviation of the heat map is adjusted to generate an adaptive key point heat map, and the defect boundary is determined according to the key point heat map, wherein the size of the defect is positively correlated with the standard deviation;

[0033] A defect output module, configured to determine and output a defect type and a defect position according to the defect boundary.

[0034] In a third aspect, the present application provides an electronic device, including: a memory, a processor;

[0035] The memory stores computer execution instructions;

[0036] The processor executes the computer-executable instructions stored in the memory, such that the processor executes the above first aspect and / or various possible implementation manners of the first aspect.

[0037] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, they are used to implement the above first aspect and / or various possible implementation manners of the first aspect.

[0038] In a fifth aspect, the present application provides a computer program product, including a computer program, which when executed by a processor, implements the above first aspect and / or various possible implementation manners of the first aspect.

[0039] The present application provides a method, device, equipment and storage medium for defect detection of a power transmission tower. The method includes: acquiring an image captured by a drone; performing multi-scale feature extraction on the image through a pre-trained improved DenseNet network to obtain a target feature map; wherein, in the improved DenseNet network, a multi-scale feature fusion module is introduced in the output layer of each dense block, and the multi-scale feature fusion module fuses low-scale features from a preset shallow layer and high-scale features from a preset deep layer to obtain a fused feature map, and adaptively adjusts the weights of each channel in the fused feature map through a channel attention mechanism; determining a defect boundary from the target feature map through an improved CornerNet algorithm; wherein, in the improved CornerNet algorithm, according to the size of the defect in the target feature map, the standard deviation of the heat map is adjusted to generate an adaptive key-point heat map, and the defect boundary is determined according to the key-point heat map, wherein the size of the defect is positively correlated with the standard deviation; determining and outputting a defect type and a defect position according to the defect boundary. Through such a target deep learning model, when extracting a feature map, low-scale texture features and high-scale shape features can be acquired, thereby improving the accuracy of feature extraction; in addition, when determining a defect boundary, the size of the heat map can be adaptively adjusted according to the size of the defect, providing suitable boundary detection conditions for defects of different sizes, improving the positioning accuracy and robustness of CornerNet for small defects, and at the same time improving the key-point pairing accuracy in a complex background. Description of the Drawings

[0040] The drawings here are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0041] Figure 1 It is a schematic flow chart of the method for defect detection of a power transmission tower provided by the present application;

[0042] Figure 2Provide a schematic diagram of the algorithm process for defect detection in this application;

[0043] Figure 3 Provide a schematic diagram of the structure of the defect detection device for power transmission towers in this application;

[0044] Figure 4 Provide a schematic diagram of the structure of an electronic device in this application.

[0045] Through the above-mentioned drawings, specific embodiments of this application have been shown, and there will be more detailed descriptions hereinafter. These drawings and text descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. Detailed implementation manners

[0046] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with this application. On the contrary, they are merely examples of devices and methods consistent with some aspects of this application as detailed in the appended claims.

[0047] Aiming at the problems of slow inspection speed and low accuracy in the field of power system equipment detection, the present invention proposes a method for detecting defects in power transmission towers based on unmanned aerial vehicles and deep learning technology, aiming to solve the problems of high labor cost, low detection efficiency and potential safety hazards in the inspection of power transmission equipment, and is widely applied to the field of power equipment inspection and maintenance.

[0048] DenseNet (Densely Connected Convolutional Networks) is a convolutional neural network architecture based on dense connections, and the main goal is to improve the feature reuse of the network. The core idea of DenseNet is to directly connect the outputs of all previous layers at each layer to form a dense connection pattern. Specifically, the input of each layer consists of the outputs of all previous layers, so as to transfer more information between different layers. This design can not only effectively reduce the problem of gradient disappearance, but also improve the representation ability of the model, reduce the number of parameters, and enhance the transmission efficiency of features. DenseNet has stronger feature reuse than traditional convolutional neural networks (such as ResNet), so it can achieve good performance in a variety of computer vision tasks.

[0049] In the field of target defect detection, DenseNet has been widely applied to various detection tasks, especially in tasks such as image classification, object detection, and image segmentation. Through its dense connection structure, DenseNet can efficiently utilize the feature information between layers, greatly enhancing the model's ability in image detail extraction and high-dimensional feature learning. In the defect detection of structures such as power transmission towers, DenseNet helps the model identify damaged areas by extracting features at different scales, whether it is surface cracks or relatively minor damages.

[0050] Especially in the defect detection task of power transmission towers, the structure of power transmission towers is complex, and defects may appear at different scales and in different backgrounds. Through efficient feature transfer and fusion, DenseNet enables effective features to be extracted at multiple levels, thus achieving good results in target defect detection.

[0051] Although DenseNet performs excellently in many visual tasks, there are still certain limitations when dealing with complex backgrounds and tiny defects.

[0052] First of all, the traditional architecture of DenseNet conducts feature transfer through direct connections between layers within the same dense block. Although this design improves the reusability of features, its detection effect on targets of different scales is relatively limited. When dealing with tiny cracks or minor damages in power transmission towers, the single-scale feature extraction ability of DenseNet may not meet the requirements of precise detection.

[0053] Secondly, DenseNet has a large computational amount. Especially during the feature transfer process, the dimensions of multiple feature maps increase rapidly, resulting in low computational efficiency of the network, which in turn affects its performance in practical applications. Therefore, in tasks such as power transmission tower defect detection, especially in scenarios with high precision requirements and high real-time requirements, the performance of DenseNet may be restricted.

[0054] To address the above problems of DenseNet, an improved version of the DenseNet Supper algorithm is proposed. By introducing multi-scale feature fusion, it extracts defect features of different sizes in power transmission tower equipment; then through techniques such as feature compression and attention mechanism, it improves the detection accuracy and computational efficiency of the system.

[0055] For the feature maps extracted by DenseNet, CornerNet (an object detection algorithm based on corner detection) can be used to determine the boundaries of defects.

[0056] For CornerNet, it locates objects by predicting the corner points of the upper left and lower right corners of the target, rather than the traditional bounding box regression method. CornerNet predicts the corner point positions of each target by generating two heatmaps and uses the association between the corner points to construct the bounding box of the target. This method utilizes the spatial relationship between the corner points to improve the detection accuracy by more accurately locating the target corner points and does not require the traditional bounding box regression step. Due to its unique corner point association strategy, CornerNet can better handle the shape and structure of the target and shows advantages in detecting complex targets and small objects.

[0057] Although CornerNet has achieved good results in many object detection tasks, there are still some limitations in specific application scenarios, especially when dealing with small defects. First, CornerNet predicts the corner points of the target through a heatmap of a fixed size, which works well in conventional object detection. However, in the detection of power tower defects, the sizes and shapes of many defects vary, especially some smaller cracks or corrosion areas. Using a heatmap of a fixed scale is likely to lead to a decrease in the accuracy of corner point localization and cannot accurately capture the edge positions of the defects.

[0058] To address the above problems, this application improves the CornerNet algorithm based on the improved DenseNet Supper network. By performing adaptive key point detection to dynamically adjust the resolution of the heatmap, it optimizes the corner point prediction according to the size and shape of the defect, thereby improving the localization accuracy and enhancing the detection ability for small defects. For small defects, the range of the heatmap is smaller, ensuring that the key points can accurately fall on the small targets and improving the detection effect.

[0059] This application trains a target deep learning model, which includes the improved DenseNet Supper network and the improved CornerNet algorithm. By inputting the images captured by the drone into the model, the defect type and the defect location can be output.

[0060] The following uses specific embodiments to elaborate in detail on the technical solution of this application and how the technical solution of this application solves the above technical problems. These several specific embodiments below can be combined with each other, and for the same or similar concepts or processes, they may not be repeated in some embodiments. The following will describe the embodiments of this application with reference to the accompanying drawings.

[0061] Figure 1 It is a schematic flowchart of the defect detection method for the power tower provided by this application, as Figure 1 shown, and includes the following steps:

[0062] S101. Obtain the images captured by the drone.

[0063] S102. Extract multi-scale features from the image through a pre-trained improved DenseNet network to obtain the target feature map. Among them, in the improved DenseNet network, a multi-scale feature fusion module is introduced in the output layer of each dense block. The multi-scale feature fusion module fuses the low-scale features from a preset shallow layer and the high-scale features from a preset deep layer to obtain a fused feature map, and adaptively adjusts the weights of each channel in the fused feature map through a channel attention mechanism.

[0064] In this step, the image is input into a pre-trained target deep learning model. First, multi-scale feature extraction is performed through the improved DenseNet network. The improved DenseNet network introduces a multi-scale feature fusion module in the output layer of each dense block. The role of this module is to fuse feature maps from different levels, especially the fusion of shallow-scale features in the low layer and high-scale features in the deep layer. Low-scale features can generally capture the basic shape and texture of the image, while high-scale features can capture larger-scale information in the image, such as the background and larger structures. By fusing these features at different scales, the network can comprehensively obtain more information in the same image, thereby improving the accuracy of defect detection.

[0065] Although the gradient propagation and feature reuse capabilities are improved through dense connections between layers, as the network depth increases, the number of channels in the feature map will increase sharply, resulting in an increase in computational complexity and memory overhead. In addition, the network cannot distinguish which features are more important for a specific detection task and is prone to interference in complex backgrounds.

[0066] Therefore, a feature compression and / or attention mechanism is introduced into the DenseNet network to design DenseNet-Supper, thereby enhancing the attention to key features and reducing computational redundancy.

[0067] Feature compression can reduce the number of channels in the fused feature map through 1×1 convolution operations, thereby reducing computational overhead.

[0068] The channel attention mechanism (i.e., the Squeeze-and-Excitation (SE) module) is introduced after feature fusion to further optimize the representation of the feature map by adaptively adjusting the weights of each channel. The channel attention mechanism can dynamically select which features are more important by learning the weights of different channels, thereby improving the accuracy and robustness of feature extraction.

[0069] S103. Determine the defect boundary from the target feature map by improving the CornerNet algorithm. Among them, in the improved CornerNet algorithm, according to the defect size in the target feature map, adjust the standard deviation of the heat map to generate an adaptive key-point heat map, and determine the defect boundary based on the key-point heat map, where the defect size is positively correlated with the standard deviation.

[0070] Use the improved CornerNet algorithm to extract and determine the boundary of the defect from the target feature map. CornerNet itself is a corner-based object detection method that locates the bounding box by predicting the upper left and lower right corners of the object. In the defect detection of power transmission towers, the shape of the object is often complex, so the corner prediction needs to be improved to adapt to defects of different sizes and shapes.

[0071] To achieve more accurate defect localization, the improved CornerNet algorithm adaptively adjusts the standard deviation of the heat map according to the defect size in the target feature map. The adjustment of the standard deviation enables the heat map to change according to the specific size of the defect instead of using a fixed size, thus more accurately predicting the defect position. The defect size is positively correlated with the standard deviation, which means that larger defects will have a higher standard deviation, while smaller defects will be adjusted to a smaller standard deviation accordingly. In this way, it can be ensured that CornerNet can provide effective boundary detection for defects of various sizes.

[0072] S104. Determine and output the defect type and defect location according to the defect boundary.

[0073] Classify the generated bounding box for defects to identify the type of defects, including but not limited to insulator damage, cracks, corrosion, etc. The output results include the coordinates of the bounding box of the defect, the defect type, and the confidence score, and the results are transmitted to the ground console or remote terminal device in the form of images and texts for the use of power maintenance personnel. The output image can be overlaid with the key-point heat map and the bounding box to provide a visual detection result, which is convenient for maintenance personnel to quickly locate the defect.

[0074] This embodiment provides a method for defect detection of power transmission towers, which includes: obtaining images captured by a drone; performing multi-scale feature extraction on the images through a pre-trained improved DenseNet network to obtain target feature maps; wherein, in the improved DenseNet network, a multi-scale feature fusion module is introduced in the output layer of each dense block, and the multi-scale feature fusion module fuses low-scale features from a preset shallow layer and high-scale features from a preset deep layer to obtain a fused feature map, and adaptively adjusts the weights of each channel in the fused feature map through a channel attention mechanism; determining defect boundaries from the target feature maps through an improved CornerNet algorithm; wherein, in the improved CornerNet algorithm, according to the size of the defects in the target feature maps, the standard deviation of the heat map is adjusted to generate an adaptive key-point heat map, and the defect boundaries are determined based on the key-point heat map, where the size of the defects is positively correlated with the standard deviation; determining and outputting the defect types and defect positions according to the defect boundaries. Through such a target deep learning model, when extracting feature maps, low-scale texture features and high-scale shape features can be obtained, thereby improving the accuracy of feature extraction; in addition, when determining defect boundaries, the size of the heat map can be adaptively adjusted according to the size of the defects, providing suitable boundary detection conditions for defects of different sizes.

[0075] Figure 2 This is a schematic diagram of the algorithm process for defect detection provided by this application. Refer to Figure 2 , the algorithm process includes:

[0076] (1) Data acquisition step:

[0077] Obtain high-definition images of power transmission towers and their insulation equipment through a camera device carried by a drone. The images should cover the surfaces of power transmission towers and equipment under different angles and lighting conditions.

[0078] The data collected by the drone is wirelessly transmitted to a ground workstation or a cloud server as input data.

[0079] (2) Image preprocessing step:

[0080] Preprocess the collected power transmission tower images. The steps include but are not limited to noise removal, brightness adjustment, contrast enhancement, and normalization, etc., to improve the image quality and provide optimized image input for feature extraction and defect detection in subsequent steps.

[0081] (3) DenseNet-Supper feature extraction step:

[0082] Use the DenseNet-Supper network to perform multi-scale feature extraction on the preprocessed images:

[0083] Multi-scale Feature Fusion Module: By cross-layer connection, it fuses feature information from different depths, extracts defect features of different sizes in power pole and tower equipment, and comprehensively extracts small defects and complex structure defects in particular.

[0084] Feature Compression and Channel Attention Mechanism: A 1×1 convolution is introduced after each Dense Block for feature compression to reduce computational complexity; and the importance of each channel in the feature map is adaptively adjusted through the SE attention mechanism to highlight key features and suppress background noise.

[0085] Output the extracted deep feature map as the input for subsequent key point detection.

[0086] (4) Adaptive Key Point Detection Step:

[0087] Use the improved CornerNet model for key point detection. Specifically, it includes:

[0088] Key Point Detection: By predicting the upper left and lower right key points of the defect, the initial position of each target defect is generated, and then the initial bounding box of each target defect is determined.

[0089] Adaptive Heat Map Generation: According to the size and shape of the initial defect, dynamically adjust the standard deviation σ of the heat map t , and generate an adaptive key point heat map to ensure more accurate key point detection for small defects.

[0090] For targets of different sizes, the heat map resolution is dynamically adjusted according to the scale St of the target to ensure adaptation to different defect types.

[0091] (5) Key Point Pairing and Bounding Box Generation Step:

[0092] Perform pairing operations on the detected upper left and lower right key points (existing operations in the CornerNet model, not elaborated here) to generate the initial bounding box of each target defect.

[0093] To ensure the accuracy of key point pairing, the system optimizes the pairing process through the key point pairing loss L pair to ensure that the key points of the same defect can be correctly matched and reduce misdetection cases.

[0094] (6) Bounding Box Regression Optimization Step:

[0095] Use the bounding box regression module to optimize the initially generated bounding box. Specifically, it includes:

[0096] Perform regression adjustment on the four boundaries (left, right, top, bottom) of the initial bounding box to ensure that the generated bounding box fits the actual defect better.

[0097] Polygon bounding box generation: For defects with complex or irregular shapes (such as cracks, corrosion areas, etc.), a polygon bounding box is generated through multiple key points to more accurately describe the actual shape of the defect.

[0098] The bounding box optimization step uses the regression loss L bbox to minimize the difference between the bounding box prediction and the actual boundary, further improving the positioning accuracy of the bounding box.

[0099] (7) Defect classification and output step:

[0100] Classify the generated bounding boxes to identify the types of defects, including but not limited to insulator damage, cracks, corrosion, etc.

[0101] The output results include the bounding box coordinates of the defect, the defect type, and the confidence score, and the results are transmitted to the ground console or remote terminal device in the form of images and text for use by power maintenance personnel.

[0102] The output image can be overlaid with a heat map of key points and a bounding box to provide a visual detection result, facilitating maintenance personnel to quickly locate the defect.

[0103] Introduction to key algorithms:

[0104] 1. Principle of the DenseNet Supper algorithm

[0105] To address the limitations of DenseNet in dealing with complex backgrounds and small defects on power transmission towers, the present invention has improved the DenseNet design to the DenseNet Supper algorithm, mainly including multi-scale feature fusion and feature compression and attention mechanism to improve the defect detection accuracy of the system and the computational efficiency of the network.

[0106] (1) Multi-scale Feature Fusion

[0107] Traditional DenseNet transfers features through direct connections between layers within the same dense block. Although this design improves the reusability of features, its detection effect on target defects of different scales is limited. The sizes of defects on power transmission towers vary greatly, especially some small cracks and minor damages, and the single-scale feature extraction ability of DenseNet will be restricted. To improve the adaptability of the model to defects of different scales, the present invention introduces multi-scale feature fusion into DenseNet. Specifically, the network adds a multi-scale feature fusion module to the output layer of each dense block to collect features from different levels (shallow and deep layers) and fuse them to capture more diverse target features.

[0108] Assume that the input of the l-th layer is x l , and the feature transfer formula of the original DenseNet is:

[0109] x l+1 = H l ([x0, x1,..., x l )

[0110] where H l is a feature extraction function formed by combining a 1×1 convolutional layer and a 3×3 convolutional layer, and the input is the cumulative feature map.

[0111] In DenseNet-Supper, the present invention introduces the fusion of multiple scale feature maps, combining low-scale features from shallower layers and high-scale features from deeper layers. The formula is expressed as:

[0112] x l+1 = F(H l ([x0, x1,..., x l ), S(x k ))

[0113] where S(x k ) represents the low-scale features extracted from a preset shallower layer k (where k is a preset multiple layers, and the specific setting is determined according to actual training needs), and F represents a feature fusion function, which usually uses convolutional operations or upsampling / downsampling operations to align and fuse different scale features.

[0114] The feature fusion function can be any one of the following:

[0115] Concatenation: Directly concatenate features from different sources or different levels to form a larger feature vector.

[0116] Weighted summation: Directly perform weighted summation on features from different sources or levels. This method assumes that all features have similar importance at different levels or different sources, and adjusts their influence through weighted parameters.

[0117] Element-wise addition: After aligning the spatial dimensions of different features, fuse the features through element-wise addition. It is usually used for feature fusion of the same scale or the same source, keeping the dimension of the features unchanged.

[0118] Cascade method: Gradually fuse features through multiple steps. Usually, rough fusion is performed first, and then fine optimization is carried out. Different fusion strategies may be applied in each step to gradually improve the quality of the fused features.

[0119] In summary, through this multi-scale feature fusion mechanism, the network can simultaneously capture the feature information of small defects and larger structural defects. Especially for small targets such as fine cracks and corrosion in power transmission towers, the detection accuracy can be significantly improved.

[0120] (2) Feature Compression and Attention Mechanism

[0121] Although DenseNet improves the gradient propagation and feature reuse capabilities through dense connections between layers, as the network depth increases, the number of channels in the feature map will increase sharply, resulting in an increase in computational complexity and memory overhead. In addition, the network cannot distinguish which features are more important for a specific detection task and is prone to interference in complex backgrounds.

[0122] Therefore, the present invention introduces a feature compression and attention mechanism into DenseNet to design DenseNet-Supper, thereby enhancing the attention to key features and reducing computational redundancy.

[0123] Feature Compression: Feature compression reduces the number of channels in the feature map through 1×1 convolution operations, thereby reducing computational overhead. Suppose the output feature map at a certain layer is:

[0124]

[0125] where H and W are the height and width of the feature map respectively, and C is the number of channels.

[0126] Through 1×1 convolution, the number of channels is reduced from C to C′, and the formula is:

[0127] F′ = W 1×1 * F

[0128] where, W 1×1 is the weight matrix of the 1×1 convolution kernel, and * represents the convolution operation, is the compressed feature map. In this way, the expression ability of the feature is retained, but the computational complexity is significantly reduced.

[0129] Channel Attention Mechanism: To further enhance the extraction of key features, the present invention also introduces a channel attention mechanism, namely the Squeeze-and-Excitation (SE) module. The SE module adaptively assigns weights to each channel, enhances the attention to key features, and suppresses irrelevant or secondary features.

[0130] Squeeze (Feature Compression): First, the information of each channel is compressed into a scalar through global average pooling to represent the importance of that channel.

[0131]

[0132] Among them, F c (i, j) is the value of the c-th channel of the feature map at the position (i, j), and z c is the global average value of this channel.

[0133] Excitation (Feature Enhancement): After passing through two fully connected networks (FC), the weight s of the channel is generated through the sigmoid activation function c :

[0134] s = σ(W2 · ReLU(W1 · z))

[0135] Among them, W1 and W2 are learnable weight matrices, σ is the sigmoid function, and the output s is the weight vector of each channel.

[0136] Feature recalibration: Finally, each channel of the feature map is multiplied by the corresponding weight to adjust the importance of each channel:

[0137] F′ c = s c · F c

[0138] Among them, s c is the weight of channel c, and F c ′ is the weighted feature map.

[0139] Through feature compression and the attention mechanism, the network can adaptively focus on the features crucial for the defect detection task, enhance the recognition ability of small defects, and reduce the interference of background noise. In addition, this mechanism reduces the computational amount, enabling the network to operate efficiently on the UAV platform.

[0140] 2. Principle of Improved CornerNet Algorithm

[0141] To enhance the performance of the CornerNet model in the power pole defect detection task, the present invention improves it, mainly including adaptive key point detection and bounding box regression optimization. These improvements improve the localization accuracy and robustness of CornerNet for small defects, and at the same time improve the key point pairing accuracy in complex backgrounds. The improved algorithm principle is described in detail below.

[0142] (1) Adaptive Keypoint Detection

[0143] CornerNet predicts the upper - left and lower - right keypoints of an object through a heatmap of a fixed size. Although this method is effective in general object detection tasks, in the detection of power tower defects, especially for small or irregular defects (such as cracks, corrosion, etc.), the scale of the fixed heatmap will affect the positioning accuracy of the keypoints. To solve this problem, the present invention proposes an adaptive keypoint detection method that dynamically adjusts the resolution of the heatmap to adapt to the size and shape of the defect, thereby improving the detection ability for small defects.

[0144] 1) Generation of Keypoint Heatmap

[0145] In adaptive keypoint detection, the standard deviation σt of the heatmap is adaptively adjusted according to the size and shape of the defect target. Specifically, for each defect, the generated heatmap K t (x, y) is given by the formula:

[0146]

[0147] where (x c , y c ) is the center position of the keypoint, σ t is the standard deviation adjusted according to the target size, which is used to control the expansion range of the heatmap. For small defects, the value of σ t is small, and the range of the heatmap is narrow, ensuring that the keypoints fall more accurately on small targets; for large defects, the value of σ t is large, allowing a larger range for keypoint prediction.

[0148] 2) Adaptive Scale Strategy

[0149] Adaptive keypoint detection dynamically adjusts the resolution of the heatmap by calculating the scale S t of each defect. The scale S t is defined as the mean of the width and height of the defect:

[0150]

[0151] where w t and h t are the width and height of the target defect respectively. According to the value of S t , the standard deviation σ t of the heatmap is dynamically set to:

[0152] σ t =α·S t

[0153] Among them, α is an adjustment coefficient used to control the expansion range of the heat map.

[0154] Through the adaptive heat map generation mechanism, the present invention can more accurately predict the defect positions of different sizes on the power transmission tower, especially the detection ability for small defects has been significantly improved.

[0155] (2) Bounding Box Regression Optimization

[0156] The original CornerNet generates the bounding box of the defect by pairing key points. However, when the defect shape is complex or the size difference is large, this method may lead to inaccurate bounding box generation. Therefore, the present invention introduces bounding box regression optimization to further optimize the generated bounding box through learning the regression model to improve the accuracy of the bounding box.

[0157] 1) Bounding Box Regression Model

[0158] The goal of bounding box regression is to make the bounding box more consistent with the boundary of the real defect by optimizing the coordinates of the predicted key points. Specifically, the regression errors of the four boundaries (left, right, top, bottom) of the bounding box are minimized. The regression objective is:

[0159]

[0160] Among them, (x i, y i ) are the coordinates of the key points predicted by the model, (x i * , y i * ) are the real key point coordinates, and N is the number of detected key points. By minimizing this loss function, the regression model can accurately adjust the four boundaries of the bounding box to make it more conform to the real defect shape and position.

[0161] 2) Polygonal Bounding Box Generation

[0162] The defects on the power transmission tower are usually irregular in shape, such as crack or corrosion areas. To better adapt to these irregular shapes, the bounding box regression optimization of the present invention is not only applicable to the traditional rectangular bounding box, but also supports the generation of polygonal bounding boxes. By connecting multiple key points, the system can generate a boundary more conforming to the defect shape.

[0163] Assume that the key point set of the defect is P = {(x1, y1), (x2, y2), …, (x n , y n )}, then its bounding box can be represented as a polygon B, and its area A(B) can be calculated by the polygon area formula:

[0164]

[0165] The polygon bounding box generated from the key point coordinates optimized by bounding box regression can better match the actual shape of the defect.

[0166] Figure 3 As shown in the structural schematic diagram of the defect detection device for power transmission towers provided in this application, Figure 3 as shown, the defect detection device 30 for power transmission towers provided in this embodiment includes:

[0167] An acquisition module 301, configured to acquire an image captured by a drone;

[0168] An improved DenseNet network module 302, configured to perform multi-scale feature extraction on the image through a pre-trained improved DenseNet network to obtain a target feature map; wherein, in the improved DenseNet network, a multi-scale feature fusion module is introduced in the output layer of each dense block, and the multi-scale feature fusion module fuses low-scale features from a preset shallow layer and high-scale features from a preset deep layer to obtain a fused feature map, and adaptively adjusts the weights of each channel in the fused feature map through a channel attention mechanism;

[0169] An improved CornerNet algorithm module 303, configured to determine a defect boundary from the target feature map through an improved CornerNet algorithm; wherein, in the improved CornerNet algorithm, according to the size of the defect in the target feature map, the standard deviation of the heat map is adjusted to generate an adaptive key point heat map, and the defect boundary is determined according to the key point heat map, wherein the size of the defect is positively correlated with the standard deviation;

[0170] A defect output module 304, configured to determine and output a defect type and a defect location according to the defect boundary.

[0171] The defect detection device for power transmission towers provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here in this embodiment.

[0172] Optionally, in the improved DenseNet network module 302, the number of channels of the fused feature map is reduced through a 1×1 convolution operation.

[0173] Optionally, the improved CornerNet algorithm module 303 is specifically configured to:

[0174] Determine the standard deviation according to the size of the defect;

[0175] Generate the heat map of the key points according to the standard deviation through the following formula:

[0176]

[0177] where (x c , y c ) is the center position of the key point, K(x, y) is the value of the heat map, representing the heat intensity at the position (x, y), and σ t is the standard deviation, which is used to control the expansion range of the heat map.

[0178] Optionally, the improved CornerNet algorithm module 303 is specifically configured to:

[0179] Determine the defect scale according to the width and height of the defect size, and the defect scale is equal to the average value of the width and the height;

[0180] Multiply the defect scale by a preset adjustment coefficient to obtain the standard deviation.

[0181] Optionally, the improved CornerNet algorithm module 303 is specifically configured to:

[0182] Determine the position of each key point through the local maximum value of the heat map of the key points;

[0183] Determine the initial bounding box of the defect according to the position of each key point;

[0184] Optimize the initial bounding box through the bounding box regression parameters obtained by pre-training to obtain the defect boundary.

[0185] Optionally, the bounding box regression parameters are trained through the following loss function:

[0186]

[0187] where (x i , y i ) are the coordinates of the key points predicted by the model, (x i * , y i * ) are the coordinates of the true key points, and N is the number of detected key points.

[0188] The defect detection device for power transmission towers provided by this application can implement the steps in the above method embodiments, and its principle and effect are similar to those of the method embodiments, which will not be elaborated here.

[0189] Figure 4 This is a schematic structural diagram of the electronic device provided by this application. As Figure 4As shown in the figure, the electronic device 50 provided in this embodiment includes: at least one processor 501 and a memory 502. Optionally, the device 50 further includes a communication component 503. Among them, the processor 501, the memory 502, and the communication component 503 are connected through a bus 504.

[0190] In the specific implementation process, at least one processor 501 executes the computer-executable instructions stored in the memory 502, so that at least one processor 501 executes the above-mentioned method.

[0191] For the specific implementation process of the processor 501, reference can be made to the above method embodiment, and its implementation principle and technical effect are similar, so they will not be elaborated here in this embodiment.

[0192] In the above embodiment, it should be understood that the processor may be a central processing unit (English: Central Processing Unit, abbreviated as: CPU), or other general-purpose processors, digital signal processors (English: Digital Signal Processor, abbreviated as: DSP), application specific integrated circuits (English: Application Specific Integrated Circuit, abbreviated as: ASIC), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0193] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0194] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0195] This application also provides a computer program product, including a computer program, which implements the above-mentioned method when executed by a processor.

[0196] The electronic device of the present application can also be integrated into a drone control device or a device that communicates with a drone remotely.

[0197] The present application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above method.

[0198] The above-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0199] An exemplary readable storage medium is coupled to a processor so that the processor can read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in a device.

[0200] The division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections between each other can be through some interfaces, and the indirect couplings or communication connections of devices or units can be in electrical, mechanical, or other forms.

[0201] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0202] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0203] If a function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs and other various media that can store program codes.

[0204] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When this program is executed, it executes the steps including the above method embodiments; and the aforementioned storage medium includes: ROMs, RAMs, magnetic disks, or optical discs and other various media that can store program codes.

[0205] Finally, it should be noted that: After considering the specification and practicing the invention disclosed herein, those skilled in the art will easily think of other implementation manners of the present invention. The present invention is intended to cover any variations, uses, or adaptive changes of the present invention. These variations, uses, or adaptive changes follow the general principles of the present invention and include common general knowledge or conventional technical means in the technical field not disclosed in the present invention. It is not limited to the exact structure already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A method for detecting defects in a power tower, characterized in that: The method comprises: Get images taken by drones; Performing multi-scale feature extraction on the image through a pre-trained improved DenseNet network to obtain a target feature map; wherein, in the improved DenseNet network, a multi-scale feature fusion module is introduced into the output layer of each dense block, and the multi-scale feature fusion module performs feature fusion on low-scale features from a preset shallow layer and high-scale features from a preset deep layer to obtain a fused feature map, and adaptively adjusts the weights of each channel in the fused feature map through a channel attention mechanism; Determine the defect boundary from the target feature map by using an improved CornerNet algorithm; wherein, in the improved CornerNet algorithm, according to the defect size in the target feature map, adjust the standard deviation of the heat map, generate an adaptive key point heat map, and determine the defect boundary according to the key point heat map, wherein the defect size is positively correlated with the standard deviation; According to the defect boundary, the defect type and defect position are determined and output.

2. The method according to claim 1, characterized in that: In the improved DenseNet network, the number of channels of the fused feature map is reduced through a 1×1 convolution operation.

3. The method according to claim 1, characterized in that The step of adjusting the standard deviation of the heat map according to the defect size in the target feature map to generate an adaptive key point heat map includes: Determining the standard deviation according to the defect size; The key point heat map is generated according to the standard deviation using the following formula: Among them, (x c ,y c ) is the center position of the key point, K(x,y) is the value of the heat map, indicating the heat intensity at the position (x,y), σ t The standard deviation is used to control the expansion range of the heat map.

4. The method according to claim 3, characterized in that Determining the standard deviation according to the defect size includes: Determine the defect scale according to the width and height of the defect size, wherein the defect scale is equal to the average value of the width and the height; The defect scale is multiplied by a preset adjustment coefficient to obtain the standard deviation.

5. The method according to any one of claims 1 to 4, characterized in that: Determining the defect boundary according to the key point heat map includes: Determine the position of each key point by the local maximum of the key point heat map; Determine an initial bounding box of the defect according to the position of each key point; The initial bounding box is optimized by using bounding box regression parameters obtained through pre-training to obtain the defect boundary.

6. The method according to claim 5, characterized in that The bounding box regression parameters are trained using the following loss function: Among them, (x i ,y i ) is the key point coordinate predicted by the model, (x i * ,y i * ) are the true key point coordinates, and N is the number of key points detected.

7. The method according to claim 6, characterized in that Determining an initial boundary box of the defect according to the position of each key point includes: Each of the key points is connected in a preset order to determine the initial bounding box.

8. A defect detection device for a power tower, characterized in that: The device comprises: An acquisition module is used to acquire images taken by a drone; An improved DenseNet network module is used to perform multi-scale feature extraction on the image through a pre-trained improved DenseNet network to obtain a target feature map; wherein, in the improved DenseNet network, a multi-scale feature fusion module is introduced into the output layer of each dense block, and the multi-scale feature fusion module performs feature fusion on low-scale features from a preset shallow layer and high-scale features from a preset deep layer to obtain a fused feature map, and adaptively adjusts the weights of each channel in the fused feature map through a channel attention mechanism; An improved CornerNet algorithm module, used for determining the defect boundary from the target feature map by using an improved CornerNet algorithm; wherein, in the improved CornerNet algorithm, the standard deviation of the heat map is adjusted according to the defect size in the target feature map, an adaptive key point heat map is generated, and the defect boundary is determined according to the key point heat map, wherein the defect size is positively correlated with the standard deviation; The defect output module is used to determine and output the defect type and defect position according to the defect boundary.

9. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images