A method, system, device and storage medium for detecting defects of a power transmission line

Abstract

The present application relates to the technical field of image analysis, in particular to a power transmission line defect detection method, system, device and storage medium, a defect detection neural network composed of a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module and an image segmentation module, which can not only significantly reduce the computing burden, but also be suitable for real-time application of unmanned aerial vehicles, and most importantly, can separate small targets from the background, improving the detection accuracy of small targets in complex background and occlusion scenarios; at the same time, in the process of training the defect detection neural network, by controlling the sum of the intersection over union loss, the bounding box coordinate loss and the bounding box shape loss to be less than the loss threshold, the matching degree of the predicted box and the real box is improved, thereby ensuring the accuracy of small target detection in the case of complex background or irregular target shape.

Description

Technical Field

[0001] The present invention relates to the field of image analysis technology, and in particular to a method, system, device and storage medium for detecting defects in a transmission line. Background Art

[0002] Core components of power transmission lines include insulators, shock absorbers, and transmission towers. These components are constantly exposed to harsh natural environments and are highly susceptible to a variety of factors, including temperature and humidity fluctuations, lightning, strong electric fields, and contamination. These defects can lead to defects such as breakage, spontaneous explosions, and contamination. These defects not only affect the stability of transmission lines but, in severe cases, can even cause widespread power outages. Therefore, regular inspections and status monitoring of transmission line equipment are crucial. With the development of drone technology, it is increasingly being used in power system inspections. Drones (UAVs) can capture large amounts of transmission line image data in a short period of time, offering the advantages of high mobility and efficiency. However, the sheer volume of drone inspection images creates a significant challenge in extracting accurate information from them.

[0003] In power system inspections, the most commonly used object detection methods are deep learning models, particularly single-stage object detection algorithms such as the YOLO family and SSD. These algorithms offer the advantages of fast detection speed and high accuracy, making them suitable for real-time detection tasks. However, these models are computationally complex and resource-intensive, making them difficult to directly apply to resource-limited devices such as drones. To meet the real-time and lightweight requirements of drone inspections, many researchers have improved algorithms such as YOLO and SSD. However, these algorithms still face challenges in the unique scenarios of power line inspections. For one thing, transmission line images feature large scenes, multiple scales, small objects, complex backgrounds, and object occlusion. Traditional detection models still perform poorly in such images, prone to missed or false detections. Furthermore, deep learning-based object detection models are parameter-intensive and typically require significant computational resources, which is particularly restrictive given the limited computing power and battery life of drones. Summary of the Invention

[0004] The object of the present invention is to provide a method, system, device and storage medium for detecting defects in a transmission line.

[0005] The technical solutions of the present invention are as follows:

[0006] A method for detecting defects in a transmission line includes the following operations:

[0007] S1. Construct a defect detection neural network based on a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module, and an image segmentation module;

[0008] The processing process in the multi-level image feature fusion analysis module is as follows: after upsampling, the deep image features are spliced ​​with the middle image features to obtain the middle-deep fusion features; the middle-deep fusion features are aggregated with multi-scale features to obtain the middle-deep multi-scale fusion features; the middle-deep multi-scale fusion features are upsampling and spliced ​​with the shallow image features to obtain the multi-level fusion features; the multi-level fusion features are aggregated with multi-scale attention features to obtain the first image semantic features; the shallow image features, middle image features and deep image features are all outputs of the multi-level image feature extraction module;

[0009] The specific operation of the multi-scale attention feature aggregation processing is as follows: the multi-level fusion feature is processed by feature convolution and character segmentation to obtain a fused convolution feature; the fused convolution feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain a first fused enhanced feature; the first fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain a second fused enhanced feature; the second fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain a third fused enhanced feature; and so on, several times of partial convolution-based attention mechanism processing and splicing processing are performed to obtain the Nth fused enhanced feature; the Nth fused enhanced feature and the fused convolution feature are spliced ​​and convolved to obtain the first image semantic feature;

[0010] S2. Using a training data set consisting of defect images of several key parts of the transmission line, a defect detection neural network is trained. When the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss, and the bounding box shape loss in the image segmentation module is less than a loss threshold, the trained defect detection neural network is obtained.

[0011] S3. The transmission line diagram to be inspected is processed by the trained defect detection neural network to obtain a defect detection result.

[0012] The processing process in the first image semantic feature enhancement module in S1 is specifically as follows: the first image semantic feature output by the multi-level image feature fusion analysis module is processed by multi-scale attention feature aggregation or feature convolution, and then sequentially processed by splicing and multi-scale attention feature aggregation with the mid-to-deep multi-scale fusion feature to obtain the second image semantic feature.

[0013] The specific processing process in the multi-level image feature extraction module in S1 is as follows: the transmission line diagram is subjected to feature convolution and two rounds of feature convolution and multi-scale feature aggregation to obtain shallow image features; the shallow image features are subjected to feature convolution and multi-scale feature aggregation to obtain middle-level image features; the middle-level image features are subjected to feature convolution, multi-scale feature aggregation and spatial pyramid pooling to obtain deep image features.

[0014] The operation of the attention mechanism processing based on partial convolution in S1 is specifically: the input features are sequentially subjected to partial convolution, point-by-point convolution for several times, attention mechanism processing, and then are subjected to splicing processing with the input features to obtain output features.

[0015] The bounding box coordinate loss in S2 is obtained by the following formula:

[0016] ,

[0017] is the bounding box coordinate loss, , are the width and height of the real box respectively, s is a scale factor, , are the center point coordinates of the predicted box and the real box respectively, c is the diagonal distance of the intersection area of the predicted box and the real box.

[0018] The bounding box shape loss in S2 is obtained by the following formula:

[0019] ,

[0020] ,

[0021] ,

[0022] is the bounding box shape loss, is a first loss coefficient, is a first loss coefficient, , are the width and height of the predicted box respectively, , are the width and height of the real box respectively, s is a scale factor.

[0023] The specific processing operations in the image semantic feature fusion analysis module in S1 are as follows: the first image semantic feature, the second image semantic feature and the third image semantic feature are respectively subjected to multi-scale feature extraction processing and then spliced ​​to obtain image semantic fusion features; the multi-scale feature extraction processing operation is as follows: the first image semantic feature or the second image semantic feature and the third image semantic feature are respectively subjected to feature convolution of different scales, multi-scale channel attention mechanism and convolution processing and then spliced ​​to obtain the first image semantic enhancement feature or the second image semantic enhancement feature or the third image semantic enhancement feature for performing the splicing processing operation; the first image semantic feature, the second image semantic feature and the third image semantic feature are respectively the outputs of the multi-level image feature fusion analysis module, the first image semantic feature enhancement module and the second image semantic feature enhancement module.

[0024] A transmission line defect detection system, used to implement the above-mentioned transmission line defect detection method, comprising:

[0025] The defect detection neural network generation module is used to construct a defect detection neural network based on the multi-level image feature extraction module, the multi-level image feature fusion analysis module, the first image semantic feature enhancement module, the second image semantic feature enhancement module, the image semantic feature fusion analysis module and the image segmentation module; the processing process in the multi-level image feature fusion analysis module is specifically as follows: the deep image features are up-sampled and then spliced ​​with the middle image features to obtain the middle-deep fusion features; the middle-deep fusion features are subjected to multi-scale feature aggregation to obtain the middle-deep multi-scale fusion features; the middle-deep multi-scale fusion features are up-sampled and then spliced ​​with the shallow image features to obtain the multi-level fusion features; the multi-level fusion features are subjected to multi-scale attention feature aggregation to obtain the first image semantic features; the shallow image features, the middle image features and the deep image features are all The output of the multi-level image feature extraction module; the operation of the multi-scale attention feature aggregation processing is specifically as follows: the multi-level fusion feature is subjected to feature convolution and character segmentation processing to obtain a fused convolution feature; the fused convolution feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain a first fused enhanced feature; the first fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain a second fused enhanced feature; the second fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain a third fused enhanced feature; and so on, performing several times of partial convolution-based attention mechanism processing and splicing processing to obtain the Nth fused enhanced feature; the Nth fused enhanced feature and the fused convolution feature are spliced ​​and convolved to obtain the first image semantic feature;

[0026] A training defect detection neural network generation module is used to train the defect detection neural network using a training data set composed of defect images of several key parts of the transmission line. When the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss, and the bounding box shape loss in the image segmentation module is less than a loss threshold, the trained defect detection neural network is obtained;

[0027] The defect detection result generation module is used to process the transmission line diagram to be inspected through the trained defect detection neural network to obtain the defect detection result.

[0028] A power transmission line defect detection device comprises a processor and a memory, wherein the processor implements the above-mentioned power transmission line defect detection method when executing a computer program stored in the memory.

[0029] A computer-readable storage medium is used to store a computer program, wherein the computer program implements the above-mentioned power transmission line defect detection method when executed by a processor.

[0030] The beneficial effects of the present invention are:

[0031] (1) Constructing a defect detection neural network consisting of a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module, and an image segmentation module. This not only significantly reduces the computational burden and is suitable for real-time applications of drones, but most importantly, it can separate small targets from the background and improve the detection accuracy of small targets in complex backgrounds and occlusion scenarios.

[0032] (2) The multi-level image feature fusion analysis module of the defect detection neural network uses multi-scale attention feature aggregation processing, which can enhance the global perception ability of small targets and effectively suppress the interference of complex backgrounds. This makes the defect detection neural network more accurate in identifying small targets such as broken insulators and bird nests, and reduces the occurrence of false detections and missed detections.

[0033] (3) During the training of the defect detection neural network, the matching degree between the predicted box and the true box is improved by controlling the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss, and the bounding box shape loss to be less than the loss threshold, thereby ensuring the accuracy of small target detection in scenes with limited computing resources and complex backgrounds or irregular target shapes;

[0034] (4) The present invention provides a method for detecting defects in power transmission lines, which is a lightweight detection technology suitable for real-time inspection by drones. It can promptly detect potential safety hazards such as insulator damage, falling shock hammers, and bird nests on towers. At the same time, through high-precision and real-time detection results, power maintenance personnel can respond quickly and prevent faults from occurring, thereby significantly improving the safety and stability of power transmission lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] By reading the detailed description of the preferred embodiment below, the solutions and advantages of the present application will become clear to those skilled in the art. The accompanying drawings are only for illustrating the preferred embodiment and are not to be considered as limiting the present invention.

[0036] In the attached figure:

[0037] Figure 1 This is a graph showing the precision-recall effect of the experimental dataset processed by the method of this embodiment;

[0038] Figure 2 2 is a confusion matrix diagram obtained by processing the experimental data set using the method of this embodiment. DETAILED DESCRIPTION

[0039] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings.

[0040] This embodiment provides a method for detecting defects in a power transmission line, including the following operations:

[0041] S1. Construct a defect detection neural network based on a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module, and an image segmentation module;

[0042] The processing process in the multi-level image feature fusion analysis module is as follows: after upsampling, the deep image features are spliced ​​with the middle image features to obtain the middle-deep fusion features; the middle-deep fusion features are aggregated with multi-scale features to obtain the middle-deep multi-scale fusion features; the middle-deep multi-scale fusion features are upsampling and spliced ​​with the shallow image features to obtain the multi-level fusion features; the multi-level fusion features are aggregated with multi-scale attention features to obtain the first image semantic features; the shallow image features, middle image features and deep image features are all outputs of the multi-level image feature extraction module;

[0043] The specific operation of the multi-scale attention feature aggregation processing is as follows: the multi-level fusion feature is processed by feature convolution and character segmentation to obtain a fused convolution feature; the fused convolution feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain a first fused enhanced feature; the first fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain a second fused enhanced feature; the second fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain a third fused enhanced feature; and so on, several times of partial convolution-based attention mechanism processing and splicing processing are performed to obtain the Nth fused enhanced feature; the Nth fused enhanced feature and the fused convolution feature are spliced ​​and convolved to obtain the first image semantic feature;

[0044] S2. Using a training data set consisting of defect images of several key parts of the transmission line, a defect detection neural network is trained. When the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss, and the bounding box shape loss in the image segmentation module is less than a loss threshold, the trained defect detection neural network is obtained.

[0045] S3. The transmission line diagram to be inspected is processed by the trained defect detection neural network to obtain a defect detection result.

[0046] S1. Construct a defect detection neural network based on a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module and an image segmentation module.

[0047] Constructing a defect detection neural network consisting of a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module and an image segmentation module can not only significantly reduce the computational burden and be suitable for real-time applications of drones; most importantly, it can separate small targets from the background and improve the detection accuracy of small targets in complex backgrounds and occlusion scenarios.

[0048] The multi-level image feature extraction module is used to extract different levels of features of the transmission line map, obtaining shallow image features with prominent edges and textures, mid-level image features with prominent object shapes and component structures, and deep image features with prominent semantic features and scene features.

[0049] The multi-level image feature extraction module performs the following processing: The power transmission line diagram undergoes feature convolution, followed by two rounds of feature convolution and multi-scale feature aggregation to obtain shallow-level image features; shallow-level image features undergo feature convolution and multi-scale feature aggregation to obtain mid-level image features; and mid-level image features undergo feature convolution, multi-scale feature aggregation, and spatial pyramid pooling to obtain deep-level image features. This feature convolution can be achieved through convolution, batch normalization, and nonlinear processing (SiLU activation function). Multi-scale feature aggregation includes, but is not limited to, the C2f module, and spatial pyramid pooling includes, but is not limited to, the SPPF module.

[0050] The multi-level image feature fusion analysis module is used to fuse the shallow image features, middle image features and deep image features output by the multi-level image feature extraction module, so that the acquired image contains both overall semantic information and local details, thereby improving the detection and recognition capabilities of subsequent small targets.

[0051] The processing process in the multi-level image feature fusion analysis module is as follows: the deep image features are upsampled and then spliced ​​with the middle image features to highlight the component structure features and obtain the middle-deep fusion features; the middle-deep fusion features are aggregated with multi-scale features to enhance the image semantic feature expression ability and obtain the middle-deep multi-scale fusion features; the middle-deep multi-scale fusion features are upsampled and then spliced ​​with the shallow image features to enhance the local detail features on the basis of highlighting the component structure features and obtain the multi-level fusion features; the multi-level fusion features are aggregated with multi-scale attention features to separate the target objects, especially small targets, from the background and obtain the first image semantic features.

[0052] The above-mentioned multi-scale attention feature aggregation processing operation is specifically as follows: the multi-level fusion features are subjected to feature convolution and character segmentation processing (which can be achieved through the Split function) to obtain fused convolution features; the fused convolution features are processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution features to perform high-quality feature extraction on the image while reducing computational redundancy and memory access to obtain the first fused enhanced feature; the first fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain the second fused enhanced feature; the second fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain the third fused enhanced feature; and so on, several times of partial convolution-based attention mechanism processing and splicing processing are performed to obtain the Nth fused enhanced feature; the Nth fused enhanced feature and the fused convolution feature are spliced ​​and convolved to obtain the first image semantic feature.

[0053] The attention mechanism based on partial convolution can be implemented as follows: the input features are sequentially subjected to partial convolution (which can be implemented through PConv convolutional layers), several point-wise convolutions (which can be implemented through PWconv convolutional layers), and then processed by the attention mechanism. The results are then concatenated with the input features to obtain the output features. Alternatively, the input features are sequentially subjected to partial convolution (which can be implemented through PConv convolutional layers), several convolutions, and processed by the attention mechanism. The results are then concatenated with the input features to obtain the output features.

[0054] The operation of the above attention mechanism is as follows: the input is grouped and the input X is divided into G groups. The number of channels in each group is C / G, and the groups are independent of each other, so as to better focus on the feature information of each group and obtain several groups of features; each group feature is pooled horizontally and vertically to obtain a horizontal tensor of size C / G×W×1 and a vertical tensor of size C / G×H×1; all horizontal tensors and all vertical tensors are concatenated, convolved and processed with Sigmoid function respectively to extract local spatial information and spatial features that can capture a small range, and then weight generation is performed (which can be achieved through Re-weight function) to obtain a spatial weight feature map; the spatial weight feature map is multiplied element-by-element with the input, and significant feature information is input while suppressing unimportant features to obtain a spatial feature map; the spatial feature map is divided The group is normalized to obtain a spatial normalized feature map; the spatial normalized feature map is processed by average pooling and Softmax function to obtain spatial nonlinear features; at the same time, all group features are convolved with a convolution kernel of 3 and then spliced ​​in the channel dimension to obtain local features; the local features are processed by average pooling and Softmax function to obtain local nonlinear features; the spatial nonlinear features and the local features are fused to obtain the first spatial fusion feature; the local nonlinear features and the spatial normalized feature map are fused to obtain the second spatial fusion feature; the first spatial fusion feature and the second spatial fusion feature are added element by element, then processed by Sigmoid function and weights are generated to obtain a spatial weight optimization map; the spatial weight optimization map is multiplied element by element with the input to obtain an output containing cross-channel and cross-space information, which is used to perform the above-mentioned splicing operation.

[0055] The first image semantic feature enhancement module is used to enhance the first image semantic features output by the multi-level image feature fusion analysis module to improve the semantic features of the power transmission line diagram. The processing in the first image semantic feature enhancement module is as follows: the first image semantic features output by the multi-level image feature fusion analysis module are subjected to multi-scale attention feature aggregation or feature convolution processing. These features are then concatenated with the mid- and deep-level multi-scale fusion features and then subjected to multi-scale attention feature aggregation processing. This process highlights the structural features of components in the power transmission line diagram, facilitating subsequent accurate component detection and identification, and resulting in the second image semantic features.

[0056] The second image semantic feature enhancement module is used to enhance the second image semantic features output by the first image semantic feature enhancement module, further improving the semantic features of the power transmission line diagram. The processing in the second image semantic feature enhancement module is as follows: the second image semantic features output by the first image semantic feature enhancement module are subjected to multi-scale attention feature aggregation or feature convolution processing. These features are then concatenated with the deep image features output by the multi-level image feature extraction module and then subjected to multi-scale attention feature aggregation processing. This process highlights the semantic features of components in the power transmission line diagram, facilitating subsequent accurate component detection and identification, and resulting in third image semantic features.

[0057] The image semantic feature fusion analysis module is used to fuse the first, second, and third image semantic features output by the multi-level image feature fusion analysis module, the first image semantic feature enhancement module, and the second image semantic feature enhancement module, respectively, to improve feature richness and generate image semantic fusion features. The processing operation in the image semantic feature fusion analysis module is as follows: after the first, second, and third image semantic features are processed through multi-scale feature extraction, they are concatenated to generate image semantic fusion features.

[0058] The specific operation of multi-scale feature extraction processing is: the first image semantic feature or the second image semantic feature and the third image semantic feature are respectively subjected to feature convolution of different scales, multi-scale channel attention mechanism and convolution processing, and then spliced ​​to obtain the first image semantic enhancement feature or the second image semantic enhancement feature or the third image semantic enhancement feature, which is used to perform the splicing processing operation.

[0059] The specific operation of the multi-scale channel attention mechanism is as follows: after the input is processed by the channel attention mechanism of different scales, the key channel information is captured and then added element-by-element to the input to obtain the first comprehensive feature; after the first comprehensive feature is processed by average pooling, full connection and channel expansion, it is multiplied element-by-element with the input to obtain the output.

[0060] The specific operations of the channel attention mechanism are as follows: the input is processed by one-dimensional embedding (which can be achieved through PatchEmbedding), GELU activation function, batch normalization and depth-wise separable convolution to obtain depth-wise separable features; the depth-wise separable features are processed by GELU activation function and batch normalization, and then added element-by-element with the depth-wise separable features to obtain depth-wise normalized fusion features; the depth-wise normalized fusion features are processed by GELU activation function and batch normalization to obtain output.

[0061] The image segmentation module is used to segment the image semantic fusion features output by the image semantic feature fusion analysis module to obtain detection results. Segmentation can be achieved through bounding box regression and classification recognition. For example, segmentation can be achieved using the RCNN object detection method.

[0062] S2. Use a training data set consisting of defect images of several key parts of the transmission line to train a defect detection neural network. When the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss, and the bounding box shape loss in the image segmentation module is less than a loss threshold, the trained defect detection neural network is obtained.

[0063] During the training of the defect detection neural network, the matching degree between the predicted box and the true box is improved by controlling the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss, and the bounding box shape loss to be less than the loss threshold, thereby ensuring the accuracy of small target detection in scenes with limited computing resources, complex backgrounds, or irregular target shapes.

[0064] Among them, the intersection-over-union loss can be achieved by the following formula:

[0065] ,

[0066] is the intersection-over-union loss, 、 They are the intersection area of ​​the predicted box and the true box, and the union area of ​​the predicted box and the true box, respectively.

[0067] The bounding box coordinate loss is obtained by the following formula:

[0068] ,

[0069] is the bounding box coordinate loss, 、 are the width and height of the real frame respectively, s is the scale factor, 、 are the center point coordinates of the predicted box and the real box respectively, cis the diagonal distance of the intersection area between the predicted box and the true box.

[0070] The bounding box shape loss is obtained by the following formula:

[0071] ,

[0072] ,

[0073] ,

[0074] is the bounding box shape loss, is the first loss coefficient, is the first loss coefficient, 、 are the width and height of the prediction box respectively, 、 are the width and height of the real frame respectively.

[0075] S3. The transmission line diagram to be inspected is processed by the trained defect detection neural network to obtain a defect detection result.

[0076] To verify the effectiveness of the detection method of this embodiment, the following experiments were conducted.

[0077] Experimental Dataset. The dataset used in the experiment is based on images taken by China Electric Power. Data augmentation methods such as background fusion, random rotation, and noise addition are used to augment the dataset. The images include small target scenes such as dropped insulators, broken insulators, bird nests, and shock-absorbing hammers. The experimental dataset contains a total of 3,518 images. The training, test, and validation sets are divided in an 8:1:1 ratio.

[0078] Experimental Environment and Parameter Configuration. The experiment was conducted on a 64-bit Windows 11 operating system using Python 3.9, CUDA 11.6, and PyTorch 2.0. The computer used was a Gen Intel(R) Core(TM) i5-11400H processor with an NVIDIA GeForce RTX3050 GPU and 4GB of RAM. The relevant parameters were set as follows: 300 training epochs, which achieves convergence; a batch size of 16, which allows the device's GPU to operate at full power; an input image size of 640×640, with images scaled and uniformly sized to improve speed; an SGD optimizer with gradient descent for learning rate adjustment; an initial learning rate of 0.01 and a minimum learning rate of 0.0001, which was gradually reduced as training progressed.

[0079] Evaluation Metrics. The experiment uses Precision (P), Recall (R), Mean Average Precision (mAP), Model Parameters (Params), and Total Floating-Point Operations (FLOPs) as evaluation metrics. Params refers to the number of parameters in the model, and FLOPs measures the complexity of the model. Precision is used to assess the accuracy of predictions, while Recall assesses the ability to find the correct sample. They are defined as follows:

[0080] ,

[0081] ,

[0082] TP means both the prediction and the actual are true; FP means the prediction is true and the actual is false; and FN means the prediction is false and the actual is true. mAP measures the performance of the model on each category. mAP is averaged across all categories to measure the performance of the model. mAP50 represents the average precision of all categories when the intersection-over-union ratio is 0.5.

[0083] Experimental results. In the experiment, the method of this embodiment and YOLOv8 processed the experimental data set respectively. It was found that the floating point operations of the method of this embodiment were only 6.5 GFLOPS, which was about 19.7% less than that of YOLOv8. In addition, the processing results of the experimental data set of this embodiment can also be seen in Figure 1 The precision-recall curve and Figure 2 The confusion matrix of .

[0084] pass Figure 1 It can be found that the detection method of this embodiment has detection accuracies of 0.994, 0.950, 0.951, and 0.993 for insulator damage, insulator string drop, shock-absorbing hammer, and bird's nest, respectively, achieving high-accuracy detection.

[0085] Figure 2 The confusion matrix in [ 0 ] describes the prediction accuracy of the detection method of this embodiment for insulator breakage, insulator string drop, shock-absorbing hammer, and bird's nest in the experimental data set, and illustrates the relationship between the predictions. Figure 2 The rows in represent the true labels, the columns represent the predicted categories, and the diagonal elements represent the correct detection rate; background positive sample false detection means that the negative sample (non-background) is classified as a positive sample, which is wrong; background negative sample false detection means that the positive sample (background) is classified as a negative sample, which is wrong; Figure 2 It can be seen that the method of this embodiment achieves high-precision detection in the detection of insulator damage, insulator string drop, shock-absorbing hammer, and bird's nest.

[0086] This embodiment further provides a transmission line defect detection system, which is used to implement the above-mentioned transmission line defect detection method, including:

[0087] The defect detection neural network generation module is used to construct a defect detection neural network based on the multi-level image feature extraction module, the multi-level image feature fusion analysis module, the first image semantic feature enhancement module, the second image semantic feature enhancement module, the image semantic feature fusion analysis module and the image segmentation module; the processing process in the multi-level image feature fusion analysis module is specifically as follows: the deep image features are up-sampled and then spliced ​​with the middle image features to obtain the middle-deep fusion features; the middle-deep fusion features are subjected to multi-scale feature aggregation to obtain the middle-deep multi-scale fusion features; the middle-deep multi-scale fusion features are up-sampled and then spliced ​​with the shallow image features to obtain the multi-level fusion features; the multi-level fusion features are subjected to multi-scale attention feature aggregation to obtain the first image semantic features; the shallow image features, the middle image features and the deep image features are all The output of the multi-level image feature extraction module; the operation of the multi-scale attention feature aggregation processing is specifically as follows: the multi-level fusion feature is subjected to feature convolution and character segmentation processing to obtain a fused convolution feature; the fused convolution feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain a first fused enhanced feature; the first fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain a second fused enhanced feature; the second fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain a third fused enhanced feature; and so on, performing several times of partial convolution-based attention mechanism processing and splicing processing to obtain the Nth fused enhanced feature; the Nth fused enhanced feature and the fused convolution feature are spliced ​​and convolved to obtain the first image semantic feature;

[0088] A training defect detection neural network generation module is used to train the defect detection neural network using a training data set composed of defect images of several key parts of the transmission line. When the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss, and the bounding box shape loss in the image segmentation module is less than a loss threshold, the trained defect detection neural network is obtained;

[0089] The defect detection result generation module is used to process the transmission line diagram to be inspected through the trained defect detection neural network to obtain the defect detection result.

[0090] This embodiment further provides a power transmission line defect detection device, comprising a processor and a memory, wherein the processor implements the above-mentioned power transmission line defect detection method when executing a computer program stored in the memory.

[0091] The embodiment also provides a computer readable storage medium for storing a computer program, wherein the computer program is executed by a processor to implement the defect detection method of the power transmission line.

[0092] The defect detection neural network constructed by the multi-level image feature extraction module, the multi-level image feature fusion analysis module, the first image semantic feature enhancement module, the second image semantic feature enhancement module, the image semantic feature fusion analysis module and the image segmentation module can not only significantly reduce the computational burden and be suitable for real-time application of the unmanned aerial vehicle, but also most importantly separate small targets from the background and improve the detection accuracy of the small targets in a complex background and a shielding scene.

[0093] The multi-scale attention feature aggregation processing used in the multi-level image feature fusion analysis module of the defect detection neural network of the embodiment can enhance the global perception ability of the small target, effectively suppress the interference of the complex background, make the defect detection neural network more accurate in the recognition of small targets such as insulator defects and bird nests, and reduce the occurrence of false detection and missed detection.

[0094] In the process of training the defect detection neural network, the sum of the intersection over union loss of the image bounding box, the bounding box coordinate loss and the bounding box shape loss is controlled to be less than the loss threshold, so that the matching degree of the prediction box and the real box is improved, and the accuracy of small target detection in the case that the background is complex or the target shape is irregular in the limited computing resource scene is ensured.

[0095] The defect detection method of the power transmission line provided by the embodiment belongs to light detection technology, is suitable for real-time inspection of the unmanned aerial vehicle, can realize timely discovery of potential safety hazards such as insulator damage, shock absorber falling, tower bird nest and the like, and through the high-precision and real-time detection result, power maintenance personnel can quickly respond and prevent faults from occurring, so that the safety and stability of the power transmission line are greatly improved.

Claims

1. A method for detecting defects in a transmission line, characterized in that: The following operations are included: S1. Construct a defect detection neural network based on a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module, and an image segmentation module; The processing process in the multi-level image feature fusion analysis module is as follows: the deep image features are upsampled and then spliced ​​with the middle image features to obtain the middle-deep fusion features; the middle-deep fusion features are subjected to multi-scale feature aggregation to obtain the middle-deep multi-scale fusion features; The mid- and deep-layer multi-scale fusion features are upsampled and then concatenated with the shallow-layer image features to obtain multi-level fusion features. The multi-level fusion features are then aggregated with the multi-scale attention features to obtain the first image semantic features. The shallow-layer image features, mid-layer image features, and deep-layer image features are all outputs of the multi-level image feature extraction module. The specific operation of multi-scale attention feature aggregation processing is as follows: multi-level fusion features are processed by feature convolution and character segmentation to obtain fused convolution features; the fused convolution features are processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution features to obtain the first fused enhanced features; The first fused enhanced feature is processed by the partial convolution-based attention mechanism and then concatenated with the fused convolution feature to obtain the second fused enhanced feature; The second fused enhanced feature is processed by the partial convolution-based attention mechanism and then concatenated with the fused convolution feature to obtain the third fused enhanced feature. Similarly, several partial convolution-based attention mechanism processing and splicing processing are performed to obtain the Nth fused enhanced feature; the Nth fused enhanced feature and the fused convolution feature are spliced ​​and convolved to obtain the first image semantic feature; S2. Using a training data set consisting of defect images of several key parts of the transmission line, a defect detection neural network is trained. When the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss, and the bounding box shape loss in the image segmentation module is less than a loss threshold, the trained defect detection neural network is obtained. The bounding box coordinate loss is obtained by the following formula: ， is the bounding box coordinate loss, 、 are the width and height of the real frame respectively, s is the scale factor, 、 are the center point coordinates of the predicted box and the real box respectively, c is the diagonal distance between the predicted box and the real box; The bounding box shape loss is obtained by the following formula: ， ， ， is the bounding box shape loss, is the first loss coefficient, is the first loss coefficient, 、 are the width and height of the prediction box respectively, 、 are the width and height of the real frame respectively, s is the scale factor; S3. The transmission line diagram to be inspected is processed by the trained defect detection neural network to obtain a defect detection result.

2. The method for detecting defects in a power transmission line according to claim 1, wherein: In S1, the processing process in the first image semantic feature enhancement module is specifically as follows: The first image semantic features output by the multi-level image feature fusion analysis module are processed by multi-scale attention feature aggregation or feature convolution, and then sequentially spliced ​​and multi-scale attention feature aggregation with the mid- and deep-level multi-scale fusion features to obtain the second image semantic features.

3. The method for detecting defects in a power transmission line according to claim 1, wherein: In S1, the processing process in the multi-level image feature extraction module is specifically as follows: The transmission line map is processed through feature convolution, two rounds of feature convolution, and multi-scale feature aggregation to obtain shallow image features. The shallow image features are processed through feature convolution and multi-scale feature aggregation to obtain middle-level image features. The middle-level image features are processed through feature convolution, multi-scale feature aggregation, and spatial pyramid pooling to obtain deep image features.

4. The method for detecting defects in a power transmission line according to claim 1, wherein: In S1, the operation of the attention mechanism processing based on partial convolution is specifically as follows: the input features are processed in sequence by partial convolution, several times of point-by-point convolution, and the attention mechanism, and then spliced ​​with the input features to obtain the output features.

5. The method for detecting defects in a power transmission line according to claim 1, wherein: In S1, the processing operations in the image semantic feature fusion analysis module are specifically as follows: The first image semantic feature, the second image semantic feature, and the third image semantic feature are respectively processed by multi-scale feature extraction and then spliced ​​to obtain image semantic fusion features; The multi-scale feature extraction processing operation is specifically as follows: the first image semantic feature or the second image semantic feature and the third image semantic feature are respectively subjected to feature convolution of different scales, multi-scale channel attention mechanism and convolution processing, and then spliced ​​to obtain the first image semantic enhancement feature or the second image semantic enhancement feature or the third image semantic enhancement feature for performing the splicing processing operation; The first image semantic feature, the second image semantic feature and the third image semantic feature are the outputs of the multi-level image feature fusion analysis module, the first image semantic feature enhancement module and the second image semantic feature enhancement module respectively.

6. A transmission line defect detection system, characterized in that: The method for detecting defects in a power transmission line according to claim 1 comprises: The defect detection neural network generation module is used to construct a defect detection neural network based on the multi-level image feature extraction module, the multi-level image feature fusion analysis module, the first image semantic feature enhancement module, the second image semantic feature enhancement module, the image semantic feature fusion analysis module and the image segmentation module; the processing process in the multi-level image feature fusion analysis module is specifically as follows: the deep image features are up-sampled and then spliced ​​with the middle image features to obtain the middle-deep fusion features; the middle-deep fusion features are subjected to multi-scale feature aggregation to obtain the middle-deep multi-scale fusion features; the middle-deep multi-scale fusion features are up-sampled and then spliced ​​with the shallow image features to obtain the multi-level fusion features; the multi-level fusion features are subjected to multi-scale attention feature aggregation to obtain the first image semantic features; the shallow image features, the middle image features and the deep image features are all The output of the multi-level image feature extraction module; the operation of the multi-scale attention feature aggregation processing is specifically as follows: the multi-level fusion feature is subjected to feature convolution and character segmentation processing to obtain a fused convolution feature; the fused convolution feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain a first fused enhanced feature; the first fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain a second fused enhanced feature; the second fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain a third fused enhanced feature; and so on, performing several times of partial convolution-based attention mechanism processing and splicing processing to obtain the Nth fused enhanced feature; the Nth fused enhanced feature and the fused convolution feature are spliced ​​and convolved to obtain the first image semantic feature; A training defect detection neural network generation module is used to train the defect detection neural network using a training data set composed of defect images of several key parts of the transmission line. When the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss, and the bounding box shape loss in the image segmentation module is less than a loss threshold, the trained defect detection neural network is obtained; The defect detection result generation module is used to process the transmission line diagram to be inspected through the trained defect detection neural network to obtain the defect detection result.

7. A transmission line defect detection device, characterized in that: The method comprises a processor and a memory, wherein when the processor executes the computer program stored in the memory, the method for detecting defects in a power transmission line according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that Used to store a computer program, wherein when the computer program is executed by a processor, the method for detecting defects in a power transmission line according to any one of claims 1 to 5 is implemented.

Images