Defect detection method, system and equipment for power transmission line and storage medium

Abstract

The invention relates to the technical field of image analysis, in particular to a defect detection method, system and device for a power transmission line and a storage medium. A defect detection neural network composed of a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module and an image segmentation module is constructed, so that the calculation burden can be remarkably reduced; the method is suitable for real-time application of the unmanned aerial vehicle, most importantly, the small target can be separated from the background, and the detection precision of the small target in a complex background and a shielding scene is improved; meanwhile, in the process of training the defect detection neural network, the matching degree of a prediction frame and a real frame is improved by controlling the sum of intersection-to-union ratio loss of image bounding boxes, coordinate loss of the bounding boxes and shape loss of the bounding boxes to be smaller than a loss threshold value, and therefore the accuracy of small target detection under the condition that the background is complex or the target shape is irregular is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and in particular to a method, system, device and storage medium for defect detection of a power transmission line. Background Art

[0002] The core components of transmission lines include insulators, shock-absorbing hammers, and transmission towers. These components are exposed to harsh natural environments for a long time and are easily affected by factors such as temperature and humidity changes, lightning, strong electric fields, and dirt, which can lead to defects such as breakage, self-explosion, and pollution. These defects not only affect the stability of transmission lines, but may even cause large-scale power outages in serious cases. Therefore, regular inspections and status monitoring of transmission line equipment are essential. With the development of drone technology, it has gradually been applied to power system inspections. It can obtain a large amount of transmission line image data in a short time, and has the advantages of strong mobility and high efficiency. However, the amount of data in drone inspection images is huge, and how to accurately extract effective information from them has become a major problem.

[0003] In power system inspection, the commonly used target detection methods are mainly deep learning models, especially single-stage target detection algorithms such as the YOLO series and SSD. These algorithms have the advantages of fast detection speed and high accuracy, and are suitable for real-time detection tasks. However, these models have high computational complexity and consume a lot of computing resources, making them difficult to be directly applied to resource-limited devices such as drones. In order to meet the real-time and lightweight requirements of drone inspections, many researchers have improved algorithms such as YOLO and SSD, but there are still some problems in the unique scenes of power inspections. On the one hand, transmission line images have the characteristics of large scenes, multiple scales, small targets, complex backgrounds, and target occlusion. The performance of traditional detection models in such images is still unsatisfactory, and it is easy to miss or misdetect; on the other hand, the target detection model based on deep learning has a large number of parameters and usually requires a lot of computing resources, which is particularly limited under the limited computing power and battery life of drones. Summary of the invention

[0004] The object of the present invention is to provide a method, system, device and storage medium for defect detection of a power transmission line.

[0005] The technical solution of the present invention is as follows: A method for detecting defects in a power transmission line comprises the following operations: S1. Constructing a defect detection neural network based on a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module and an image segmentation module; The processing process in the multi-level image feature fusion analysis module is as follows: the deep image features are spliced ​​with the middle image features after upsampling to obtain the middle-deep fusion features; the middle-deep fusion features are aggregated by multi-scale features to obtain the middle-deep multi-scale fusion features; the middle-deep multi-scale fusion features are spliced ​​with the shallow image features after upsampling to obtain the multi-level fusion features; the multi-level fusion features are aggregated by multi-scale attention features to obtain the first image semantic features; the shallow image features, the middle image features and the deep image features are all outputs of the multi-level image feature extraction module; The operation of multi-scale attention feature aggregation processing is specifically as follows: multi-level fusion features are processed by feature convolution and character segmentation to obtain fused convolution features; the fused convolution features are processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution features to obtain the first fused enhanced feature; the first fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution features to obtain the second fused enhanced feature; the second fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution features to obtain the third fused enhanced feature; and so on, several times of attention mechanism processing and splicing processing based on partial convolution are performed to obtain the Nth fused enhanced feature; the Nth fused enhanced feature and the fused convolution feature are spliced ​​and convoluted to obtain the first image semantic feature; S2. Using a training data set composed of defect images of key parts of a plurality of transmission lines, a defect detection neural network is trained. When the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss, and the bounding box shape loss in the image segmentation module is less than a loss threshold, a trained defect detection neural network is obtained; S3. The transmission line diagram to be inspected is processed by the trained defect detection neural network to obtain defect detection results.

[0006] The processing process in the first image semantic feature enhancement module in S1 is specifically as follows: the first image semantic feature output by the multi-level image feature fusion analysis module is processed by multi-scale attention feature aggregation or feature convolution, and then processed by splicing and multi-scale attention feature aggregation with the mid- and deep-level multi-scale fusion features in sequence to obtain the second image semantic feature.

[0007] The specific processing process in the multi-level image feature extraction module in S1 is as follows: the transmission line diagram is subjected to feature convolution and two times of feature convolution and multi-scale feature aggregation to obtain shallow image features; the shallow image features are subjected to feature convolution and multi-scale feature aggregation to obtain middle-level image features; the middle-level image features are subjected to feature convolution, multi-scale feature aggregation and spatial pyramid pooling to obtain deep image features.

[0008] The specific operation of the partial convolution-based attention mechanism processing in S1 is as follows: the input features are processed in sequence by partial convolution, several point-by-point convolutions, and the attention mechanism, and then concatenated with the input features to obtain the output features.

[0009] The bounding box coordinate loss in S2 is obtained by the following formula: , is the bounding box coordinate loss, , are the width and height of the real frame respectively, s is the scale factor, , are the center point coordinates of the predicted box and the real box respectively, c It is the diagonal distance of the intersection area between the predicted box and the true box.

[0010] The bounding box shape loss in S2 is obtained by the following formula: , , , is the bounding box shape loss, is the first loss coefficient, is the first loss coefficient, , are the prediction box width and height respectively, , are the width and height of the real frame respectively, s is the scale factor.

[0011] The processing operation in the image semantic feature fusion analysis module in S1 is specifically as follows: the first image semantic feature, the second image semantic feature and the third image semantic feature are respectively processed by multi-scale feature extraction and then spliced ​​to obtain image semantic fusion features; the multi-scale feature extraction processing operation is specifically as follows: the first image semantic feature or the second image semantic feature and the third image semantic feature are respectively processed by feature convolution of different scales, multi-scale channel attention mechanism and convolution and then spliced ​​to obtain the first image semantic enhancement feature or the second image semantic enhancement feature or the third image semantic enhancement feature for performing the splicing processing operation; the first image semantic feature, the second image semantic feature and the third image semantic feature are respectively the outputs of the multi-level image feature fusion analysis module, the first image semantic feature enhancement module and the second image semantic feature enhancement module.

[0012] A transmission line defect detection system, used to implement the above-mentioned transmission line defect detection method, comprising: The defect detection neural network generation module is used to construct a defect detection neural network based on a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module and an image segmentation module; the processing process in the multi-level image feature fusion analysis module is specifically as follows: the deep image features are up-sampled and then spliced ​​with the middle image features to obtain the middle-deep fusion features; the middle-deep fusion features are aggregated by multi-scale features to obtain the middle-deep multi-scale fusion features; the middle-deep multi-scale fusion features are up-sampled and then spliced ​​with the shallow image features to obtain the multi-level fusion features; the multi-level fusion features are aggregated by multi-scale attention features to obtain the first image semantic features; the shallow image features, the middle image features and the deep image features are all The output of the multi-level image feature extraction module; the operation of the multi-scale attention feature aggregation processing is specifically as follows: the multi-level fusion feature is processed by feature convolution and character segmentation to obtain the fused convolution feature; the fused convolution feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain the first fused enhanced feature; the first fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain the second fused enhanced feature; the second fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain the third fused enhanced feature; and so on, perform several times of attention mechanism processing and splicing processing based on partial convolution to obtain the Nth fused enhanced feature; the Nth fused enhanced feature and the fused convolution feature are spliced ​​and convoluted to obtain the first image semantic feature; A training defect detection neural network generation module is used to train the defect detection neural network using a training data set composed of defect images of key parts of a plurality of transmission lines. When the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss and the bounding box shape loss in the image segmentation module is less than a loss threshold, a training defect detection neural network is obtained; The defect detection result generation module is used to process the transmission line diagram to be inspected through the trained defect detection neural network to obtain the defect detection result.

[0013] A power transmission line defect detection device comprises a processor and a memory, wherein the processor implements the above-mentioned power transmission line defect detection method when executing a computer program stored in the memory.

[0014] A computer-readable storage medium is used to store a computer program, wherein the computer program implements the above-mentioned power transmission line defect detection method when executed by a processor.

[0015] The beneficial effects of the present invention are: (1) Constructing a defect detection neural network consisting of a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module, and an image segmentation module, which can not only significantly reduce the computational burden and be suitable for real-time application of UAVs, but also, most importantly, can separate small targets from the background and improve the detection accuracy of small targets in complex backgrounds and occlusion scenes; (2) The multi-level image feature fusion analysis module of the defect detection neural network uses multi-scale attention feature aggregation processing, which can enhance the global perception ability of small targets and effectively suppress the interference of complex backgrounds. This makes the defect detection neural network more accurate in identifying small targets such as broken insulators and bird nests, and reduces the occurrence of false detections and missed detections. (3) In the process of training the defect detection neural network, the matching degree between the predicted box and the true box is improved by controlling the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss, and the bounding box shape loss to be less than the loss threshold, thereby ensuring the accuracy of small target detection in scenes with limited computing resources and complex backgrounds or irregular target shapes. (4) The present invention provides a method for detecting defects in power transmission lines, which is a lightweight detection technology suitable for real-time inspection by drones. It can realize the timely discovery of potential safety hazards such as insulator damage, fall-off of anti-vibration hammers, and bird nests on towers. At the same time, through high-precision and real-time detection results, power maintenance personnel can respond quickly and prevent faults from occurring, thereby greatly improving the safety and stability of power transmission lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] By reading the detailed description of the preferred embodiment below, the scheme and advantages of the present application will become clear to those skilled in the art. The accompanying drawings are only for the purpose of illustrating the preferred embodiment and are not to be considered as limiting the present invention.

[0017] In the attached picture: Figure 1 This is a graph showing the accuracy-recall rate obtained by processing the experimental data set using the method of this embodiment in the embodiment; Figure 2 : This is a confusion matrix diagram obtained by processing the experimental data set using the method of this embodiment. DETAILED DESCRIPTION

[0018] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings.

[0019] This embodiment provides a method for detecting defects in a power transmission line, including the following operations: S1. Constructing a defect detection neural network based on a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module and an image segmentation module; The processing process in the multi-level image feature fusion analysis module is as follows: the deep image features are spliced ​​with the middle image features after upsampling to obtain the middle-deep fusion features; the middle-deep fusion features are aggregated by multi-scale features to obtain the middle-deep multi-scale fusion features; the middle-deep multi-scale fusion features are spliced ​​with the shallow image features after upsampling to obtain the multi-level fusion features; the multi-level fusion features are aggregated by multi-scale attention features to obtain the first image semantic features; the shallow image features, the middle image features and the deep image features are all outputs of the multi-level image feature extraction module; The operation of multi-scale attention feature aggregation processing is specifically as follows: multi-level fusion features are processed by feature convolution and character segmentation to obtain fused convolution features; the fused convolution features are processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution features to obtain the first fused enhanced feature; the first fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution features to obtain the second fused enhanced feature; the second fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution features to obtain the third fused enhanced feature; and so on, several times of attention mechanism processing and splicing processing based on partial convolution are performed to obtain the Nth fused enhanced feature; the Nth fused enhanced feature and the fused convolution feature are spliced ​​and convoluted to obtain the first image semantic feature; S2. Using a training data set composed of defect images of key parts of a plurality of transmission lines, a defect detection neural network is trained. When the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss, and the bounding box shape loss in the image segmentation module is less than a loss threshold, a trained defect detection neural network is obtained; S3. The transmission line diagram to be inspected is processed by the trained defect detection neural network to obtain defect detection results.

[0020] S1. Construct a defect detection neural network based on a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module and an image segmentation module.

[0021] Constructing a defect detection neural network consisting of a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module and an image segmentation module can not only significantly reduce the computational burden and be suitable for real-time applications of drones; most importantly, it can separate small targets from the background and improve the detection accuracy of small targets in complex backgrounds and occluded scenes.

[0022] The multi-level image feature extraction module is used to extract different levels of features of the transmission line map, obtaining shallow image features with prominent edges and textures, middle-level image features with prominent object shapes and component structures, and deep image features with prominent semantic features and scene features.

[0023] The processing operations in the multi-level image feature extraction module are as follows: the transmission line diagram is subjected to feature convolution and two feature convolutions and multi-scale feature aggregation to obtain shallow image features; shallow image features are subjected to feature convolution and multi-scale feature aggregation to obtain middle image features; middle image features are subjected to feature convolution, multi-scale feature aggregation and spatial pyramid pooling to obtain deep image features. The above feature convolution can be achieved through convolution, batch normalization and nonlinear processing (SiLU activation function). Multi-scale feature aggregation processing includes but is not limited to being achieved through the C2f module, and spatial pyramid pooling processing includes but is not limited to being achieved through the SPPF module.

[0024] The multi-level image feature fusion analysis module is used to fuse the shallow image features, middle image features and deep image features output by the multi-level image feature extraction module, so that the acquired image contains both overall semantic information and local details, thereby improving the detection and recognition capabilities of subsequent small targets.

[0025] The processing process in the multi-level image feature fusion analysis module is as follows: the deep image features are upsampled and then spliced ​​with the middle image features to highlight the component structure features and obtain the middle-deep fusion features; the middle-deep fusion features are aggregated by multi-scale features to enhance the image semantic feature expression capability and obtain the middle-deep multi-scale fusion features; the middle-deep multi-scale fusion features are upsampled and then spliced ​​with the shallow image features to enhance the local detail features on the basis of highlighting the component structure features and obtain the multi-level fusion features; the multi-level fusion features are aggregated by multi-scale attention features to separate the target objects, especially small targets, from the background and obtain the first image semantic features.

[0026] The above-mentioned multi-scale attention feature aggregation processing operation is specifically as follows: the multi-level fusion features are subjected to feature convolution and character segmentation processing (which can be achieved through the Split function) to obtain fused convolution features; the fused convolution features are processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution features to perform high-quality feature extraction on the image while reducing computational redundancy and memory access to obtain the first fused enhanced feature; the first fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain the second fused enhanced feature; the second fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain the third fused enhanced feature; and so on, several times of partial convolution-based attention mechanism processing and splicing processing are performed to obtain the Nth fused enhanced feature; the Nth fused enhanced feature and the fused convolution feature are spliced ​​and convolved to obtain the first image semantic feature.

[0027] Among them, the operation of the attention mechanism processing based on partial convolution can be: the input features are sequentially subjected to partial convolution (which can be realized by PConv convolution layer), several times of point-by-point convolution (which can be realized by PWconv convolution layer), and attention mechanism processing, and then spliced ​​with the input features to obtain the output features. It can also be: the input features are sequentially subjected to partial convolution (which can be realized by PConv convolution layer), several times of convolution, and attention mechanism processing, and then spliced ​​with the input features to obtain the output features.

[0028] The operation of the above attention mechanism is as follows: the input is grouped and processed, the input X is divided into G groups, the number of channels in each group is C / G, and the groups are independent of each other, so as to better focus on the feature information of each group and obtain several groups of features; each group feature is pooled horizontally and vertically respectively, and a horizontal tensor of size C / G×W×1 and a vertical tensor of size C / G×H×1 are obtained; all horizontal tensors and all vertical tensors are concatenated, convolved and processed with Sigmoid function respectively, local spatial information and spatial features that can capture a small range are extracted, and then weight generation is performed (which can be achieved through Re-weight function) to obtain a spatial weight feature map; the spatial weight feature map is element-by-element multiplied with the input, and significant feature information is input, while unimportant features are suppressed to obtain a spatial feature map; the spatial feature map is divided The group is normalized to obtain a spatial normalized feature map; the spatial normalized feature map is processed by average pooling and Softmax function to obtain spatial nonlinear features; at the same time, all group features are processed by convolution with a convolution kernel of 3 and then spliced ​​in the channel dimension to obtain local features; the local features are processed by average pooling and Softmax function to obtain local nonlinear features; the spatial nonlinear features and local features are fused to obtain the first spatial fused features; the local nonlinear features and the spatial normalized feature map are fused to obtain the second spatial fused features; the first spatial fused features and the second spatial fused features are added element by element, processed by Sigmoid function and weights are generated to obtain a spatial weight optimization map; the spatial weight optimization map is multiplied element by element with the input to obtain an output containing cross-channel and cross-space information, which is used to perform the above-mentioned splicing operation.

[0029] The first image semantic feature enhancement module is used to enhance the first image semantic features output by the multi-level image feature fusion analysis module to improve the semantic features of the power transmission line diagram. The processing process in the first image semantic feature enhancement module is as follows: the first image semantic features output by the multi-level image feature fusion analysis module are subjected to multi-scale attention feature aggregation processing or feature convolution processing, and then sequentially subjected to splicing processing and multi-scale attention feature aggregation processing with the mid- and deep-level multi-scale fusion features to highlight the structural features of the components in the power transmission line diagram, so as to facilitate subsequent accurate component detection and identification, and obtain the second image semantic features.

[0030] The second image semantic feature enhancement module is used to enhance the second image semantic features output by the first image semantic feature enhancement module, so as to further improve the semantic features of the power transmission line diagram. The processing process in the second image semantic feature enhancement module is as follows: the second image semantic features output by the first image semantic feature enhancement module are subjected to multi-scale attention feature aggregation processing or feature convolution processing, and then the deep image features in the output of the multi-level image feature extraction module are subjected to splicing processing and multi-scale attention feature aggregation processing in sequence, so as to highlight the semantic features of the components in the power transmission line diagram, facilitate subsequent accurate component detection and identification, and obtain the third image semantic features.

[0031] The image semantic feature fusion analysis module is used to fuse the first image semantic features, the second image semantic features and the third image semantic features respectively output by the multi-level image feature fusion analysis module, the first image semantic feature enhancement module and the second image semantic feature enhancement module, so as to improve the richness of the features and obtain image semantic fusion features. The processing operation in the image semantic feature fusion analysis module is as follows: the first image semantic features, the second image semantic features and the third image semantic features are respectively processed by multi-scale feature extraction and then spliced ​​to obtain image semantic fusion features.

[0032] The specific operation of multi-scale feature extraction processing is as follows: the first image semantic feature or the second image semantic feature and the third image semantic feature are respectively subjected to feature convolution of different scales, multi-scale channel attention mechanism and convolution processing, and then spliced ​​to obtain the first image semantic enhancement feature or the second image semantic enhancement feature or the third image semantic enhancement feature, which is used to perform the splicing processing operation.

[0033] The specific operations of the multi-scale channel attention mechanism are as follows: after the input is processed by the channel attention mechanisms of different scales and the key channel information is captured, it is added element by element with the input to obtain the first comprehensive feature; after the first comprehensive feature is processed by average pooling, full connection and channel expansion, it is multiplied element by element with the input to obtain the output.

[0034] The specific operations of the channel attention mechanism are as follows: the input is processed by one-dimensional embedding (which can be achieved through PatchEmbedding), GELU activation function, batch normalization and depth-wise separable convolution to obtain depth-wise separable features; the depth-wise separable features are processed by GELU activation function and batch normalization, and then added element-by-element with the depth-wise separable features to obtain depth-wise normalized fused features; the depth-wise normalized fused features are processed by GELU activation function and batch normalization to obtain output.

[0035] The image segmentation module is used to segment the image semantic fusion features output by the image semantic feature fusion analysis module to obtain the detection results. The segmentation process can be implemented by bounding box regression and classification recognition processing. For example, the segmentation process can be implemented by the RCNN target detection method.

[0036] S2. Using a training data set consisting of defect images of several key parts of the transmission line, a defect detection neural network is trained. When the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss, and the bounding box shape loss in the image segmentation module is less than a loss threshold, a trained defect detection neural network is obtained.

[0037] In the process of training the defect detection neural network, the matching degree between the predicted box and the true box is improved by controlling the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss, and the bounding box shape loss to be less than the loss threshold, thereby ensuring the accuracy of small target detection in scenes with limited computing resources and complex backgrounds or irregular target shapes.

[0038] Among them, the intersection-over-intersection loss can be achieved by the following formula: , is the intersection-and-join loss, , They are the intersection area of ​​the predicted box and the true box, and the union area of ​​the predicted box and the true box, respectively.

[0039] The bounding box coordinate loss is obtained by the following formula: , is the bounding box coordinate loss, , are the width and height of the real frame respectively, s is the scale factor, , are the center point coordinates of the predicted box and the real box respectively, c It is the diagonal distance of the intersection area between the predicted box and the true box.

[0040] The bounding box shape loss is obtained by the following formula: , , , is the bounding box shape loss, is the first loss coefficient, is the first loss coefficient, , are the width and height of the prediction box respectively, , are the real frame width and height respectively.

[0041] S3. The transmission line diagram to be inspected is processed by the trained defect detection neural network to obtain defect detection results.

[0042] In order to verify the effect of the detection method of this embodiment, the following experiment was conducted.

[0043] Experimental data set. The data set used in the experiment is a picture taken by China Electric Power. At the same time, the data set is expanded by using data enhancement methods such as background fusion, random rotation, and adding noise. The images contain small target scenes such as insulator string drop, insulator damage, bird nests, and shockproof hammers. The experimental data set contains a total of 3518 pictures. In the experiment, the training set, test set, and validation set are divided in a ratio of 8:1:1.

[0044] Experimental environment and parameter configuration. The experiment is based on a 64-bit Windows 11 operating system, with Python 3.9, CUDA 11.6, and Pytorch 2.0 as the experimental environment. The computer used in the experiment is Gen Intel (R) Core (TM) i5-11400H, and the GPU is NVIDIA GeForce RTX3050, with 4GB of running memory. The relevant parameters are set as follows: the number of training rounds is 300, at which point convergence can be achieved; the batch size is 16, and the device GPU can run at full power; the input image size is 640 × 640, and the image is scaled, the size is unified, and the speed is improved; the optimizer is SGD, and the gradient descent method is used to adjust the learning rate; the initial learning rate is 0.01, the minimum learning rate is 0.0001, and the learning rate is continuously reduced as the training progresses.

[0045] Evaluation indicators. The experiment uses accuracy (Precision, P), recall (Recall, R), mean average precision (mAP), model parameters (Params), and total floating point operations (FLOPs) as evaluation indicators. Params refers to the number of parameters in the model, and FLOPs is used to measure the complexity of the model. Accuracy is used to evaluate the accuracy of the prediction, and recall evaluates the ability to find the correct sample. The definition is shown in the following formula: , , TP means that both the prediction and the actual are true; FP means that the prediction is true and the actual is false; FN means that the prediction is false and the actual is true. mAP measures the performance of the model on each category. mAP takes the average of all categories to measure the performance of the model on all categories. mAP50 represents the average accuracy of all categories when the intersection-over-union ratio is 0.5.

[0046] Experimental results. In the experiment, the method of this embodiment and YOLOv8 processed the experimental data set respectively, and it was found that the floating point operations of the method of this embodiment were only 6.5 GFLOPS, which was about 19.7% less than that of YOLOv8. In addition, the processing results of the experimental data set of this embodiment can also be seen in Figure 1 The precision-recall curve of Figure 2 The confusion matrix of .

[0047] pass Figure 1 It can be found that the detection method of this embodiment has detection accuracies of 0.994, 0.950, 0.951, and 0.993 for insulator damage, insulator string loss, shock-absorbing hammer, and bird's nest, respectively, achieving high-accuracy detection.

[0048] Figure 2 The confusion matrix in describes the prediction accuracy of the detection method of this embodiment for insulator damage, insulator string drop, shock hammer, and bird nest in the experimental data set, and illustrates the relationship between the predictions. Figure 2 The rows in represent the true labels, the columns represent the predicted categories, and the diagonal elements represent the correct detection rate; background positive sample false detection means that the negative sample (non-background) is classified as a positive sample, which is wrong; background negative sample positive detection means that the positive sample (background) is classified as a negative sample, which is wrong; Figure 2 It can be seen that the method of this embodiment achieves high-precision detection in the detection of insulator damage, insulator string drop, shock-absorbing hammer, and bird nest.

[0049] This embodiment further provides a transmission line defect detection system, which is used to implement the above-mentioned transmission line defect detection method, including: The defect detection neural network generation module is used to construct a defect detection neural network based on a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module and an image segmentation module; the processing process in the multi-level image feature fusion analysis module is specifically as follows: the deep image features are up-sampled and then spliced ​​with the middle image features to obtain the middle-deep fusion features; the middle-deep fusion features are aggregated by multi-scale features to obtain the middle-deep multi-scale fusion features; the middle-deep multi-scale fusion features are up-sampled and then spliced ​​with the shallow image features to obtain the multi-level fusion features; the multi-level fusion features are aggregated by multi-scale attention features to obtain the first image semantic features; the shallow image features, the middle image features and the deep image features are all The output of the multi-level image feature extraction module; the operation of the multi-scale attention feature aggregation processing is specifically as follows: the multi-level fusion feature is processed by feature convolution and character segmentation to obtain the fused convolution feature; the fused convolution feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain the first fused enhanced feature; the first fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain the second fused enhanced feature; the second fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain the third fused enhanced feature; and so on, perform several times of attention mechanism processing and splicing processing based on partial convolution to obtain the Nth fused enhanced feature; the Nth fused enhanced feature and the fused convolution feature are spliced ​​and convoluted to obtain the first image semantic feature; A training defect detection neural network generation module is used to train the defect detection neural network using a training data set composed of defect images of key parts of a plurality of transmission lines. When the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss and the bounding box shape loss in the image segmentation module is less than a loss threshold, a training defect detection neural network is obtained; The defect detection result generation module is used to process the transmission line diagram to be inspected through the trained defect detection neural network to obtain the defect detection result.

[0050] This embodiment further provides a power transmission line defect detection device, including a processor and a memory, wherein the processor implements the above-mentioned power transmission line defect detection method when executing a computer program stored in the memory.

[0051] This embodiment further provides a computer-readable storage medium for storing a computer program, wherein the computer program implements the above-mentioned power transmission line defect detection method when executed by a processor.

[0052] This embodiment constructs a defect detection neural network consisting of a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module and an image segmentation module. It can not only significantly reduce the computational burden and is suitable for real-time applications of drones, but most importantly, it can separate small targets from the background and improve the detection accuracy of small targets in complex backgrounds and occluded scenes.

[0053] The multi-level image feature fusion analysis module of the defect detection neural network of this embodiment uses multi-scale attention feature aggregation processing, which can enhance the global perception ability of small targets and effectively suppress the interference of complex backgrounds, making the defect detection neural network more accurate in identifying small targets such as insulator defects and bird nests, and reducing the occurrence of false detections and missed detections.

[0054] In the process of training the defect detection neural network, this embodiment improves the matching degree between the predicted box and the real box by controlling the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss, and the bounding box shape loss to be less than the loss threshold, thereby ensuring the accuracy of small target detection in scenes with limited computing resources and complex backgrounds or irregular target shapes.

[0055] A defect detection method for a transmission line provided in this embodiment is a lightweight detection technology, which is suitable for real-time inspection by drones. It can realize the timely discovery of potential safety hazards such as insulator damage, detachment of shock-absorbing hammers, and bird nests on towers. At the same time, through high-precision and real-time detection results, power maintenance personnel can respond quickly and prevent faults from occurring, thereby greatly improving the safety and stability of transmission lines.

Claims

1. A method for detecting defects in a power transmission line, characterized in that: The following operations are included: S1. Constructing a defect detection neural network based on a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module and an image segmentation module; The processing process in the multi-level image feature fusion analysis module is as follows: the deep image features are upsampled and then concatenated with the middle image features to obtain the middle-deep fusion features; the middle-deep fusion features are aggregated by multi-scale features to obtain the middle-deep multi-scale fusion features; The mid- and deep-layer multi-scale fusion features are upsampled and concatenated with the shallow-layer image features to obtain multi-level fusion features. The multi-level fusion features are aggregated with the multi-scale attention features to obtain the first image semantic features. The shallow-layer image features, the mid-layer image features, and the deep-layer image features are all outputs of the multi-level image feature extraction module. The specific operation of multi-scale attention feature aggregation processing is as follows: multi-level fusion features are processed by feature convolution and character segmentation to obtain fused convolution features; the fused convolution features are processed by the attention mechanism based on partial convolution and then concatenated with the fused convolution features to obtain the first fused enhanced features; The first fused enhanced feature is processed by the attention mechanism based on partial convolution and then concatenated with the fused convolution feature to obtain the second fused enhanced feature; The second fused enhanced feature is processed by the attention mechanism based on partial convolution and then concatenated with the fused convolution feature to obtain the third fused enhanced feature; By analogy, the partial convolution-based attention mechanism processing and splicing processing are performed several times to obtain the Nth fused enhanced feature; the Nth fused enhanced feature and the fused convolution feature are spliced ​​and convolved to obtain the first image semantic feature; S2. Using a training data set composed of defect images of key parts of a plurality of transmission lines, a defect detection neural network is trained. When the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss, and the bounding box shape loss in the image segmentation module is less than a loss threshold, a trained defect detection neural network is obtained; S3. The transmission line diagram to be inspected is processed by the trained defect detection neural network to obtain defect detection results.

2. The method for detecting defects in a power transmission line according to claim 1, characterized in that: In S1, the processing process in the first image semantic feature enhancement module is specifically as follows: The first image semantic features output by the multi-level image feature fusion analysis module are processed by multi-scale attention feature aggregation or feature convolution, and then sequentially processed by splicing and multi-scale attention feature aggregation with the mid- and deep-level multi-scale fusion features to obtain the second image semantic features.

3. The method for detecting defects in a power transmission line according to claim 1, characterized in that: In S1, the processing process in the multi-level image feature extraction module is specifically as follows: The transmission line map is processed by feature convolution, second feature convolution and multi-scale feature aggregation to obtain shallow image features; the shallow image features are processed by feature convolution and multi-scale feature aggregation to obtain middle-level image features; the middle-level image features are processed by feature convolution, multi-scale feature aggregation and spatial pyramid pooling to obtain deep image features.

4. The method for detecting defects in a power transmission line according to claim 1, characterized in that: In S1, the operation of the partial convolution-based attention mechanism processing is specifically as follows: the input features are processed in sequence by partial convolution, several times of point-by-point convolution, and the attention mechanism, and then concatenated with the input features to obtain the output features.

5. The method for detecting defects in a power transmission line according to claim 1, characterized in that: In S2, the bounding box coordinate loss is obtained by the following formula: ， is the bounding box coordinate loss, , are the width and height of the real frame respectively, s is the scale factor, , are the center point coordinates of the predicted box and the real box respectively, c It is the diagonal distance of the intersection area between the predicted box and the true box.

6. The method for detecting defects in a power transmission line according to claim 1, characterized in that: In S2, the bounding box shape loss is obtained by the following formula: ， ， ， is the bounding box shape loss, is the first loss coefficient, is the first loss coefficient, , are the prediction box width and height respectively, , are the width and height of the real frame respectively, s is the scale factor.

7. The method for detecting defects in a power transmission line according to claim 1, characterized in that: In S1, the processing operations in the image semantic feature fusion analysis module are specifically as follows: The first image semantic feature, the second image semantic feature and the third image semantic feature are respectively processed by multi-scale feature extraction, and then spliced ​​to obtain an image semantic fusion feature; The multi-scale feature extraction processing operation is specifically as follows: the first image semantic feature or the second image semantic feature and the third image semantic feature are respectively subjected to feature convolution of different scales, multi-scale channel attention mechanism and convolution processing, and then spliced ​​to obtain the first image semantic enhancement feature or the second image semantic enhancement feature or the third image semantic enhancement feature, which is used to perform the splicing processing operation; The first image semantic feature, the second image semantic feature and the third image semantic feature are outputs of the multi-level image feature fusion analysis module, the first image semantic feature enhancement module and the second image semantic feature enhancement module respectively.

8. A transmission line defect detection system, characterized in that: The method for detecting defects in a power transmission line according to claim 1 comprises: The defect detection neural network generation module is used to construct a defect detection neural network based on a multi-level image feature extraction module, a multi-level image feature fusion analysis module, a first image semantic feature enhancement module, a second image semantic feature enhancement module, an image semantic feature fusion analysis module and an image segmentation module; the processing process in the multi-level image feature fusion analysis module is specifically as follows: the deep image features are up-sampled and then spliced ​​with the middle image features to obtain the middle-deep fusion features; the middle-deep fusion features are aggregated by multi-scale features to obtain the middle-deep multi-scale fusion features; the middle-deep multi-scale fusion features are up-sampled and then spliced ​​with the shallow image features to obtain the multi-level fusion features; the multi-level fusion features are aggregated by multi-scale attention features to obtain the first image semantic features; the shallow image features, the middle image features and the deep image features are all The output of the multi-level image feature extraction module; the operation of the multi-scale attention feature aggregation processing is specifically as follows: the multi-level fusion feature is processed by feature convolution and character segmentation to obtain the fused convolution feature; the fused convolution feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain the first fused enhanced feature; the first fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain the second fused enhanced feature; the second fused enhanced feature is processed by the attention mechanism based on partial convolution and then spliced ​​with the fused convolution feature to obtain the third fused enhanced feature; and so on, perform several times of attention mechanism processing and splicing processing based on partial convolution to obtain the Nth fused enhanced feature; the Nth fused enhanced feature and the fused convolution feature are spliced ​​and convoluted to obtain the first image semantic feature; A training defect detection neural network generation module is used to train the defect detection neural network using a training data set composed of defect images of key parts of a plurality of transmission lines. When the sum of the image bounding box intersection-over-union loss, the bounding box coordinate loss and the bounding box shape loss in the image segmentation module is less than a loss threshold, a training defect detection neural network is obtained; The defect detection result generation module is used to process the transmission line diagram to be inspected through the trained defect detection neural network to obtain the defect detection result.

9. A defect detection device for a power transmission line, characterized in that: The method comprises a processor and a memory, wherein the processor implements the power transmission line defect detection method according to any one of claims 1 to 7 when executing the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the defect detection method for a power transmission line according to any one of claims 1 to 7 is implemented.

Images