Power transmission line small target defect identification method based on visual angle of unmanned aerial vehicle

Abstract

The invention discloses a power transmission line small target defect identification method based on an unmanned aerial vehicle visual angle, and relates to the field of intelligent operation and maintenance of power equipment, and the method comprises the steps: carrying out the enhancement processing of a defect feature data set based on a multi-dimensional feature analysis technology, and obtaining a defect sample library; based on the defect sample library, constructing and training a power transmission line defect detection network model based on a depth drift convolution algorithm, and inputting a to-be-detected power transmission line image into the trained power transmission line defect detection network model to obtain a power transmission line defect detection result; and constructing a deep residual error classification network model, and classifying small target defect detection results with low confidence in the power transmission line defect detection results by using the deep residual error classification network model to obtain a small target defect identification result. According to the invention, the detection precision under the situations of complex small targets and the like is improved, so that the reliability and practicability of the electric power inspection system are improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent operation and maintenance of power equipment, and more specifically, to a method for identifying small target defects in transmission lines from the perspective of unmanned aerial vehicles (UAVs). Background Technology

[0002] Transmission lines are a core component of power transmission systems, a critical element, and the carrier for transmitting electricity across regions. Therefore, the safety and reliability of transmission lines are of paramount importance, as they are crucial for ensuring the stable operation of the power system. Applying drone technology to inspect transmission lines enables rapid detection of potential hazards and confirmation of defects, achieving the goal of intelligent analysis of the transmission network.

[0003] Currently, the common operation and maintenance method for power transmission lines is to use drone inspections combined with manual review. Mainly, maintenance personnel conduct regular inspections using drones and manually judge whether there are any abnormalities in the power transmission lines. This type of inspection often relies on the personal experience of on-site personnel. Moreover, due to the long lines and the large number of pictures taken, manual review is very slow and prone to omissions and misidentifications. In some special scenarios such as missing pins or small target defects in insulator strings, the cost of manual review is even higher.

[0004] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0005] To address the problems in related technologies, this invention proposes a method for identifying small target defects in power transmission lines from the perspective of unmanned aerial vehicles (UAVs), thereby overcoming the aforementioned technical problems in existing related technologies.

[0006] Therefore, the specific technical solution adopted by the present invention is as follows:

[0007] A method for identifying small target defects in power transmission lines from the perspective of unmanned aerial vehicles (UAVs), the method comprising:

[0008] S1. Acquire images of power transmission lines in the drone scene and construct a defect feature dataset. Enhance the defect feature dataset based on multi-dimensional feature analysis technology to obtain a defect sample library.

[0009] S2. Based on the defect sample library, construct and train a transmission line defect detection network model based on the depth-drift convolution algorithm. Input the transmission line image to be detected into the trained transmission line defect detection network model to obtain the transmission line defect detection result.

[0010] S3. Construct a deep residual classification network model and use the deep residual classification network model to classify the low-confidence small target defect detection results in the transmission line defect detection results to obtain the small target defect identification results.

[0011] Preferably, the step of acquiring power line images in a drone scenario and constructing a defect feature dataset, and then enhancing the defect feature dataset based on multi-dimensional feature analysis technology to obtain a defect sample library includes:

[0012] S11. Obtain images of power transmission lines at the corresponding waypoint locations based on UAV waypoint information, and integrate the power transmission line images collected at each power transmission station to obtain a defect feature dataset.

[0013] S12. Perform geometric transformation, mosaic enhancement, and hybrid enhancement processing on the defect feature dataset in sequence to achieve data enhancement of the defect feature dataset;

[0014] S13. Introduce a logarithmic function on the inverse proportion of the number of defect samples in the defect feature dataset to smooth and enhance the probability distribution of defect samples in the defect feature dataset, and construct a defect sample library based on the enhanced defect feature dataset.

[0015] Preferably, the step of constructing and training a transmission line defect detection network model based on a defect sample library and using a depth-drift convolution algorithm, and inputting the transmission line image to be detected into the trained transmission line defect detection network model to obtain the transmission line defect detection result includes:

[0016] S21. Construct an adaptive feature extraction network model for the power transmission line scenario, and input the defect samples in the defect sample library into the adaptive feature extraction network model for feature extraction to obtain the defect feature map;

[0017] S22. Construct a feature fusion network model based on a collection-allocation mechanism, and use the feature fusion network model to fuse the defect feature maps to obtain multi-scale defect features;

[0018] S23. Integrate scale-dimensional attention, spatial-dimensional attention, and task-dimensional attention to generate an improved self-attention recognition head. Use the improved self-attention recognition head to identify multi-scale defect features and obtain the transmission line defect detection results.

[0019] Preferably, the construction of the adaptive feature extraction network model for the power transmission line scenario, and the input of defect samples from the defect sample library into the adaptive feature extraction network model for feature extraction to obtain the defect feature map, includes:

[0020] S211. Construct a variable-position drift convolution module, specifically including:

[0021] Select the center point of the preset convolutional kernel structure, spread out several sampling positions from the center point to obtain the sampling point offset parameters, and construct the position weight parameters to constrain the sampling position offset.

[0022] Add sampling point offset parameters and position weight parameters to each sampling point in the preset convolutional kernel structure to obtain a variable position drift convolutional module; distinguish whether the current sampling point region is the target region. If the current sampling point region is not the target region, set the position weight parameter to 0.

[0023] S212. A lightweight object detection deep learning model is used as the target architecture, and a variable position drift convolution module is introduced into the target architecture to obtain an adaptive feature extraction network model.

[0024] S213. Use an adaptive feature extraction network model to extract features from defect samples in the defect sample library to obtain a defect feature map optimized by a variable position drift convolution module.

[0025] Preferably, the feature fusion network model based on the collection-allocation mechanism includes a low-level collection-allocation branch and a high-level collection-allocation branch, and each branch consists of a feature unification module, a feature information fusion module and a feature information injection module.

[0026] Preferably, the construction of a feature fusion network model based on a collection-allocation mechanism, and the fusion of defect feature maps using the feature fusion network model to obtain multi-scale defect features, includes:

[0027] S221. In the low-level collection-allocation branch, obtain the scale of each defect feature map, and divide the defect feature map into semantic feature map and detail feature map based on the defect feature map scale.

[0028] S222. In the feature unification module, the semantic feature map is upsampled by bilinear interpolation and the detail feature map is downsampled by average pooling to obtain the upsampled features and downsampled features respectively.

[0029] S223. In the feature fusion module, the upsampled features and downsampled features are fused at the first level using a multi-layer reparameterized convolution module. The first-level fused features are divided into global features, and then the global features are fused with the hierarchical features to obtain the second-level fused features.

[0030] S224. In the feature information injection module, the bilinear interpolation method is used to perform tertiary fusion of the secondary fusion features output by the feature fusion module with the local features at the current input scale, and the tertiary fusion features are output.

[0031] Preferably, in the feature information injection module, after performing a three-level fusion of the global features output by the feature fusion module and the local features at the current input scale using bilinear interpolation, and outputting the three-level fusion result, the method further includes:

[0032] S225. The three-level fused features are input to the high-level collection-allocation branch, and the feature unification module unifies the size of the input features through average pooling.

[0033] S226. The feature fusion module fuses the input features after unification of size through the Transformer module and divides the fusion result into global features.

[0034] S227. After fusing global features with local features, the feature information injection module distributes the fused global features to different levels to obtain the final multi-scale defect features.

[0035] Preferably, the scale-dimensional attention consists of an average pooling layer, a 1×1 convolution, a ReLU activation function, and a hardsigmoid activation function, and the scale-dimensional attention is used to adapt to objects of different scales in the horizontal dimension of the image;

[0036] The spatial dimension attention is used to learn the positional relationships of target objects in the spatial dimension in order to identify defects at different target locations;

[0037] The task-dimensional attention consists of an average pooling layer, two fully connected layers, and a ReLU activation function, which is used to dynamically control the switching of feature map channels according to different tasks.

[0038] Preferably, the construction of a deep residual classification network model, and the use of the deep residual classification network model to classify the low-confidence small target defect detection results in the transmission line defect detection results, to obtain the small target defect identification results, includes:

[0039] S31. Using bilinear interpolation, the pin category images in the transmission line defect detection results are scaled to a preset size, and the scaled pin category images are then fed into a deep residual classification network model.

[0040] The deep residual classification network model is the adaptive feature extraction network in S2, and the global average pooling layer of the adaptive feature extraction network model is replaced with a fully connected layer.

[0041] S32. Use a deep residual classification network model to perform secondary feature classification on pin category images, and use the pin defect category obtained after classification as the small target defect recognition result.

[0042] Preferably, the expression for the probability distribution of the defective samples is:

[0043]

[0044] In the formula, P i N represents the enhancement probability of defect category i. iLet represent the number of faults in the current i-th category, α represent the smoothing function, and c represent the c-th category.

[0045] The beneficial effects of this invention are as follows:

[0046] 1. This invention improves the feature extraction capability for defects of different sizes and shapes in UAV images by using an adaptive feature extraction network based on variable position drift convolution. In order to enhance the network's ability to fuse multi-scale features, a feature fusion algorithm based on the "collection-distribution" mechanism is combined into the traditional feature pyramid network to achieve lossless cross-layer transmission of feature information, thereby effectively solving the problems of large target scale differences and strong background interference in power transmission line defect detection.

[0047] 2. This invention introduces a unified self-attention dynamic recognition head in the feature recognition stage, which integrates multiple attention mechanisms in a coherent manner from three dimensions: scale, space, and task. This enhances the ability to perceive complex defects in UAV images. Furthermore, a two-stage residual classification network is used for secondary recognition in scenarios involving missing pins and insulator strings to obtain the true defect category. This improves the detection accuracy in scenarios involving complex small targets, thereby enhancing the reliability and practicality of the power inspection system. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a flowchart of a method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) according to an embodiment of the present invention.

[0050] Figure 2 This is a schematic diagram illustrating a specific implementation of a method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) according to an embodiment of the present invention.

[0051] Figure 3 This is a network architecture diagram of a transmission line defect detection network in a method for identifying small target defects in transmission lines based on the perspective of an unmanned aerial vehicle (UAV) according to an embodiment of the present invention.

[0052] Figure 4 This is a schematic diagram of missing pins in a method for identifying small target defects in power transmission lines based on the perspective of an unmanned aerial vehicle (UAV) according to an embodiment of the present invention.

[0053] Figure 5This is a schematic diagram of a pin not being installed properly in a method for identifying small target defects in power transmission lines based on the perspective of an unmanned aerial vehicle (UAV) according to an embodiment of the present invention.

[0054] Figure 6 This is a schematic diagram of pin corrosion in a method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) according to an embodiment of the present invention.

[0055] Figure 7 This is a schematic diagram of a normal pin in a method for identifying small target defects in power transmission lines based on the perspective of an unmanned aerial vehicle (UAV) according to an embodiment of the present invention.

[0056] Figure 8 This is a schematic diagram of a variable position drift convolution module in a method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) according to an embodiment of the present invention.

[0057] Figure 9 This is a schematic diagram of a feature fusion network structure based on a collection-allocation mechanism in a method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) according to an embodiment of the present invention.

[0058] Figure 10 This is a schematic diagram of an improved self-attention recognition head in a method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) according to an embodiment of the present invention.

[0059] Figure 11 This is a schematic diagram of a deep residual classification network in a method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) according to an embodiment of the present invention.

[0060] Figure 12 This is a comparison chart of the accuracy of each category before and after data augmentation in a method for identifying small target defects in power transmission lines based on the perspective of an unmanned aerial vehicle (UAV) according to an embodiment of the present invention. Detailed Implementation

[0061] To further illustrate the various embodiments, the present invention provides accompanying drawings, which are part of the disclosure of the present invention. These drawings are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention. The components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0062] According to an embodiment of the present invention, a method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) is provided.

[0063] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figures 1-2As shown, the method for identifying small target defects in power transmission lines based on the perspective of an unmanned aerial vehicle (UAV) according to an embodiment of the present invention includes the following:

[0064] S1. Acquire images of power transmission lines in the drone scene and construct a defect feature dataset. Enhance the defect feature dataset based on multi-dimensional feature analysis technology to obtain a defect sample library.

[0065] The process of acquiring power transmission line images in a drone scenario and constructing a defect feature dataset, followed by enhancing the defect feature dataset using multi-dimensional feature analysis technology to obtain a defect sample library, includes:

[0066] S11. Obtain images of power transmission lines at the corresponding waypoint locations based on UAV waypoint information, and integrate the power transmission line images collected at each power transmission station to obtain a defect feature dataset.

[0067] S12. Perform geometric transformation, mosaic enhancement, and hybrid enhancement processing on the defect feature dataset in sequence to achieve data enhancement of the defect feature dataset;

[0068] S13. Introduce a logarithmic function on the inverse proportion of the number of defect samples in the defect feature dataset to smooth and enhance the probability distribution of defect samples in the defect feature dataset, and construct a defect sample library based on the enhanced defect feature dataset.

[0069] It should be noted that a defect feature dataset for drone scenarios is established, and the samples are enhanced based on multi-dimensional feature analysis. A training sample library containing multiple perspectives and scenarios is constructed, and annotation rules are specified based on real-world scenarios. Specifically:

[0070] As shown in Table 1, the present invention first obtains the transmission line image at the corresponding waypoint location based on the UAV waypoint information. The UAV waypoint is the prior information planned in the actual inspection process. Then, this process is repeated for different sites to obtain the initial training sample data. The specific defect statistics are shown in the second and third columns of Table 1.

[0071] Table 1: Defect Characteristics Table

[0072]

[0073] Based on existing sample data, offline data augmentation was performed using three methods: geometric transformation, Mosaic (puzzle augmentation), and Mixup (hybrid augmentation). Due to the uneven distribution of the data samples, selective augmentation was performed based on the number of defective samples. Here, a logarithmic function was introduced based on the inverse proportion of the number of defects to smooth the augmentation probability distribution and avoid extreme weight allocation. The core formula for geometric transformation augmentation is as follows:

[0074]

[0075] In the formula, (x,y) represents the original image coordinates, (x',y') represents the enhanced coordinates, a, b, c, and d control rotation, scaling, and cropping, and t x , t y Both represent the amount of translation.

[0076] The Mosaic method trains the model by stitching four training images together into one, reducing the scale of the original four images to one-quarter. This enhancement increases the model's contextual understanding and its ability to detect small targets.

[0077] Mixup is a data augmentation technique based on linear interpolation that generates new training samples by mixing two input images and their labels. Its expression is:

[0078]

[0079] In the formula, λ∈(0,1) represents the hyperparameter controlling the mixing intensity, (x i ,y i (x) represents the coordinates of the first image. j ,y j (x', y') represents the coordinate position of the second image, and (x', y') represents the enhanced coordinate position.

[0080] Given that there are N in the defect feature dataset c There are N defect categories, and the original number of samples for each category is N1, N2, ..., N. c , where N i Let P represent the number of faults in the current i-th category, and define the augmentation probability P for category i. i for:

[0081]

[0082] In the formula, α represents the smoothing function, which is set to 1, related to the properties of the logarithmic function. This ensures that when there are fewer defect categories, the probability of data augmentation is higher. After obtaining the probability function, the number of times the category to be augmented is obtained by multiplying the probability of each category by the total number of samples, thus solving the problem of imbalanced line samples, as shown in columns 4 and 5 of Table 1.

[0083] S2. Based on the defect sample library, construct and train a transmission line defect detection network model based on the depth-drift convolution algorithm. Input the transmission line image to be detected into the trained transmission line defect detection network model to obtain the transmission line defect detection result.

[0084] It should be noted that, as Figure 3As shown, this invention designs a multi-scale deep learning network architecture based on a depth-drift convolution algorithm using a defect sample library. It improves and optimizes the three stages of feature extraction, feature fusion, and feature recognition in traditional deep learning methods, and constructs a feature-oriented power line defect detection network suitable for high-resolution UAV imagery to obtain the detected defect categories and coordinates.

[0085] exist Figure 3 In the middle, (1) input represents the input module, where the input is the original training data; (2) Multi-scale feature extraction module, which uses CBL (Bonv: convolution, BN: normalization, Leakyrelu: activation function): variable position drift convolution, kh, kw: represent the width and height of the convolution kernel (conv), MxPool+ELAN: respectively represent pooling + efficient layer aggregation of the features after convolution to obtain multi-scale Edge + Texture + color + Brightness Variation, SPPCSP: spatial pyramid pooling and cross-stage channel connection, gather: feature aggregation; (3) Multi-scale feature fusion module, which includes Low-GD: low-level collection-allocation mechanism and High-GD: high-level collection-allocation mechanism, where Inject: feature information injection module, IFM: feature information fusion module, FAM: feature unification module, feature split: feature separation, feature fusion: feature fusion, feature align: feature alignment, mean pooling: average pooling, transformer: a feature extraction method, (4) Unified self-sttention identification head: unified self-sttention identification head, including scale aware: scale dimension, spatial aware: spatial dimension, taskware: task dimension, where bounding box: detection rectangle, center point: center point coordinates, cornerpoint: corner point coordinates, (5) output: output layer, where defect identification and location: defect category and coordinates.

[0086] The transmission line defect detection network mainly consists of three parts: (2) Multi-scale feature extraction: an adaptive feature extraction network based on variable position drift convolution; (3) Multi-scale feature fusion: a multi-scale feature fusion network based on the "collection-allocation" mechanism; and (4) Unified self-attention identification head: a unified self-attention identification head. Figure 3 This paper demonstrates the entire process of feature extraction, fusion, and fault identification of the input power transmission line image in the algorithm network presented in this paper. In stage (2), the input high-resolution power transmission line image from the UAV is used for feature extraction through fixed-size convolution and variable convolution. Variable convolution uses additional convolutional layers to learn offsets from the extracted feature maps and adaptively adjusts its sampling position. In stage (3), the multi-scale feature maps extracted by the backbone are uniformly collected, fused as needed, and then distributed to different layers, solving the problem of feature information loss during cross-layer transmission in traditional feature pyramid networks based on "recursion". Finally, in stage (4) of the model, a recognition head that simultaneously embeds scale awareness, spatial awareness, and task awareness is used to enhance the network's ability to perceive defects in small targets.

[0087] Specifically, the process involves constructing and training a transmission line defect detection network model based on a defect sample library and a depth-drift convolution algorithm. The image of the transmission line to be detected is then input into the trained network model to obtain the transmission line defect detection results, including:

[0088] S21. Construct an adaptive feature extraction network model for the power transmission line scenario, and input the defect samples in the defect sample library into the adaptive feature extraction network model for feature extraction to obtain the defect feature map.

[0089] The process of constructing an adaptive feature extraction network model for the power transmission line scenario, and inputting defect samples from the defect sample library into the adaptive feature extraction network model for feature extraction to obtain a defect feature map includes:

[0090] S211. Construct a variable-position drift convolution module, specifically including:

[0091] Select the center point of the preset convolutional kernel structure, spread out several sampling positions from the center point to obtain the sampling point offset parameters, and construct the position weight parameters to constrain the sampling position offset.

[0092] Add sampling point offset parameters and position weight parameters to each sampling point in the preset convolutional kernel structure to obtain a variable position drift convolutional module; distinguish whether the current sampling point region is the target region. If the current sampling point region is not the target region, set the position weight parameter to 0.

[0093] S212. A lightweight object detection deep learning model is used as the target architecture, and a variable position drift convolution module is introduced into the target architecture to obtain an adaptive feature extraction network model.

[0094] S213. Use an adaptive feature extraction network model to extract features from defect samples in the defect sample library to obtain a defect feature map optimized by a variable position drift convolution module.

[0095] It should be noted that this invention uses a lightweight object detection deep learning model (YOLOv11-S deep learning model) as the basic model architecture to construct a deep learning-based adaptive feature extraction network for power transmission line scenes based on surface features. However, traditional feature extraction networks are limited by using convolutional kernels of fixed size, and cannot form a matching kernel structure when facing targets of different sizes. The expression for this is shown below:

[0096]

[0097] In the formula, taking a 3×3 convolution kernel as an example, each output y1(p0) needs to sample 9 positions at the center position x of the convolution kernel. These 9 positions are fixed shapes obtained by spreading outward from the center position x(p0).

[0098] To enable the network to adjust its receptive field and adapt to defect categories of varying sizes and shapes, using variable-position-drift convolutions instead of the original fixed convolution kernels is a better solution. The variable-position-drift convolution is represented as follows:

[0099]

[0100] In the formula, a learnable sampling point offset parameter Δp is added for each sampling point in the convolution. n and position weight parameter Δm n Each output y2(p0) is diffused outwards from the center point x(p0) and sampled at 9 locations to obtain the parameter ΔP. n Then, convolution has non-fixed sampling positions, where w represents each p n The learnable parameter weights corresponding to the points, weight Δm nThe purpose is to constrain the drift of the convolution, because overly free drift of the convolution sampling position may introduce contextual information from non-target objects, interfering with the feature extraction quality of the network. Therefore, while learning the offset of the variable convolution, a weight parameter is assigned to the position after the drift to distinguish whether the region is the target region of the current task. If the region of the sampling point is a non-target region, the weight is set to 0, specifically as follows: Figure 8 As shown.

[0101] exist Figure 8 In Figure (a), Deformable convolution is an adaptive sampling map based on variable position drift convolution. When facing defective targets with spatial size differences in the image, the variable convolution can adaptively learn the position and weight of sampling points. Sampling points spreading outward means expanding the sampling points, and feature map means the feature map. Figure (b) shows "ELAN with DCN": an efficient layer aggregation module design based on variable position drift convolution. Convolutions 1, 2, and 5 are 1×1 fixed-size convolution kernels for channel compression; convolution 3 is a 3×3 fixed-size convolution kernel for feature map downsampling. CBL means regular convolution + bn normalization + leakyrelu activation, and DCN means variable drift convolution + normalization + activation.

[0102] S22. Construct a feature fusion network model based on the collection-allocation mechanism, and use the feature fusion network model to fuse the defect feature maps to obtain multi-scale defect features.

[0103] The feature fusion network model based on the collection-allocation mechanism includes a low-level collection-allocation branch and a high-level collection-allocation branch, and each branch consists of a feature unification module, a feature information fusion module and a feature information injection module.

[0104] The construction of a feature fusion network model based on a collection-allocation mechanism, and the fusion of defect feature maps using the feature fusion network model to obtain multi-scale defect features, includes:

[0105] S221. In the low-level collection-allocation branch, obtain the scale of each defect feature map, and divide the defect feature map into semantic feature map and detail feature map based on the defect feature map scale.

[0106] S222. In the feature unification module, the semantic feature map is upsampled by bilinear interpolation and the detail feature map is downsampled by average pooling to obtain the upsampled features and downsampled features respectively.

[0107] S223. In the feature fusion module, the upsampled features and downsampled features are fused at the first level using a multi-layer reparameterized convolution module. The first-level fused features are divided into global features, and then the global features are fused with the hierarchical features to obtain the second-level fused features.

[0108] S224. In the feature information injection module, the bilinear interpolation method is used to perform tertiary fusion of the secondary fusion features output by the feature fusion module with the local features at the current input scale, and the tertiary fusion features are output.

[0109] The feature information injection module, after performing a three-level fusion of the global features output by the feature fusion module and the local features at the current input scale using bilinear interpolation and outputting the three-level fusion result, further includes:

[0110] S225. The three-level fused features are input to the high-level collection-allocation branch, and the feature unification module unifies the size of the input features through average pooling.

[0111] S226. The feature fusion module fuses the input features after unification of size through the Transformer module and divides the fusion result into global features.

[0112] S227. After fusing global features with local features, the feature information injection module distributes the fused global features to different levels to obtain the final multi-scale defect features.

[0113] It should be noted that, as Figure 9 As described above, based on the optimized defect features from the variable-position drift convolution module, this invention employs a feature fusion network based on a "collection-allocation mechanism" to fuse the extracted multi-scale features. In its implementation, it abandons the traditional "recursive" feature fusion method, using a unified module to collect and fuse feature map information from different levels in the backbone feature extraction network, thus solving the problem of information loss during transmission in traditional feature pyramid networks. The entire process relies on low-level collection-allocation branches (Low-GD) and high-level collection-allocation branches (High-GD), each branch consisting of three modules: a feature unification module (FAM), a feature information fusion module (IFM), and a feature information injection module (Inject).

[0114] exist Figure 9In the middle, the neck is based on a feature fusion network structure of "collection-distribution mechanism", where backbone: the base network, multi-scale feature extraction: multi-scale feature extraction module, feature fusion: feature fusion, inject: feature information injection module, gather: feature aggregation, distribute: feature distribution, Low-GD: low-level collection-distribution mechanism, High-GD: high-level collection-distribution mechanism, IFM: feature information fusion module, and FAM: feature unification module.

[0115] In the low-level GD branch (low-level collection-allocation branch), the multi-scale feature maps extracted by the adaptive feature extraction network are first received, denoted as M1, M2, and M3. The scale of each feature map is represented as follows:

[0116] M i ∈N×C Mi ×R Mi ;

[0117] In the formula, M i Let represent the i-th multi-scale feature map, N represent the batch size of the input feature maps, C represent the number of channels, and R represent the size.

[0118] In the feature unification module, small input feature maps (semantic feature maps) are upsampled using bilinear interpolation, while large feature maps (detail feature maps) are downsampled using average pooling. This unification of feature maps of different sizes is achieved, and the output result F is obtained. align :

[0119] F align =Low_FAM([M1,M2,M3]);

[0120] The feature fusion module uses multi-layer reparameterized convolutional blocks to process the output F. align Feature fusion is performed, and the result is then divided into global features F in the channels. inj_P3 and F inj_P4 Then, it is fused with features from different levels:

[0121] F fuse =RepBlock(F align );

[0122] F inj_p3 ,F inj_p4 =Split(F fuse );

[0123] In the formula, F alignRepresents the output feature maps after upsampling and downsampling respectively. RepBlock represents the feature fusion operation, Split represents the feature separation operation, and F... fuse This represents the features after being fused by multiple layers of reparameterized convolutional modules.

[0124] The feature information injection module injects the global features output by the previous module, such as F. inj_P3 The local features at the current input scale, such as M3, are amplified by bilinear interpolation or shrunk by average pooling and then merged again. Finally, the RepBlock module (feature fusion operation) outputs the results, which are denoted as P3, P4 and P5.

[0125] The high-level GD branch (high-level collection-allocation branch) is responsible for fusing the feature results output from the low-level GD branch. The feature unification module uses average pooling to unify the input feature size to obtain F. align The feature fusion module processes the unified input features F through the Transformer module. align The fusion result F is obtained. fuse And divided into different global features F inj_P4 and F inj_P5 After being fused with local features, the feature information injection module distributes them to different levels:

[0126] F align =High_FAM([P3,P4,P5]);

[0127] F fuse =Transformer(F align );

[0128] F inj_N4 ,F inj_N5 =Split(conv1×1(F fuse ));

[0129] In the formula, F align F represents the input features after average pooling. fuse This represents the fusion result generated after processing the input features through the Transformer module, and Split represents the feature separation operation.

[0130] S23. Integrate scale-dimensional attention, spatial-dimensional attention, and task-dimensional attention to generate an improved self-attention recognition head. Use the improved self-attention recognition head to identify multi-scale defect features and obtain the transmission line defect detection results.

[0131] The scale-dimensional attention consists of an average pooling layer, a 1×1 convolution, a ReLU activation function, and a hardsigmoid activation function. The scale-dimensional attention is used to adapt objects of different scales in the horizontal dimension of the image.

[0132] The spatial dimension attention is used to learn the positional relationships of target objects in the spatial dimension in order to identify defects at different target locations;

[0133] The task-dimensional attention consists of an average pooling layer, two fully connected layers, and a ReLU activation function, which is used to dynamically control the switching of feature map channels according to different tasks.

[0134] It should be noted that after acquiring multi-scale features, a unified and improved self-attention recognition head is used for defect identification. The recognition head is a key component of the network, responsible for identifying the input sample feature map and providing the location and category information of defects in the image. To address the problem of feature loss of small target defects in the captured images as the drone's flight altitude increases, this invention introduces a unified self-attention dynamic recognition head in the feature recognition stage.

[0135] like Figure 10 As shown, the unified self-attention dynamic recognition head coherently integrates multiple attention mechanisms across three dimensions: scale, spatial, and task. This avoids the significant computational cost of directly learning attention across all dimensions, significantly improving the network's ability to recognize defects in small targets. Feature maps at different levels in the feature pyramid are first upsampled or downsampled to near-average sizes. The scaled feature maps can be represented as an L×S×C three-dimensional tensor, which is then used as input for continuous attention processing.

[0136] The scale-dimensional attention mechanism consists of four parts: average pooling, 1×1 convolution, ReLU activation function, and hard sigmoid activation function, which are used to adapt to objects of different scales in the horizontal dimension of the image.

[0137] Spatial dimension learning is achieved by using convolution operations to sparsify attention learning. Then, features are aggregated across layers at the same spatial location to learn the positional relationships of target objects in the spatial dimension (height × width), helping the network to better identify defects that appear in different shapes and locations.

[0138] Task-oriented attention consists of average pooling, two fully connected layers, and a ReLU activation function. Depending on the different representations of the recognition task, such as bounding boxes, center points, and corner points, task-oriented attention can dynamically control the switching on and off of different feature map channels to achieve task perception.

[0139] exist Figure 10 In this context, the input is a 3D tensor of size L×S×C, where L represents the number of feature maps at different levels in the feature pyramid, S = H×W represents the height and width of the feature map, and C represents the number of channels. Here, Images represents the input image, feature pyramid represents the feature pyramid, and general view represents the feature view. Scale-aware attention, spatial-aware attention, and task-aware attention are also defined. Input is the feature view of the input, average pooling is the average pooling layer, conv(1x1) is the convolutional layer, ReLU is the activation layer, hard sigmoid is the relevance activation function, index is the spatial attention index, offset is the offset layer, average pooling is the average pooling layer, fully connected is the fully connected layer, and normalize is the normalization process.

[0140] S3. Construct a deep residual classification network model and use the deep residual classification network model to classify the low-confidence small target defect detection results in the transmission line defect detection results to obtain the small target defect identification results.

[0141] The construction of a deep residual classification network model, and the use of this model to classify low-confidence small target defect detection results in the transmission line defect detection results, to obtain small target defect identification results, include:

[0142] S31. Use bilinear interpolation technology to scale the pin category images in the transmission line defect detection results to a preset size, and then send the scaled pin category images to the deep residual classification network model.

[0143] The deep residual classification network model is the adaptive feature extraction network in S2, and the global average pooling layer of the adaptive feature extraction network model is replaced with a fully connected layer.

[0144] S32. Use a deep residual classification network model to perform secondary feature classification on pin category images, and use the pin defect category obtained after classification as the small target defect recognition result.

[0145] It should be noted that after S2 obtains the defect location and category information, a two-stage defect type subdivision is performed on the pin category image. The pin failure subdivision result is as follows: Figures 4-7As shown, the detected pin-related faults were further categorized into four types: missing pins, improperly installed pins, corroded pins, and normal pins. The captured pin-related local images were scaled to a fixed size (224×224) using bilinear interpolation to eliminate scale differences.

[0146] like Figure 11 As shown, the prepared data is fed into a deep residual classification network for model training. The deep residual classification network uses the feature extraction network part in S2. Here, in order to reduce the number of parameters, a global average pooling layer (GAP) is used instead of a fully connected layer. After defect subdivision, the accuracy of pin category can be greatly improved.

[0147] exist Figure 11 In this context, GAP represents the parameter changes relative to FC, where FC stands for fully connected layer, GAP stands for globally average pooling layer, and softmax represents the activation layer.

[0148] Furthermore, after constructing a deep residual classification network model and using it to classify low-confidence small target defect detection results in transmission line defect detection results to obtain small target defect identification results, the process also includes:

[0149] The image to be detected is input into a pre-trained deep learning network architecture based on a depth-drift convolution algorithm combined with a multi-scale deep learning network architecture to obtain the detection results of each branch network; the detection results that need to be classified in the second stage are then input into a pre-trained classification network for further classification to obtain the final detection result.

[0150] The defect target is cropped from the corresponding image based on the location information of the detection result that requires secondary classification. After standardization, it is fed into the trained classification network to obtain further classification results. The category of the detection result is then updated as the final detection result.

[0151] The following section provides further explanation of the method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) provided by this invention, using specific implementation data:

[0152] This invention achieves accurate identification of defective targets in complex scenarios by constructing a collaboratively optimized two-stage detection framework, including:

[0153] First, a defect feature dataset for UAV scenarios is established. Based on multi-dimensional feature analysis, the samples are enhanced to construct a training sample library containing multiple perspectives and scenarios. Labeling rules are specified based on real-world scenarios.

[0154] Secondly, based on the defect sample library, a feature-oriented power line defect detection network based on the depth-drift convolution algorithm combined with a multi-scale deep learning network was designed. The feature extraction, feature fusion and feature recognition stages in the traditional deep learning method were improved and optimized, and a feature-oriented power line defect detection network suitable for high-resolution UAV images was constructed.

[0155] A deep residual classification network is further constructed to refine the features of low-confidence small sample detection results to obtain accurate small sample recognition results; finally, the optimized detection results are output through the decision-level fusion module.

[0156] Experimental data shows that the average detection accuracy of this invention reaches 70.7% in scenarios involving complex small targets such as insulator strings and pins, initially reaching the commercial level in the industry and significantly improving the reliability and practicality of power line inspection systems. This invention innovatively integrates attention mechanisms with ensemble learning, effectively solving technical challenges in transmission line defect detection such as large target scale differences and strong background interference by constructing a "detection-verification" two-stage inference architecture. Specifically:

[0157] Data augmentation is performed on the basic data, and the accuracy before and after augmentation is compared, such as... Figure 12 As shown, the accuracy of each category was significantly improved after data augmentation, especially in the scenarios of pins and insulators.

[0158] Based on the baseline model, as shown in Table 2, the baseline represents the benchmark experiment without adding any additional methods. Experiment A represents replacing the traditional feature extraction network of the baseline with VPDC (Adaptive Feature Extraction Network), significantly improving the mAP50 (overall accuracy) from 65.7% to 68.4%, and also improving the recall and precision by 1.2% and 2.1%, respectively, demonstrating the capability of the variable position drift convolution method. Experiment B represents using the GD (collection-allocation mechanism) module on top of the baseline. Experiment A+B, building on Experiment A, shows that using the "collection-allocation mechanism" feature fusion network significantly enhances the network model's multi-scale feature fusion capability, thereby improving the network's accuracy in identifying transmission line defects. Experimental results show that this network based on the "collection-allocation mechanism" achieves further improvements in various metrics compared to the previous step, with mAP50 improving by 1.4%, recall by 0.7%, and precision by 0.9%. Finally, Experiment C represents adding a Head (self-attention recognition head) to the baseline. In Experiments A+B+C, the introduction of a unified self-attention recognition head also improved the network's mAP50 by 0.9% and F1-score by 0.01. Ultimately, FO-Net improved the mAP50 by 5% and the F1-score by 0.05 compared to the baseline, proving that the network design and construction process is reasonable and that the various structures are mutually compatible. Furthermore, the remaining experimental results in the ablation experiments also show that all three methods used in this invention can bring about improvements in accuracy.

[0159] Table 2: Accuracy Comparison Before and After Data Augmentation

[0160]

[0161] The specific interpretations in Table 2 are as follows: In the first column, Experiment represents the experiment performed, baseline represents the benchmark experiment, A represents adding VPDC operation to the baseline, B represents adding GD operation to the baseline, C represents adding Head operation to the baseline, and A+B, A+C, B+C, and A+B+C represent different fusion operations; the second column, Method, represents different methods, specifically VPDC for adaptive feature extraction network, GD for collection-allocation mechanism, and Head for self-attention recognition head; the third column, Metrics, represents the specific model performance results, including mAP50 comprehensive evaluation results, Precision for accuracy, Recall for recall, F1 for comprehensive score, and Parameters for the number of model parameters.

[0162] Accordingly, as shown in Table 3, the algorithm in this invention is also compared with the previous YOLO series algorithms. Experimental results show that the model achieves improved accuracy while maintaining a low number of parameters.

[0163] Table 3: Comparison of YOLO series algorithms before and after improvement

[0164]

[0165] In Table 3, the first column "Models" represents the model column, "YOLO*" represents the corresponding YOLO models of different series, and "ours" represents our improved algorithm, which is the result after A+B+C based on the baseline in Table 2. The second column "Metrics" represents the specific model performance results, including the mAP50 comprehensive evaluation result, "Precision" represents accuracy, "Recall" represents recall, "F1" represents the comprehensive score, "Parameters" represents the number of model parameters, "GFLOP" represents the model computation cost, and "Layers" represents the number of model layers.

[0166] Finally, a two-stage defect type subdivision was performed on the pin category images. The results of the direct output of the detection model and the results after the classification network were compared as follows:

[0167] Table 4: Comparison of direct output results from the detection model and results processed by the classification network

[0168] Methods accuracy Use the test results directly 0.218 Classification network results 0.623

[0169] As shown in Table 4, the accuracy rate was significantly improved in scenarios with small targets such as pins after the classification network model was applied.

[0170] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV), characterized in that, The method includes: S1. Acquire images of power transmission lines in the drone scene and construct a defect feature dataset. Enhance the defect feature dataset based on multi-dimensional feature analysis technology to obtain a defect sample library. S2. Based on the defect sample library, construct and train a transmission line defect detection network model based on the depth-drift convolution algorithm. Input the transmission line image to be detected into the trained transmission line defect detection network model to obtain the transmission line defect detection result. S3. Construct a deep residual classification network model and use the deep residual classification network model to classify the low-confidence small target defect detection results in the transmission line defect detection results to obtain the small target defect identification results.

2. The method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, The process involves acquiring images of power transmission lines in a drone-based scenario and constructing a defect feature dataset. This dataset is then enhanced using multi-dimensional feature analysis techniques to obtain a defect sample library, which includes: S11. Obtain images of power transmission lines at the corresponding waypoint locations based on UAV waypoint information, and integrate the power transmission line images collected at each power transmission station to obtain a defect feature dataset. S12. Perform geometric transformation, mosaic enhancement, and hybrid enhancement processing on the defect feature dataset in sequence to achieve data enhancement of the defect feature dataset; S13. Introduce a logarithmic function on the inverse proportion of the number of defect samples in the defect feature dataset to smooth and enhance the probability distribution of defect samples in the defect feature dataset, and construct a defect sample library based on the enhanced defect feature dataset.

3. The method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, The aforementioned method involves constructing and training a transmission line defect detection network model based on a defect sample library and a depth-drift convolution algorithm. The image of the transmission line to be detected is then input into the trained network model to obtain the transmission line defect detection results, including: S21. Construct an adaptive feature extraction network model for the power transmission line scenario, and input the defect samples in the defect sample library into the adaptive feature extraction network model for feature extraction to obtain the defect feature map; S22. Construct a feature fusion network model based on a collection-allocation mechanism, and use the feature fusion network model to fuse the defect feature maps to obtain multi-scale defect features; S23. Integrate scale-dimensional attention, spatial-dimensional attention, and task-dimensional attention to generate an improved self-attention recognition head. Use the improved self-attention recognition head to identify multi-scale defect features and obtain the transmission line defect detection results.

4. The method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) as described in claim 3, characterized in that, The adaptive feature extraction network model for the power transmission line scenario is constructed, and defect samples from the defect sample library are input into the adaptive feature extraction network model for feature extraction, resulting in defect feature maps including: S211. Construct a variable-position drift convolution module, specifically including: Select the center point of the preset convolutional kernel structure, spread out several sampling positions from the center point to obtain the sampling point offset parameters, and construct the position weight parameters to constrain the sampling position offset. Add sampling point offset parameters and position weight parameters to each sampling point in the preset convolutional kernel structure to obtain a variable position drift convolutional module; distinguish whether the current sampling point region is the target region. If the current sampling point region is not the target region, set the position weight parameter to 0. S212. A lightweight object detection deep learning model is used as the target architecture, and a variable position drift convolution module is introduced into the target architecture to obtain an adaptive feature extraction network model. S213. Use an adaptive feature extraction network model to extract features from defect samples in the defect sample library to obtain a defect feature map optimized by a variable position drift convolution module.

5. The method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) as described in claim 4, characterized in that, The feature fusion network model based on the collection-allocation mechanism includes a low-level collection-allocation branch and a high-level collection-allocation branch, and each branch consists of a feature unification module, a feature information fusion module, and a feature information injection module.

6. The method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) as described in claim 5, characterized in that, The construction of a feature fusion network model based on a collection-allocation mechanism, and the fusion of defect feature maps using this model to obtain multi-scale defect features, includes: S221. In the low-level collection-allocation branch, obtain the scale of each defect feature map, and divide the defect feature map into semantic feature map and detail feature map based on the defect feature map scale. S222. In the feature unification module, the semantic feature map is upsampled by bilinear interpolation and the detail feature map is downsampled by average pooling to obtain the upsampled features and downsampled features respectively. S223. In the feature fusion module, the upsampled features and downsampled features are fused at the first level using a multi-layer reparameterized convolution module. The first-level fused features are divided into global features, and then the global features are fused with the hierarchical features to obtain the second-level fused features. S224. In the feature information injection module, the bilinear interpolation method is used to perform tertiary fusion of the secondary fusion features output by the feature fusion module with the local features at the current input scale, and the tertiary fusion features are output.

7. The method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) as described in claim 6, characterized in that, The feature information injection module, after performing a three-level fusion of the global features output by the feature fusion module and the local features at the current input scale using bilinear interpolation, and outputting the three-level fusion result, also includes: S225. The three-level fused features are input to the high-level collection-allocation branch, and the feature unification module unifies the size of the input features through average pooling. S226. The feature fusion module fuses the input features after unification of size through the Transformer module and divides the fusion result into global features. S227. After fusing global features with local features, the feature information injection module distributes the fused global features to different levels to obtain the final multi-scale defect features.

8. The method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) as described in claim 7, characterized in that, The scale-dimensional attention consists of an average pooling layer, a 1×1 convolution, a ReLU activation function, and a hard sigmoid activation function. The scale-dimensional attention is used to adapt objects of different scales in the horizontal dimension of the image. The spatial dimension attention is used to learn the positional relationships of target objects in the spatial dimension in order to identify defects at different target locations; The task-dimensional attention consists of an average pooling layer, two fully connected layers, and a ReLU activation function, which is used to dynamically control the switching of feature map channels according to different tasks.

9. The method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, The construction of a deep residual classification network model, and the use of this model to classify low-confidence small target defect detection results in the transmission line defect detection results, yields the following small target defect identification results: S31. Using bilinear interpolation, the pin category images in the transmission line defect detection results are scaled to a preset size, and the scaled pin category images are then fed into a deep residual classification network model. The deep residual classification network model is the adaptive feature extraction network in S2, and the global average pooling layer of the adaptive feature extraction network model is replaced with a fully connected layer. S32. Use a deep residual classification network model to perform secondary feature classification on pin category images, and use the pin defect category obtained after classification as the small target defect recognition result.

10. A method for identifying small target defects in power transmission lines from the perspective of an unmanned aerial vehicle (UAV) according to claim 2, characterized in that, The expression for the probability distribution of the defective samples is: In the formula, P i N represents the enhancement probability of defect category i. i Let represent the number of faults in the current i-th category, α represent the smoothing function, and c represent the c-th category.

Images