Overhead transmission line defect detection method based on multi-scale feature fusion

Abstract

The invention provides an overhead transmission line defect detection method based on multi-scale feature fusion, and the method comprises the steps: carrying out the correction and registration of a lead image and a laser point cloud image, carrying out the data enhancement, and constructing a detection data set; the method comprises the following steps: respectively inputting two images into a target detection network for training, adding an attention mechanism in a feature extraction network Backbone so as to carry out correlation calculation on a target, and meanwhile, adopting a multi-scale feature fusion detection method in a backbone part of a YOLOV8 operation network, so that the improvement of precision and accuracy is realized on the premise of ensuring the size of a model, and the accuracy of the model is improved. And the requirement of actual overhead transmission conductor defect detection is met.

Description

Technical Field

[0001] The present invention relates to the technical field of power transmission line conductor detection, and in particular to an overhead power transmission line defect detection method based on multi-scale feature fusion. Background Art

[0002] At present, my country mostly adopts manual line inspection and drone inspection for the quality inspection of ultra-high voltage and extra-high voltage overhead line installation. Manual line inspection requires inspection personnel to walk along the line to inspect the quality of the line and related hardware. This inspection method has problems such as harsh working environment, high safety risks, high labor intensity, low work efficiency, and manual inspection has visual blind spots, which easily lead to the omission of quality defects. Drones can greatly improve the efficiency of inspection, but drone inspection is generally designed to check whether the environment around the overhead transmission line, such as buildings, trees, hillsides, roads, rivers, etc., meets the specifications. It is easily affected by environmental and weather factors, and needs to carry relevant acceptance equipment. There is insufficient battery life, and it is impossible to conduct 360° all-round and multi-feature inspection of the conductors and their accessories, resulting in the omission of quality defects. The above problems have laid hidden dangers for the safe and stable operation of the line in the future, so it is urgent to research, develop and use new technologies to solve them.

[0003] The invention patent with patent number CN202310740759 mainly introduces a remote sensing image semantic change detection method and system that combines change detection and semantic segmentation tasks. It uses weak supervision and triple attention mechanism, which combines image channel attention, adjacent pixel block attention and similar pixel block attention to more effectively explore the semantic change information of image pairs at the same location and different periods. This method can improve the detection performance of semantic changes in remote sensing images. Its disadvantage is that most of the existing change detection methods based on deep learning rely on a large amount of labeled data. This may be due to the small area of ​​the change area in the image pair and the subjectivity of the labeled data set, resulting in low utilization of remote sensing image data. The invention patent with patent number CN202410482951 mainly introduces a defect detection method based on YOLOv8. This method introduces a new C2f module in the YOLOv8 computing network, which uses a dilated convolution layer to reduce the dimensionality of the original feature map and extract local and boundary information. In addition, the method also integrates the attention mechanism to enhance the key channels and spatial positions in the fused feature map to identify defects. Overall, this method provides a fast and accurate defect detection solution using YOLOv8. The downside is that although the new C2f module can improve performance, there may still be challenges when processing images with very complex or subtle defects, especially in the presence of high background noise. This method may require large computing resources and storage capacity, especially when applied to large-scale real-time inspection systems. Although this method has made progress in balancing inspection speed and accuracy, further optimization may be required in extreme cases (such as extremely high speed requirements or extreme accuracy requirements). Summary of the invention

[0004] In order to solve the problems of the prior art, the present invention provides an overhead transmission line defect detection method based on multi-scale feature fusion, the conductor image and the laser point cloud image are corrected and registered and data enhancement is performed to construct a detection data set; the two images are respectively input into the target detection network for training, and an attention mechanism is added to the feature extraction network Backbone to calculate the correlation of the target, and at the same time, a multi-scale feature fusion detection method is adopted in the backbone part of the YOLOV8 computing network, so as to achieve improvement of precision and accuracy while ensuring the size of the model, so as to meet the needs of actual overhead transmission line defect detection.

[0005] The present application embodiment provides a wire quality defect detection method based on the YOLO model, and the wire quality defect detection method includes: Acquire overhead transmission line conductor images and laser cloud point images.

[0006] The internal and external parameters of the annular array camera device used to obtain the wire image and the laser cloud point image are input into the pre-trained target YOLO model, the wire image is gridded, and each grid is subjected to target detection, and the edge rectangular frame center position information, width and height information, confidence information and target category information of the quality defects in each grid are output; wherein the quality defects include wire damage and deformation, skewness of the anti-vibration hammer, skewness of the spacer bar, non-vertical phases, loose winding and inadequate positioning of the aluminum tape, etc. The annular array camera device is arranged on the robot annular camera array.

[0007] (1) In the target YOLO model, position detection is performed based on the wire image and the quality defects in the internal and external parameter grids of the annular camera matrix device, and the position information of the noted wire quality defects in each grid in the wire image relative to the annular array camera device is output.

[0008] (2) Construct an overhead power line defect detection model based on the improved YOLOV8, including the Backbone module, CDS-Neck module and Head module after adding the attention mechanism; The Backbone module introduces the downsampling module CGNeck_Down for feature extraction and residual learning, and introduces the Triplet-Attention module for cross-dimensional interaction. By capturing the interaction between the spatial dimension and the input tensor channel dimension, the CBAM model based on Spatial and Channel alleviates the problem of spatial interdependence.

[0009] The CDS-Neck module is first introduced into the backbone network through the feature map of the P2 layer to further enhance the feature extraction capability, and the lightweight sampling operator CARAFE module is introduced to improve network performance, reduce one Concat layer, and add a CARAFE layer; and change the original connection method to achieve lightweight while ensuring feature extraction capability.

[0010] The Triplet-Attention module is used to capture more complex feature relationships in images. It improves feature representation capabilities by calculating attention in three different directions: horizontal, vertical, and channel.

[0011] The CIoU loss function in the original YOLOv8 is replaced by the EIoU loss function. By calculating the Euclidean distance between the center point of the predicted box and the true box, the position difference of the bounding box can be more accurately measured. By considering the width and height difference between the predicted box and the true box, the shape and scale mismatch problem can be better handled.

[0012] (3) The dataset is used to train the overhead transmission line defect detection model constructed based on the improved YOLOv8; the trained model is used for detection to realize the detection of various defects in transmission lines.

[0013] Furthermore, the data set in step (1) includes five defects: wire damage and deformation, anti-vibration hammer skew, spacer bar skew, phases are not vertical, and aluminum tape is not tightly wrapped and not in place.

[0014] Furthermore, the image size in the data set in step (1) is scaled and adjusted to 640×640 pixels; the data folder is Data, which contains two subfolders, images and labels, where images stores the training images and labels are stored in the xml files corresponding to the training images; the training set image path is stored in train.txt, which stores the relative paths of the training images and labels.

[0015] Furthermore, the Backbone module in step (2) includes a Conv module, four CGNeck_Down modules, a Triplet-Attention module, four C2f-DySnakeConv modules and a spatial pyramid pooling module SPPF.

[0016] Furthermore, the Backbone module implementation process in step (2) is as follows: The four layers of C2f-DySnakeConv in the backbone network are 3, 5, 7, and 9 respectively. The feature information extracted from the 3, 5, and 7 layers are input into the CDS-Neck module for further fusion of feature information. The input image first enters a Conv layer for feature extraction, and then enters the Triplet-Attention module to effectively process the context information while retaining the local description and outputting it to the Neck layer, and then performs the following loop operations: enters the CGNeck_Down module to capture the context features of all stages from both the semantic level and the spatial level, and performs sampling operations; enters the C2f-DySnakeConv module for feature extraction and outputs it to the Neck layer; repeats this process, and finally enters the SPPF module to perform pooling operations on feature maps of different scales without changing the size of the feature map.

[0017] The CGNeck_Down module includes a local feature extractor floc(*), a surrounding context extractor fsur(*), a joint feature extractor fjoi(*) and a global feature extractor fglo(*); the floc(*) is implemented by conventional convolution calculation; the fsur(*) is implemented by dilated convolution calculation; the fjoi(*) is a concatenation layer, followed by a BN layer and PreLU function activation; the fglo(*) is a global pooling layer followed by two fully connected layers to extract features, and obtain a weight vector, which is used to guide the joint feature fusion.

[0018] Furthermore, CGNeck in the CGNeck_Down module adopts residual learning; two methods, local residual learning and global residual learning, are adopted.

[0019] Furthermore, the Triplet-Attention module is an attention mechanism used in convolutional neural networks to enhance feature expression by performing attention calculations in three orthogonal directions to capture more comprehensive feature information. It includes channel attention, spatial attention, and height attention, and performs global average pooling and global maximum pooling on the input feature map in the spatial dimension (H×W) to obtain two channel description vectors.

[0020] The channel attention calculation process in the Triplet-Attention module is as follows: (1) Global average pooling and maximum pooling: Let the input feature map be X, with a shape of C×H×W; global average pooling: , the shape is C×1×1; global maximum pooling: , the shape is also C×1×1; (2) Fully connected layer: The results of global average pooling and maximum pooling are passed through a fully connected layer with shared weights to obtain the importance score of each channel; , ; (3) Weighted summation and activation: After weighted summation of the output of the fully connected layer, the Sigmoid function is used for activation to obtain the channel attention weight: ; (4) Reweighting: Multiply the input feature map by the channel attention weight to obtain an enhanced feature map: ; The spatial attention calculation process in the Triplet-Attention module is as follows: (1) Channel dimension aggregation: Aggregate the input feature map on the channel dimension (C) to obtain two two-dimensional feature maps; let the input feature map be X, with a shape of C×H×W, and channel average pooling: , shape is 1×H×W; channel maximum pooling: , the shape is also 1×H×W; (2) Convolution calculation: The aggregated two-dimensional feature maps are concatenated and a convolution operation is performed to generate a spatial attention map. The following formula is used in the concatenation process: , the shape is 2×H×W; then the convolution operation is performed, and the shape after the operation is 1×H×W; (3) Activation and reweighting: Activate the attention map through the Sigmoid function to obtain the spatial attention weight: , multiply the input feature map by the spatial attention weight to obtain the enhanced feature map: ; The high attention calculation process in the Triplet-Attention module is as follows: (1) Channel dimension aggregation: Aggregate the input feature map in the height dimension (H) to obtain two height feature maps; let the input feature map be X, with a shape of C×H×W; height average pooling , shape is C×1×W; height maximum pooling , the shape is C×1×W; (2) Convolution calculation: The aggregated height feature maps are spliced ​​and a convolution operation is performed to generate a height attention map. The following formula is used in the splicing process: , the shape is C×2×W; then the convolution operation is performed, and the shape after the operation is C×1×W; (3) Activation and reweighting: Activate the attention map through the Sigmoid function to obtain the spatial attention weight: , multiply the input feature map by the spatial attention weight to obtain the enhanced feature map: ; The outputs of the above three attention modules are weighted summed to obtain the final enhanced feature map.

[0021] The implementation process of the Triplet-Attention module is as follows: The triple attention consists of three parallel branches, two of which are responsible for capturing the cross-dimensional interaction between the channel dimension C and the spatial dimension H or W; the outputs of all three branches are aggregated by a simple averaging method, which solves this defect by capturing the interaction between the spatial dimension and the channel dimension of the input tensor; cross-dimensional interaction is introduced in the triple attention, and the dependencies between the (C, H), (C, W) and (H, W) dimensions of the input tensor are captured by three branches respectively; the Z-pool layer is responsible for reducing the zeroth dimension of the tensor to two dimensions by connecting the average pooling and maximum pooling features; We define triple attention as a three-branch module, namely the triple attention module, which receives an input tensor and outputs a refined tensor of the same shape. Given an input tensor χ∈RC×H×W, it is passed to each of the three branches of the proposed triple attention module: In the first branch, an interaction is established between the height dimension and the channel dimension, rotating the input χ by 90 counterclockwise along the H axis. ◦ ; Then, χ 1 Will pass through Zpool and be restored to χ ∗1 , whose shape is (2×H×C); ∗1 It then passes through a standard convolutional layer with a kernel size of k×k and a batch normalization layer, providing an intermediate output of dimension (1×H×C). The tensor is then passed through a sigmoid activation layer (σ) to generate attention weights. The generated attention weights are then applied to χ 1 , then rotate 90 degrees clockwise along the H axis ◦ , to preserve the original input shape of χ; In the second branch, we rotate χ by 90 degrees counterclockwise about the W axis. ◦ ; The rotated tensor χ 2 The dimension of is (H×C×W) and passes through the Z pooling layer; thus, the tensor is reduced to a χ with shape (2×C×W) ∗2 ; ∗2 After a standard convolutional layer, which is defined by a k×k kernel size, and then a batch normalization layer, the output is a tensor of shape (1×C×W); this tensor is then passed through a sigmoid activation layer (σ) to obtain the attention weights, which are simply applied to χ 2 , then rotate the output 90 degrees clockwise along the W axis ◦ , to keep the same shape as the input χ; In the last branch, the channels of the input tensor χ are reduced to two by Z-pooling; the reduced tensor χ 3 The shape of is (2×H×W), which passes through a standard convolutional layer defined by the kernel size k, followed by a batch normalization layer; the output passes through a sigmoid activation layer (σ) to generate attention weights of shape (1×H×W), which are then applied to the input χ; the shape (C×H×W) refinement tensors generated by the three branches are summarized by simple averaging.

[0022] Furthermore, the implementation process of the CDS-Neck module is as follows: first, the feature maps output by the 3rd, 5th, and 7th layers in the backbone enter the concat layer of the neck to realize the splicing of the input data, and the feature map of the 10th layer first enters the lightweight operator CARAFE upsampling module to enlarge the image, and then enters the Concat layer of the 13th layer for splicing after extracting the features; after the two C2f-DySnakeConv modules of the 14th and 15th layers extract the features, the C2f-DySnakeConv module of the 14th layer first outputs to the Detect head of the head, and then enters the 16th CARAFE layer for upsampling operation, and the output feature image enters the first layer together with the feature image input from the 5th layer. The 17th Concat layer performs the splicing operation, and then enters the 18th C2f-DySnakeConv module for feature extraction and outputs it to the Detect head at the head; then enters the 19th Conv layer convolution layer to reduce the amount of calculation, and then enters the 20th CARAFE layer for upsampling operation, and then enters the 21st Concat layer together with the features input from the 3rd layer for splicing, connecting feature tensors of different sizes to improve the performance of the model; enters the 22nd C2f-DySnakeConv module for feature extraction and fusion; finally, it is output to the Detect head at the head.

[0023] The beneficial effects of the present invention are: 1. Aiming at the situation that the computing power of low-end devices is limited and the prediction accuracy is not high, the present invention replaces the BottleNeck in the Backbone network and the C2f module in the Neck in the YOLOv8 network with the Dynami-Snake module. The C2f_DySnakeConv module achieves high accuracy and is more suitable for focusing on the characteristics of slender defects, which can extract feature information more effectively. 2. In terms of small target detection accuracy, the feature map of the P2 layer in the backbone network is also introduced into the Neck. An upsampling layer is added to the original Neck to enlarge the image size and improve the feature extraction capability, thereby improving the detection accuracy of small targets in defect detection. 3. In terms of overall detection accuracy, the downsampling module of the Backbone network in the YOLOv8 network is replaced with the CGNeck_Down module to perform feature extraction fusion and residual learning to promote information transmission, further enhance the feature extraction capability, reduce the interference of complex background on defect detection, suppress redundant and interfering information, improve the accuracy of the model for defect detection and classification, increase the model's attention to important features, and reduce the complexity of the model; the improved C2f_DySnakeConv module also pays better attention to the characteristics of slender defects, accurately locates defects and retains precise location information; in terms of design, it pays more attention to the modeling of the global context, and uses the large kernel self-attention mechanism to interact information in the entire image, local residual learning (LRL) and global residual learning (GRL). This helps the model better understand the context and association of the entire image, not just the local area, and improves the overall accuracy of defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 This is a schematic diagram of the structure of the overhead transmission line defect detection model based on YOLOv8 multi-scale feature fusion; Figure 2 This is the schematic diagram of the CGNeck_Down module; Figure 3 The schematic diagram of the Triplet-Attention module. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The present invention will be described in further detail below in conjunction with the accompanying drawings.

[0028] like Figure 1As shown, the present invention provides an overhead transmission line defect detection method based on YOLOv8 multi-scale feature fusion, firstly, a video signal is acquired through a camera, the video signal is intercepted into an image, and then the image is preprocessed, and a defect detection model is used to detect defects on the image and the detection result is drawn on the image. Specifically, the following steps are included: Step 1: Use a camera to capture the defect video stream signal on the surface of the wire, intercept the video stream signal into an image, pre-process the image, and form a data set.

[0029] This dataset collects five common defects, including wire damage and deformation, anti-vibration hammer skew, spacer skew, non-vertical phases, and loose aluminum wrapping. There are 300 pictures for each defect, totaling 1500 pictures. The input image size is resized to 640×640 pixels. The data folder is data, which includes two subfolders, images and labels. Images stores training images, and labels stores xml files corresponding to training images. The ratio of training set to test set is 7:3. The training set image path is saved in train.txt; the test set image path is saved in test.txt, which stores the relative path of training images and labels.

[0030] Step 2: Figure 1 As shown in the figure, a wire surface defect detection model based on multi-scale feature fusion of YOLOv8 is constructed, including Backbone module, CDS-Neck module and Head module. The backbone network module introduces the lightweight downsampling module CGNeck_Down to perform feature extraction and residual learning, introduces the Triplet-Attention module for context information interaction, and introduces the C2f- DySnakeConv module to further enhance the feature extraction capability for the features of slender defects. The CDS-Neck module first introduces the feature map of the P2 layer in the backbone network to further enhance the feature extraction capability, introduces the lightweight upsampling operator CARAFE module to improve the network performance, reduces one Concat layer (connection layer), and adds a CARAFE layer. And the original connection method is changed. At the same time, the original C2f module is modified to the C2f- DySnakeConv module, which achieves lightweight while ensuring the feature extraction capability.

[0031] The backbone network includes a Conv module, four CGNeck_Down modules, a Triplet-Attention module, four C2f-DySnakeConv modules and an SPPF module. The backbone network extracts feature information of defects through various modules. As the size of the input image becomes smaller and smaller, more and more feature information is extracted. The four layers of C2f-DySnakeConv (3, 5, 7, and 9 layers) in the backbone network are used to output the feature information extracted by the 3, 5, and 7 layers to the Neck module for further fusion of feature information. The image with the size adjusted to 640×640 pixels is input, first enters a Conv layer for feature extraction, and then enters the Triplet-Attention module to effectively process context information while retaining local descriptions. Then it enters the CGNeck_Down module to capture context features of all stages from both the semantic level and the spatial level, and performs sampling operations (increasing the receptive field and reducing the amount of calculation), then enters the CGNeck_Down module for downsampling operations, and then enters the C2f-DySnakeConv module for feature extraction and outputs to the Neck layer. This process is repeated, and finally the SPPF module (spatial pyramid pooling) is entered to perform pooling operations on feature maps of different scales without changing the size of the feature map, thereby improving the detection performance.

[0032] like Figure 2 As shown in the figure, the CGNeck_Down module consists of 4 parts: 1) Local feature extractor floc(*): Use standard convolutional layers to learn local features. 2) Surrounding context feature extractor fsur(*): Use dilated convolution (DConv) to learn surrounding context with a larger receptive field. 3〉Joint feature extractor fjoi(*): Use connection layers and batch normalization (BN) and parameterized ReLU (PReLU) operations to fuse the outputs of local features and surrounding context to obtain joint features. 4〉Global feature extractor fglo(*): Context guidance block: The global average pooling layer (GAP layer) is followed by two fully connected layers (FC layers) to extract features, and a weight vector can be obtained to guide the fusion of joint features. The global residual learning (GRL) method is also used here to help learn complex features and improve the back propagation of gradients during training.

[0033] The channel attention calculation process in the Triplet-Attention module is as follows: (1) Global average pooling and maximum pooling: Let the input feature map be X, with a shape of C×H×W; global average pooling: , the shape is C×1×1; global maximum pooling: , the shape is also C×1×1; (2) Fully connected layer: The results of global average pooling and maximum pooling are passed through a fully connected layer with shared weights to obtain the importance score of each channel; , ; (3) Weighted summation and activation: After weighted summation of the output of the fully connected layer, the Sigmoid function is used for activation to obtain the channel attention weight: ; (4) Reweighting: Multiply the input feature map by the channel attention weight to obtain an enhanced feature map: ; The spatial attention calculation process in the Triplet-Attention module is as follows: (1) Channel dimension aggregation: Aggregate the input feature map on the channel dimension (C) to obtain two two-dimensional feature maps; let the input feature map be X, with a shape of C×H×W, and channel average pooling: , shape is 1×H×W; channel maximum pooling: , the shape is also 1×H×W; (2) Convolution calculation: The aggregated two-dimensional feature maps are concatenated and a convolution operation is performed to generate a spatial attention map. The following formula is used in the concatenation process: , the shape is 2×H×W; then the convolution operation is performed, and the shape after the operation is 1×H×W; (3) Activation and reweighting: Activate the attention map through the Sigmoid function to obtain the spatial attention weight: , multiply the input feature map by the spatial attention weight to obtain the enhanced feature map: ; The high attention calculation process in the Triplet-Attention module is as follows: (1) Channel dimension aggregation: Aggregate the input feature map in the height dimension (H) to obtain two height feature maps; let the input feature map be X, with a shape of C×H×W; height average pooling , shape is C×1×W; height maximum pooling , the shape is C×1×W; (2) Convolution calculation: The aggregated height feature maps are spliced ​​and a convolution operation is performed to generate a height attention map. The following formula is used in the splicing process: , the shape is C×2×W; then the convolution operation is performed, and the shape after the operation is C×1×W; (3) Activation and reweighting: Activate the attention map through the Sigmoid function to obtain the spatial attention weight: , multiply the input feature map by the spatial attention weight to obtain the enhanced feature map: ; The outputs of the above three attention modules are weighted summed to obtain the final enhanced feature map.

[0034] Furthermore, the implementation process of the Triplet-Attention module is as follows: The triple attention consists of three parallel branches, two of which are responsible for capturing the cross-dimensional interaction between the channel dimension C and the spatial dimension H or W; the outputs of all three branches are aggregated by a simple averaging method, which solves this defect by capturing the interaction between the spatial dimension and the channel dimension of the input tensor; cross-dimensional interaction is introduced in the triple attention, and the dependencies between the (C, H), (C, W) and (H, W) dimensions of the input tensor are captured by three branches respectively; the Z-pool layer is responsible for reducing the zeroth dimension of the tensor to two dimensions by connecting the average pooling and maximum pooling features; We define triple attention as a three-branch module, namely the triple attention module, which receives an input tensor and outputs a refined tensor of the same shape. Given an input tensor χ∈RC×H×W, it is passed to each of the three branches of the proposed triple attention module: In the first branch, an interaction is established between the height dimension and the channel dimension, rotating the input χ by 90 counterclockwise along the H axis. ◦ ; Then, χ 1 Will pass through Zpool and be restored to χ ∗1 , whose shape is (2×H×C); ∗1 It then passes through a standard convolutional layer with a kernel size of k×k and a batch normalization layer, providing an intermediate output of dimension (1×H×C). The tensor is then passed through a sigmoid activation layer (σ) to generate attention weights. The generated attention weights are then applied to χ 1 , then rotate 90 degrees clockwise along the H axis ◦ , to preserve the original input shape of χ; In the second branch, we rotate χ by 90 degrees counterclockwise about the W axis. ◦ ; The rotated tensor χ 2 The dimension of is (H×C×W) and passes through the Z pooling layer; thus, the tensor is reduced to a χ with shape (2×C×W) ∗2 ; ∗2 After a standard convolutional layer, which is defined by a k×k kernel size, and then a batch normalization layer, the output is a tensor of shape (1×C×W); this tensor is then passed through a sigmoid activation layer (σ) to obtain the attention weights, which are simply applied to χ 2 , then rotate the output 90 degrees clockwise along the W axis◦ , to keep the same shape as the input χ; In the last branch, the channels of the input tensor χ are reduced to two by Z-pooling; the reduced tensor χ 3 The shape of is (2×H×W), which passes through a standard convolutional layer defined by the kernel size k, followed by a batch normalization layer; the output passes through a sigmoid activation layer (σ) to generate attention weights of shape (1×H×W), which are then applied to the input χ; the shape (C×H×W) refinement tensors generated by the three branches are summarized by simple averaging.

[0035] Step 3: Use the data set to train the wire surface defect detection model constructed based on the improved YOLOv8; use the trained model to perform detection to realize wire surface defect detection.

[0036] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, the above is only a preferred embodiment of the present invention. Since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited to this. Any technician familiar with the technical field is within the technical scope disclosed by the present invention. For ordinary technicians in the technical field, changes or replacements that can be easily thought of should be covered within the protection scope of the present invention without departing from the principle of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for detecting defects in overhead transmission lines based on multi-scale feature fusion, characterized in that The following steps are involved: 1) Obtain the conductor image and laser point cloud image of the overhead transmission line, calibrate and register the conductor image and laser point cloud image, perform data enhancement, and build a detection dataset; 2) Inputting the internal and external parameters of the annular array camera device used to obtain the conductor image and the laser cloud point image into the pre-trained target YOLO model, gridding the conductor image, and performing target detection on each grid to output the position information of the conductor quality defect noted in each grid in the conductor image relative to the annular array camera device; 3) Build a wire surface defect detection model based on multi-scale feature fusion of YOLOv8, including Backbone module, CDS-Neck module and Head module; replace the CIoU loss function in the original YOLOv8 with the EIoU loss function, measure the position difference of the bounding box by calculating the Euclidean distance between the center point of the predicted box and the true box, and deal with the shape and scale mismatch problem by considering the width and height difference between the predicted box and the true box; The Backbone module introduces the downsampling module CGNeck_Down for feature extraction and residual learning, and introduces the Triplet-Attention module for cross-dimensional interaction to capture the interaction between the spatial dimension and the input tensor channel dimension, and improves the feature representation capability by calculating attention in three different directions: horizontal, vertical, and channel. The CDS-Neck module is first introduced into the backbone network through the feature map of the P2 layer to enhance the feature extraction capability, and the lightweight sampling operator CARAFE module is introduced to improve the network performance, reduce one Concat layer, add one CARAFE layer, and change the original connection mode; 4) The constructed wire surface defect detection model is trained using the data set, and the trained model is used for detection to realize wire surface defect detection.

2. The method for detecting defects in overhead power transmission lines based on multi-scale feature fusion according to claim 1, characterized in that: Step 1) The conductor image is obtained by photographing a circular array camera device arranged on the robot circular camera array; the data set includes five defects: conductor damage and deformation, anti-vibration hammer skew, spacer bar skew, each phase is not vertical, and aluminum tape is not tightly wrapped and not in place.

3. The method for detecting defects in overhead power transmission lines based on multi-scale feature fusion according to claim 1, characterized in that: Step 2) The position information includes the center position information of the edge rectangle frame of the quality defect in the grid, width and height information, confidence information and target category information.

4. The method for detecting defects in overhead power transmission lines based on multi-scale feature fusion according to claim 1, characterized in that: Step 3) The Backbone module specifically includes a Conv module, four CGNeck_Down modules, a Triplet-Attention module, three C2f-DySnakeConv modules and a spatial pyramid pooling module SPPF.

5. The method for detecting defects in overhead power transmission lines based on multi-scale feature fusion according to claim 4 is characterized in that: The implementation process of the Backbone module is as follows: the four layers of C2f-DySnakeConv in the backbone network are 3, 5, 7, and 9 respectively. The feature information extracted from the 3, 5, and 7 layers is input into the CDS-Neck module for further fusion of the feature information; the input image first enters a Conv layer for feature extraction, and then enters the Triplet-Attention module to effectively process the context information while retaining the local description and outputting it to the Neck layer, and then performs the following loop operations: enters the CGNeck_Down module to capture the context features of all stages from both the semantic level and the spatial level, and performs sampling operations; enters the C2f-DySnakeConv module to extract features and output them to the Neck layer; repeats this process, and finally enters the SPPF module to perform pooling operations on feature maps of different scales without changing the size of the feature map.

6. The method for detecting defects in overhead power transmission lines based on multi-scale feature fusion according to claim 4 or 5, characterized in that: The CGNeck_Down module includes a local feature extractor floc(*), a surrounding context extractor fsur(*), a joint feature extractor fjoi(*) and a global feature extractor fglo(*); the floc(*) is implemented by conventional convolution calculation; the fsur(*) is implemented by dilated convolution calculation; the fjoi(*) is a concatenation layer, followed by a BN layer and PreLU function activation; the fglo(*) is a global pooling layer followed by two fully connected layers to extract features, and obtain a weight vector, which is used to guide the joint feature fusion.

7. The method for detecting defects in overhead power transmission lines based on multi-scale feature fusion according to claim 4 or 5, characterized in that: In the CGNeck_Down module, CGNeck adopts residual learning, including local residual learning and global residual learning.

8. The method for detecting defects in overhead power transmission lines based on multi-scale feature fusion according to claim 4 or 5, characterized in that: The Triplet-Attention module is used in the convolutional neural network to enhance the attention mechanism of feature expression, and captures more comprehensive feature information by performing attention calculations in three orthogonal directions respectively; Including channel attention, spatial attention, and height attention, global average pooling and global maximum pooling are performed on the input feature map in the spatial dimension (H×W) to obtain two channel description vectors; The channel attention calculation process in the Triplet-Attention module is as follows: (1) Global average pooling and maximum pooling: Let the input feature map be X, with a shape of C×H×W; global average pooling: , the shape is C×1×1; global maximum pooling: , the shape is also C×1×1; (2) Fully connected layer: The results of global average pooling and maximum pooling are passed through a fully connected layer with shared weights to obtain the importance score of each channel; , ; (3) Weighted summation and activation: After weighted summation of the output of the fully connected layer, the Sigmoid function is used for activation to obtain the channel attention weight: ; (4) Reweighting: Multiply the input feature map by the channel attention weight to obtain an enhanced feature map: ; The spatial attention calculation process in the Triplet-Attention module is as follows: (1) Channel dimension aggregation: Aggregate the input feature map on the channel dimension (C) to obtain two two-dimensional feature maps; let the input feature map be X, with a shape of C×H×W, and channel average pooling: , shape is 1×H×W; channel maximum pooling: , the shape is also 1×H×W; (2) Convolution calculation: The aggregated two-dimensional feature maps are concatenated and a convolution operation is performed to generate a spatial attention map. The following formula is used in the concatenation process: , the shape is 2×H×W; then the convolution operation is performed, and the shape after the operation is 1×H×W; (3) Activation and reweighting: Activate the attention map through the Sigmoid function to obtain the spatial attention weight: , multiply the input feature map by the spatial attention weight to obtain the enhanced feature map: ; The high attention calculation process in the Triplet-Attention module is as follows: (1) Channel dimension aggregation: Aggregate the input feature map in the height dimension (H) to obtain two height feature maps; let the input feature map be X, with a shape of C×H×W; height average pooling , shape is C×1×W; height maximum pooling , the shape is C×1×W; (2) Convolution calculation: The aggregated height feature maps are spliced ​​and a convolution operation is performed to generate a height attention map. The following formula is used in the splicing process: , the shape is C×2×W; then the convolution operation is performed, and the shape after the operation is C×1×W; (3) Activation and reweighting: Activate the attention map through the Sigmoid function to obtain the spatial attention weight: , multiply the input feature map by the spatial attention weight to obtain the enhanced feature map: ; The outputs of the above three attention modules are weighted summed to obtain the final enhanced feature map.

9. The method for detecting defects in overhead power transmission lines based on multi-scale feature fusion according to claim 8, characterized in that: The implementation process of the Triplet-Attention module is as follows: The triple attention consists of three parallel branches, two of which are responsible for capturing the cross-dimensional interaction between the channel dimension C and the spatial dimension H or W; the outputs of all three branches are aggregated by a simple averaging method, which solves this defect by capturing the interaction between the spatial dimension and the channel dimension of the input tensor; cross-dimensional interaction is introduced in the triple attention, and the dependencies between the (C, H), (C, W) and (H, W) dimensions of the input tensor are captured by three branches respectively; the Z-pool layer is responsible for reducing the zeroth dimension of the tensor to two dimensions by connecting the average pooling and maximum pooling features; We define triple attention as a three-branch module, namely the triple attention module, which receives an input tensor and outputs a refined tensor of the same shape. Given an input tensor χ∈RC×H×W, it is passed to each of the three branches of the proposed triple attention module: In the first branch, an interaction is established between the height dimension and the channel dimension, rotating the input χ by 90 counterclockwise along the H axis. ◦ ; Then, χ 1 Will pass through Zpool and be restored to χ ∗1 , whose shape is (2×H×C); ∗1 It then passes through a standard convolutional layer with a kernel size of k×k and a batch normalization layer to provide an intermediate output of dimension (1×H×C). The tensor is then passed through a sigmoid activation layer (σ) to generate attention weights. The generated attention weights are then applied to χ 1 , then rotate 90 degrees clockwise along the H axis ◦ , to preserve the original input shape of χ; In the second branch, we rotate χ by 90 degrees counterclockwise about the W axis. ◦ ; The rotated tensor χ 2 The dimension of is (H×C×W) and passes through the Z pooling layer; thus, the tensor is reduced to a χ with shape (2×C×W) ∗2 ; ∗2 After a standard convolutional layer, which is defined by a k×k kernel size, and then a batch normalization layer, the output is a tensor of shape (1×C×W); this tensor is then passed through a sigmoid activation layer (σ) to obtain the attention weights, which are simply applied to χ 2 , then rotate the output 90 degrees clockwise along the W axis ◦ , to keep the same shape as the input χ; In the last branch, the channels of the input tensor χ are reduced to two by Z-pooling; the reduced tensor χ 3 The shape of is (2×H×W), which passes through a standard convolutional layer defined by the kernel size k, followed by a batch normalization layer; the output passes through a sigmoid activation layer (σ) to generate attention weights of shape (1×H×W), which are then applied to the input χ; the shape (C×H×W) refinement tensors generated by the three branches are summarized by simple averaging.

10. The method for detecting defects in overhead power transmission lines based on multi-scale feature fusion according to claim 1, characterized in that: The implementation process of the CDS-Neck module described in step 3) is as follows: first, the feature maps output by the 3rd, 5th, and 7th layers in the backbone enter the concat layer of the neck to realize the splicing of the input data. The feature map of the 10th layer first enters the lightweight operator CARAFE upsampling module to enlarge the image, extracts the features, and then enters the Concat layer of the 13th layer for splicing; after the features are extracted by the two C2f-DySnakeConv modules of the 14th and 15th layers, the C2f-DySnakeConv module of the 14th layer first outputs to the Detect head of the head, and then enters the 16th CARAFE layer for upsampling operation, and the output feature image enters the first layer together with the feature image input from the 5th layer. The 17th Concat layer performs the splicing operation, and then enters the 18th C2f-DySnakeConv module for feature extraction and outputs it to the Detect head at the head; then enters the 19th Conv layer convolution layer to reduce the amount of calculation, and then enters the 20th CARAFE layer for upsampling operation, and then enters the 21st Concat layer together with the features input from the 3rd layer for splicing, connecting feature tensors of different sizes to improve the performance of the model; enters the 22nd C2f-DySnakeConv module for feature extraction and fusion; finally, it is output to the Detect head at the head.

Images