Unmanned aerial vehicle defect detection method based on channel space fusion cross attention

Abstract

The invention relates to an unmanned aerial vehicle defect detection method based on channel space fusion cross attention. The method comprises the following steps: planning a waypoint sampling route of a tower pole in a power transmission line; a high-resolution visible light image shot by the unmanned aerial vehicle at a low altitude is obtained and preprocessed; improving the YOLOv8 network model to obtain a YOLOv8-based unmanned aerial vehicle defect detection model, and training and verifying the YOLOv8-based unmanned aerial vehicle defect detection model; and identifying the defect positions and types of the copper-aluminum transition wire clamp and the pin missing in the tower. According to the invention, a multi-scale feature extraction network in a YOLOv8 network model is replaced by an ESSF module to solve the problem of severe scale change, shallow information and deep information are fully fused, more small target information can be captured, and a small target detection layer is added to enhance the perception ability of the model to a small object; a CSFCA attention mechanism module is integrated, global and local information can be fully utilized, background noise can be filtered, key information can be highlighted, and the efficiency of defect detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) power inspection, and in particular to a UAV defect detection method based on channel space fusion cross attention. Background Art

[0002] As the scale of the power grid continues to expand, the construction of power equipment is also increasing, and the number and scale of equipment that needs to be maintained are gradually increasing. The equipment components in high-voltage power lines are complex and widely distributed, especially in areas with harsh outdoor environments, which brings many challenges to traditional manual inspection methods. Traditional inspections rely on manual operations and manual records, which not only consume a lot of manpower, but also easily lead to problems such as untimely information transmission, low inspection efficiency, and high costs. In addition, due to the high-risk environment and operation scenarios involved in the inspection process, safety accidents occur frequently, making the disadvantages of manual inspections more and more obvious. By using drones for power inspections, the efficiency and safety of inspections are improved, while reducing costs.

[0003] Object detection from the perspective of drones faces many unique challenges, including the dense distribution of a large number of small objects, significant changes in object scale, and complex background noise. These problems greatly increase the difficulty of object detection in drone images, making it difficult for traditional detectors used in natural scenes to be directly applicable to such scenes. Especially in drone images, the key difficulty lies in how to deal with small objects smaller than 32 pixels and the significant changes in object scale in different scenes. In addition, the complex background noise and sparse target features in drone datasets further increase the difficulty of detection. To solve this problem, Lim et al. proposed a scheme based on an attention mechanism, which aims to filter out environmental noise information and focus on the feature information of small objects. However, existing attention mechanisms, such as CBAM, SE, CA, ECA, etc., are usually processed by pooling and aggregating global feature information. Although the global information is enhanced, it may also cause the feature information of small objects to be masked by complex backgrounds. Most of these existing attention mechanism methods rely on the independent effects of channel attention and spatial attention, and fail to fully explore the potential correlation between channel information and spatial information. Therefore, the ability to focus on small objects in complex backgrounds is still limited.

[0004] Therefore, in the task of drone image target detection, a more effective technical solution is urgently needed to cope with challenges such as dense distribution of tiny targets, scale changes and background interference, so as to improve the performance of detectors in complex scenes. Summary of the invention

[0005] In view of the problems of dense distribution of tiny targets, significant changes in target scales, and serious interference from complex background noise in UAV defect detection, the purpose of the present invention is to provide a UAV defect detection method based on channel space fusion and cross-attention, which can enhance the model's perception of tiny objects by adding a small target detection layer, filter background noise and highlight key information, and improve the efficiency of defect detection.

[0006] To achieve the above object, the present invention adopts the following technical solution: a method for detecting defects of unmanned aerial vehicles based on channel space fusion cross attention, the method comprising the following steps in order:

[0007] (1) Planning the waypoint sampling route of the transmission line tower: using a drone equipped with a high-pixel, zoom visible light camera, and planning the waypoint flight sampling points and paths for the towers at different heights and locations;

[0008] (2) Obtain high-resolution visible light images taken by a drone at low altitude, preprocess the high-resolution visible light images, and divide the preprocessed images into a test set and a training set;

[0009] (3) The YOLOv8 network model is improved to obtain a YOLOv8-based UAV defect detection model, wherein the YOLOv8-based UAV defect detection model includes a backbone network, an ESSF module, a CSFCA attention mechanism module, and a detection head; the training set is input into the YOLOv8-based UAV defect detection model for training, and the test set is used to verify the accuracy of the YOLOv8-based UAV defect detection model;

[0010] (4) Identify the defect locations and categories of copper-aluminum excessive wire clips and missing pins in pole towers: Input the high-resolution visible light image taken by the drone to be inspected into the YOLOv8-based drone defect detection model to output the specific locations of the copper-aluminum excessive wire clips and missing pins.

[0011] In step (1), the effective pixel of the zoom visible light camera is better than 20 million, and the zoom ratio is greater than or equal to 4 times; the sampling points are set by an average sampling method, and 25 sampling points are set in a 5*5 manner; the parameters of the sampling points include serial number, longitude, latitude, altitude, direction angle and zoom ratio.

[0012] The step (2) specifically comprises the following steps:

[0013] (2a) Two photos were taken at each sampling point for later selection of available high-resolution photos;

[0014] (2b) Annotate the copper-aluminum excessive wire clips and missing pins on the high-resolution visible light images;

[0015] (2c) The labeled high-resolution visible light images are preprocessed by horizontal mirror flipping, vertical mirror flipping, mosaic data enhancement, mixed enhancement, and random perturbation. The images of all sampling points after preprocessing are evenly divided into 5 groups, 4 of which are used as training sets and the remaining 1 is used as a test set.

[0016] In step (3), the improvement of the YOLOv8 network model specifically includes the following steps:

[0017] (3a) The YOLOv8 network model is improved, wherein the YOLOv8 network model includes a backbone network, a multi-scale feature extraction network and a detection head; the SSFF module and the TFE module together form a multi-scale feature fusion module, and the P2 feature layer is added to the SSFF module. At this time, the multi-scale feature fusion module forms an ESSF module; the multi-scale feature extraction network in the YOLOv8 network model is replaced by the ESSF module;

[0018] (3b) A CSFCA attention mechanism module is added to the detection head of the YOLOv8 network model. The CSFCA attention mechanism module is added before each detection head. The CSFCA attention mechanism module includes a local attention calculation module and a global attention calculation module. The local attention calculation module includes multiple patch block cutting modules, multiple CSF modules and multiple patch block splicing modules. The number of the patch block cutting modules, the number of the CSF modules and the number of the patch block splicing modules are all p; the global attention calculation module includes a CSF module and an anti-pooling layer;

[0019] First, the input feature F(B, H, W, C) is cut through the Patch block cutting module, where B represents the number of samples in each batch, H and W represent the height and width of the image respectively, and C represents the number of channels of the image. The input feature F is cut into patches of the same size, the number of patches is set to p, and the feature dimension of each patch is (B, H / / p, W / / p, C). Then, they are spliced ​​on the first dimension, B, and the feature dimension after splicing is (B*p*p, H / / p, W / / p, C). Then, each patch block is calculated for local attention by the CSF module, and the patch block features obtained by cutting the input feature F are multiplied to obtain the weighted feature block to focus on the detailed features in the local range. Then, the patch features after local attention calculation and weighting are spliced ​​and restored to F through the Patch block splicing module. ′ (B, H, W, C); then perform global attention calculation, perform attention calculation between each patch block, and calculate F ′The patch blocks in the image are pooled separately, and the feature dimension is F″(B, p, p, C). The global attention is calculated using the CSF module, and the unpooling operation is performed to make the dimension of the attention map the same as the input feature F, thereby ensuring that the attention weights of the local area are consistent. ′ Perform a dot multiplication with the global attention to obtain the final output feature map after weighting by local attention and global attention calculation.

[0020] The step (4) specifically comprises the following steps:

[0021] (4a) The backbone network of the UAV defect detection model based on YOLOv8 is used to extract features from the defect image and generate a series of feature maps;

[0022] (4b) Using the ESSF module to combine multi-scale information, multi-scale features are obtained;

[0023] (4c) Using the CSFCA attention mechanism module to filter background information of the extracted multi-scale features and highlight key features;

[0024] (4d) Four detection heads are used, corresponding to four different scales: (20,20), (40,40), (80,80) and (160,160), to obtain detection results of objects of different sizes. Subsequently, non-maximum suppression is used to filter multiple detection results, remove prediction boxes with high overlap, and retain the bounding box with the highest confidence. The detection head outputs the bounding box and category confidence of the target. The bounding box and category confidence of the target are the detection results, where the bounding box defines the position of the target in the image.

[0025] In step (3b), the CSF module works as follows: First, for a given input feature F∈R C×H×W , where H and W represent the height and width of the image, respectively, and C represents the number of channels of the image. Pooling is performed in the spatial direction and channel direction to obtain spatial descriptors respectively. and channel descriptors Space Descriptor Aggregate all channel information, channel descriptor Aggregate all spatial information; then Reshape to (1, 1, H*W), after reshape and Concatenate in the channel dimension to get And through 1x1 two-dimensional convolution, the dimension is reduced to obtain the intermediate feature map F∈R 1 ×1×(C+H*W) / r, r is the reduction factor; then, batch normalization and ReLU function are used to encode the potential connection between spatial information and channel information, and then the dimension is restored through 1x1 two-dimensional convolution, and it is split into spatial descriptors and channel descriptors in the channel dimension; one-dimensional convolution is used to calculate the channel attention of the channel descriptor, and then the Sigmoid activation function is used to finally obtain the channel attention F C ∈R 1×1×C ; Use 3x3 two-dimensional convolution to calculate the spatial attention of the spatial descriptor, and generate the spatial attention F through the Sigmoid activation function S ∈R H×W×1 ; Final F C and F S Perform point multiplication to obtain channel space fusion attention, expressed as F CS ∈R H×W×C ; The CSF module is used within the patch block to calculate the local attention, and the CSF module is used between the patch blocks to calculate the global attention.

[0026] It can be seen from the above technical scheme that the beneficial effects of the present invention are: first, the present invention uses the YOLOv8 model as the basis for improvement, and proposes a UAV defect detection model based on YOLOv8, and replaces the multi-scale feature extraction network in the YOLOv8 network model with the ESSF module to solve the problem of drastic scale changes, fully integrates shallow information and deep information, can capture more small target information, and increase the small target detection layer to enhance the model's perception of tiny objects; second, the CSFCA attention mechanism module is integrated in the detection head, which can make full use of global and local information, filter background noise and highlight key information; third, the present invention realizes the automatic detection of defects in power line components, which has the advantages of fast, convenient, accurate and objective, improves the efficiency of defect detection, reduces work costs, and improves the engineering and automation level of traditional manual inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flow chart of the method of the present invention;

[0028] Figure 2 This is a schematic diagram of the detection process of the drone defect detection model based on YOLOv8;

[0029] Figure 3 This is the framework diagram of the drone defect detection model based on YOLOv8;

[0030] Figure 4 It is the structural diagram of the ESSF module;

[0031] Figure 5 This is the structural diagram of the CSFCA attention mechanism module;

[0032] Figure 6 It is the structural diagram of the CSF channel space fusion method;

[0033] Figure 7 This is a comparison chart of the experimental PR of the drone defect detection model based on YOLOv8 and the YOLOv8 series;

[0034] Figure 8 It is a graph of the test results of different target detection algorithms on the data set. DETAILED DESCRIPTION

[0035] like Figure 1 , Figure 2 As shown, a method for detecting defects in unmanned aerial vehicles based on channel space fusion cross attention is provided, and the method comprises the following steps in order:

[0036] (1) Planning the waypoint sampling route of the transmission line tower: using a drone equipped with a high-pixel, zoom visible light camera, and planning the waypoint flight sampling points and paths for the towers at different heights and locations;

[0037] (2) Obtain high-resolution visible light images taken by a drone at low altitude, preprocess the high-resolution visible light images, divide the preprocessed images into a test set and a training set, and use the test set to verify the accuracy of the model;

[0038] (3) The YOLOv8 network model is improved to obtain a YOLOv8-based drone defect detection model, namely, the CSFCANet model. The YOLOv8-based drone defect detection model includes a backbone network, an ESSF module, a CSFCA attention mechanism module, and a detection head. The training set is input into the YOLOv8-based drone defect detection model for training, and the test set is used to verify the accuracy of the YOLOv8-based drone defect detection model.

[0039] (4) Identify the defect locations and categories of copper-aluminum excessive wire clips and missing pins in pole towers: Input the high-resolution visible light image taken by the drone to be inspected into the YOLOv8-based drone defect detection model to output the specific locations of the copper-aluminum excessive wire clips and missing pins.

[0040] In step (1), the effective pixel of the zoom visible light camera is better than 20 million, and the zoom ratio is greater than or equal to 4 times; the sampling points are set by an average sampling method, and 25 sampling points are set in a 5*5 manner; the parameters of the sampling points include serial number, longitude, latitude, altitude, direction angle and zoom ratio.

[0041] The step (2) specifically comprises the following steps:

[0042] (2a) Two photos were taken at each sampling point for later selection of available high-resolution photos;

[0043] (2b) Annotate the copper-aluminum excessive wire clips and missing pins on the high-resolution visible light images;

[0044] (2c) The labeled high-resolution visible light images are preprocessed by horizontal mirror flipping, vertical mirror flipping, mosaic data enhancement, mixed enhancement, and random perturbation. The images of all sampling points after preprocessing are evenly divided into 5 groups, 4 of which are used as training sets and the remaining 1 is used as a test set.

[0045] In step (3), if Figure 3 As shown, the improvement of the YOLOv8 network model specifically includes the following steps:

[0046] (3a) The YOLOv8 network model is improved, and the YOLOv8 network model includes a backbone network, a multi-scale feature extraction network and a detection head; the SSFF module and the TFE module together constitute a multi-scale feature fusion module, the SSFF module (Scalesequence feature fusion module) is a scale sequence feature fusion module, and the TFE module (Triple feature encoding module) is a triple feature encoding module. The P2 feature layer is added to the SSFF module, and the multi-scale feature fusion module forms an ESSF module; the multi-scale feature extraction network in the YOLOv8 network model is replaced by the ESSF module, as shown in FIG. Figure 4 As shown in the figure; the ESSF module is used to obtain rich and valuable multi-scale feature information, which enhances the network's ability to extract features from tiny objects; the SSFF module is used to extract scale-invariant features of the target, and the TFE module splices three different sizes of features, namely large, medium and small, in the spatial dimension to perform feature fusion to capture detailed information about small targets; the ESSF module also adds a tiny target detection layer, which can effectively enhance the network's perception of tiny objects. This layer is specially designed to capture the detailed features of tiny objects in images, and by processing high-resolution image information, it can more keenly perceive the existence of tiny objects.

[0047] (3b) To address the problem of complex background noise of the target, a CSFCA attention mechanism module is added to the detection head of the YOLOv8 network model, such as Figure 5As shown, the CSFCA attention mechanism module is added before each detection head. The CSFCA attention mechanism module includes a local attention calculation module and a global attention calculation module. The local attention calculation module includes multiple patch block cutting modules, multiple CSF modules and multiple patch block splicing modules. The number of the patch block cutting modules, the CSF modules and the patch block splicing modules is p; the global attention calculation module includes a CSF module and an anti-pooling layer;

[0048] First, the input feature F(B, H, W, C) is cut through the Patch block cutting module, where B represents the number of samples in each batch, H and W represent the height and width of the image respectively, and C represents the number of channels of the image. The input feature F is cut into patches of the same size, the number of patches is set to p, and the feature dimension of each patch is (B, H / / p, W / / p, C). Then, they are spliced ​​on the first dimension, B, and the feature dimension after splicing is (B*p*p, H / / p, W / / p, C). Then, each patch block is calculated for local attention by the CSF module, and the patch block features obtained by cutting the input feature F are multiplied to obtain the weighted feature block to focus on the detailed features in the local range. Then, the patch features after local attention calculation and weighting are spliced ​​and restored to F through the Patch block splicing module. ′ (B, H, W, C); then perform global attention calculation, perform attention calculation between each patch block, and calculate F ′ The patch blocks in the image are pooled separately, and the feature dimension is F″(B, p, p, C). The global attention is calculated using the CSF module, and the unpooling operation is performed to make the dimension of the attention map the same as the input feature F, thereby ensuring that the attention weights of the local area are consistent. ′ Perform a dot multiplication with the global attention to obtain the final output feature map after weighting by local attention and global attention calculation.

[0049] The step (4) specifically comprises the following steps:

[0050] (4a) The backbone network of the UAV defect detection model based on YOLOv8 is used to extract features from the defect image and generate a series of feature maps;

[0051] (4b) Using the ESSF module to combine multi-scale information, multi-scale features are obtained;

[0052] (4c) Using the CSFCA attention mechanism module to filter background information of the extracted multi-scale features and highlight key features;

[0053] (4d) Four detection heads are used, corresponding to four different scales: (20,20), (40,40), (80,80) and (160,160), to obtain detection results of objects of different sizes. Subsequently, non-maximum suppression is used to filter multiple detection results, remove prediction boxes with high overlap, and retain the bounding box with the highest confidence. The detection head outputs the bounding box and category confidence of the target. The bounding box and category confidence of the target are the detection results, where the bounding box defines the position of the target in the image.

[0054] like Figure 6 As shown, in step (3b), the working method of the CSF module is as follows: First, for a given input feature F∈R C×H×W , where H and W represent the height and width of the image, respectively, and C represents the number of channels of the image. Pooling is performed in the spatial direction and channel direction to obtain spatial descriptors respectively. and channel descriptors Space Descriptor Aggregate all channel information, channel descriptor Aggregate all spatial information; then Reshape to (1, 1, H*W), after reshape and Concatenate in the channel dimension to get And through 1x1 two-dimensional convolution, the dimension is reduced to obtain the intermediate feature map F∈R 1 ×1×(C+H*W) / r , r is the reduction factor, which is used for feature compression. Then, batch normalization and ReLU function are used to encode the potential connection between spatial information and channel information, and then the dimension is restored by 1x1 two-dimensional convolution, and it is split into spatial descriptors and channel descriptors in the channel dimension. One-dimensional convolution is used to calculate the channel attention of the channel descriptor, and then the Sigmoid activation function is used to finally obtain the channel attention F. C ∈R 1×1×C ; Use 3x3 two-dimensional convolution to perform spatial attention calculation on the spatial descriptor to highlight important spatial features and suppress less important spatial information. Generate spatial attention F through Sigmoid activation function S ∈R H×W×1 ; Final F C and F S Perform point multiplication to obtain channel space fusion attention, expressed as F CS ∈R H×W×C ; The CSF module is used within the patch block to calculate the local attention, and the CSF module is used between the patch blocks to calculate the global attention.

[0055] In the YOLOv8 network model, there are only three detection heads, P3 (80*80), P4 (40*40), and P5 (20*20), which are prone to missed detection and false detection for detecting small targets. In order to solve this problem, the present invention adds a small object prediction head P2 (160*160) detection head to the YOLOv8 network model to enhance the network's perception and detection performance for small targets, and by increasing the fine-grained receptive field, better capture target detail information and achieve more accurate detection.

[0056] In order to solve the problem of significant changes in the scale of targets in drone aerial images, traditional methods often rely on feature pyramid structures for feature fusion. However, these techniques are often implemented through simple summation or cascading, and fail to fully utilize the relationship between different feature maps. To solve this problem, the ESSF module is proposed. The ESSF module extracts scale-invariant semantic features (P3*) from multi-scale feature maps such as P3, P4, and P5 through the SSFF module. The SSFF module effectively integrates the high-dimensional information of deep features and the detailed information of shallow features by horizontally stacking features and applying 3D convolution, realizing the extraction of scale sequence features. Although the SSFF module has made progress in feature fusion, there is still room for improvement in capturing the detailed information of small targets. In particular, the features of the P2 layer are particularly critical for small target detection. The ASF attention scale sequence fusion does not fully tap the potential of the P2 layer and fails to extract scale-invariant features from it. Based on ASF, the ESSF module particularly strengthens the utilization of the P2 layer features. By combining the P3* and P4* features, the scale-invariant features P2* of the P2 layer are further extracted. This improvement significantly enhances the ability to extract high-resolution image information and increases its applicability for small target detection while maintaining feature richness and representation capabilities.

[0057] Table 1 Performance comparison of YOLOv8s and YOLOv8s-ESSF

[0058]

[0059] By using the ESSF module, we can extract the multi-scale invariant feature information of small targets, and add a small target detection layer, which improves the detection of dense targets and small targets. As shown in Table 1, mAP 50 Improved by 12.3%, mAP 50-95 The improvement is 6.4%, and the number of parameters is reduced by 2.08M, but more convolution calculations are added, resulting in an increase in the amount of calculation.

[0060] In drone aerial images, the target objects are small and the background noise is high, which significantly interferes with target detection. The attention mechanism can effectively filter background information and enhance the focus on the target object. However, the existing attention mechanisms have limitations. Channel attention mechanisms such as SE and ECA ignore the spatial information of each channel, resulting in the loss of feature information of low-weight channels. Pixel-level attention mechanisms such as CBAM perform feature compression on a global scale, which can easily lose feature information of small targets. In addition, CBAM calculates channel and spatial attention independently, and does not fully utilize the connection between the two. To address these problems, a CSFCA attention mechanism module is proposed, such as Figure 5 As shown in the figure, the CSFCA attention mechanism module first calculates the attention of the local block to highlight the local key information, and then performs global attention calculation to highlight the information of the key area, thereby retaining more feature information.

[0061] Table 2 Performance comparison of YOLOv8s and YOLOv8s-ESSF-CSFCA

[0062]

[0063]

[0064] As shown in Table 2, after adding the CSFCA attention mechanism module, its mAP 50 Improved by 1.7%, mAP 50-95 The improvement is 0.6%, and the number of parameters increases by 0.11M, but more convolution calculations are added, resulting in an increase in the amount of calculation.

[0065] The present invention is compared with a series of attention mechanisms, including SE, ECA, CBAM, and CA, on the power line defect dataset. These attention mechanisms are integrated into YOLOv8s-ESSF for training, and the training settings are kept consistent with the present invention.

[0066] Table 3 Experimental comparison of different attention mechanisms

[0067]

[0068] As shown in Table 3, the CSFCA attention mechanism module reduces mAP 50 Improved by 1.7% in mAP 50-95 It is improved by 0.6%. Compared with other attention mechanisms, the CSFCA attention mechanism module has the best effect, with a slight improvement in parameters. However, considering the improvement in detection performance, the improvement in parameters is acceptable.

[0069] In order to verify the performance advantage of CSFCANet over YOLOv8 in the UAV image object detection task, the s, m, l, and x of the CSFCANet and YOLOv8 series are used for comparison.

[0070] Table 4 Performance comparison of CSFCANet and YOLOv8 series

[0071]

[0072]

[0073] The results are shown in Table 4. CSFCANet achieves 56.7% precision P, 46.5% recall R, and 45.6% mAP. 50 , 20% mAP 50-95 , the effect even exceeds YOLOv8x, the parameters are the smallest, and the computational complexity is improved. This shows that CSFCANet reduces the computational complexity while improving the detection accuracy, achieving a win-win situation in terms of accuracy and model lightweight. Figure 7 The PR curves of CSFCANet and YOLOv8 series on the power line defect dataset are shown, providing a clearer visualization of the performance advantage of CSFCANet.

[0074] The performance of CSFCANet is verified by comparing different target detection algorithms and conducting comparative experiments with YOLOv8s. Figure 8 As shown, Figure 8 The detection results of the CSFCANet model are as follows Figure 8 As shown in the upper and lower images on the left, the detection results of YOLOv8s are as follows Figure 8 As shown in the upper and lower images on the right, Figure 8 The two pictures in the middle and top are scenes of rural power towers. Figure 8 The two pictures below are scenes of urban power towers. The detection box shows the category of the detected target. 5 represents the copper-aluminum excessive wire clamp category, and 6 represents the pin missing category. 0.60 represents the confidence level, which means the credibility of the target in the pre-selected box. Figure 8 The detection effect of the CSFCANet model is better than that of the YOLOv8 model, and more defects are detected.

[0075] Table 5 Performance comparison of different target detection algorithms

[0076]

[0077] The experimental results are shown in Table 5. The mAP of the CSFCANet model 50 and mAP 50-95The highest results were achieved in all the above, surpassing YOLOv8s. These experimental results verify that CSFCANet has good performance.

[0078] When the input resolution of the model is set to 2048*2048, the detection accuracy of CSFCANet will be greatly improved.

[0079] Table 6 Detection effect with a resolution of 2048*2048

[0080]

[0081] It can be seen from Table 6 that when the resolution is increased to 2048*2048, the detection accuracy reaches a very high level.

[0082] In general, the defect images taken by drones in real power lines have very small defect targets, making them difficult to observe and monitor. After optimizing the algorithm, the present invention improves the recognition effect to a certain extent.

[0083] In summary, the present invention uses the YOLOv8 model as the basis for improvement, and proposes a UAV defect detection model based on YOLOv8. The multi-scale feature extraction network in the YOLOv8 network model is replaced by the ESSF module to solve the problem of drastic scale changes, and the shallow information and deep information are fully integrated to capture more small target information. Adding a small target detection layer can enhance the model's perception of tiny objects; the CSFCA attention mechanism module is integrated in the detection head, which can make full use of global and local information, filter background noise and highlight key information; the present invention realizes the automatic detection of defects in power line components, which has the advantages of fast, convenient, accurate and objective, improves the efficiency of defect detection, reduces work costs, and improves the engineering and automation level of traditional manual inspections.

Claims

1. A UAV defect detection method based on channel space fusion cross attention, characterized by: The method comprises the following steps in order: (1) Planning the waypoint sampling route of the transmission line tower: using a drone equipped with a high-pixel, zoom visible light camera, and planning the waypoint flight sampling points and paths for the towers at different heights and locations; (2) Obtain high-resolution visible light images taken by a drone at low altitude, preprocess the high-resolution visible light images, and divide the preprocessed images into a test set and a training set; (3) The YOLOv8 network model is improved to obtain a YOLOv8-based UAV defect detection model, wherein the YOLOv8-based UAV defect detection model includes a backbone network, an ESSF module, i.e., a neck module, a CSFCA attention mechanism module, and a detection head; the training set is input into the YOLOv8-based UAV defect detection model for training, and the test set is used to verify the accuracy of the YOLOv8-based UAV defect detection model; (4) Identify the defect locations and categories of copper-aluminum excessive wire clips and missing pins in pole towers: Input the high-resolution visible light image taken by the drone to be inspected into the YOLOv8-based drone defect detection model to output the specific locations of the copper-aluminum excessive wire clips and missing pins.

2. The method for detecting defects in unmanned aerial vehicles based on channel space fusion and cross attention according to claim 1 is characterized by: In step (1), the effective pixel of the zoom visible light camera is better than 20 million, and the zoom ratio is greater than or equal to 4 times; the sampling points are set by an average sampling method, and 25 sampling points are set in a 5*5 manner; the parameters of the sampling points include serial number, longitude, latitude, altitude, direction angle and zoom ratio.

3. The method for detecting defects in unmanned aerial vehicles based on channel space fusion and cross attention according to claim 1 is characterized in that: The step (2) specifically comprises the following steps: (2a) Two photos were taken at each sampling point for later selection of available high-resolution photos; (2b) Annotate the copper-aluminum excessive wire clips and missing pins on the high-resolution visible light images; (2c) The labeled high-resolution visible light images are preprocessed by horizontal mirror flipping, vertical mirror flipping, mosaic data enhancement, mixed enhancement, and random perturbation. The images of all sampling points after preprocessing are evenly divided into 5 groups, 4 of which are used as training sets and the remaining 1 is used as a test set.

4. The method for detecting defects in unmanned aerial vehicles based on channel space fusion and cross attention according to claim 1 is characterized in that: In step (3), the improvement of the YOLOv8 network model specifically includes the following steps: (3a) The YOLOv8 network model is improved, wherein the YOLOv8 network model includes a backbone network, a multi-scale feature extraction network and a detection head; the SSFF module and the TFE module together form a multi-scale feature fusion module, and the P2 feature layer is added to the SSFF module. At this time, the multi-scale feature fusion module forms an ESSF module; the multi-scale feature extraction network in the YOLOv8 network model is replaced by the ESSF module; (3b) A CSFCA attention mechanism module is added to the detection head module of the YOLOv8 network model. The CSFCA attention mechanism module is added before each detection head. The CSFCA attention mechanism module includes a local attention calculation module and a global attention calculation module. The local attention calculation module includes multiple patch block cutting modules, multiple CSF modules and multiple patch block splicing modules. The number of the patch block cutting modules, the number of the CSF modules and the number of the patch block splicing modules are all p; the global attention calculation module includes a CSF module and an anti-pooling layer; First, the input feature F(B, H, W, C) is cut through the Patch block cutting module, where B represents the number of samples in each batch, H and W represent the height and width of the image respectively, and C represents the number of channels of the image. The input feature F is cut into patches of the same size, the number of patches is set to p, and the feature dimension of each patch is (B, H / / p, W / / p, C). Then, they are spliced ​​on the first dimension, B, and the feature dimension after splicing is (B*p*p, H / / p, W / / p, C). Then, each patch block is calculated for local attention by the CSF module, and the patch block features obtained by cutting the input feature F are multiplied to obtain the weighted feature block to focus on the detailed features in the local range. Then, the patch features after local attention calculation and weighting are spliced ​​and restored to F through the Patch block splicing module. ′ (B, H, W, C); then perform global attention calculation, perform attention calculation between each patch block, and calculate F ′ The patch blocks in the image are pooled separately, and the feature dimension is F″(B, p, p, C). The global attention is calculated using the CSF module, and the unpooling operation is performed to make the dimension of the attention map the same as the input feature F, thereby ensuring that the attention weights of the local area are consistent. ′ Perform a dot multiplication with the global attention to obtain the final output feature map after weighting by local attention and global attention calculation.

5. The method for detecting defects in unmanned aerial vehicles based on channel space fusion and cross attention according to claim 1 is characterized in that: The step (4) specifically comprises the following steps: (4a) The backbone network of the UAV defect detection model based on YOLOv8 is used to extract features from the defect image and generate a series of feature maps; (4b) Using the ESSF module to combine multi-scale information, multi-scale features are obtained; (4c) Using the CSFCA attention mechanism module to filter background information of the extracted multi-scale features and highlight key features; (4d) Four detection heads are used, corresponding to four different scales: (20,20), (40,40), (80,80) and (160,160), to obtain detection results of objects of different sizes. Subsequently, non-maximum suppression is used to filter multiple detection results, remove prediction boxes with high overlap, and retain the bounding box with the highest confidence. The detection head outputs the bounding box and category confidence of the target. The bounding box and category confidence of the target are the detection results, where the bounding box defines the position of the target in the image.

6. The method for detecting defects in unmanned aerial vehicles based on channel space fusion and cross attention according to claim 4 is characterized in that: In step (3b), the CSF module works as follows: First, for a given input feature F∈R C×H×W , where H and W represent the height and width of the image, respectively, and C represents the number of channels of the image. Pooling is performed in the spatial direction and channel direction to obtain spatial descriptors respectively. and channel descriptors Space Descriptor Aggregate all channel information, channel descriptor Aggregate all spatial information; then Reshape to (1, 1, H*W), after reshape and Concatenate in the channel dimension to get And through 1x1 two-dimensional convolution, the dimension is reduced to obtain the intermediate feature map F∈R 1 ×1×(C+H*W) / r , r is the reduction factor; then, batch normalization and ReLU function are used to encode the potential connection between spatial information and channel information, and then the dimension is restored through 1x1 two-dimensional convolution, and it is split into spatial descriptors and channel descriptors in the channel dimension; one-dimensional convolution is used to calculate the channel attention of the channel descriptor, and then the Sigmoid activation function is used to finally obtain the channel attention F C ∈R 1×1×C ; Use 3x3 two-dimensional convolution to calculate the spatial attention of the spatial descriptor, and generate the spatial attention F through the Sigmoid activation function S ∈R H×W×1 ; Final F C and F S Perform point multiplication to obtain channel space fusion attention, expressed as F CS ∈R H×W×C ; The CSF module is used within the patch block to calculate the local attention, and the CSF module is used between the patch blocks to calculate the global attention.

Images