A kite line detection method based on frequency domain enhancement and domain adaptive migration

Abstract

The application discloses a kind of based on frequency domain enhancement and domain self-adapting migration's kite line detection method, it is related to transmission line intelligent inspection technical field.The method takes YOLO26 as baseline detection network, integrates wavelet transform convolution module in backbone network shallow layer, carries out two-dimensional discrete wavelet decomposition to feature map and reconstructs after depth separable convolution is applied on high-frequency subband, to enhance the horizontal and vertical high-frequency edge response of kite line;Cross-layer frequency domain attention fusion module is inserted in neck network P3 and P4 scale output node, and multi-scale convolution aggregation, wavelet domain high-frequency channel attention weighting and residual fusion are sequentially executed;Two-stage domain self-adapting migration strategy is used, the first stage freezes backbone shallow layer and fine-tunes deep layer and neck to adapt low-frequency background distribution, and the second stage unfreezes all networks and enables wavelet transform convolution to reduce learning rate and adapt kite line high-frequency feature;NWD-SIoU hybrid loss is used as boundary box regression loss;And the image containing kite line in training set is improved sampling weight, and kite line area replication pasting enhancement is implemented.The application effectively solves the problem of missed detection and inaccurate positioning caused by extreme aspect ratio, minimum size and cross-domain semantic gap of kite line, and is suitable for unmanned aerial vehicle transmission line intelligent inspection in kite line automatic identification.

Description

Technical Field

[0001] This invention relates to the fields of intelligent inspection of power transmission lines and computer vision technology, specifically to a method, device, and storage medium for kite string detection in drone aerial photography scenarios. Background Technology

[0002] Overhead power transmission lines are exposed to the outdoor environment for extended periods, and kite string entanglement is one of the main foreign object hazards causing short circuits, power outages, and even fires. Using drones equipped with visible light cameras for line inspection has become a mainstream method, with a complete operational loop consisting of "target detection → motion tracking → gimbal aiming → laser ablation." In this loop, the recall rate of the kite string detection module directly determines the initialization quality of the downstream tracking loop, and its positioning accuracy directly affects the alignment effect of the gimbal servo and laser triggering. Kite string detection, in computer vision, is a task involving the detection of extremely small, aspect ratio targets, presenting three complex technical challenges: First, loss of frequency domain features. In aerial images, kite strings often appear as elongated structures with a width of less than 10 pixels, and their discriminative information is mainly concentrated in the high-frequency components in the horizontal and vertical directions. Repeated downsampling and spatial pooling operations in deep convolutional networks are equivalent to low-pass filtering, which continuously attenuates these high-frequency components, causing the kite string to be smoothed or even disappear in deep feature maps. Second, unstable regression gradients. Due to the extreme aspect ratio of kite strings, their predicted and ground truth bounding boxes are prone to non-intersection in the early stages of training. Standard IoU-type losses have zero or non-differentiable gradients when there is no intersection, making regression learning of the kite string branch difficult; even when there is intersection, a small center shift can cause drastic changes in IoU, resulting in high gradient signal noise. Third, cross-domain semantic gap. The feature space of general vision pre-trained models consists of natural images, with a ground-level perspective geometry and target scale distribution mainly consisting of medium to large targets such as cars and pedestrians. Aerial images of power transmission lines are taken from a top-down or oblique perspective, with the background consisting mainly of sky, vegetation, and towers, and the targets mainly being slender, elongated objects. There are significant differences in the characteristic manifolds between the two. Direct fine-tuning is insufficient to fully activate the high-frequency discrimination channels required for kite strings. In the prior art, Chinese patent application CN118470564A discloses a foreign object detection method for transmission lines based on an improved YOLOv8. This method enhances small target detection capabilities by adding an attention module, but it only performs feature recalibration in the spatial domain, failing to address frequency domain enhancement for the high-frequency characteristics of kite strings and neglecting to solve the regression gradient problem in non-intersecting cases. Chinese patent application CN117765398A discloses a power line inspection target detection method based on transfer learning, employing a frozen backbone transfer strategy. However, this freezing strategy limits the domain adaptation capability of high-level semantic features and does not design a specific regression loss for targets with extreme aspect ratios. Therefore, there is an urgent need for a kite string detection method that can simultaneously enhance kite string features in the frequency domain, stabilize the regression gradient of small targets, and achieve cross-domain progressive adaptation. Summary of the Invention

[0003] Technical problems to be solved This invention aims to solve the following three technical problems: (1) The horizontal and vertical high-frequency edge features of kite lines are attenuated during the downsampling process of deep convolutional networks, resulting in the loss of kite line discrimination information in deep feature maps; (2) Due to the extreme aspect ratio of kite lines, the predicted box and the real box often have no intersection, and the gradient of the standard IoU class regression loss is unstable; (3) There is a semantic gap between the COCO natural image pre-training domain and the aerial inspection domain of power transmission lines in terms of perspective geometry, target scale distribution and background statistics, and direct fine-tuning is difficult to fully activate the kite line discrimination channel. Technical solution To address the aforementioned technical problems, this invention provides a kite string detection method based on frequency domain enhancement and domain adaptive migration. The overall network structure is as follows Figure 1 As shown, the detection framework based on YOLO26m includes a backbone network, a neck network, and a detection head. The backbone network adopts a CSPDarknet structure with five stages. Wavelet transform convolutional modules (WTConv) are integrated at the outputs of stages 2 and 3. The neck network adopts a PAN-FPN structure, with a cross-layer frequency domain attention fusion module (CAFM-FD) inserted at the P3 and P4 scale output nodes. The detection head uses the DFL-free decoupled detection head of YOLO26, including classification and regression branches. The overall process of the detection method is as follows: Figure 2 As shown, it includes the following steps: Step S1: Construct a frequency domain enhancement detection network. Wavelet transform convolutional modules are integrated into the shallow stages (stages 2 and 3) of the backbone network, with the structure as follows: Figure 3As shown, the wavelet transform convolution module works as follows: First, a two-dimensional discrete wavelet transform is performed on the input feature map X, decomposing it into four sub-bands: LL low-frequency sub-band, LH horizontal high-frequency sub-band, HL vertical high-frequency sub-band, and HH diagonal high-frequency sub-band. The kite line in the image mainly exhibits the high-frequency response of the LH and HL sub-bands because its edges extend primarily along the horizontal and vertical directions. Second, a 3×3 depthwise separable convolution is applied to the LH, HL, and HH high-frequency sub-bands respectively, while the LL low-frequency sub-band remains unchanged. The computational cost of depthwise separable convolution is approximately 1 / 8 to 1 / 9 of that of standard convolution, maintaining lightweight computation while enhancing high-frequency details. Then, the enhanced three high-frequency sub-bands and the unprocessed LL low-frequency sub-band are reconstructed into a spatial domain feature map using an inverse wavelet transform. Finally, the reconstructed feature map is added element-wise to the input feature map X to form a residual output, avoiding performance degradation caused by noise introduced during the early stages of training due to high-frequency enhancement. Step S2: Construct the cross-layer frequency domain attention fusion module. Insert the cross-layer frequency domain attention fusion module (CAFM-FD) into the P3-scale output node and P4-scale output node of the neck network, respectively, as shown in the following structure. Figure 4 As shown. The reason for choosing scales P3 and P4 is that P3 corresponds to an 80×80 resolution feature map, which is suitable for detecting kite strings with a width of less than 10 pixels; P4 corresponds to a 40×40 resolution feature map, which is suitable for detecting the overall direction of longer kite strings. The processing flow of the cross-layer frequency domain attention fusion module is as follows: (a) Multi-scale aggregation: The input feature X is processed by 1×1 convolution, 3×3 convolution and 5×5 convolution respectively, and the three outputs are added element by element to obtain the aggregated feature U. The 1×1 convolution branch captures the relationship between channels, the 3×3 convolution branch captures local edges, and the 5×5 convolution branch captures the direction context of longer line segments. The three outputs are added instead of spliced ​​to avoid parameter inflation caused by the increase in the number of channels. (b) Frequency domain calibration: A two-dimensional discrete wavelet transform is performed on the aggregated feature U, and the channel attention weights are calculated for the high-frequency subbands obtained by decomposition. The channel attention weights are calculated as follows: global average pooling is performed on the high-frequency subbands to obtain channel descriptors, which are then processed through two fully connected bottleneck networks (channel reduction ratio of 4) and a sigmoid activation function to generate weights. After weighting, the calibrated features U' are reconstructed using inverse wavelet transform. This step enables the network to selectively enhance high-frequency channels related to kite strings and suppress high-frequency channels related to background textures. (c) Residual fusion: The calibrated features U' are added element-wise to the module input features X to obtain the output features Y = X + U'. Residual connections ensure that the module can be inserted into existing architectures without causing performance degradation: if the attention branch learns an output close to zero, the module degenerates into an identity mapping. Step S3: Two-stage domain adaptive transfer training. For example... Figure 5As shown, the transfer training is divided into two stages: Stage 1 (Low-frequency background adaptation): Loading the pre-trained YOLO26m weights from COCO. Freezing the first two stages of the backbone network (i.e., the shallow edge detector), and fine-tuning the remaining stages of the backbone network, the neck network, and the detector head. Training for 300 epochs with an initial learning rate of 0.01, using cosine annealing scheduling. Wavelet transform convolutional modules are not enabled in this stage. The reason for freezing the shallow layers is that the shallow edge detector captures general edge and texture features, exhibiting good transferability between natural and aerial images; while deep semantic features are strongly correlated with specific domains and require thorough fine-tuning. This stage allows the model to first adapt to the low-frequency background distribution (sky, vegetation, tower structures) of the power transmission line scene. Stage 2 (High-frequency feature adaptation): Unfreezing all network parameters, enabling the wavelet transform convolutional module, and continuing training for another 100 epochs with a learning rate of 0.001. The reason for reducing the learning rate is that the high-frequency enhancement branch is prone to overfitting specific linear patterns in the training set when the background understanding is insufficient. The lower learning rate allows it to smoothly learn the high-frequency features of the kite line on the already adapted background representation. Step S4: Use NWD-SIoU hybrid loss. Replace the bounding box regression loss with NWD-SIoU hybrid loss, which takes the following form: L reg = α · L NWD + (1 - α) · L SIoU α is set to 0.5. The NWD component is calculated as follows: the predicted box and the ground truth box are modeled as two-dimensional Gaussian distributions, with their means being the center coordinates of the boxes, and the covariance matrix determined by the width and height of the boxes. The Wasserstein distance between the two Gaussian distributions is calculated and obtained as a similarity measure after exponential normalization. The key advantage of NWD is that even if the predicted box and the ground truth box have no intersection, there is still a Wasserstein distance between the two Gaussian distributions, so the gradient always exists and is smooth, solving the problem of gradient vanishing caused by no intersection in the early stage of kite string training. The SIoU component includes four terms: angle penalty term, distance penalty term, shape penalty term, and IoU term. The angle penalty term is used to constrain the orientation consistency between the predicted box and the ground truth box. For targets with extreme aspect ratios like kite strings, orientation consistency is more important than precise boundary positions—as long as the orientations of the predicted box and the ground truth box are aligned, subsequent convergence can be achieved by fine-tuning the center position; while orientation misalignment will cause the regression to get stuck in a local optimum. Focal Loss is used for classification loss to alleviate the extreme foreground-background imbalance in a single image where hundreds of background regions correspond to a single kite string. Step S5: Kite String Small Sample Enhancement. Since kite string samples may account for a relatively small proportion of the collected data, two enhancement strategies are implemented: First, class-specific resampling: increasing the sampling weight of training images containing kite strings, with a weight coefficient set to 2.0. Second, kite string region copy-paste enhancement: cropping the kite string bounding box region from the labeled image and randomly pasting it onto a power transmission line background image without kite strings. During pasting, brightness perturbation (simulating different lighting conditions) and scale perturbation (simulating different shooting distances) are applied to the cropped region, and Poisson fusion is performed on the pasted boundary to eliminate stitching artifacts. Beneficial effects The present invention has the following advantages over the prior art: (1) By integrating wavelet transform convolution modules into the shallow layers of the backbone network, high-frequency subbands are enhanced in a targeted manner, so that the horizontal and vertical high-frequency edge features of the kite string can be preserved in the deep feature map. The reversibility of wavelet transform ensures that the enhancement process does not introduce spatial information loss, and depth-separable convolution ensures lightweight design. (2) By inserting a cross-layer frequency domain attention fusion module into the neck network, channel attention is applied to the high-frequency subband in the wavelet domain, enabling the network to distinguish between "kite string high frequency" and "background texture high frequency", thus avoiding false enhancement of background noise. (3) By using a two-stage domain adaptive migration strategy, the first stage adapts to the low-frequency background and the second stage adapts to the high-frequency target, avoiding the introduction of noise by the high-frequency enhancement branch when the background is mismatched. At the same time, the full model fine-tuning is better than the frozen backbone strategy. (4) By using the NWD-SIoU hybrid loss, the NWD component solves the gradient vanishing problem in the case of no intersection, and the SIoU angle penalty term constrains the direction consistency of the kite string. The two are complementary. (5) By enhancing the copy-paste of the kite string region and class-specific resampling, the class imbalance problem under small sample conditions is alleviated. Attached Figure Description Figure 1 This is a schematic diagram of the overall structure of the detection network of the present invention; Figure 2 This is an overall flowchart of the kite string detection method of the present invention; Figure 3 This is a schematic diagram of the wavelet transform convolution module of the present invention; Figure 4 This is a schematic diagram of the cross-layer frequency domain attention fusion module of the present invention; Figure 5 This is a schematic diagram of the two-stage domain adaptive migration strategy of the present invention. Detailed Implementation The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. The following embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Example 1: Overall Network Architecture Reference Figure 1 The detection network in this embodiment uses YOLO26m as the baseline and includes a backbone network, a neck network, and a detection head. The backbone network adopts a CSPDarknet structure with a total of 5 stages. Wavelet transform convolutional modules are integrated at the outputs of stages 2 and 3. The output feature map resolution of stage 2 is 160×160, and the output feature map resolution of stage 3 is 80×80. The neck network adopts a PAN-FPN structure. A cross-layer frequency domain attention fusion module is inserted at the P3 scale output node (corresponding to 80×80 resolution) and the P4 scale output node (corresponding to 40×40 resolution). The detection head uses the DFL-free decoupled detection head of YOLO26, including classification and regression branches. The inference stage enables an end-to-end output mode without NMS. The input image is uniformly scaled to 640×640 pixels. Example 2: Specific implementation of the wavelet transform convolution module Reference Figure 3 The wavelet transform convolution module in this embodiment includes the following processing steps: Step A1, the input feature map X has a dimension of C×H×W. A two-dimensional discrete wavelet transform is performed using the Haar wavelet basis to decompose it into four sub-bands: LL, LH, HL, and HH, each with a dimension of C×H / 2×W / 2. Step A2, 3×3 depthwise separable convolutions are applied to the three high-frequency sub-bands LH, HL, and HH respectively. The depthwise separable convolution consists of channel-wise convolution and pointwise convolution: channel-wise convolution applies a 3×3 convolution kernel independently to each channel, and pointwise convolution applies a 1×1 convolution to achieve information exchange between channels. Step A3, the enhanced LH, HL, and HH sub-bands and the unprocessed LL sub-band are reconstructed using inverse wavelet transform to obtain a spatial domain feature map with a dimension of C×H×W. Step A4, the reconstructed feature map is added element-wise to the input feature map X to obtain the output. In this embodiment, the kite string exhibits a significant high-frequency response in the LH and HL sub-bands. After step A2 enhancement, the response amplitude at the edge of the kite string is increased, while the response of low-frequency background areas such as the sky and vegetation remains basically unchanged. Example 3: Specific Implementation of the Cross-Layer Frequency Domain Attention Fusion Module Reference Figure 4The cross-layer frequency domain attention fusion module of this embodiment includes the following processing steps: Step B1, the input feature X is processed by 1×1 convolution, 3×3 convolution, and 5×5 convolution respectively. All three convolutions include batch normalization and SiLU activation. The three outputs are added element-wise to obtain the aggregated feature U. Step B2, a two-dimensional discrete wavelet transform is performed on the aggregated feature U to obtain four sub-bands: LL, LH, HL, and HH. Step B3, global average pooling is performed on the three high-frequency sub-bands LH, HL, and HH respectively to obtain channel descriptors. The channel descriptors are reduced to C / 4 by a first fully connected layer, activated by ReLU, increased to C by a second fully connected layer, and activated by Sigmoid to obtain channel weights. The channel weights are multiplied by the corresponding high-frequency sub-bands channel by channel. Step B4, the weighted high-frequency sub-bands and LL sub-bands are reconstructed by inverse wavelet transform to obtain the calibration feature U'. Step B5, the calibration feature U' is added element-wise to the input feature X to obtain the output feature Y. Example 4: Specific implementation of NWD-SIoU hybrid loss The calculation process for the NWD components is as follows: The predicted bounding box is modeled as a two-dimensional Gaussian distribution, with its mean being the coordinates of the box center and its covariance matrix determined by the width and height of the box. Similarly, the ground truth bounding box is modeled as a corresponding Gaussian distribution. The two-dimensional Gaussian distribution modeling of the predicted and ground truth bounding boxes is as follows: The second-order Wasserstein distance between two Gaussian distributions is: The normalized similarity is: Where C is a scale constant, and in this embodiment, it is taken as the median size of the kite string targets in the TL-FOD dataset. The NWD loss is: The calculation of the SIoU components includes angle penalty Λ, distance penalty Δ, shape penalty Ω, and an IoU term, with the specific formula using the original definition of SIoU. The mixed loss is: The classification loss is Focal Loss, with a focusing parameter γ of 2.0 and a balance factor α of 0.25. Example 5: Specific Implementation of Two-Stage Transfer Training Reference Figure 5The two-stage transfer learning in this embodiment includes: Stage 1: Loading the pre-trained YOLO26m weights from COCO. Freezing Stages 1 and 2 of the backbone network. Training for 300 epochs on the TL-FOD dataset. The optimizer uses MuSGD with an initial learning rate of 0.01, momentum of 0.937, and weight decay of 5 × 10⁻⁶. -4 The batch size is 32. Cosine annealing is used for learning rate scheduling. Data augmentation includes Mosaic, random HSV dithering, and horizontal flipping. The wavelet transform convolution module is not enabled in this stage. Second stage: Unfreeze all network parameters and enable the wavelet transform convolution module. The learning rate is reduced to 0.001, and training continues for 100 epochs. Mosaic enhancement is disabled in this stage to avoid discontinuous annotations caused by kite lines crossing boundaries in the stitched images. The sampling weight for images containing kite lines is set to 2.0. Example 6: Inference Deployment After training, the model can be exported in RKNN format and deployed on edge computing platforms such as Rockchip RK3576. During the inference phase, an end-to-end output mode without NMS is enabled, taking a 640×640 image as input and outputting a kite-string bounding box and confidence score.

Claims

1. A kite string detection method based on frequency domain enhancement and domain adaptive migration, characterized in that, Includes the following steps: Step S1: Construct a frequency domain enhanced detection network: Using YOLO26 as the baseline detection framework, the baseline detection framework includes a backbone network, a neck network, and a detection head; integrate a wavelet transform convolution module in the shallow stage of the backbone network, the wavelet transform convolution module performs a two-dimensional discrete wavelet transform on the input feature map to obtain a low-frequency sub-band and three high-frequency sub-bands, apply depthwise separable convolution only to the three high-frequency sub-bands and then perform inverse wavelet transform reconstruction, and add the reconstruction result to the residual of the input feature map; Step S2, construct a cross-layer frequency domain attention fusion module: insert the cross-layer frequency domain attention fusion module into the P3 scale output node and the P4 scale output node of the neck network respectively, and the cross-layer frequency domain attention fusion module sequentially performs multi-scale aggregation, frequency domain calibration and residual fusion; Step S3, two-stage domain adaptive transfer training: In the first stage, COCO pre-trained weights are loaded, the first two stages of the backbone network are frozen, and the remaining stages of the backbone network, the neck network and the detection head are fine-tuned to make the model adapt to the low-frequency background distribution of the aerial photography scene of the power transmission line. In the second stage, all network parameters are unfrozen and the wavelet transform convolution module is enabled to continue fine-tuning with a learning rate lower than that in the first stage, so that the high-frequency enhancement branch can be adapted to the kite string feature. Step S4: The NWD-SIoU hybrid loss is used as the bounding box regression loss. The NWD-SIoU hybrid loss is composed of a weighted sum of the normalized Wasserstein distance component and the SIoU component. Step S5: Increase the sampling weight of images containing kite strings in the training set, and perform kite string region copy-paste enhancement.

2. The method according to claim 1, characterized in that, The wavelet transform convolution module includes: A two-dimensional discrete wavelet transform unit is used to decompose the input feature map into LL low-frequency subband, LH horizontal high-frequency subband, HL vertical high-frequency subband and HH diagonal high-frequency subband; The high-frequency enhancement unit is used to apply 3×3 depth-separable convolution to the LH horizontal high-frequency subband, the HL vertical high-frequency subband and the HH diagonal high-frequency subband respectively, while keeping the LL low-frequency subband unchanged; The inverse wavelet transform unit is used to reconstruct a spatial domain feature map by combining the enhanced three high-frequency sub-bands with the LL low-frequency sub-band. The residual addition unit is used to add the spatial domain feature map to the input feature map element by element.

3. The method according to claim 1, characterized in that, The cross-layer frequency domain attention fusion module includes: The multi-scale aggregation unit is used to process the input features through 1×1 convolution, 3×3 convolution and 5×5 convolution respectively, and add the three outputs element by element to obtain the aggregated feature U; The frequency domain calibration unit is used to perform a two-dimensional discrete wavelet transform on the aggregated feature U, calculate and weight the channel attention weights of the high-frequency sub-bands obtained by decomposition, and then reconstruct the calibration feature U' through inverse wavelet transform. The residual fusion unit is used to add the calibration feature U' to the input feature X of the cross-layer frequency domain attention fusion module element by element to obtain the output feature Y, i.e., Y = X + U'.

4. The method according to claim 3, characterized in that, The channel attention weights in the frequency domain calibration unit are calculated as follows: global average pooling is performed on the high-frequency subband to obtain channel descriptors, and channel weights are generated through two layers of fully connected bottleneck networks and a Sigmoid activation function. The channel reduction ratio of the two layers of fully connected bottleneck networks is 4.

5. The method according to claim 1, characterized in that, The NWD-SIoU hybrid loss takes the form of: L_reg = α · L_NWD + (1 - α) · L_SIoU Where L_NWD is the normalized Wasserstein distance loss, L_SIoU is the SIoU loss, and α is the balance coefficient, with a value ranging from 0.4 to 0.

6.

6. The method according to claim 5, characterized in that, The L_NWD model the predicted box and the ground truth box as two-dimensional Gaussian distributions, calculates the Wasserstein distance between the two Gaussian distributions, and obtains the similarity measure by exponential normalization; the L_SIoU includes an angle penalty term, a distance penalty term, a shape penalty term, and an IoU term, wherein the angle penalty term is used to constrain the directional consistency between the kite string predicted box and the ground truth box.

7. The method according to claim 1, characterized in that, In step S3, the learning rate for the first stage is 0.01, and the training lasts for 300 epochs; the learning rate for the second stage is 0.001, and the training lasts for 100 epochs.

8. The method according to claim 1, characterized in that, The kite string region copy and paste enhancement in step S5 includes: cropping the kite string bounding box region from the labeled image, randomly pasting it onto the background image of the power transmission line without kite string, and applying brightness perturbation and scale perturbation to the pasted region.

9. The method according to claim 1, characterized in that, The detection head adopts the DFL-free decoupled detection head of YOLO26, and the inference stage adopts the end-to-end output mode without NMS; the shallow stages of the backbone network are the second stage and the third stage, and the wavelet transform convolution module is integrated at the output of the second stage and the third stage.

10. A kite string detection device based on frequency domain enhancement and domain adaptive migration, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method of any one of claims 1 to 9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 9.

Images