Target detection method for patrol inspection of broken strand conductor of power transmission line by unmanned aerial vehicle

Abstract

The invention relates to the technical field of image processing, in particular to a target detection method for routing inspection of a broken strand conductor of a power transmission line by an unmanned aerial vehicle, and the method comprises the steps: expanding a data set of the broken strand power transmission line through employing a conventional data enhancement method: further expanding the data set of the broken strand power transmission line through employing a diffusion model, comprising the following steps: performing background restoration and expansion on a data set by using a diffusion model and a corresponding cue word vector, and generating a broken stock picture from a power transmission line picture by using the generation capability of the diffusion model; using a diffusion model to generate data sets under different styles and different cameras; the method comprises the following steps: positioning a key region of interest of a power transmission line, performing high-resolution feature extraction on a data set of the key region of interest to realize target detection of broken strands, and performing rapid rough detection on a non-key region by using an auxiliary detection network; and classifying detection results. Compared with the prior art, the method has the advantages of high transmission line broken strand detection accuracy, good real-time performance and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a target detection method for a broken conductor of a power transmission line during an unmanned aerial vehicle inspection. Background Art

[0002] Detecting broken strands in transmission line conductors, a crucial technology for ensuring the safe operation of power grids, has garnered considerable attention in recent years. Existing technologies primarily rely on image processing and deep learning methods, using drones or inspection robots to capture conductor images and combining them with target detection algorithms to automatically identify broken strands. These methods have made progress in improving detection efficiency and accuracy, particularly with relatively stable performance for low- and medium-resolution images. Furthermore, various convolutional neural network architectures have been introduced to enhance the model's feature extraction capabilities and improve broken strand detection. For example, the YOLO family of common target detection algorithms, influenced by their network structure and pre-trained weights, performs well at 512*512 resolution, and newer versions now support 1024*1024 resolution.

[0003] However, current technology still has significant shortcomings. First, the power industry is relatively closed, and high-quality labeled samples of broken transmission line strands are relatively scarce. This makes it difficult for the model to fully learn the diverse characteristics of broken strands during training, limiting the generalization and robustness of the detection algorithm. Second, transmission line images are usually high-resolution to ensure clear details of broken strands, but the processing of high-resolution images incurs huge computational and storage overhead, making it difficult to achieve real-time and efficient detection on devices with limited hardware resources in the actual field. In this case, how to effectively control hardware resource consumption while ensuring detection accuracy remains a core problem in current research.

[0004] After searching, China Invention Patent Application Publication No. CN118799563A discloses a transmission line foreign body detection method based on YOLOv9 and diffusion model, including: S1, obtaining transmission line foreign body images and performing data annotation to obtain transmission line foreign body image data set; S2, using the Wensheng graph diffusion model to perform multimodal data enhancement to obtain the image data set after data enhancement, and divide it into a training set and a verification set; S3, using a public data set to pre-train the improved YOLOv9 model to obtain the best pre-training weight; S4, inputting the training set obtained in step S2 into the pre-trained improved YOLOv9 model for training, verifying the verification set in each round, obtaining the best weights and hyperparameters, and obtaining a transmission line foreign body detection model; S5, configuring the transmission line foreign body detection model on the edge device to realize transmission line foreign body detection; wherein, the backbone network, neck of the improved YOLOv9 model The network and the head network are connected in sequence to form the main branch, the programmable gradient information module is connected to the backbone network, and the programmable gradient information module serves as an auxiliary branch. The neck network adopts a high-level screening feature fusion pyramid; the MobileNetv4-Small network is used as the backbone network, and the backbone network includes the MNv4-1 module, MNv4-2 module, MNv4-3 module, MNv4-4 module, and MNv4-5 module arranged in sequence. The MNv4-1 module includes a convolution normalization module, which includes ordinary convolution and depthwise separable convolution. First, the image features are extracted through ordinary convolution, and then the spatial features are extracted again using depthwise separable convolution. The extracted feature channels are then split into two parts, one of which remains unchanged and the other part undergoes point convolution operation. The point convolution fuses the channel features, and then the features of the two branches are further fused through element-by-element multiplication. This existing patent application has the problems of requiring large hardware resource overhead and low detection accuracy.

[0005] How to achieve accurate and efficient detection of broken conductors in transmission lines has become a technical problem that needs to be solved. Summary of the Invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a target detection method for broken strands of power transmission line during drone inspection.

[0007] The purpose of the present invention can be achieved by the following technical solutions: According to one aspect of the present invention, a method for detecting broken conductors in a transmission line during drone inspection is provided, the method comprising: Use traditional data augmentation methods to expand the dataset of broken transmission lines: The diffusion model was used to further expand the broken transmission line dataset, including: using the diffusion model and corresponding hint word vectors to perform background restoration and expansion on the dataset; using the diffusion model's generative capabilities to generate broken strand photos from transmission line images; and further expanding the dataset using the diffusion model to generate datasets with different styles and camera settings. Locate key areas of the transmission line, use the main detection network to perform high-resolution feature extraction on the data set of key areas to achieve target detection of broken strands, and use the auxiliary detection network to perform fast and rough detection on non-key areas; The test results are classified into broken strand areas, possible broken strand areas and normal areas.

[0008] Preferably, the key area division adopts the method of detecting targets and area expansion, wherein the detection targets include insulator strings, suspension clamps, tension clamps and conductor connection points; the area expansion includes fixed pixel expansion, proportional expansion, adaptive expansion and shape perception expansion.

[0009] Preferably, the auxiliary detection network includes a shallow feature extraction module and an attention enhancement module connected in sequence; The shallow feature extraction module uses three layers of shallow convolution for shallow features; The attention enhancement module adopts a dual attention design, including a channel attention submodule and a spatial attention submodule, to extract attention features.

[0010] More preferably, the auxiliary detection network further includes an anomaly scoring module and a conditional trigger detection module after the attention enhancement module; The anomaly scoring module concatenates shallow features and attention features, and generates anomaly scores through global average pooling and fully connected layers; The conditional trigger detection module dynamically allocates computing resources according to the anomaly score, directly skips the low anomaly score path, and feeds back the high anomaly score path to the main detection network to trigger secondary fine detection.

[0011] Preferably, the diffusion model is a stable diffusion3 network, which includes an encoder and a backbone network, wherein the encoder includes a VAE encoder and a multimodal text encoder; The input image passes through the VAE encoder and outputs a potential representation; Text prompts are fed into a multimodal text encoder, which outputs features as conditional embeddings. The input of the backbone network includes latent representation, temporal embedding and conditional embedding, and the output is predicted noise and velocity field.

[0012] More preferably, the diffusion model adopts Diffusion Transformer as a backbone network, and the backbone network includes multiple layers of Transformer blocks to process the potential representation of the input.

[0013] More preferably, the backbone network includes three tracking modules connected in series, namely a text tracking module, a visual tracking module and a hybrid tracking module. The text tracking module and the visual tracking module include a self-attention layer, a feedforward network and layer normalization, and the hybrid tracking module includes a cross-attention layer, a feedforward network and layer normalization.

[0014] More preferably, the encoder is a dual-branch conditional encoder, which independently encodes the source image and the editing condition through the source image branch and the conditional branch, and then fuses the information through a spatially aware cross-attention mechanism, and uses an adaptive weight mechanism to dynamically adjust the fusion strength according to the mask information and feature similarity; The source image branch adopts a convolutional neural network to capture spatial hierarchical feature representation, and the conditional branch uses a multimodal encoder to process different types of editing instructions.

[0015] More preferably, the method further comprises improving the VAE encoder by adding a structure-preserving attention mechanism after inputting the image; The structure-preserving attention mechanism guides attention calculation by explicitly modeling the structural information of the image. The process includes the following steps: Step 1, structural feature extraction: The system uses a multi-scale edge detection operator to extract structural features of different scales of the source image; Step 2, structural importance assessment: calculate the structural importance of each pixel position and identify key structures; Step 3, adaptive weight generation: adaptively adjust according to the editing intensity to generate structural weights; Step 4: Attention mechanism fusion: Combine the structural weights with the traditional attention mechanism to form a structure-aware attention distribution; Step 5: Diffusion process guidance: Apply structure-preserving constraints during the denoising process of the diffusion model and output the structure-preserving image editing results.

[0016] More preferably, an adaptive region mask attention mechanism is introduced in the encoder part, and the adaptive region mask attention mechanism includes a semantic segmentation network, a text encoder, an intent understanding module and an adaptive mask generation module, including: Use the pre-trained semantic segmentation network to accurately divide the input image into semantic regions and identify different object and region categories; Analyze the user's editing instructions through the text encoder and extract the semantic concepts involved in the editing intention; The intent understanding module calculates the relevance score of each semantic region to the editing intent, automatically identifying the regions that need to be modified and those that need to be retained; The adaptive mask generation module combines the basic mask and semantic relevance analysis results to automatically expand the mask to semantically related areas. At the same time, it uses a differentiable soft mask to replace the hard binary mask and dynamically adjusts the attention weight of each spatial position according to the soft mask.

[0017] Compared with the prior art, the present invention has the following beneficial effects: 1) This invention fully leverages the diffusion model's ability to understand real-world images, envisions possible variations, and effectively expands the broken-strand dataset of power transmission line conductors through realistic redrawing. This addresses the data shortage in this niche scenario, which leads to degraded target detection performance and insufficient generalization. It greatly accelerates the development of automated detection capabilities for broken-strand conductors in power transmission lines. The main and auxiliary detection networks are then used to perform target detection in key and non-key areas, respectively, achieving a balance between detection accuracy and efficiency and reducing the required computing resources.

[0018] 2) For the scenario of broken strands in transmission lines, the present invention divides key areas by detecting targets and expanding the area. The small target of detecting broken strands is first converted into a larger target of detecting conductors. Then, by dividing the key areas around the conductors, high-resolution and large computing resources are invested in detecting key areas, while downsampling and reducing the resolution of targets are used for detection in non-key areas. This solves the contradiction between high-resolution image computing resources and computing accuracy during the inspection process, concentrates the use of limited computing resources, and improves the efficiency of computing power.

[0019] 3) In this paper, the encoder of the diffusion model is improved into a dual-branch conditional encoder, which independently encodes the source image and editing conditions, and then fuses information through a spatially aware cross-attention mechanism. An adaptive weight mechanism is used to dynamically adjust the fusion strength according to the mask information and feature similarity. This can significantly improve the editing accuracy and image quality while reducing unnecessary modifications, thus contributing to both detection accuracy and efficiency.

[0020] 4) The encoder part of the present invention also adopts a structure-preserving attention mechanism and a multi-scale semantic alignment network to achieve more accurate semantic positioning and modification, adaptively adjust the structural weight according to the strength of the structural editing, reduce the degree of modification in complex areas, enhance the modification in simple areas, and further achieve a balance between accuracy and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 Schematic diagram of the process of the target detection method of the present invention; Figure 2 Schematic diagram of the structure of the diffusion model in the present invention; Figure 3 Schematic diagram of the structure of the dual-branch detection network in the present invention; Figure 4 Schematic diagram of the structure of the multi-scale semantic alignment network in the present invention; Figure 5 Schematic diagram of the structure of the dual-branch conditional encoder in the present invention; Figure 6 Schematic diagram of the structure of the structure-preserving attention mechanism in the present invention; Figure 7 Schematic diagram of the structure of the adaptive region mask attention in the present invention. DETAILED DESCRIPTION

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0023] Example 1 This embodiment relates to a method for detecting broken conductors in a transmission line during drone inspection. Figure 1 ,include: S1, obtain a dataset of broken transmission lines and use traditional data augmentation methods to expand the existing dataset: Traditional data augmentation methods such as occlusion, scale transformation, and illumination transformation are used to expand the dataset and increase the diversity of the dataset.

[0024] S2, Diffusion models, as generative models, are based on the stochastic process of Markov chains. Diffusion models can achieve extremely high diversity in their generation. Using a larger, general diffusion model, the generated data has high validity. Diffusion models are used in small-sample learning scenarios to alleviate the problem of insufficient samples by generating a rich set of auxiliary samples. Furthermore, some research uses diffusion models for cross-domain data augmentation, improving the model's adaptability to different environments, devices, or perspectives.

[0025] Using diffusion model as a data enhancement tool, for the broken transmission line dataset, such as Figure 2 , the dataset is further expanded using the following three methods: a) Use the diffusion model and corresponding cue word vectors to perform background restoration and expansion on the dataset. For example, replace the forest scene with a barren mountain in the background of a power transmission line, remove the houses in the background, etc. For example, in S1, after partially masking the image with a black block, the diffusion model can be used to re-complete the image based on different cues. For example, the image expansion method can be used to expand the frame of a close-up shot to a distant wide-angle shot.

[0026] b) Directly use the generative capability of the diffusion model to generate broken strand photos based on a large number of normal transmission line images.

[0027] This step requires a large diffusion model with good performance. A leading model can be used to ensure the effectiveness of the generated images for the object detection algorithm. When using an open-source model, fine-tuning can be performed using real-world photos of power transmission lines to enhance the model's expertise in this field.

[0028] c) Using the diffusion model, we can further generate datasets for different camera styles and camera setups, expanding the data's usability across a wide range of devices. Advanced diffusion models can simulate the effects of shooting with various aperture settings, lenses, and cameras, and even simulate changes in the shooting angle within a certain range. This model goes beyond simple color processing such as light intensity and Gaussian blur in traditional object detection projects, allowing us to train diffusion models using datasets of photos captured by a small number of devices.

[0029] The present invention adopts a method of modifying images using basic images and text descriptions. The artificially synthesized dataset generated based on real transmission line photos is closer to reality and more effective for training target detection projects, rather than unrealistic and invalid data.

[0030] The diffusion model is a stable diffusion3 (SD3) network, which uses the Diffusion Transformer (DiT) as the backbone network, replacing the traditional U-Net architecture. The backbone network includes multiple Transformer blocks to process the potential representation of the input. The Transformer blocks include self-attention layers (Self-Attention), cross-attention layers (Cross-Attention with text), feed-forward networks (FFN), residual connections, and layer normalization, supporting better long-range dependency modeling and scalability.

[0031] The stable diffusion3 network includes: 1) VAE encoder (latent space encoder): Input image and noise, pass through the latent space encoder and output the latent representation.

[0032] 2) Multimodal Text Encoder: After the text is input into the multimodal text encoder (CLIP-L / 14, CLIP-G / 14, T5-XXL), the output features of the three encoders are fused through the attention mechanism and used as conditional embedding.

[0033] CLIP L / 14: Provides basic text-visual alignment capabilities; CLIP G / 14: Enhanced visual-linguistic understanding; T5-XXL: Enhances plain text comprehension and complex instruction following.

[0034] The latent space encoder and the multimodal text encoder together form the encoder.

[0035] 3) Diffusion Transformer (DiT) backbone network: Input: latent representation, temporal embedding and conditional embedding, the backbone network consists of multiple layers of Transformer blocks, and outputs predicted noise / velocity field.

[0036] The stable diffusion3 network uses Flow Matching training, which uses straight paths to connect noise and data distributions, simplifying the sampling process and improving training stability.

[0037] S3, for the detection of broken strands of transmission line conductors, uses a dual-branch detection network for target detection, deep detection of key areas and shallow detection of non-key areas, such as Figure 3 shown.

[0038] For the scenario of detecting broken strands on transmission lines, the broken strands are generally located along the transmission line itself. Therefore, an algorithm can be designed to first detect the skeleton structure of the transmission line, including: First, use target detection methods or semantic segmentation models to locate the transmission line area to form the focus area (ROI), which includes the conductor connection points, insulator areas and areas with high historical fault incidence. Use the YOLO V8 or YOLO V9 network model for high-precision detection.

[0039] Subsequently, significant computing resources were invested in this key area, using the main detection network to perform high-resolution feature extraction to meticulously identify small targets such as broken strands. This approach effectively balanced computing resources with detection accuracy by leveraging the features of the larger target (the transmission line).

[0040] ROI key area demarcation uses target detection and region expansion. Targets include insulators, suspension clamps, tension clamps, and conductor joints. Region expansion options include fixed pixel expansion (±50px), proportional expansion (1.2-1.5x), adaptive expansion (target size), and shape-aware expansion (ellipse / rectangle).

[0041] Furthermore, to further reduce the computational overhead and risk of missed detection in non-critical areas (such as ordinary wire segments and background areas), a lightweight auxiliary detection network can be designed by combining convolution operations and attention mechanisms. This network performs rapid and rough detection of non-critical areas and implements an early warning trigger mechanism for detected anomalies. This "one-shot" strategy ensures the integrity of the overall detection while avoiding the computational bottlenecks associated with high-precision detection of the entire image.

[0042] In summary, this algorithm achieves efficient and accurate target detection from high-resolution photos by first focusing on the transmission line trunk and then locally inspecting broken strands. This is particularly suitable for edge computing platforms requiring real-time processing, such as when a robot needs to automatically repair a broken conductor using visual servoing. Leveraging limited local computing resources, this algorithm performs high-speed and high-precision target detection, laying the foundation for real-time automated robotic operation.

[0043] Common object detection methods in high-resolution images include two-stage detectors based on convolutional neural networks (CNNs), such as Faster R-CNN, and single-stage detectors, such as YOLO and SSD. These methods effectively recognize both large and small objects by learning multi-scale features. However, in ultra-high-resolution images, performing comprehensive detection on the entire image consumes enormous computational resources, and small object detection is easily affected by background interference, resulting in reduced accuracy. In the task of detecting broken strands in power transmission lines, the broken strands are often distributed along the transmission line itself, a typical "small target embedded in target" problem. Directly searching for all broken strands in the entire high-resolution image is both computationally intensive and prone to reduced detection accuracy due to complex backgrounds. To address this issue, a transmission line detection module can be designed to first locate the spatial distribution of the entire transmission line and generate a mask or bounding box of the transmission line.

[0044] This approach is equivalent to first identifying the "large target"—the transmission line—and narrowing the search space for subsequent small target detection to this area. High-resolution, multi-scale feature extraction and fine-grained analysis can then be performed on this key area, using deeper or more complex network structures to precisely detect small broken strands. This algorithm fully utilizes the spatial and morphological characteristics of the transmission line, concentrating computing resources on areas most likely to experience broken strands. This significantly improves the recall and accuracy of small target detection while reducing overall computational costs.

[0045] While key area detection can effectively focus on broken strands, non-key areas of the image may still contain interference or other anomalies. Completely ignoring these areas may result in missed detections. To address this, a lightweight auxiliary detection network can be designed to leverage shallow features of convolutional neural networks or attention mechanisms to quickly and roughly scan non-key areas.

[0046] The auxiliary detection network adopts a lightweight, multi-layered design approach and consists of five core modules: a preprocessing module, a shallow feature extraction module, an attention enhancement module, an anomaly scoring module, and a conditional trigger detection module. The design goal of this entire branch is to achieve rapid scanning of non-critical areas and anomaly warning with minimal computational overhead.

[0047] The following is a detailed description of each module in the auxiliary detection branch: 1) Preprocessing module, including downsampling and data normalization. Images in non-key areas are downsampled from their original resolution (H × W) to (H / 4 × W / 4), significantly reducing computational effort. Input images are normalized to improve model stability.

[0048] 2) Shallow feature extraction module: This module uses three layers of shallow convolution to avoid the computational burden of overly deep networks. It is used to capture shallow features such as edges, textures, and local patterns.

[0049] The first shallow convolution layer: 3→16 channels, 3×3 convolution, ReLU activation; Second layer of shallow convolution: 16→32 channels, 3×3 convolution, ReLU activation; The third shallow convolution layer: 32→64 channels, 3×3 convolution, ReLU activation.

[0050] 3) Attention enhancement module, which adopts a dual attention design, including a channel attention submodule and a spatial attention submodule, to extract attention features.

[0051] The channel attention submodule includes global average pooling (GAP), fully connected layers, and sigmoid activation to learn the importance weights of different feature channels. Spatial attention is used to highlight the spatial locations of potential abnormal areas in the image.

[0052] 4) Anomaly scoring module, which concatenates shallow features and attention features and generates anomaly scores through global average pooling and fully connected layers.

[0053] 5) The conditional trigger detection module dynamically allocates computing resources based on anomaly scores to achieve an optimal balance between efficiency and accuracy. For example, low-anomaly-scoring paths are skipped to save computing resources; high-anomaly-scoring paths trigger a secondary, refined detection, adjusting the network depth and using a more complex detection network.

[0054] The auxiliary detection network works in conjunction with the main detection network. The main branch focuses on high-precision detection of the ROI area, while the auxiliary branch is responsible for rapid scanning of non-ROI areas. The auxiliary branch acts as a safety net, reducing the risk of missed detections due to inaccurate ROI delineation.

[0055] Anomaly signals from the auxiliary detection network are fed back to the ROI extraction module to optimize the region partitioning strategy and dynamically allocate computing resources based on anomaly scoring. If the auxiliary branch indicates potential anomalies in certain areas, a second, more refined inspection is performed; if no anomalies are apparent, this process is skipped, conserving computing resources. Convolution operations are performed using smaller models or lower resolutions to maintain detection speed while still capturing some anomaly features.

[0056] This combined primary and auxiliary detection strategy, with deep detection in key areas and shallow detection in non-key areas, significantly improves the system's computational efficiency and practicality while ensuring the integrity of transmission line broken strand detection.

[0057] S4: Classify the test results and attempt to confirm them using further methods. For example, they can be divided into areas with a certain degree of breakage, areas with a potential for breakage, and areas with a certain degree of normality. The areas with a potential for breakage are further confirmed, while the other two areas are filed away.

[0058] For areas where strands may be broken, relevant images can be manually confirmed, or sensors with physical distance detection, such as lidar or structured light laser sensors, can be used to further confirm the wire condition at close range.

[0059] Example 2 This embodiment also relates to a method for detecting broken conductors in transmission line inspections using drones. The diffusion model has been enhanced in terms of its image generation capabilities, including: 1) Improve the encoder to a dual-branch conditional encoder: The dual-branch conditional encoder adopts a separate processing strategy, designing independent encoding paths for the source image and the editing condition (i.e., the text prompt of the generated image) (source image branch and conditional branch respectively), and then performs information fusion through the spatial perception cross attention mechanism, such as Figure 5The source image branch focuses on extracting and preserving original visual features, using a convolutional neural network to capture spatial hierarchical feature representations. The conditional branch uses a multimodal encoder to process different types of editing instructions, including text descriptions, region masks, edge maps, and other control signals.

[0060] The two branches exchange information through a carefully designed cross-attention mechanism. Unlike simple feature fusion, this approach incorporates spatially aware cross-attention computation, enabling the model to understand the correspondence between editing conditions and spatial locations in the image. An adaptive weighting mechanism dynamically adjusts the fusion strength based on mask information and feature similarity, ensuring that conditional guidance is strengthened in areas requiring editing and that external interference is mitigated in areas requiring preservation. The adaptive weight W is specifically: W = α*M + β*S, where α and β are weight coefficients, M is the mask information, and S is feature similarity.

[0061] Traditional image-to-image models often simply concatenate or add the source image and editing conditions, a practice that can easily lead to information confusion and feature interference. The source image contains rich visual details and structural information, while the editing conditions (such as text, masks, and control signals) convey the user's editing intent. Mixing these two completely different types of information makes it difficult for the model to accurately understand which parts need to be preserved and which parts need to be modified. The dual-branch conditional encoder significantly improves editing accuracy and image quality. Compared to traditional methods, the dual-branch structure can reduce unnecessary modifications by approximately 30% while improving the semantic consistency of the edited area. This is particularly effective in preserving background details in complex scene editing.

[0062] 2) Improve the VAE encoder by adding a structure-preserving attention mechanism after the input: The structure-preserving attention mechanism guides the attention calculation by explicitly modeling the structural information of the image. The structure of the structure-preserving attention mechanism is as follows Figure 6 , the process includes the following steps: Step 1: Structural feature extraction: The system uses multi-scale edge detection operators (including Sobel, Canny, etc.) to extract structural features of the source image at different scales. These features can accurately represent key structural information such as object boundaries, texture direction, and geometric shape.

[0063] Step 2: Structural importance evaluation: Calculate the structural importance of each pixel position and identify key structures such as edges and corners.

[0064] Step 3: Adaptive Weight Generation: The β parameter is dynamically adjusted based on the edit intensity. This parameter adaptively adjusts to generate structural weights. In structurally complex areas (such as edges and corners), the model reduces the aggressiveness of modifications; in structurally simple areas (such as uniform textures), it allows for more innovative modifications. When the user requests a major modification, the structural constraints are appropriately relaxed; when subtle adjustments are required, the structural constraints are tightened.

[0065] Step 4: Attention mechanism fusion: Combine the structural weights with the traditional attention mechanism to form a structure-aware attention distribution.

[0066] Step 5: Diffusion process guidance: Apply structure-preserving constraints during the denoising process of the diffusion model and output the structure-preserving image editing results.

[0067] The structure-preserving attention mechanism excels at preserving the structural integrity of images, particularly in applications requiring high structural accuracy, such as architectural photography, portrait editing, and industrial design. Editing results using structure-preserving attention achieve a 45% improvement in structural consistency scores over baseline methods, while maintaining flexibility and creativity.

[0068] 3) Improve the encoder and introduce a multi-scale semantic alignment network: The multi-scale semantic alignment network solves this problem from the perspective of hierarchical semantics. Figure 4 The system simultaneously extracts image features and semantic representations at multiple spatial resolutions, building a semantic understanding hierarchy from coarse to fine. At the coarse scale, the model focuses on overall semantic categories and layout relationships; at the fine scale, the model focuses on local detail features and boundary accuracy.

[0069] The feature encoder at each scale is trained to extract the optimal semantic representation at the corresponding resolution. This allows for more targeted and precise editing than using a single feature encoder alone. For example, when a text instruction is issued and an image of a broken wire is to be generated, the diffusion model must first understand where the wires are in the image before generating the broken wires. Multi-scale feature encoders facilitate this understanding.

[0070] Semantic similarity calculation uses cosine similarity to measure the degree of match between source image features and target semantics: high similarity area (Sim>0.8): keep the original features and only make minor adjustments; low similarity area (Sim<0.3): require major modifications to achieve the target semantics.

[0071] Semantic alignment is achieved by calculating the cosine similarity between multi-scale features. Regions with high similarity indicate a close match between the source image and the target semantics, and the modification intensity in these regions will be reduced accordingly; regions with low similarity require a greater degree of modification to achieve the target semantics.

[0072] The feature modulation mechanism dynamically adjusts the feature representation based on similarity. The modulation strength γ is calculated through a gating mechanism, which comprehensively considers the source features, target semantics, and similarity information. This enables the model to flexibly adapt to different editing needs while maintaining semantic consistency.

[0073] Multi-scale semantic alignment significantly improves editing accuracy and semantic consistency. In quantitative evaluations, this method achieves significant improvements in semantic segmentation accuracy, boundary sharpness, and content preservation. In particular, it enables more precise semantic region localization and modification in complex scene editing. This approach is particularly well-suited for image editing tasks requiring precise semantic control, such as style transfer, object replacement, and scene modification.

[0074] 4) Introducing adaptive region mask attention to the encoder part: The adaptive region mask attention mechanism integrates three core components: semantic segmentation, intent understanding, and adaptive mask generation, such as Figure 7 .

[0075] The system uses a pre-trained semantic segmentation network to accurately divide the input image into semantic regions and identify different object and region categories.

[0076] The text encoder analyzes the user's editing instructions and extracts the semantic concepts involved in the editing intent. The intent analysis module calculates the relevance score of each semantic region to the editing intent and automatically identifies areas that need to be modified and those that need to be maintained.

[0077] The mask generation process combines the user-provided base mask (if any) with the results of semantic relevance analysis. The system automatically expands the mask to semantically related regions and replaces the hard binary mask with a differentiable soft mask to ensure a smooth transition between edit boundaries.

[0078] The attention modulation mechanism dynamically adjusts the attention weights at each spatial location based on the soft mask. Within the edited region, the model prioritizes the guidance of conditional information; within the protected region, the model prioritizes preserving the original features. This refined attention control ensures precise and natural editing.

[0079] The adaptive region mask attention mechanism significantly improves the intelligence and quality of mask-guided editing. Users no longer need to draw precise masks; the system automatically understands the editing intent and generates appropriate editing regions.

[0080] Example 3 The electronic device of the present invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0081] Many components in a device are connected to the I / O interface, including: input units, such as a keyboard and mouse; output units, such as various types of displays and speakers; storage units, such as magnetic disks and optical disks; and communication units, such as network cards, modems, and wireless communication transceivers. The communication unit allows the device to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks.

[0082] The processing unit performs the various methods and processes described above. For example, in some embodiments, the method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program can be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of the method described above can be performed. Alternatively, in other embodiments, the CPU can be configured to execute the method in any other appropriate manner (e.g., by means of firmware).

[0083] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0084] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0085] In the context of the present invention, machine-readable medium can be a tangible medium that can contain or store a program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0086] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for detecting broken conductors in transmission lines during drone inspection, characterized in that: The method involves augmenting the broken transmission line dataset using traditional data augmentation methods: The diffusion model was used to further expand the broken transmission line dataset, including: using the diffusion model and corresponding hint word vectors to perform background restoration and expansion on the dataset; using the diffusion model's generative capabilities to generate broken strand photos from transmission line images; and further expanding the dataset using the diffusion model to generate datasets with different styles and camera settings. Key areas of the transmission line are located. The main detection network is used to perform high-resolution feature extraction on the data sets of key areas to achieve target detection of broken strands. The auxiliary detection network is used to perform fast and rough detection on non-key areas. The detection results are classified into broken strand areas, possible broken strand areas, and normal areas.

2. The target detection method for broken conductors in transmission lines inspected by drones according to claim 1 is characterized in that: The key area division adopts the method of detecting targets and area expansion, wherein the detection targets include insulator strings, suspension clamps, tension clamps and conductor connection points; the area expansion includes fixed pixel expansion, proportional expansion, adaptive expansion and shape perception expansion.

3. The target detection method for broken conductors in transmission lines inspected by drones according to claim 1 is characterized in that: The auxiliary detection network includes a shallow feature extraction module and an attention enhancement module connected in sequence; The shallow feature extraction module uses three layers of shallow convolution for shallow features; The attention enhancement module adopts a dual attention design, including a channel attention submodule and a spatial attention submodule, to extract attention features.

4. The target detection method for broken conductors in transmission lines during inspection by a UAV according to claim 3 is characterized in that: The auxiliary detection network further includes an anomaly scoring module and a conditional trigger detection module after the attention enhancement module; The anomaly scoring module concatenates shallow features and attention features, and generates anomaly scores through global average pooling and fully connected layers; The conditional trigger detection module dynamically allocates computing resources according to the anomaly score, directly skips the low anomaly score path, and feeds back the high anomaly score path to the main detection network to trigger secondary fine detection.

5. The target detection method for broken conductors in transmission lines during inspection by a UAV according to claim 1 is characterized in that: The diffusion model is a stable diffusion3 network, which includes an encoder and a backbone network, wherein the encoder includes a VAE encoder and a multimodal text encoder; The input image passes through the VAE encoder and outputs a potential representation; Text prompts are fed into a multimodal text encoder, which outputs features as conditional embeddings. The input of the backbone network includes latent representation, temporal embedding and conditional embedding, and the output is predicted noise and velocity field.

6. The target detection method for broken conductors in transmission lines during inspection by a UAV according to claim 5 is characterized in that: The diffusion model adopts Diffusion Transformer as the backbone network, which includes multiple layers of Transformer blocks to process the potential representation of the input.

7. The target detection method for broken conductors in transmission lines during inspection by a UAV according to claim 5 is characterized in that: The backbone network includes three tracking modules connected in series, namely a text tracking module, a visual tracking module and a hybrid tracking module. The text tracking module and the visual tracking module include a self-attention layer, a feedforward network and layer normalization, and the hybrid tracking module includes a cross-attention layer, a feedforward network and layer normalization.

8. The target detection method for broken conductors in transmission lines during inspection by a UAV according to claim 5 is characterized in that: The encoder is a dual-branch conditional encoder that independently encodes the source image and editing conditions through the source image branch and the conditional branch, and then fuses information through a spatially aware cross-attention mechanism. The adaptive weight mechanism dynamically adjusts the fusion strength based on the mask information and feature similarity. The source image branch adopts a convolutional neural network to capture spatial hierarchical feature representation, and the conditional branch uses a multimodal encoder to process different types of editing instructions.

9. The target detection method for broken conductors in transmission lines during inspection by a UAV according to claim 5, characterized in that: The method also includes improving the VAE encoder by adding a structure-preserving attention mechanism after the input image; The structure-preserving attention mechanism guides attention calculation by explicitly modeling the structural information of the image. The process includes the following steps: Step 1, structural feature extraction: The system uses a multi-scale edge detection operator to extract structural features of different scales of the source image; Step 2, structural importance assessment: calculate the structural importance of each pixel position and identify key structures; Step 3, adaptive weight generation: adaptively adjust according to the editing intensity to generate structural weights; Step 4: Attention mechanism fusion: Combine the structural weights with the traditional attention mechanism to form a structure-aware attention distribution; Step 5: Diffusion process guidance: Apply structure-preserving constraints during the denoising process of the diffusion model and output the structure-preserving image editing results.

10. The target detection method for broken conductors in transmission lines during inspection by a UAV according to claim 5, characterized in that: The adaptive region mask attention mechanism is introduced in the encoder part. The adaptive region mask attention mechanism includes a semantic segmentation network, a text encoder, an intent understanding module, and an adaptive mask generation module, including: Use the pre-trained semantic segmentation network to accurately divide the input image into semantic regions and identify different object and region categories; Analyze the user's editing instructions through the text encoder and extract the semantic concepts involved in the editing intention; The intent understanding module calculates the relevance score of each semantic region to the editing intent, automatically identifying the regions that need to be modified and those that need to be retained; The adaptive mask generation module combines the basic mask and semantic relevance analysis results to automatically expand the mask to semantically related areas. At the same time, it uses a differentiable soft mask to replace the hard binary mask and dynamically adjusts the attention weight of each spatial position according to the soft mask.

Images