A method and system for detecting defects of key components of a power transmission line

Abstract

The application discloses a power transmission line key component defect detection method and system, and relates to the technical field of power transmission line defect detection.The method comprises the following steps: generating a text description of a power transmission line inspection image; performing multi-scale feature extraction on the power transmission line inspection image and the text description, performing same-level alignment and cross-level alignment on visual features and text features, and obtaining multi-scale semantic features; identifying power transmission line key components according to the multi-scale semantic features, obtaining a component candidate set, extracting a corresponding detection area for each component, and performing defect detection in the detection area; determining a defect area and a component area corresponding to the defect area for each defect detection candidate result, extracting structure appearance features of the two areas, and obtaining a final defect detection result through consistency analysis of spatial position relationship and matching degree analysis of the structure appearance features. The method realizes reliable detection and analysis of physical state abnormalities of power transmission line key components.

Description

Technical Field

[0001] This invention relates to the field of power transmission line defect detection technology, and in particular to a method and system for detecting defects in key components of power transmission lines. Background Technology

[0002] During long-term operation, defects in critical components of transmission lines are essentially accompanied by changes in the physical state of materials or structures. These changes typically manifest as abnormalities in observable features such as surface morphology, structural integrity, geometric boundaries, texture distribution, and color characteristics. Examples include missing or cracked features caused by insulator damage, changes in surface color and contour due to corrosion of grading rings, and structural displacement caused by loosening or wear of vibration dampers. These appearance changes directly reflect the deterioration of the physical state of materials or structures and are important evidence for monitoring the operational status of transmission lines.

[0003] In recent years, with the development of drone inspection and intelligent image recognition, more and more automated inspection methods based on target detection models or image recognition algorithms have been applied to the field of power transmission line defect detection. However, existing technologies still have the following shortcomings: (1) Lack of multi-level semantic understanding of transmission lines. Most existing detection models are based on single-scale local target features for identification, which makes it difficult to comprehensively consider the attributes of the components themselves, the relationship of the neighborhood structure and the semantics of the overall operation scene of the tower. This results in unstable detection performance in complex backgrounds, small target details and defects, occlusion and multi-component coexistence scenarios.

[0004] (2) Insufficient reliability and logical reliability of detection results. Existing methods usually output detection results directly, lacking a mechanism to judge the semantic consistency and verify the logical rationality of defect detection results. They cannot determine whether the detection results match the semantics of the component category, the relationship of the surrounding structure and the overall environment, which easily leads to false detections and semantically unreasonable detection conclusions.

[0005] (3) Limited adaptability to complex inspection environments. In real power transmission line inspection environments with large changes in lighting, complex shooting angles, significant changes in target size, and strong background interference, conventional detection models are not robust enough and cannot meet the needs of long-term stable power inspection. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes a method and system for detecting defects in key components of transmission lines. It introduces a visual feature modeling approach with multimodal and multigranular data constraints, combined with a step-by-step detection and consistency verification mechanism based on the physical dependencies of components, to achieve reliable detection and analysis of abnormal physical states of key components of transmission lines.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for detecting defects in key components of transmission lines, comprising: Acquire images of power transmission line inspections; Generating text descriptions for images of power transmission line inspections; Multi-scale feature extraction was performed on the images and text descriptions of power transmission line inspections. The obtained visual and text features were then aligned at the same level and cross-level from top to bottom to obtain multi-scale semantic features. Based on multi-scale semantic features, key components of transmission lines are identified to obtain a component candidate set. The component candidate set is traversed, and for each component, the corresponding detection area is extracted. Defect detection is performed within the detection area to obtain defect detection candidate results. The defect detection candidate results for each component are traversed. For each defect detection candidate result, the defect area and its corresponding component area are determined, and the structural appearance features of the two are extracted. The final defect detection result is obtained through the consistency analysis of the spatial position relationship between the two and the matching degree analysis of the structural appearance features.

[0008] As an alternative implementation method, a trained large model is used to generate text descriptions; wherein the training process of the large model includes: The transmission line inspection image samples are decoupled into component-level images. Each component-level image contains only the area where the defective component is located, and a question-and-answer pair is designed based on a text prompt. Using complete power line inspection image samples, design visual question-and-answer pairs for multiple components. Provide a global image description as a prompt for each complete image, and design multiple question-and-answer pairs for each complete image based on this. A stepwise learning strategy is adopted, selecting images containing single components to help the large model focus on the detailed features of the components, and then introducing multi-component images containing multiple components and complex structures to enable the large model to understand the spatial relationships, functional connections and overall semantic structure between components.

[0009] As an alternative implementation method, the process of multi-scale feature extraction for transmission line inspection images includes: adopting a multi-scale feature decoupling approach to split a transmission line inspection image into a single-component image, a multi-component image, and a global image; wherein, the single-component image is directly cropped from the original image with detection box annotations according to the location of the defective component; the multi-component image is randomly expanded and cropped at a ratio of 0.5 to 0.7 times that of the original image, centered on the marked faulty component area, including both the faulty component and the normal components that coexist with it; the global image is input at its original size; the images of the three granularities are processed by three CLIP visual encoders to extract visual features respectively, and then the three visual features are projected onto the same dimensional space by a visual projection layer to obtain the visual embedding; The process of multi-scale feature extraction for text descriptions includes: dividing the text description into global semantic text, multi-part semantic text and single-part semantic text according to the visual scale; using the CLIP text encoder to extract the features of each text respectively; and then projecting them into the same dimensional space as the visual projection layer to obtain the text embedding.

[0010] As an alternative implementation, peer alignment refers to one-to-one matching within the same semantic granularity, that is, aligning the visual features of a single-part image with the text features of the defect description text, aligning the visual features of a multi-part image with the text features of the attribute text and the text features of the spatial relationship text, and aligning the visual features of a global image with the text features of the global description text. Cross-level alignment includes: aligning global text features with visual features of multi-part images and single-part images respectively, and then aligning multi-part text features with visual features of single-part images.

[0011] As an alternative implementation method, the consistency analysis of spatial positional relationships includes: determining whether the defective area is located within the effective structural range of the target component area; if the defective area is significantly deviated from the main structural area of ​​the corresponding component, it is determined that the spatial consistency requirement is not met. The matching degree analysis of structural appearance features includes: comparing whether the defect area conforms to the typical defect characteristics of this type of component in terms of shape distribution, scale ratio and texture variation, thereby determining whether the defect is consistent with the physical structural characteristics of the target component. When the defective area and its corresponding component area meet the consistency requirements in terms of spatial position and structural appearance, the test result is considered a valid defect test result and is retained.

[0012] As an alternative implementation method, after obtaining the final defect detection results, a defect detection report is generated, which includes the defect category, spatial location and its associated component information, as well as the confidence level, and is presented visually. Based on this, the final defect detection results are statistically analyzed, and the number and distribution characteristics of defects are summarized by line, tower or component dimension, and the risk level of the defects is classified.

[0013] Secondly, the present invention provides a defect detection system for key components of transmission lines, comprising: The acquisition module is configured to acquire images of power transmission line inspections; The text generation module is configured to generate text descriptions for images of power transmission line inspections. The alignment module is configured to extract multi-scale features from the images and text descriptions of the power transmission line inspection, and then perform same-level alignment and top-down cross-level alignment on the obtained visual and text features to obtain multi-scale semantic features. The defect detection module is configured to identify key components of the transmission line based on multi-scale semantic features, obtain a component candidate set, traverse the component candidate set, extract the corresponding detection area for each component, perform defect detection within the detection area, and obtain defect detection candidate results. The matching module is configured to traverse the defect detection candidate results for each component, determine the defect area and its corresponding component area for each defect detection candidate result, and extract the structural appearance features of the two. Through the consistency analysis of the spatial position relationship between the two and the matching degree analysis of the structural appearance features, the final defect detection result is obtained.

[0014] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0015] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0016] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: To address the problems of high false alarm rates and difficulty in accurately reflecting the physical state of components during transmission line inspections, which are easily affected by complex background interference, critical component defect detection is difficult to achieve. This invention proposes a method and system for detecting defects in critical transmission line components based on multi-level semantic feature modeling and consistency constraints. By introducing a visual feature modeling approach with multi-modal and multi-granular data constraints, and combining it with a step-by-step detection and consistency verification mechanism based on the physical dependencies of components, reliable detection and analysis of physical state anomalies in critical transmission line components can be achieved. Furthermore, based on the visual representation of the physical state of critical transmission line components, by acquiring transmission line inspection images, the surface state, structural integrity, and abnormal morphologies of critical components are tested and analyzed to determine whether the components have physical state anomalies or defects. This method is designed for actual transmission line operation scenarios and has the advantages of enhanced semantic expression capabilities, improved defect detection accuracy, and guaranteed reliability of detection results.

[0018] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1 This is a typical appearance example of a defect in a key component of a power transmission line provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart of the defect detection method for key components of power transmission lines provided in Embodiment 1 of the present invention; Figure 3 A flowchart for generating a text description provided in Embodiment 1 of the present invention; Figure 4 A schematic diagram of a multi-level feature alignment visual-semantic model provided in Embodiment 1 of the present invention; Figure 5 This is a schematic diagram of the defect detection results on the transmission line inspection image provided in Embodiment 1 of the present invention. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0022] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0023] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms “comprising” and “including”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0025] Example 1 This embodiment addresses the problems of high false alarm rates and difficulty in accurately reflecting the physical state of components during transmission line inspections, which are easily affected by complex background interference. It proposes a defect detection method based on multi-level semantic feature modeling and consistency constraints. By introducing a visual feature modeling approach with multi-modal and multi-granular data constraints, and combining it with a step-by-step detection and consistency verification mechanism based on the physical dependencies of components, reliable detection and analysis of physical state anomalies in key transmission line components can be achieved. Based on the visual representation of the physical state of key transmission line components, by acquiring transmission line inspection images, the surface state, structural integrity, and abnormal morphologies of key components are tested and analyzed to determine whether the components have physical state anomalies or defects.

[0026] As shown in Table 1 and Figure 1 As shown, the defects generated in different key components of transmission lines during operation essentially correspond to abnormal physical states of their materials or structures. This embodiment uses key components of transmission lines as the detection object, summarizing and defining common abnormal physical states during their operation, providing a basis for judging the object and its state for subsequent detection and analysis. By modeling and identifying the appearance features corresponding to different components and their typical defects, effective detection of the operating state of key components of transmission lines is achieved.

[0027] Table 1. Key components of transmission lines and their corresponding defects; .

[0028] like Figure 2 As shown, it includes the following steps: S1: Acquire images of power transmission line inspections.

[0029] In this step, drones are used to inspect and take close-up photos of the transmission line towers to obtain images that show the surface condition and location structure of key components of the transmission line. All transmission line inspection images are then subjected to rigorous preprocessing to remove low-quality images.

[0030] S2: Generate text descriptions for images of power transmission line inspections.

[0031] like Figure 3 The diagram illustrates the process of generating text descriptions of multimodal, multi-granularity transmission line component defects based on a general large-scale model. A multimodal dataset of transmission line component defects is constructed using this general large-scale model, providing a reliable data foundation for subsequent model training.

[0032] Before generating image text descriptions, to improve the accuracy of the general large model in generating text descriptions of transmission line inspection images, a batch of representative transmission line inspection image samples were selected. The transmission line inspection image samples can include all key components and their defects in various forms, materials, angles, and backgrounds. In addition, visual question-and-answer pairs from simple to complex were set up in combination with the content of the transmission line inspection images to guide the large model to gradually learn knowledge of the transmission line defect domain.

[0033] Specifically: First, each selected transmission line inspection image is decoupled into component-level images, meaning each component-level image only contains the area where the defective component is located. Targeted question-and-answer pairs are then designed, with a text prompt provided beforehand, such as "This is an image of a transmission line equipotential ring." Based on this, simple question-and-answer pairs are designed, such as "What is the defect in the image?" and the corresponding answer. This forms a complete set of single-component visual question-and-answer pairs. The same question-and-answer pairs are designed as prompts for each component and its corresponding defect in the sample, providing a framework for the general large-scale model to learn. The goal is for the large-scale model to comprehensively understand transmission line components and defect types through single-component visual question-and-answer pairs.

[0034] Next, using complete images, we continue designing visual question-and-answer pairs for multiple components. Similarly, we first provide a global image description for each image as a prompt. Based on this, we design multiple question-and-answer pairs for each image. The question "What power transmission line components are included in the image?" clarifies the component categories; the question "What spatial relationships exist between these components?" analyzes the spatial distribution and positional relationships between components; the question "What are the common attributes of these components?" extracts key information such as material, color, and shape; and the question "Are there any defects? If so, what are the types, locations, and states of the defects?" provides a detailed description of the defects in the image. This process provides ample contextual support for the general large model, ensuring that it can more accurately grasp the semantics when generating image text descriptions.

[0035] Furthermore, to further optimize the general-purpose large-scale model's accurate understanding of transmission line inspection images, a step-by-step learning strategy is proposed, guiding the model's learning from simple to complex. First, images containing a single target component are selected to help the large-scale model focus on the component's detailed features, enabling it to initially recognize transmission line components. Subsequently, multi-component images containing multiple components and complex structures are introduced, allowing the large-scale model to understand the spatial relationships, functional connections, and overall semantic structure between components. Through this gradual learning process, the large-scale model can progressively adapt to increasing input complexity, thereby improving its multi-dimensional image analysis capabilities and enhancing the accuracy and depth of image description.

[0036] Building upon this foundation, text descriptions will be generated for all transmission line inspection images based on the prior knowledge learned by the general large-scale model. Throughout the generation process, a multi-round question-guided Prompt strategy will be employed to progressively deepen and broaden the questions, ensuring that the text descriptions generated by the large-scale model are logically rigorous, comprehensive, and detailed. Through this strategy, the large-scale model can gradually construct a comprehensive description of components, relationships, attributes, and defects in the images. Each round of generated text descriptions will undergo manual verification to ensure accuracy and high quality. These high-quality image-text pairs will provide a solid data foundation for the intelligent operation and maintenance of transmission lines and will also lay an important foundation for the further development of intelligent defect detection and diagnosis systems.

[0037] S3: Multi-scale feature extraction is performed on the images and text descriptions of the power transmission line inspection. The obtained visual features and text features are then aligned at the same level and cross-level from top to bottom to obtain multi-scale semantic features.

[0038] Due to factors such as the shooting angle, height, and component complexity during inspections, defects in the same component of a power transmission line exhibit significant differences in target size, detail, and context in different images, resulting in scale diversity. Existing models struggle to construct a unified and robust multi-scale feature space under multimodal conditions, which greatly limits the precise detection and identification of defects.

[0039] To address this challenge, this embodiment utilizes the multimodal, multi-granular dataset constructed in step S2 and designs a collaborative learning framework for global and local features using a multimodal contrastive learning algorithm. In global feature learning, global contextual information is extracted through alignment and constraints between the complete inspection image and the overall text description. In local feature learning, fine-grained feature modeling is achieved through alignment between local component images and text descriptions. A unified multimodal, multi-scale feature space is constructed through the collaborative optimization of global and local features.

[0040] like Figure 4 As shown, it mainly includes three steps: visual feature extraction, semantic feature extraction, and multi-level feature alignment.

[0041] Specifically: (1) Visual feature extraction. Using the idea of ​​multi-scale feature decoupling, a defect image of a transmission line component is divided into three scales: single component, multi-component, and global, to explicitly adapt to inputs with different receptive fields and different semantic granularities.

[0042] Among them, single component defect images Obtained directly by cropping from the ground truth detection bounding box (can be directly cropped according to the location of the defective component); multi-component image. Centered on the marked faulty component area, the image is randomly expanded and cropped at a ratio of 0.5 to 0.7 times that of the original image, including both the faulty component and the normal components that coexist with it; the global image retains its original size, as shown below: Images of three different granularities will be processed by three CLIP visual encoders. Visual features were extracted separately. , , : ;in, These represent three granularities: single-component defects, multi-component defects, and global defects, respectively.

[0043] Next, the three visual features are fed into the three projection layers respectively. Projecting them onto the same dimensional space yields visual embeddings. , , : ;in, , and These represent visual feature representations of single-component defects, multi-component defects, and global defects, respectively.

[0044] (2) Semantic Feature Extraction. For multi-scale visual input, a multi-granularity text description corresponding to the structure is constructed to reduce semantic mismatches caused by "the image is local, but the text is global" or "the image is global, but the text is too detailed." Specifically, the text description generated by the general large model is divided into three levels according to the visual scale: including global semantic text... Multi-component semantic text and Single-component semantic text .

[0045] Correspondingly, three CLIP text encoders are used. Extract features from the four text descriptions corresponding to the three granularities to obtain , , : ;in, , Indicates a single component, Indicates multiple component attributes. Indicates the spatial relationship between multiple components. This represents the global context, specifically four texts at three different granularities.

[0046] Next, the four text features at three different granularities are fed into three projection layers respectively. The text embeddings are obtained by projecting them onto the same dimensional space as the visual projection layer. , , : ; .

[0047] Among them, the attributes and spatial relationships obtained at the multi-component level are simply averaged after projection to obtain the multi-target semantic embedding, which is used to align with the multi-target visual embedding.

[0048] Among them, attribute text describes the shape, material, color and other general identifiable attributes of all parts appearing in the same image, which helps large models to identify "who is who" in multi-part scenes; spatial relationship text explicitly describes the spatial topological relationships between each part, such as up and down, left and right, inside and outside, adjacent and connected, to supplement similar parts that are difficult to distinguish by appearance features alone.

[0049] Defect description text at the single-component level focuses on depicting details such as the type, morphological changes, and surface condition of the abnormal transmission line components in the image, providing direct semantic supervision for small target and single-component detection.

[0050] Global description text at the global level summarizes the entire image of the transmission line components, covering foreground targets and background environment, enabling the model to have scene-level discrimination capabilities.

[0051] Since the semantic alignment capability of visual-language models relies on large-scale image-text pre-training, full fine-tuning is computationally expensive and may disrupt the original general representation space. Therefore, a parameter-efficient fine-tuning strategy is adopted for lightweight adaptation: on the one hand, a learnable cue vector is introduced in the text encoder to replace the fixed prompt, enabling the model to adaptively learn domain semantics; on the other hand, only the parts with a very small number of parameters but sensitive to distribution adjustment are unfrozen, namely, the training layer normalization (LayerNorm, LN) layer, the frozen feed-forward network (FFN) layer and the self-attention (Attn) layer in the visual encoder, and only the bias of the FFN layer is trained in the text encoder, while the linear layer and the QuickGELU activation function are frozen, to achieve efficient parameter fine-tuning.

[0052] During the training phase, the model simultaneously acquires consistency constraints between images and text at the same scale and collaborative constraints between multiple scales. This not only improves the fine-grained recognition capability of defective targets, but also maintains the unity and robustness of the multimodal and multi-scale feature space in complex inspection scenarios.

[0053] (3) Multi-level feature alignment.

[0054] After projecting the visual features obtained from the three levels and their corresponding text features onto a unified contrast space, two types of feature alignment are performed in this space: sibling alignment and cross-level alignment.

[0055] Feature alignment is a common method in multimodal models. For example, the alignment of features between image and text modalities is essentially achieved through contrastive learning. In a unified embedding space, it brings matching image-text pairs closer together and pushes away mismatched pairs, thereby achieving cross-modal semantic alignment. In short, feature alignment uses contrastive learning to calculate the similarity between image and text features.

[0056] Sibling alignment aligns the features of images and text at the same level, while cross-level alignment aligns the features based on the semantic inclusion relationship between images and text at different levels, that is, aligning high-level text features with low-level image features. Finally, sibling and cross-level alignment are constrained by corresponding loss functions.

[0057] Specifically: Peer alignment involves one-to-one matching within the same semantic granularity. This means aligning single-component image features with defect description text features, multi-component image features with attribute and relation text features, and global image features with global description text features, to ensure that each scale can learn a semantically consistent representation.

[0058] Considering that different levels of text have a relationship of inclusion and being included: global descriptions often involve scene background, component attributes and defect targets at the same time, and multi-component descriptions may also contain detailed descriptions of a single component. In order to make reverse use of this richer linguistic information, a top-down cross-level alignment is further designed, that is: the text features at the global level are aligned with the visual features of multi-component and single-component respectively, and the text features at the multi-component level are then aligned with the visual features of single-component, thereby gradually pushing high-level semantics into the local visual representation and improving the semantic sufficiency and consistency of local features.

[0059] The entire feature alignment process described above is optimized within a unified space through comparative constraints.

[0060] To ensure that images and text of different granularities can be aligned and that cross-scale semantics can be propagated downwards, and to avoid destroying the general representation obtained by CLIP pre-training during fine-tuning, the loss of this model consists of two parts: adaptive weighted fusion multi-granularity alignment loss and feature consistency constraint loss.

[0061] First, the multi-granularity alignment loss of adaptive weighted fusion.

[0062] First, at the same granularity semantic level, the model directly aligns visual features and text features of the same granularity, i.e., at the global level. and Multi-component level and and single-component level and A one-to-one correspondence exists between them. This process ensures the basic semantic consistency of visual representations at corresponding semantic levels. For any granularity... The loss for sibling alignment is: .

[0063] Furthermore, considering that high-level semantics often implicitly constrain low-level visual representations, this paper introduces a cross-granularity semantic alignment mechanism based on peer-level alignment. Specifically, this mechanism is achieved by aligning global semantic features... Visual features of multiple parts and visual features of single components Alignment is performed, and multi-part semantic features are combined. Further constrain the visual features of individual components We construct a multi-granularity semantic constraint path from global to local, and gradually strengthen the semantic response of local defect areas under the guidance of high-level semantics.

[0064] The corresponding cross-level alignment loss is: .

[0065] To comprehensively consider the importance of different levels and alignment paths, all intra-level alignment and cross-level alignment losses are adaptively weighted and fused to obtain the final multi-level alignment loss: ; in, The weights for each alignment branch can be set or learned based on the emphasis on fine-grained defect identification or global semantic preservation in the task.

[0066] Secondly, the loss due to feature consistency constraints.

[0067] Because the parameters of the CLIP visual encoder and text encoder were not fully unfrozen, but only a small portion of the parameters were trained, relying entirely on the newly learned projection features for alignment can easily lead to task specialization and a significant deviation from the original CLIP semantic coordinate system. Therefore, feature consistency constraint loss was introduced at each granularity level to maintain the pre-trained semantics and mitigate overfitting.

[0068] Taking the single-object vision branch as an example, the trained visual features Corresponding features to the frozen CLIP output After normalization, the cosine similarity is calculated, and the consistency loss is defined as: .

[0069] Both the visual and textual sides apply the same constraints to all three granularities to ensure that the fine-tuned semantic vectors still fall near the semantic manifold pre-trained by CLIP.

[0070] The consistency constraint loss after synthesis is: .

[0071] The final training objective is obtained by weighted summation of alignment and consistency terms: ; in, This is a balancing factor used to control the weights between making the model more suitable for the current task and maintaining the stability of the pre-trained semantics.

[0072] S4: Identify key components of transmission lines based on multi-scale semantic features to obtain a candidate set of components. Traverse the candidate set of components, extract the corresponding detection area for each component, perform defect detection within the detection area, and obtain candidate detection results.

[0073] After completing the training of a visual-semantic model with semantic expression capabilities in the power field in step S3, a hierarchical detection framework for transmission line inspection images is established on this basis to achieve collaborative identification of key component detection and defect detection.

[0074] First, the inspection images Input to visual-semantic model Extracting multi-scale semantic feature sets The features at different scales simultaneously encode structural information and electrical semantic information, providing a unified and discriminative expression for subsequent detection.

[0075] Based on the above characteristics, through the component inspection head The key components of the transmission line were identified, and a candidate set of components was obtained. ,in Indicates the first The bounding box of each component. Indicates the first The categories of each component, Indicates the first The confidence level of each component, where n is the number of components.

[0076] because Generated by a visual-semantic model, this stage of detection not only relies on visual texture features but is also constrained by the electrical semantic space, thus maintaining a high detection rate and positioning accuracy even under conditions such as long-distance shooting, large proportion of small targets, and complex background interference.

[0077] After obtaining relatively stable component detection results, the detection process further enters the component-constraint-based defect identification stage. This method does not directly perform unconstrained defect search across the entire image, but instead constructs a defect detection search space centered on each component candidate region. Specifically: for each component detection result... By using regional feature extraction operators Extract the corresponding detection region from the multi-scale feature map. And enter it into the part category Matching defect detection branch A set of candidate defect detection results is obtained. ,in For the defect box, Defect type The confidence level for defect detection.

[0078] Through a hierarchical detection mechanism, defect identification is performed only within areas with physical dependencies and semantic relationships. This narrows the detection space, significantly reduces the probability of false detections, and ultimately forms a system... The structured inspection results, presented as "component-defect" units, provide a basis for subsequent consistency assessment.

[0079] In summary, after acquiring images of transmission line inspections, key components are first detected and located, and the corresponding inspection areas for each key component are determined. Subsequently, physical anomalies of components are detected only within the inspection areas, thereby generating candidate defect detection results for physical anomalies.

[0080] S5: Traverse the defect detection candidate results for each component. For each defect detection candidate result, determine the defect area and its corresponding component area, and extract the structural appearance features of the two. Through the consistency analysis of the spatial position relationship between the two and the matching degree analysis of the structural appearance features, the final defect detection result is obtained.

[0081] After completing defect detection based on component region constraints, due to the complex background and dense components in the inspection images, there may still be cases where the defect detection results do not match the actual physical structure of the components. To address this, a consistency verification mechanism for detection results is further introduced to perform a secondary screening of candidate defect detection results, thereby suppressing false alarms caused by semantically unreasonable or physically invalid results.

[0082] Specifically: (1) For each defect detection candidate result, first obtain its corresponding defect area and the component area associated with the defect area. By analyzing the consistency of the two in spatial position, determine whether the defect area is within the effective structural range of the target component area.

[0083] The effective structural range is defined as the detection box region of the component recognition detection box. Since the component regions in the training images are labeled before training the component recognition model, the detection box of each component contains the complete component region. Therefore, the detection box region of the component recognition result can be defined as the effective structural range of the component region.

[0084] (2) If the defect area deviates significantly from the effective structural range of the corresponding component, the test result is deemed not to meet the spatial consistency requirements.

[0085] Under normal circumstances, a defect in a component should appear on the component itself, such as broken insulators or rusted vibration dampers. Therefore, if the defect area corresponding to the component is not within the component's area or only a small part of it is within the component's area, it is defined as the defect area significantly deviating from the effective structural range of the corresponding component. It can be understood that both the component area and the defect area refer to the detection box area in the results of the component identification model and the defect detection model.

[0086] For all defects, firstly, the spatial position of the detection box in the component identification and defect detection results is used to determine if the defect area is not within the corresponding component area. If the defect area is not within the corresponding component area, it is determined that the defect area is significantly deviated from the component area. For some defect areas that are likely to appear on the edge of the component, such as small-scale corrosion of hardware, the area of ​​the overlapping area between the defect area and the component area is calculated. If the area of ​​the overlapping area is less than 40% of the area of ​​the defect area, it is determined that the defect area is significantly deviated from the corresponding component area.

[0087] (3) On this basis, further evaluate the degree of matching between the defect area and the corresponding component area in terms of local appearance features and structural morphology; for example, by comparing whether the defect area conforms to the typical defect characteristics of the component in terms of shape distribution, scale ratio and texture variation, it can be determined whether the defect is consistent with the physical structural characteristics of the target component.

[0088] The method for determining whether a component meets the criteria is as follows: First, select a region in the component area that is in the same location as the corresponding defect area, and use a convolutional neural network to extract features from this region and the defect area respectively. Then, calculate the feature similarity between the region and the defect area. If the similarity score is greater than 90%, the defect is determined to meet the typical defect characteristics of this type of component; otherwise, it does not meet the criteria.

[0089] (4) When the defect area and its corresponding component area meet the consistency requirements in terms of spatial position and structural appearance, the test result shall be regarded as a valid defect test result and retained; otherwise, for test results that do not meet the consistency requirements, they may be marked as results to be manually reviewed according to preset rules, thereby ensuring the recall capability of the test, effectively reducing the false alarm rate and improving the reliability of the overall test results.

[0090] In summary, during the consistency verification stage, candidate defect detection results are first obtained, and the defect area and its corresponding component area are determined. Subsequently, the structural appearance features of the defect area and the component area are extracted, and the consistency analysis of their spatial position relationship and the matching degree analysis of their structural appearance features are performed. When both meet the preset requirements, the detection result is retained; otherwise, the detection result is transferred to the manual verification process, thereby improving the reliability of the final detection result.

[0091] In this embodiment, after completing the semantic consistency verification, the final defect detection results are structurally integrated to form a defect detection report. This report includes not only the defect category, spatial location, and information about the component to which it belongs, but also the detection confidence level. Subsequently, the detection results are visualized by overlaying component frames and defect frames onto the original inspection image or its magnified local area. Different colors or annotation methods can be used to distinguish different types of defects and different risk levels, making the defect results highly intuitive and interpretable.

[0092] Based on this, the defect detection results are further statistically analyzed, and the number and distribution characteristics of defects are summarized according to the dimensions of lines, towers or components. The risk level of defects is classified in combination with the confidence level, and high-risk defects or key targets of concern are automatically identified.

[0093] like Figure 5 The image shown is a schematic diagram of defect detection results on a power transmission line inspection image. Among them, Figure 5 The English labels in the text are as follows: insulator damage; insulator bunch-drop; shockproof hammer intersection; gradingring damage; and shielded ring corrosion. Using these methods, abnormal physical conditions of key components such as insulators, grading rings, and shockproof hammers can be effectively detected under complex background conditions, and the spatial location of defects can be accurately determined, thus providing a reliable basis for subsequent operational status assessment and maintenance decisions.

[0094] Example 2 This embodiment provides a defect detection system for key components of power transmission lines, including: The acquisition module is configured to acquire images of power transmission line inspections; The text generation module is configured to generate text descriptions for images of power transmission line inspections. The alignment module is configured to extract multi-scale features from the images and text descriptions of the power transmission line inspection, and then perform same-level alignment and top-down cross-level alignment on the obtained visual and text features to obtain multi-scale semantic features. The defect detection module is configured to identify key components of the transmission line based on multi-scale semantic features, obtain a component candidate set, traverse the component candidate set, extract the corresponding detection area for each component, perform defect detection within the detection area, and obtain defect detection candidate results. The matching module is configured to traverse the defect detection candidate results for each component, determine the defect area and its corresponding component area for each defect detection candidate result, and extract the structural appearance features of the two. Through the consistency analysis of the spatial position relationship between the two and the matching degree analysis of the structural appearance features, the final defect detection result is obtained.

[0095] It should be noted that the above modules correspond to the steps described in Embodiment 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0096] In further embodiments, the following is also provided: An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0097] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0098] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0099] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in Embodiment 1.

[0100] The method in Example 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0101] A computer program product includes a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0102] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0103] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0104] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0105] Those skilled in the art will recognize that the units and algorithm steps described in connection with the various examples of this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0106] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for detecting defects in key components of transmission lines, characterized in that, include: Acquire images of power transmission line inspections; Generating text descriptions for images of power transmission line inspections; Multi-scale feature extraction was performed on the images and text descriptions of power transmission line inspections. The obtained visual and text features were then aligned at the same level and cross-level from top to bottom to obtain multi-scale semantic features. Based on multi-scale semantic features, key components of transmission lines are identified to obtain a component candidate set. The component candidate set is traversed, and for each component, the corresponding detection area is extracted. Defect detection is performed within the detection area to obtain defect detection candidate results. The defect detection candidate results for each component are traversed. For each defect detection candidate result, the defect area and its corresponding component area are determined, and the structural appearance features of the two are extracted. The final defect detection result is obtained through the consistency analysis of the spatial position relationship between the two and the matching degree analysis of the structural appearance features.

2. The method for detecting defects in key components of a transmission line as described in claim 1, characterized in that, The trained large model is used to generate text descriptions; the training process of the large model includes: The transmission line inspection image samples are decoupled into component-level images. Each component-level image contains only the area where the defective component is located, and a question-and-answer pair is designed based on a text prompt. Using complete power line inspection image samples, design visual question-and-answer pairs for multiple components. Provide a global image description as a prompt for each complete image, and design multiple question-and-answer pairs for each complete image based on this. A stepwise learning strategy is adopted, selecting images containing single components to help the large model focus on the detailed features of the components, and then introducing multi-component images containing multiple components and complex structures to enable the large model to understand the spatial relationships, functional connections and overall semantic structure between components.

3. The method for detecting defects in key components of a transmission line as described in claim 1, characterized in that, The process of multi-scale feature extraction for transmission line inspection images includes: adopting a multi-scale feature decoupling approach to split a transmission line inspection image into single-component images, multi-component images, and a global image; the single-component image is directly cropped from the original image with detection boxes labeled according to the location of the defective component; the multi-component image is randomly expanded and cropped at a ratio of 0.5 to 0.7 times the original image, centered on the labeled faulty component area, including both the faulty component and the normal components that coexist with it; the global image is input at its original size; the images of the three granularities are processed by three CLIP visual encoders to extract visual features, and then the three visual features are projected onto the same dimensional space by a visual projection layer to obtain the visual embedding; The process of multi-scale feature extraction for text descriptions includes: dividing the text description into global semantic text, multi-part semantic text and single-part semantic text according to the visual scale; using the CLIP text encoder to extract the features of each text respectively; and then projecting them into the same dimensional space as the visual projection layer to obtain the text embedding.

4. The method for detecting defects in key components of a transmission line as described in claim 3, characterized in that, Peer alignment refers to one-to-one matching within the same semantic granularity, that is, aligning the visual features of a single-part image with the text features of the defect description text, aligning the visual features of a multi-part image with the text features of the attribute text and the text features of the spatial relationship text, and aligning the visual features of a global image with the text features of the global description text. Cross-level alignment includes: aligning global text features with visual features of multi-part images and single-part images respectively, and then aligning multi-part text features with visual features of single-part images.

5. The method for detecting defects in key components of a transmission line as described in claim 1, characterized in that, The consistency analysis of spatial location relationships includes: determining whether the defect area is located within the effective structural range of the target component area; if the defect area deviates significantly from the main structural area of ​​the corresponding component, it is determined that the spatial consistency requirement is not met. The matching degree analysis of structural appearance features includes: comparing whether the defect area conforms to the typical defect characteristics of this type of component in terms of shape distribution, scale ratio and texture variation, thereby determining whether the defect is consistent with the physical structural characteristics of the target component. When the defective area and its corresponding component area meet the consistency requirements in terms of spatial position and structural appearance, the test result is considered a valid defect test result and is retained.

6. The method for detecting defects in key components of a transmission line as described in claim 1, characterized in that, After obtaining the final defect detection results, a defect detection report is generated, which includes the defect category, spatial location and information of the component to which it belongs, as well as the confidence level, and is presented visually. Based on this, the final defect detection results are statistically analyzed, and the number and distribution characteristics of defects are summarized by line, tower or component dimension, and the risk level of the defects is classified.

7. A defect detection system for key components of transmission lines, characterized in that, include: The acquisition module is configured to acquire images of power transmission line inspections; The text generation module is configured to generate text descriptions for images of power transmission line inspections. The alignment module is configured to extract multi-scale features from the images and text descriptions of the power transmission line inspection, and then perform same-level alignment and top-down cross-level alignment on the obtained visual and text features to obtain multi-scale semantic features. The defect detection module is configured to identify key components of the transmission line based on multi-scale semantic features, obtain a component candidate set, traverse the component candidate set, extract the corresponding detection area for each component, perform defect detection within the detection area, and obtain defect detection candidate results. The matching module is configured to traverse the defect detection candidate results for each component, determine the defect area and its corresponding component area for each defect detection candidate result, and extract the structural appearance features of the two. Through the consistency analysis of the spatial position relationship between the two and the matching degree analysis of the structural appearance features, the final defect detection result is obtained.

8. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the method described in any one of claims 1-6.

Images