Power transmission line pin defect semantic generation method, system, device and storage medium

Abstract

This application provides a method, system, device, and storage medium for generating semantic information about pin defects in transmission lines, relating to the field of transmission line inspection technology. The method includes: acquiring a target image of the transmission line; inputting the target image into an insulator string detection model to obtain insulator string detection results; performing multi-level region cropping on the target image based on the insulator string detection results to obtain a first target image sequence and a second target image sequence; performing pin defect detection on the first target image sequence to obtain pin defect detection results; performing spatial structure detection on the second target image sequence to obtain spatial structure detection results; and performing component relationship binding and spatial structure binding based on the insulator string detection results, pin defect detection results, and spatial structure detection results to generate pin defect semantic information. This application achieves information conversion from visual images to natural language, significantly improving the business accessibility of pin defect localization in transmission lines.

Description

Technical Field

[0001] This application relates to the field of power transmission line testing technology, specifically to a method, system, device, and storage medium for generating semantics of pin defects in power transmission lines. Background Technology

[0002] In the operation and maintenance of power transmission lines, pin defects directly threaten the structural safety of insulator strings, making them a key task for power transmission drone inspections. Defect detection in power transmission lines often employs one-stage or multi-stage detection algorithms. However, given the diverse types of pin defects, their small regional proportions, and the complex spatial structure of the components they pertain to, directly using one-stage or multi-stage detection algorithms often results in low defect detection rates and high false alarm rates.

[0003] Currently, pin defect detection based on UAV images mostly outputs rectangular boxes and category labels. Although it can locate specific defect targets, maintenance personnel are not sensitive to the pixel coordinates of the defect targets and need to repeatedly confirm the location of the defect before they can carry out defect repair tasks, thereby reducing the efficiency of power transmission line maintenance. Summary of the Invention

[0004] In view of the above-mentioned shortcomings of the prior art, this application provides a method, system, device and storage medium for generating semantics of pin defects in transmission lines, which effectively solves the problem that existing pin defect detection methods require maintenance personnel to repeatedly confirm the location of the defect, resulting in low maintenance efficiency of transmission lines.

[0005] In a first aspect, this application provides a method for generating semantic representations of pin defects in transmission lines, the method comprising: Acquire a target image of the transmission line, input the target image into the insulator string detection model, and obtain the insulator string detection result; Based on the insulator string detection results, the target image is cropped at multiple levels to obtain a first target image sequence and a second target image sequence. The first target image sequence is sequentially input into the pin defect detection model, which outputs multiple defect rectangles and corresponding defect types. Based on the defect types, the defect rectangles are subjected to confidence filtering and area non-maximum suppression to obtain the pin defect detection results. The second target image sequence is sequentially input into the spatial structure detection model, which outputs multiple spatial structure bounding boxes and corresponding spatial structure types. Confidence filtering and non-maximum suppression processing are performed based on the spatial structure types to obtain spatial structure detection results. Based on the insulator string detection results, the pin defect detection results, and the spatial structure detection results, component relationship binding and spatial structure binding are performed to generate pin defect semantic information.

[0006] In an optional implementation, the step of binding component relationships and spatial structures based on the insulator string detection results, the pin defect detection results, and the spatial structure detection results to generate pin defect semantic information includes: Based on the insulator string detection results and the pin defect detection results, the variation overlap rate is calculated to obtain the component variation overlap rate; The component variation overlap rate and the component variation overlap rate threshold are compared to obtain the component binding relationship result, and the pin defect relationship detection result is determined based on the component binding relationship result. Calculate the center point coordinates of each spatial structure rectangle based on the spatial structure detection results, and determine the spatial structure sequence detection results based on the center point coordinates. Based on the detection results of the pin defect relationship and the detection results of the spatial structure sequence, the variation overlap rate is calculated to obtain the structural variation overlap rate. The structural binding relationship result is obtained by comparing the structural variation overlap rate and the structural variation overlap rate threshold, and the pin defect semantic detection result is determined based on the structural binding relationship result. The pin defect semantic information is generated based on the pin defect semantic detection results.

[0007] In an optional implementation, the formula for calculating the variation overlap rate based on the insulator string detection results and the pin defect detection results is as follows:

[0008] In the above formula, This indicates that the defect type in the pin defect detection result is... The A defective rectangular box The insulator string type in the insulator string detection results is The A rectangular frame of an insulator string. The component variation overlap rate, This represents the intersection operation of rectangles. The area operator for a rectangle; The step of comparing the component variation overlap rate with the component variation overlap rate threshold to obtain the component binding relationship result, and determining the pin defect relationship detection result based on the component binding relationship result, includes: If the component variation overlap rate is greater than or equal to the component variation overlap rate threshold, then the defect rectangle and the insulator string rectangle corresponding to the component variation overlap rate have a component binding relationship. The defect rectangles containing all the components with the aforementioned binding relationship, along with their corresponding defect types, are used as the pin defect relationship detection results.

[0009] In an optional implementation, the formula for calculating the variation overlap rate based on the pin defect relationship detection result and the spatial structure sequence detection result is as follows:

[0010] In the formula, This indicates that the defect type in the pin defect relationship detection result is... The A defective rectangular box The spatial structure type in the spatial structure sequence detection results is The A spatial structure rectangle The overlap rate of structural variations This represents the intersection operation of rectangles. The area operator for a rectangle; The step of comparing the structural variation overlap rate with a structural variation overlap rate threshold to obtain a structural binding relationship result, and determining a pin defect semantic detection result based on the structural binding relationship result, includes: If the structural variation overlap rate is greater than or equal to the structural variation overlap rate threshold, then the defect rectangle and the spatial structure rectangle corresponding to the structural variation overlap rate have a structural binding relationship. The pin defect semantic detection result is defined as the complete set of defect rectangles containing the structural binding relationship and the corresponding defect type.

[0011] In an optional implementation, the pin defect detection model includes a backbone network, a feature pyramid network, and a decoupling network. The feature pyramid network includes a self-attention convolution module and a channel attention interaction module. The step of sequentially inputting the first target image sequence into the pin defect detection model and outputting multiple defect bounding boxes and corresponding defect types includes: The first target image in the first target image sequence is input into the backbone network for feature extraction to obtain the first layer feature map, the second layer feature map and the third layer feature map; The second layer feature map and the third layer feature map are input into the self-attention convolution module to obtain the first self-attention convolution feature map; The first self-attention convolutional feature map is concatenated with the second layer feature map to obtain the first fused feature map; The first fused feature map and the first layer feature map are input into the self-attention convolution module to obtain the second self-attention convolution feature map; The second self-attention convolutional feature map is concatenated with the first layer feature map to obtain the second fused feature map; The second fused feature map is convolved to generate a first intermediate feature map with the same resolution as the first fused feature map, and the first intermediate feature map is concatenated with the first fused feature map to obtain a third fused feature map. The third fused feature map is convolved to generate a second intermediate feature map with the same resolution as the third layer feature map, and the second intermediate feature map is concatenated with the third layer feature map to obtain a fourth fused feature map; The second fused feature map, the third fused feature map, and the fourth fused feature map are respectively input into the channel attention interaction module for feature enhancement to obtain the first target feature map, the second target feature map, and the third target feature map; The first target feature map, the second target feature map, and the third target feature map are input into the decoupled network for classification and regression processing to obtain defect types and defect bounding boxes. Multiple first target images yield multiple defect types and defect bounding boxes.

[0012] In an optional implementation, the step of performing confidence filtering and area non-maximum suppression processing on the defect rectangle based on the defect type to obtain the pin defect detection result includes: Different confidence thresholds are set according to different defect types, and the defect rectangles corresponding to those less than the confidence thresholds are deleted to obtain the initial pin defect detection results; For different defect types in the initial pin defect detection results, different minimum area overlap rate thresholds are set; Sort the areas of all the defect rectangles corresponding to each defect type, select the defect rectangle with the largest area as the registration rectangle, and use the remaining defect rectangles as the first sequence of rectangles to be tested. Calculate the minimum area overlap rate between the test rectangles in the first test rectangle sequence and the registration rectangle; If the minimum area overlap rate is greater than the minimum area overlap rate threshold, then the corresponding rectangle to be tested is deleted to obtain the target rectangle sequence. In the target rectangle sequence, the target rectangle with the largest area is selected again as the registration rectangle, and the remaining target rectangles are used as the second sequence of rectangles to be tested. The minimum area overlap rate calculation and threshold comparison are repeatedly performed based on the second test rectangle sequence and the registration rectangle until the minimum area overlap rate of all test rectangles and the selected registration rectangle is less than or equal to the minimum area overlap rate threshold. The selected registration rectangles and their corresponding defect types are used as the pin defect detection results.

[0013] In an optional implementation, the target image is cropped at multiple levels based on the insulator string detection results to obtain a first target image sequence and a second target image sequence, including: The rectangular frames of all insulator strings in the insulator string detection results are merged to obtain the merged insulator string frame; Within the range of the insulator string merging frame, a sliding window image selection is performed to obtain multiple sliding window regions; The target image is cropped according to each of the sliding window regions to obtain the first target image sequence; The target image is cropped according to the rectangular frames of each insulator string to obtain the second target image sequence.

[0014] Secondly, this application provides a semantic generation system for transmission line pin defects, the system comprising: The insulator string detection module is used to acquire a target image of the transmission line, input the target image into the insulator string detection model, and obtain the insulator string detection result. An insulator string trimming module is used to perform multi-level region trimming on the target image based on the insulator string detection results to obtain a first target image sequence and a second target image sequence; The pin defect detection module is used to input the first target image sequence into the pin defect detection model in sequence, output multiple defect rectangles and corresponding defect types, and perform confidence filtering and area non-maximum suppression processing on the defect rectangles based on the defect types to obtain the pin defect detection results. The spatial structure detection module is used to input the second target image sequence into the spatial structure detection model in sequence, output multiple spatial structure rectangles and corresponding spatial structure types, and perform confidence filtering and non-maximum suppression processing based on the spatial structure types to obtain spatial structure detection results. The defect semantic generation module is used to perform component relationship binding and spatial structure binding based on the insulator string detection results, the pin defect detection results, and the spatial structure detection results, and generate pin defect semantic information.

[0015] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for generating semantics of transmission line pin defects as described in the first aspect of this application.

[0016] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for generating semantics of transmission line pin defects as described in the first aspect of this application.

[0017] The method, system, equipment, and storage medium for generating semantic information about transmission line pin defects provided in this application generate pin defect semantic information through a progressive architecture of insulator string detection, multi-level region trimming, dual detection of pin defects and spatial structures, and semantic binding. This transforms the subordinate relationships and spatial structural relationships of defective components of concern to maintenance into executable semantic information that can be directly used for power defect maintenance and repair. It achieves information conversion from visual images to natural language and information dimensionality upgrade from pixel coordinates to business coordinates, providing powerful methodological support for intelligent maintenance. Simultaneously, it significantly improves the business accessibility of pin defect location in transmission lines, reduces the burden of manual interpretation, supports automated work order generation and accurate defect elimination, and effectively reduces the false alarm rate in actual measurements. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the method for generating semantics of transmission line pin defects provided in an embodiment of this application; Figure 2 This is a schematic diagram of the pin defect detection model in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the self-attention convolution module in the embodiments of this application; Figure 4 This is a schematic diagram of the channel attention interaction module in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of the semantic generation system for transmission line pin defects provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0020] Explanation of key component symbols: 200. Transmission line pin defect semantic generation system; 210. Insulator string detection module; 220. Insulator string cutting module; 230. Pin defect detection module; 240. Spatial structure detection module; 250. Defect semantic generation module; 300. Electronic equipment; 310. Processor; 320. Communication interface; 330. Memory; 340. Communication bus. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be further described clearly and completely below with reference to the accompanying drawings of the embodiments. It should be noted that the described embodiments are merely some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0022] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0024] Currently, pin defect detection in transmission lines often employs one-stage or multi-stage detection algorithms. However, transmission line pin defects are characterized by numerous types, small area coverage, and complex spatial structures of the components. Therefore, directly using one-stage or multi-stage detection algorithms often results in low detection rates and high false alarm rates. Even if a specific defect target can be located, maintenance personnel are not sensitive to the pixel coordinates of the defect target and need to repeatedly confirm the location of the defect before carrying out defect repair tasks, thus reducing the efficiency of transmission line operation and maintenance.

[0025] Example 1 This application provides a method for generating semantics of pin defects in transmission lines, which effectively solves the problem that existing pin defect detection methods require maintenance personnel to repeatedly confirm the location of the defect, resulting in low maintenance efficiency of transmission lines.

[0026] It should be noted that the pins in this application embodiment can be M-pins. M-pins are a key type of hardware used in transmission lines to connect and fix insulator strings, mainly distributed between insulators and between insulators and other hardware. It is understood that other types of pins can also use the method provided in this application embodiment to generate defect semantic information. Figure 1 This is a schematic diagram of the semantic generation method for transmission line pin defects provided in an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps: S100. Obtain the target image of the transmission line, input the target image into the insulator string detection model, and obtain the insulator string detection result.

[0027] In this embodiment, since M-pins are mainly distributed between insulators and between insulators and fittings, in order to generate accurate defect semantic information, it is necessary to refine the names of insulator strings of specific materials from an operation and maintenance perspective. Simultaneously, in order to capture pin defects generated within these insulator strings of specific materials and to identify candidate regions for subsequent pin defect detection, it is necessary to inspect these insulator strings of specific materials.

[0028] In one implementation, drone images of the transmission line are acquired through drone inspections and used as target images. These target images are then input into an insulator string detection model to obtain insulator string detection results. These results include multiple insulator string rectangles and their corresponding insulator string types, including but not limited to glass insulator strings, porcelain insulator strings, and composite insulator strings. If no insulator string rectangles are found in the detection results, the subsequent steps of generating semantics for transmission line pin defects cease. If insulator string rectangles are found, the subsequent steps of the transmission line pin defect semantics generation method continue.

[0029] As an optional implementation method in this application, the insulator string detection model can be based on a real-time end-to-end target detection model. RT-DETR-l The model is obtained through training, which includes the following steps: First, the original image library of transmission line insulator strings is acquired. Augmentation processing is then performed on the original image library to construct an augmented image library of transmission line insulator strings. Augmentation processing includes, but is not limited to, left / right flipping, angle rotation, Gaussian noise, and image blending. Then, the original image library and the augmented image library of transmission line insulator strings are merged to obtain a training sample library. Finally, the training sample library is divided into a training set, a validation set, and a test set. The training set is then used to train the transmission line insulator string. RT-DETR-l The model learns from samples and their corresponding labels; it adjusts the model during training using the validation set. RT-DETR-lThe model's hyperparameters are determined, and preliminary evaluation and selection are performed; after all model training and tuning are completed, a final evaluation is conducted using a test set. RT- DETR-l The model's generalization ability is used to ultimately obtain the optimal model as the insulator string detection model.

[0030] Based on this, insulator string detection, as a prerequisite for defect semantic generation, not only accurately identifies the insulator string type and outputs type labels, but also constructs the foundational dimensions for subsequent defect attribution through material differentiation. RT-DETR-l The model training process yields an insulator string detection model that balances detection accuracy and real-time performance, avoiding missed detections of long and small-scale insulator strings. The detection results directly drive subsequent multi-level region trimming and component binding, ensuring that the semantics of pin defects are always anchored to the actual material carrier, thus guaranteeing the physical interpretability and operational reliability of the semantic description from the source.

[0031] S200. Based on the insulator string detection results, perform multi-level region cropping on the target image to obtain the first target image sequence and the second target image sequence.

[0032] In this embodiment of the application, in order to accelerate the subsequent detection of pin defects, reduce irrelevant background information, and further characterize the spatial relationship between pin defects and insulator strings, multi-level region trimming can be performed on the insulator strings, specifically including the following steps: S210. Merge all the rectangular frames of insulator strings in the insulator string detection results to obtain the merged frame of insulator strings.

[0033] S220. Select multiple sliding window regions within the range of the insulator string merging frame by performing sliding window image selection.

[0034] For example, the sliding window can be 1500 pixels × 1500 pixels. The sliding step size can be set according to actual needs. Based on the sliding window and the sliding step size, image selection is performed within the range of the insulator string merging box of the target image to obtain multiple 1500 pixel × 1500 pixel sliding window areas.

[0035] S230. Crop the target image according to each sliding window region to obtain the first target image sequence.

[0036] In one implementation, each sliding window region is used as a first action box list, and the target image is cropped according to the first action box list to obtain multiple first target images, which constitute a first target image sequence. This first target image sequence is mainly used for pin defect detection of insulator strings.

[0037] S240. The target image is cropped according to the rectangular frame of each insulator string to obtain the second target image sequence.

[0038] In one embodiment, the rectangular frames of each insulator string are used as a second action frame list, and the target image is cropped according to the second action frame list to obtain multiple second target images, which constitute a second target image sequence. This second target image sequence is mainly used for spatial structure detection of insulator strings.

[0039] Based on this, the insulator string trimming adopts a dual-path design that generates a first target image sequence based on the sliding window of the insulator string merging frame, and a second target image sequence based on the rectangular frames of each insulator string. This design not only eliminates background interference and improves the signal-to-noise ratio of small target detection, but also preserves the local geometric integrity of the insulator string. This provides a scale-adaptive and semantically aligned input basis for subsequent accurate binding of defects and spatial structures, and significantly improves the robustness of joint modeling of component relationships and spatial relationships.

[0040] S300. Input the first target image sequence into the pin defect detection model in sequence, output multiple defect rectangles and corresponding defect types, perform confidence filtering and area non-maximum suppression processing on the defect rectangles based on the defect types, and obtain the pin defect detection results.

[0041] In this embodiment of the application, in order to characterize the impact of pin defects on insulator strings of different materials and further establish the spatial structural relationship between pin defects and insulator strings of different materials, pin defect detection, confidence filtering and area non-maximum suppression processing can be performed on each first target image in the first target image sequence to obtain pin defect detection results.

[0042] As an optional implementation of this application, given the complex and varied scenarios of transmission lines, the diverse types of defects, and the varying sizes of targets, this application employs an improved YOLO12-m target detection algorithm to build a pin defect detection model in order to quickly and accurately detect pin defects such as missing pins, pin pull-out, and pin corrosion. Algorithmically, the pin defect detection model can effectively cover the scale range of various pin defects and effectively perceive the detailed features of each pin defect. In engineering, the pin defect detection model, through Tensorrt model transformation, can quickly complete the pin defect detection of transmission lines.

[0043] Understandably, the improved YOLO12-m target detection model is trained to obtain the pin defect detection model. The specific model training is as follows: First, multiple original images of pin defects in the transmission line are acquired. Each original image is then labeled to generate a labeling tag, resulting in a pin defect original image library. This labeling tag includes not only the pin defect bounding box and its corresponding defect type, but also the insulator string bounding box and its corresponding insulator string type. The labeled insulator string bounding boxes are then merged to obtain a merged insulator string bounding box. Image cropping is performed using a sliding window, for example, setting the cropping resolution to 1500 pixels × 1500 pixels, thus obtaining the pin defect original sub-image library.

[0044] Then, augmentation processing is performed on the original image library of pin defects to construct an augmented image library of pin defects. Augmentation processing includes, but is not limited to, left and right flipping, angle rotation, Gaussian noise, and image blending. The bounding boxes of insulator strings in the augmented image library of pin defects are merged to obtain merged insulator string boxes. The images are then cropped using a sliding window with a cropping resolution of 1500 pixels × 1500 pixels, thus obtaining the augmented sub-image library of pin defects.

[0045] Finally, the original and augmented sub-image libraries of pin defects are merged to obtain the pin defect training sample library. This library is divided into training, validation, and test sets. The training set is used to learn the improved YOLO12-m object detection model using samples and their corresponding labels. The validation set is used to adjust the hyperparameters of the improved YOLO12-m object detection model during training, and to perform preliminary evaluation and selection. After all model training and tuning are complete, the test set is used to finally evaluate the generalization ability of the improved YOLO12-m object detection model, and the optimal model is ultimately selected as the pin defect detection model.

[0046] Figure 2 This is a schematic diagram of the pin defect detection model in the embodiments of this application, as shown below. Figure 2 As shown, the pin defect detection model includes a backbone network, a feature pyramid network, and a decoupling network. To accurately capture detailed defect features and quickly extract defect targets, a self-attention convolution module is introduced into the feature pyramid network, making the network focus more on the pin defect region. A channel attention interaction module is introduced after three layers of fused feature maps, enabling the feature pyramid network to capture fine-grained features of the pin defect. Each first target image in the first target image sequence is sequentially input into the pin defect detection model. Each first target image outputs one or more defect bounding boxes and their corresponding defect types. After defect detection is completed for the entire first target image sequence, multiple defect bounding boxes and their corresponding defect types are obtained. For example, taking a specific first target image from the first target image sequence as an example, the specific steps for inputting it into the pin defect detection model for defect detection include the following: S301. Input the first target image in the first target image sequence into the backbone network for feature extraction to obtain the first layer feature map, the second layer feature map and the third layer feature map.

[0047] In this embodiment, the backbone network is jointly constructed by the C3K2 module and the A2C2F module. The C3K2 module enhances the local texture modeling capability and adapts to the edge and rust texture features of small defects in M-shaped pins. The A2C2F module integrates channel attention and dilated convolution, enhancing multi-scale context awareness while maintaining the receptive field. Together, they form a lightweight and efficient backbone network, including three layers: Conv3, Conv4, and Conv5. After the first target image is input into the backbone network for feature extraction, Conv3 outputs a high-resolution and strongly localized first-layer feature map F3, Conv4 outputs a second-layer feature map F4 with moderate semantics, and Conv5 outputs a third-layer feature map F5 with strong semantics and weak localization. This provides a structurally complementary input basis for the subsequent cross-layer fine-grained fusion of the feature pyramid network, balancing the requirements of small target detection accuracy and real-time performance.

[0048] S302. Input the second and third layer feature maps into the self-attention convolution module to obtain the first self-attention convolution feature map.

[0049] The second-layer feature map F4 and the third-layer feature map F5 are input into the self-attention convolutional module SACM. SACM uses the third-layer feature map F5 as the global context to guide the weighted aggregation of the second-layer feature map F4, resulting in the first self-attention convolutional feature map SACF1. This process allows the second-layer feature map F4 to retain its high-resolution localization capabilities while explicitly injecting prior knowledge of the insulator string structure contained in the third-layer feature map F5, such as the fact that M-pins are often located at string ends or between sheets. This significantly enhances the local sensitivity to small and low-contrast M-pin defects, overcoming the detail attenuation problem caused by unidirectional information flow in traditional feature networks.

[0050] In this embodiment, the self-attention convolution module uses a cross-layer self-attention mechanism to guide detail localization based on structural priors. It is specifically designed for small pin targets, cluttered backgrounds, and strong positional regularity in power transmission lines. Without increasing significant computational overhead, it significantly improves the detection rate and localization robustness of minute defects. Figure 3 This is a schematic diagram of the structure of the self-attention convolution module in an embodiment of this application, as shown below. Figure 3 As shown, the second-layer feature map F4 serves as a low-level high-resolution feature map, and the third-layer feature map F5 serves as a high-level low-resolution feature map. The specific processing flow of the self-attention convolution module is as follows: First, the low-level high-resolution feature maps are subjected to 1×1 convolution and 1×1 convolution respectively to obtain projection matrix Q and projection matrix K. The projection matrix Q and projection matrix K are then matched for similarity to obtain a similarity matching matrix. Finally, after passing through an activation function, the attention weighting matrix W is obtained.

[0051] Meanwhile, the high-level low-resolution feature map is upsampled through 1x1 convolution to generate an upsampled feature map with the same resolution as the low-level high-resolution feature map. Then, the upsampled feature map is convolved with 1×1 to obtain the projection matrix V.

[0052] Finally, the projection matrix V and the attention weighting matrix W are multiplied to obtain the self-attention convolution feature map. The specific formula for obtaining the self-attention convolution feature map is as follows:

[0053] In the above formula, This represents the self-attention convolution feature map. Represents the projection matrix The vector dimension for similarity matching.

[0054] S303. Concatenate the first self-attention convolutional feature map with the second layer feature map to obtain the first fused feature map.

[0055] The first self-attention convolutional feature map SACF1 and the second-layer feature map F4 are concatenated along the channel dimension to obtain the first fused feature map ~P4. This first fused feature map ~P4 retains the inherent high-resolution spatial details of the second-layer feature map F4, such as the edge of the M-pin and the texture of rust spots, while also incorporating the high-level semantic guidance information introduced by the first self-attention convolutional feature map SACF1, such as the topological constraints of the insulator string. This fusion method avoids feature collapse, enhances the orthogonality and robustness of feature representation, and provides a joint representation foundation with both localization accuracy and semantic discriminative power for subsequent accurate regression and classification of the location of minute defects in the M-pin.

[0056] S304. Input the first fused feature map and the first layer feature map into the self-attention convolution module to obtain the second self-attention convolution feature map.

[0057] Similarly, the first layer feature map F3 is used as the low-level high-resolution feature map, and the first fused feature map ~P4 is used as the high-level low-resolution feature map. The self-attention convolution module uses the first fused feature map ~P4 as the global context to guide the first layer feature map F3 to focus on the candidate positions of M pins in key areas such as between insulator sheets and on the side of the tower, and performs weighted aggregation to obtain the second self-attention convolution feature map SACF2.

[0058] S305. Concatenate the second self-attention convolutional feature map with the first layer feature map to obtain the second fused feature map.

[0059] The second self-attention convolutional feature map SACF2 is concatenated with the first layer feature map F3 to obtain the second fused feature map P3. The second fused feature map P3 retains the original high-resolution spatial details of the first layer feature map F3, such as the M pin outline and rust spots, and injects structured attention guided by the second self-attention convolutional feature map SACF2, which clearly points to the key connection positions inside the insulator string, such as the inter-laminate gaps and end fittings.

[0060] S306. Convolve the second fused feature map to generate a first intermediate feature map with the same resolution as the first fused feature map, and concatenate the first intermediate feature map with the first fused feature map to obtain a third fused feature map.

[0061] Furthermore, the second fused feature map P3 is convolved using a Conv layer to generate a first intermediate feature map PAF1 with the same resolution as the first fused feature map ~P4, achieving feature enhancement from the high-resolution detail layer to the mid-level semantic layer. The first intermediate feature map PAF1 is concatenated with the first fused feature map ~P4 to obtain the third fused feature map P4. While maintaining semantic discriminative power, it possesses more reliable defect coordinate regression capabilities, laying a multi-scale consistent feature foundation for the fine-grained modeling of the subsequent channel attention interaction module.

[0062] S307. Convolve the third fused feature map to generate a second intermediate feature map with the same resolution as the third layer feature map, and concatenate the second intermediate feature map with the third layer feature map to obtain the fourth fused feature map.

[0063] The third fused feature map P4 is convolved using a convolutional layer Conv to generate a second intermediate feature map PAF2 with the same resolution as the third feature map F5. The second intermediate feature map PAF2 is then concatenated with the third feature map F5 to obtain the fourth fused feature map P5. This fourth fused feature map P5 maintains strong semantic discriminative ability while embedding more precise spatial constraints, such as the M-pin only existing in a specific region of a specific material string. This design enhances the ability of the fourth fused feature map P5 to model the dependency relationship between defects and components, directly supporting component binding judgment in subsequent pin defect semantic generation, and improving the business accuracy and interpretability of work order semantic output.

[0064] S308. Input the second fused feature map, the third fused feature map and the fourth fused feature map into the channel attention interaction module for feature enhancement to obtain the first target feature map, the second target feature map and the third target feature map.

[0065] In this embodiment, the second fused feature map P3, the third fused feature map P4, and the fourth fused feature map P5 are respectively enhanced by the channel attention interaction module CAIM to obtain cross-level fine-grained feature maps from top to bottom and bottom to top, including the first target feature map T3, the second target feature map T4, and the third target feature map T5.

[0066] The channel attention interaction module generates fine-grained attention weights by modeling the semantic correlation between features of different channels, so that the same defect presents semantic consistency in features of different scales. This provides a robust and traceable feature foundation for the subsequent decoupling network to output high-confidence defect types and coordinates, as well as for the structural binding in the semantic generation of pin defects. Figure 4 This is a schematic diagram of the channel attention interaction module in an embodiment of this application, as shown below. Figure 4 As shown, the specific processing flow of the fused feature map in the channel attention interaction module is as follows: First, the fused feature maps are split in half according to channels to obtain encoded feature maps X and Y. Encoded feature map X is then convolved with a 1×1 matrix to obtain the projection matrix R, and encoded feature map Y is convolved with a 1×1 matrix to obtain the projection matrix S. Then, similarity matching is performed between the projection matrices R and S to obtain a similarity matching matrix, which is then passed through an activation function to obtain the attention weighting matrix T.

[0067] At the same time, the fused feature map is convolved by 1×1 to generate a half-channel fused feature map with half the number of channels. Then, the half-channel fused feature map is convolved by 1×1 to obtain the projection matrix U.

[0068] Finally, the projection matrix U and the attention weighting matrix T are multiplied to obtain the channel attention interaction feature map. The specific formula for this channel attention interaction feature map is as follows:

[0069] In the above formula, Represents the channel attention interaction feature map. Represents the projection matrix The vector dimension for similarity matching.

[0070] S309. Input the first target feature map, the second target feature map and the third target feature map into the decoupled network for classification and regression processing respectively to obtain the defect type and defect bounding box. Multiple first target images can be used to obtain multiple defect types and defect bounding boxes.

[0071] In one implementation, the decoupled network connects the first target feature map T3, the second target feature map T4, and the third target feature map T5 to independent classification and regression branches for classification and regression processing, thereby decoupling the parameters of defect category and location prediction, avoiding gradient interference between tasks, obtaining classification targets and regression targets, and obtaining initial defect bounding boxes and corresponding defect types.

[0072] Based on this, the pin defect detection model uses a self-attention convolution module to give high-level feature maps attention to details in low-level feature maps, and uses a channel attention interaction module to perform fine-grained learning of target features, thereby enhancing the high-level semantic feature representation of the target, accelerating feature computation efficiency, and ensuring the accuracy and speed of defect detection.

[0073] It is understandable that the coordinates of multiple initial defect rectangles are mapped back to the coordinates of the original image, thereby obtaining multiple defect rectangles and their corresponding defect types. These defect types include, but are not limited to, missing M pins, M pin withdrawal, and M pin corrosion.

[0074] Furthermore, since the pin defect detection model detects sliding window images, some sliding window images contain the complete target contour, while others only contain a partial target contour. Thus, after detection by the pin defect detection model, we obtain bounding boxes of both the complete and partial target contours. Therefore, we need to use an area non-maximum suppression (AMS) algorithm to retain the bounding boxes of the complete target contour output by the pin defect detection model and delete the bounding boxes of the partial target contours. Specifically, based on each defect type, confidence filtering and AMS processing are applied to the defect bounding boxes to obtain the pin defect detection results. The specific steps include: S310. Set different confidence thresholds according to different defect types, delete the defect rectangles corresponding to those less than the confidence threshold, and obtain the initial pin defect detection results.

[0075] In one implementation, different confidence thresholds are set according to different defect types. For each defect type, the confidence of each defect rectangle is calculated, and defect rectangles with confidence scores lower than the corresponding confidence threshold are deleted. For example, for the defect type of missing M pins, the confidence threshold is set to 0.30. If the confidence score of a defect rectangle of the defect type of missing M pins is less than 0.30, then the defect rectangle is removed.

[0076] S320. Set different minimum area overlap rate thresholds for different defect types in the initial pin defect detection results.

[0077] Understandably, the initial pin defect detection results also include different defect types. Therefore, a minimum area overlap threshold is set for each defect type, for example, 0.8.

[0078] S330. Sort the areas of all defect rectangles corresponding to each defect type, select the defect rectangle with the largest area as the registration rectangle, and take the remaining defect rectangles as the first sequence of rectangles to be tested.

[0079] For example, for all defect rectangles corresponding to a certain type of M-pin corrosion defect, all defect rectangles are sorted from largest to smallest area, and the defect rectangle with the largest area is selected as the registration rectangle. The remaining defect rectangles, excluding the largest one, are used as the first sequence of rectangles to be tested. Similarly, for defect types such as missing M-pins and M-pin exit defects, the area is also sorted, and the registration rectangles and the first sequence of rectangles to be tested are selected.

[0080] S340. Calculate the minimum area overlap rate between the test rectangles in the first test rectangle sequence and the registration rectangles.

[0081] For example, suppose the bounding box to be measured is rectangle ABCD, the registration bounding box is rectangle EFGH, and the top-left point of rectangle ABCD is... The bottom right point is The top left point of rectangle EFGH is The bottom right point is The formula for calculating the minimum area overlap rate between rectangles ABCD and EFGH is as follows:

[0082]

[0083]

[0084]

[0085] In the above formulas, This represents the minimum area overlap between rectangles ABCD and EFGH. This represents the area of ​​rectangle ABCD. This represents the area of ​​the rectangle EFGH. This represents the area of ​​the intersection of rectangles ABCD and EFGH.

[0086] S350. If the minimum area overlap rate is greater than the minimum area overlap rate threshold, then delete the corresponding test rectangle and obtain the target rectangle sequence.

[0087] By calculating the minimum area overlap rate, each test rectangle in the first test rectangle sequence is assigned a minimum area overlap rate. If the minimum area overlap rate is greater than the minimum area overlap rate threshold, the corresponding test rectangle is deleted. If the minimum area overlap rate is less than or equal to the minimum area overlap rate threshold, the corresponding test rectangle is retained, thus obtaining the target rectangle sequence.

[0088] S360. Select the target rectangle with the largest area from the target rectangle sequence again as the registration rectangle, and use the remaining target rectangles as the second sequence of rectangles to be tested.

[0089] Understandably, the target rectangles in the target rectangle sequence are reordered by area, and the target rectangle with the largest area is selected as the registration rectangle. The other target rectangles excluding the registration rectangle are used as the second sequence of rectangles to be tested.

[0090] S370. Repeat the minimum area overlap rate calculation and threshold comparison based on the second test rectangle sequence and the registration rectangle until the minimum area overlap rate of all test rectangles and the selected registration rectangle is less than or equal to the minimum area overlap rate threshold.

[0091] Calculate the minimum area overlap rate between the test rectangles in the second test rectangle sequence and the registration rectangles. Delete the target rectangles whose minimum area overlap rate is greater than the minimum area overlap rate threshold, and retain the test rectangles whose minimum area overlap rate is less than or equal to the minimum area overlap rate threshold. Continue to repeat the above steps S350-S370 until the minimum area overlap rate of all test rectangles and the selected registration rectangles is less than or equal to the minimum area overlap rate threshold.

[0092] S380. Select all the registration rectangles and their corresponding defect types as the pin defect detection results.

[0093] Finally, a corresponding registration rectangle was selected for each defect type. The registration rectangle is the defect rectangle, and all defect rectangles and their corresponding defect types are used as the pin defect detection results. If no defect rectangle is found in the pin defect detection results, the subsequent steps of generating the transmission line pin defect semantics are terminated. If a defect rectangle is found in the pin defect detection results, the subsequent steps of generating the transmission line pin defect semantics continue.

[0094] Based on this, the pin defect detection is combined with the pin defect detection model to detect defects, effectively suppressing background interference and improving the robustness of small target recognition. Then, through confidence filtering and area-aware suppression strategies, the complete defect outline is accurately preserved, reducing fragmented false detections. Finally, a high-confidence defect rectangle and defect type are output, providing a stable and reliable visual foundation for subsequent pin defect semantic generation.

[0095] S400. Input the second target image sequence into the spatial structure detection model in sequence, output multiple spatial structure rectangles and corresponding spatial structure types, perform confidence filtering and non-maximum suppression processing based on the spatial structure types, and obtain the spatial structure detection results.

[0096] In this embodiment of the application, in order to refine the spatial structure of insulator strings of different materials on the tower side, insulator sheet side, and conductor side, and to further enrich the semantic information of pin defects, spatial structure detection can be performed on insulator strings of different materials, specifically including the following steps: First, all the second target images in the second target image sequence are sequentially input into the spatial structure detection model. After spatial structure detection is performed on each second target image, the model may output one or more spatial structure bounding boxes and their corresponding spatial structure types. The second target image sequence can obtain multiple initial spatial structure bounding boxes and their corresponding spatial structure types. The coordinates of the multiple initial spatial structure bounding boxes are then mapped back to the original coordinates of the target image, thereby obtaining multiple spatial structure bounding boxes and their corresponding spatial structure types.

[0097] Then, different confidence thresholds are set for different spatial structure types. Based on the confidence threshold for each spatial structure type, confidence filtering is performed, deleting spatial structure rectangles with confidence levels below the threshold to obtain an initial set of spatial structure rectangles. The confidence threshold can be set according to actual needs and human experience, for example, to 0.35.

[0098] Finally, different overlap rate thresholds are set for different spatial structure types. Based on the overlap rate threshold for each spatial structure type, a non-maximum suppression algorithm is used to calculate the overlap rate of each spatial structure rectangle in the initial set of spatial structure rectangles. Spatial structure rectangles with an overlap rate greater than the overlap rate threshold in the initial set of spatial structure rectangles are deleted, resulting in the spatial structure detection results. This overlap rate threshold can also be set according to actual needs and human experience, for example, to 0.75.

[0099] It is understood that the spatial structure detection result includes multiple spatial structure rectangles and corresponding spatial structure types, including but not limited to insulator tower side, insulator sheet side, and insulator conductor side. If there are no spatial structure rectangles in the spatial structure detection result, the subsequent steps of generating semantics for transmission line pin defects will end; if there are spatial structure rectangles in the spatial structure detection result, the subsequent steps of generating semantics for transmission line pin defects will continue.

[0100] As an optional implementation of this application, a spatial structure detection model can be obtained by training the model based on the YOLO12-m model. The model training specifically includes the following steps: First, an original image library of the spatial structure of the transmission line is obtained. The labels of this original image library include not only the spatial structure rectangles and their corresponding spatial structure types, but also the insulator string rectangles and their corresponding insulator string types. The labeled rectangles of each insulator string in the original image library are cropped to obtain the original sub-image library of the spatial structure.

[0101] Then, augmentation processing is performed on the original spatial structure image library to construct an augmented spatial structure image library for transmission lines. This augmentation processing includes, but is not limited to, left and right flipping, angle rotation, Gaussian noise, and image mixing. The labeled rectangle of each insulator string in the augmented spatial structure image library is cropped to obtain an augmented spatial structure sub-image library.

[0102] Finally, the original spatial structure sub-database and the augmented spatial structure sub-database are merged to obtain the spatial structure training sample database. The spatial structure training sample database is divided into a training set, a validation set, and a test set. The YOLO12-m model is used to learn from the samples and corresponding labels in the training set using the training set. The hyperparameters of the YOLO12-m model are adjusted during training using the validation set, and preliminary evaluation and selection are performed. After all model training and tuning are completed, the generalization ability of the YOLO12-m model is finally evaluated using the test set, and the optimal model is finally obtained as the spatial structure detection model.

[0103] Based on this, spatial structure detection focuses on characterizing the internal topological relationships of insulator strings, which can accurately identify the hierarchical spatial units of insulator strings, providing millimeter-level spatial coordinate system mapping basis for pin defect semantics, enabling semantic information generation to be accurate to specific spatial structures, and truly realizing business-level positioning of defect locations.

[0104] S500: Based on the insulator string inspection results, pin defect inspection results, and spatial structure inspection results, perform component relationship binding and spatial structure binding to generate pin defect semantic information.

[0105] In this embodiment, the M-pin defect is typically located inside the insulator string. Specific structural locations include the tower side of the glass insulator, inside the glass insulator sheet, the conductor side of the glass insulator, the tower side of the porcelain insulator, inside the porcelain insulator sheet, the conductor side of the porcelain insulator, the tower side of the composite insulator, and the conductor side of the composite insulator. However, due to its unique structure, the composite insulator string does not have M-pins installed inside, and therefore will not have M-pin defects. To determine whether the M-pin defect is inside the insulator string and to pinpoint its spatial location within the insulator string, further generating semantic information about the pin defect, it is necessary to perform component relationship binding and spatial structure binding on the M-pin defect. Specifically, component relationship binding and spatial structure binding include the following steps: S510. Calculate the variation overlap rate based on the insulator string inspection results and pin defect inspection results to obtain the component variation overlap rate.

[0106] In this embodiment of the application, the formula for calculating the variation overlap rate based on the insulator string detection results and the pin defect detection results is as follows:

[0107] In the above formula, This indicates that the defect type in the pin defect detection result is... The A defective rectangular box The insulator string type in the insulator string test results is The A rectangular frame of an insulator string. The component variation overlap rate, This represents the intersection operation of rectangles. The area operator for a rectangle.

[0108] S520. Compare the component variation overlap rate and the component variation overlap rate threshold to obtain the component binding relationship result, and determine the pin defect relationship detection result based on the component binding relationship result.

[0109] In this embodiment, the component variation overlap rate and the component variation overlap rate threshold are compared using a component binding relationship formula, the specific expression of which is as follows:

[0110] In the above formula, This indicates that the defect type in the pin defect detection result is... The A defective rectangular box The insulator string type in the insulator string test results is The A rectangular frame of an insulator string. Whether there is a binding relationship. If it is 1, it means there is a binding relationship. If it is 0, it means there is no binding relationship. This indicates that the defect type in the pin defect detection result is... The A defective rectangular box The insulator string type in the insulator string test results is The A rectangular frame of an insulator string. The component variation overlap rate; This indicates that the defect type in the pin defect detection result is... The component variation overlap rate threshold.

[0111] According to the above formula, if the component variation overlap rate is greater than or equal to the component variation overlap rate threshold, the result is 1, indicating that the defect rectangle corresponding to the component variation overlap rate and the insulator string rectangle have a component binding relationship; otherwise, the result is 0, indicating that the defect rectangle corresponding to the component variation overlap rate and the insulator string rectangle do not have a component binding relationship. The component binding relationship of each defect rectangle in the pin defect detection results is determined, pin defect rectangles without component binding relationships are removed, and pin defect rectangles with component binding relationships and their corresponding defect types are taken as the pin defect relationship detection results.

[0112] Therefore, through the above component binding relationship formula, we can also obtain the insulator string rectangle and insulator string type bound to each pin defect rectangle in the pin defect relationship detection results. For example, the M pin defect is located in a glass insulator string, or the M pin defect is located in a porcelain insulator string, or the M pin defect is located in a composite insulator string.

[0113] S530. Calculate the center point coordinates of each spatial structure rectangle based on the spatial structure detection results, and determine the spatial structure sequence detection results based on the coordinates of each center point.

[0114] In one implementation, for each spatial structure rectangle in the spatial structure detection results, the coordinates of the center point of each spatial structure rectangle are calculated, and then the spatial structure detection results are obtained in an ordered manner according to the distance, which is used as the spatial structure sequential detection results.

[0115] For example, if there is a rectangular frame on the insulator tower side, the rectangular frames are numbered sequentially based on the distance from the center point of the remaining spatial structure rectangular frames to the center point of the insulator tower side rectangular frame, following either the order "Insulator Tower Side, Insulator Disc 1, Insulator Disc 2, Insulator Disc n, Insulator Conductor Side" or "Insulator Tower Side, Insulator Disc 1, Insulator Disc 2, Insulator Disc n". If there is no rectangular frame on the insulator tower side, the rectangular frames are numbered sequentially based on the distance from the center point of the remaining spatial structure rectangular frames to the center point of the insulator conductor side rectangular frame, following the order "Insulator Conductor Side, Insulator Disc 1, Insulator Disc 2, Insulator Disc n". This results in a sequential spatial structure inspection result.

[0116] S540. Calculate the variation overlap rate based on the detection results of pin defect relationship and spatial structure sequence to obtain the structural variation overlap rate.

[0117] In this embodiment, the formula for calculating the variation overlap rate based on the pin defect relationship detection results and the spatial structure sequence detection results is as follows:

[0118] In the formula, This indicates that the defect type in the pin defect relationship detection results is... The A defective rectangular box The spatial structure type in the spatial structure sequence detection results is The A spatial structure rectangle The overlap rate of structural variations This represents the intersection operation of rectangles. The area operator for a rectangle.

[0119] S550. Based on the comparison between the structural variation overlap rate and the structural variation overlap rate threshold, the structural binding relationship result is obtained, and the semantic detection result of the pin defect is determined based on the structural binding relationship result.

[0120] In this embodiment, the structural variation overlap rate and the structural variation overlap rate threshold are compared using a structural binding relationship formula, the expression of which is as follows:

[0121] In the above formula, This indicates that the defect type in the pin defect relationship detection results is... The A rectangular frame The spatial structure type in the spatial structure sequence detection results is The A rectangular frame Whether there is a binding relationship. If it is 1, it means there is a binding relationship. If it is 0, it means there is no binding relationship. This indicates that the defect type in the pin defect relationship detection results is... The A defective rectangular box The spatial structure type in the spatial structure sequence detection results is The A spatial structure rectangle The overlap rate of structural variations; This indicates that the defect type in the pin defect relationship detection results is... The structural variation overlap rate threshold.

[0122] According to the above formula, if the structural variation overlap rate is greater than or equal to the structural variation overlap rate threshold, the result is 1, indicating that the defect rectangle corresponding to the structural variation overlap rate and the spatial structure rectangle have a structural binding relationship; otherwise, the result is 0, indicating that the defect rectangle corresponding to the structural variation overlap rate and the spatial structure rectangle do not have a structural binding relationship. The structural binding relationship of each pin defect rectangle in the pin defect relationship detection results is determined. Pin defect rectangles without structural binding relationships are removed, and pin defect rectangles with structural binding relationships and their corresponding defect types are used as pin defect semantic detection results. In these results, the pin defect rectangles are bound not only to the insulator string rectangle but also to the spatial structure rectangles within the insulator string.

[0123] Therefore, through the above structural binding relationship formula, we can obtain the spatial structure rectangle and spatial structure type bound to each pin defect rectangle, such as the M pin defect being located on the insulator tower side, the M pin defect being located on which insulator piece, or the M pin defect being located on the insulator conductor side.

[0124] S560. Generate pin defect semantic information based on the pin defect semantic detection results.

[0125] In this embodiment, each pin defect rectangle in the pin defect semantic detection result is bound not only to the insulator string rectangle but also to the spatial structure rectangle within the insulator string. Combined with the scene name from the UAV image of the transmission line, pin defect semantic information can be generated. One pin defect corresponds to one pin defect semantic information, such as: a missing M-pin on the glass insulator tower side of the 220kV "xxx" line; the M-pin of the second insulator on the porcelain insulator tower side of the 220kV "xxx" line has come loose; or M-pin corrosion has occurred on the conductor side of the composite insulator of the 220kV "xxx" line.

[0126] Understandably, if only the pin defect detection results are available without the corresponding insulator string detection results, maintenance personnel would need to analyze the inspection images to determine the material and location of the defect within the insulator string before proceeding to the corresponding location to eliminate the pin defect. Conversely, if the pin defect detection results only include the corresponding insulator string detection results, this information is ambiguous for maintenance personnel. By binding the pin defect detection results to the spatial structure detection results, the location of the M-pin defect—whether it's on the insulator tower side, within a specific insulator section, or on the insulator conductor side—can be further pinpointed. This generates clear semantic information about the pin defect, such as "the M-pin defect is located on the insulator tower side," "the M-pin defect is located within a specific insulator section," or "the M-pin defect is located on the insulator conductor side." Maintenance personnel then no longer need to analyze the relative location of the pin defect based on the inspection images and can directly target the corresponding location to eliminate the defect.

[0127] Based on this, the pin defect semantic generation transforms the subordinate relationships and spatial structural relationships of pin defect components that are of concern to maintenance personnel into visual semantic generation information from the perspective of operation and maintenance. This realizes the information transformation from visual images to natural language and the information dimensionality upgrade from pixel coordinates to business coordinates, providing powerful methodological support for intelligent operation and maintenance.

[0128] The method, system, device, and storage medium for generating semantic information of pin defects in transmission lines provided in this application generate such information through a progressive architecture of insulator string detection, multi-level region trimming, dual detection of pin defects and spatial structure, and semantic binding. This information can be directly used for power defect operation and maintenance.

[0129] Example 2 Based on the same technical concept as Embodiment 1 above, this application provides a semantic generation system for pin defects in transmission lines. Figure 5 This is a schematic diagram of the semantic generation system for transmission line pin defects provided in an embodiment of this application, as shown below. Figure 5As shown, the transmission line pin defect semantic generation system 200 includes: The insulator string detection module 210 is used to acquire the target image of the transmission line, input the target image into the insulator string detection model, and obtain the insulator string detection result.

[0130] The insulator string trimming module 220 is used to perform multi-level region trimming on the target image based on the insulator string detection results to obtain a first target image sequence and a second target image sequence.

[0131] The pin defect detection module 230 is used to input the first target image sequence into the pin defect detection model in sequence, output multiple defect rectangles and corresponding defect types, and perform confidence filtering and area non-maximum suppression processing on the defect rectangles based on the defect types to obtain the pin defect detection results.

[0132] The spatial structure detection module 240 is used to input the second target image sequence into the spatial structure detection model in sequence, output multiple spatial structure rectangles and corresponding spatial structure types, and perform confidence filtering and non-maximum suppression processing based on the spatial structure types to obtain spatial structure detection results.

[0133] The defect semantic generation module 250 is used to perform component relationship binding and spatial structure binding based on the insulator string detection results, pin defect detection results and spatial structure detection results, and generate pin defect semantic information.

[0134] The semantic generation system for transmission line pin defects provided in this application embodiment can significantly improve the business accessibility of pin defect location, reduce the burden of manual interpretation, support automated work order generation and accurate defect elimination, and effectively reduce the false alarm rate of actual measurement.

[0135] It is understood that the implementation method of the transmission line pin defect semantic generation method in Embodiment 1 above is also applicable to this embodiment and can achieve the same technical effect, so it will not be described again here.

[0136] Example 3 Based on the same concept, this application also provides an electronic device. Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 6 As shown, the electronic device 300 may include a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute the steps of the transmission line pin defect semantic generation method as described in the above embodiments. For example, it includes: S100. Obtain the target image of the transmission line, input the target image into the insulator string detection model, and obtain the insulator string detection result. S200. Based on the insulator string detection results, perform multi-level region cropping on the target image to obtain the first target image sequence and the second target image sequence. S300. Input the first target image sequence into the pin defect detection model in sequence, output multiple defect rectangles and corresponding defect types, perform confidence filtering and area non-maximum suppression on the defect rectangles based on the defect types, and obtain the pin defect detection results. S400: Input the second target image sequence into the spatial structure detection model in sequence, output multiple spatial structure rectangles and corresponding spatial structure types, perform confidence filtering and non-maximum suppression processing based on the spatial structure types, and obtain the spatial structure detection results. S500: Based on the insulator string inspection results, pin defect inspection results, and spatial structure inspection results, perform component relationship binding and spatial structure binding to generate pin defect semantic information.

[0137] The processor 310 can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0138] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0139] The memory 330 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0140] Example 4 Based on the same concept, embodiments of this application also provide a computer-readable storage medium storing a computer program containing at least one piece of code executable by a master control device to control the master control device to implement the steps of the transmission line pin defect semantic generation method as described in the above embodiments. For example, it includes: S100. Obtain the target image of the transmission line, input the target image into the insulator string detection model, and obtain the insulator string detection result. S200. Based on the insulator string detection results, perform multi-level region cropping on the target image to obtain the first target image sequence and the second target image sequence. S300. Input the first target image sequence into the pin defect detection model in sequence, output multiple defect rectangles and corresponding defect types, perform confidence filtering and area non-maximum suppression on the defect rectangles based on the defect types, and obtain the pin defect detection results. S400: Input the second target image sequence into the spatial structure detection model in sequence, output multiple spatial structure rectangles and corresponding spatial structure types, perform confidence filtering and non-maximum suppression processing based on the spatial structure types, and obtain the spatial structure detection results. S500: Based on the insulator string inspection results, pin defect inspection results, and spatial structure inspection results, perform component relationship binding and spatial structure binding to generate pin defect semantic information.

[0141] Based on the same technical concept, this application also provides a computer program, which, when executed by a main control device, is used to implement the above-described method embodiments.

[0142] The computer program may be stored, in whole or in part, on a computer-readable storage medium packaged with the processor, or in part or in whole on a memory not packaged with the processor.

[0143] Based on the same technical concept, this application also provides a processor for implementing the above-described method embodiments. The processor can be a chip.

[0144] In summary, the method, system, equipment, and storage medium for generating semantic information about transmission line pin defects provided in this application generate such information through a progressive architecture involving insulator string detection, multi-level region trimming, dual detection of pin defects and spatial structures, and semantic binding. This transforms the subordinate relationships and spatial structural relationships of defective components of concern to maintenance into executable semantic information that can be directly used for power system defect maintenance and repair. It achieves information conversion from visual images to natural language and information dimensionality upgrade from pixel coordinates to business coordinates, providing strong methodological support for intelligent maintenance. Simultaneously, it significantly improves the business accessibility of pin defect localization in transmission lines, reduces the burden of manual interpretation, supports automated work order generation and accurate defect elimination, and effectively reduces the false alarm rate in actual measurements.

[0145] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0146] The embodiments described above are merely examples of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for generating semantic representations of pin defects in transmission lines, characterized in that, The method includes: Acquire a target image of the transmission line, input the target image into the insulator string detection model, and obtain the insulator string detection result; Based on the insulator string detection results, the target image is cropped at multiple levels to obtain a first target image sequence and a second target image sequence. The first target image sequence is sequentially input into the pin defect detection model, which outputs multiple defect rectangles and corresponding defect types. Based on the defect types, the defect rectangles are subjected to confidence filtering and area non-maximum suppression to obtain the pin defect detection results. The second target image sequence is sequentially input into the spatial structure detection model, which outputs multiple spatial structure bounding boxes and corresponding spatial structure types. Confidence filtering and non-maximum suppression processing are performed based on the spatial structure types to obtain spatial structure detection results. Based on the insulator string detection results, the pin defect detection results, and the spatial structure detection results, component relationship binding and spatial structure binding are performed to generate pin defect semantic information.

2. The method for generating semantic representations of transmission line pin defects according to claim 1, characterized in that, The step of binding component relationships and spatial structures based on the insulator string detection results, the pin defect detection results, and the spatial structure detection results to generate pin defect semantic information includes: Based on the insulator string detection results and the pin defect detection results, the variation overlap rate is calculated to obtain the component variation overlap rate; The component variation overlap rate and the component variation overlap rate threshold are compared to obtain the component binding relationship result, and the pin defect relationship detection result is determined based on the component binding relationship result. Calculate the center point coordinates of each spatial structure rectangle based on the spatial structure detection results, and determine the spatial structure sequence detection results based on the center point coordinates. Based on the detection results of the pin defect relationship and the detection results of the spatial structure sequence, the variation overlap rate is calculated to obtain the structural variation overlap rate. The structural binding relationship result is obtained by comparing the structural variation overlap rate and the structural variation overlap rate threshold, and the pin defect semantic detection result is determined based on the structural binding relationship result. The pin defect semantic information is generated based on the pin defect semantic detection results.

3. The method for generating semantic representations of transmission line pin defects according to claim 2, characterized in that, The formula for calculating the variation overlap rate based on the insulator string detection results and the pin defect detection results is as follows: In the above formula, This indicates that the defect type in the pin defect detection result is... The A defective rectangular box The insulator string type in the insulator string detection results is The A rectangular frame of an insulator string. The component variation overlap rate, This represents the intersection operation of rectangles. The area operator for a rectangle; The step of comparing the component variation overlap rate with the component variation overlap rate threshold to obtain the component binding relationship result, and determining the pin defect relationship detection result based on the component binding relationship result, includes: If the component variation overlap rate is greater than or equal to the component variation overlap rate threshold, then the defect rectangle and the insulator string rectangle corresponding to the component variation overlap rate have a component binding relationship. The defect rectangles containing all the components with the aforementioned binding relationship, along with their corresponding defect types, are used as the pin defect relationship detection results.

4. The method for generating semantic representations of transmission line pin defects according to claim 2, characterized in that, The formula for calculating the variation overlap rate based on the pin defect relationship detection results and the spatial structure sequence detection results is as follows: In the formula, This indicates that the defect type in the pin defect relationship detection result is... The A defective rectangular box The spatial structure type in the spatial structure sequence detection results is The A spatial structure rectangle The overlap rate of structural variations This represents the intersection operation of rectangles. The area operator for a rectangle; The step of comparing the structural variation overlap rate with a structural variation overlap rate threshold to obtain a structural binding relationship result, and determining a pin defect semantic detection result based on the structural binding relationship result, includes: If the structural variation overlap rate is greater than or equal to the structural variation overlap rate threshold, then the defect rectangle and the spatial structure rectangle corresponding to the structural variation overlap rate have a structural binding relationship. The pin defect semantic detection result is defined as the complete set of defect rectangles containing the structural binding relationship and the corresponding defect type.

5. The method for generating semantic representations of transmission line pin defects according to claim 1, characterized in that, The pin defect detection model includes a backbone network, a feature pyramid network, and a decoupled network. The feature pyramid network includes a self-attention convolution module and a channel attention interaction module. The first target image sequence is sequentially input into the pin defect detection model, which outputs multiple defect rectangles and corresponding defect types, including: The first target image in the first target image sequence is input into the backbone network for feature extraction to obtain the first layer feature map, the second layer feature map and the third layer feature map; The second layer feature map and the third layer feature map are input into the self-attention convolution module to obtain the first self-attention convolution feature map; The first self-attention convolutional feature map is concatenated with the second layer feature map to obtain the first fused feature map; The first fused feature map and the first layer feature map are input into the self-attention convolution module to obtain the second self-attention convolution feature map; The second self-attention convolutional feature map is concatenated with the first layer feature map to obtain the second fused feature map; The second fused feature map is convolved to generate a first intermediate feature map with the same resolution as the first fused feature map, and the first intermediate feature map is concatenated with the first fused feature map to obtain a third fused feature map. The third fused feature map is convolved to generate a second intermediate feature map with the same resolution as the third layer feature map, and the second intermediate feature map is concatenated with the third layer feature map to obtain a fourth fused feature map; The second fused feature map, the third fused feature map, and the fourth fused feature map are respectively input into the channel attention interaction module for feature enhancement to obtain the first target feature map, the second target feature map, and the third target feature map; The first target feature map, the second target feature map, and the third target feature map are input into the decoupled network for classification and regression processing to obtain defect types and defect bounding boxes. Multiple first target images yield multiple defect types and defect bounding boxes.

6. The method for generating semantic representations of transmission line pin defects according to claim 1, characterized in that, The process of performing confidence filtering and area non-maximum suppression on the defect rectangle based on the defect type to obtain the pin defect detection result includes: Different confidence thresholds are set according to different defect types, and the defect rectangles corresponding to those less than the confidence thresholds are deleted to obtain the initial pin defect detection results; For different defect types in the initial pin defect detection results, different minimum area overlap rate thresholds are set; Sort the areas of all the defect rectangles corresponding to each defect type, select the defect rectangle with the largest area as the registration rectangle, and use the remaining defect rectangles as the first sequence of rectangles to be tested. Calculate the minimum area overlap rate between the test rectangles in the first test rectangle sequence and the registration rectangle; If the minimum area overlap rate is greater than the minimum area overlap rate threshold, then the corresponding rectangle to be tested is deleted to obtain the target rectangle sequence. In the target rectangle sequence, the target rectangle with the largest area is selected again as the registration rectangle, and the remaining target rectangles are used as the second sequence of rectangles to be tested. The minimum area overlap rate calculation and threshold comparison are repeatedly performed based on the second test rectangle sequence and the registration rectangle until the minimum area overlap rate of all test rectangles and the selected registration rectangle is less than or equal to the minimum area overlap rate threshold. The selected registration rectangles and their corresponding defect types are used as the pin defect detection results.

7. The method for generating semantic representations of transmission line pin defects according to claim 1, characterized in that, Based on the insulator string detection results, the target image is cropped at multiple levels to obtain a first target image sequence and a second target image sequence, including: The rectangular frames of all insulator strings in the insulator string detection results are merged to obtain the merged insulator string frame; Within the range of the insulator string merging frame, a sliding window image selection is performed to obtain multiple sliding window regions; The target image is cropped according to each of the sliding window regions to obtain the first target image sequence; The target image is cropped according to the rectangular frames of each insulator string to obtain the second target image sequence.

8. A semantic generation system for pin defects in transmission lines, characterized in that, The system includes: The insulator string detection module is used to acquire a target image of the transmission line, input the target image into the insulator string detection model, and obtain the insulator string detection result. An insulator string trimming module is used to perform multi-level region trimming on the target image based on the insulator string detection results to obtain a first target image sequence and a second target image sequence; The pin defect detection module is used to input the first target image sequence into the pin defect detection model in sequence, output multiple defect rectangles and corresponding defect types, and perform confidence filtering and area non-maximum suppression processing on the defect rectangles based on the defect types to obtain the pin defect detection results. The spatial structure detection module is used to input the second target image sequence into the spatial structure detection model in sequence, output multiple spatial structure rectangles and corresponding spatial structure types, and perform confidence filtering and non-maximum suppression processing based on the spatial structure types to obtain spatial structure detection results. The defect semantic generation module is used to perform component relationship binding and spatial structure binding based on the insulator string detection results, the pin defect detection results, and the spatial structure detection results, and generate pin defect semantic information.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the semantic generation method for transmission line pin defects as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the semantic generation method for transmission line pin defects as described in any one of claims 1-7.

Images