Power transmission line insulator string defect positioning system and method based on cascade detection strategy

Abstract

The invention belongs to the technical field of image processing, and discloses a power transmission line insulator chain defect positioning system and method based on a cascade detection strategy, and the system comprises an insulator chain candidate region positioning network and an instance segmentation network. The insulator string candidate area positioning network is used for identifying an insulator string in a to-be-processed image and marking a bounding box to position an insulator string candidate area, and the instance segmentation network is used for performing defect identification and instance segmentation on the insulator string candidate area. According to the invention, a two-stage cascade detection strategy is provided, so that the defects of the insulator chain can be positioned more accurately; particularly, a BiELAN module is introduced into the instance segmentation network, more efficient multi-scale feature fusion is realized through a double-layer routing attention mechanism and a self-adaptive query mechanism, and the capability of identifying insulator strings of different sizes is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, relates to equipment monitoring in electric power engineering, and in particular to a transmission line insulator string defect positioning system and method based on a cascade detection strategy. Background Art

[0002] With the rapid development of automation and intelligence of power systems, intelligent detection of power lines has become a research hotspot in the field of power inspection. Although traditional manual inspection and detection methods based on image processing can provide certain support in some scenarios, they have problems such as low detection efficiency, poor accuracy, and being greatly affected by environmental conditions. Especially in complex environments, traditional methods often have difficulty in coping with complex power equipment and diverse environmental changes.

[0003] In this context, computer vision and deep learning technologies are widely used in intelligent inspection of power lines, especially target detection and instance segmentation technologies. As an advanced instance segmentation model, SOLOv2 network performs well in many visual inspection tasks. With its unique structural design and efficient learning mechanism, SOLOv2 network can effectively detect and segment targets in more complex scenarios and has strong performance.

[0004] However, although the SOLOv2 network has achieved good results in many target detection tasks, it still faces some challenges. First, when dealing with dense targets or small objects, the SOLOv2 network may be affected by blurred boundaries, resulting in decreased segmentation accuracy. Second, the SOLOv2 network's ability to handle long-tail distribution data still needs to be optimized, especially when some categories in the dataset are scarce, the model may be biased or missed. In addition, the SOLOv2 network has a high computational complexity and may face greater computational and storage pressure when deployed on resource-limited embedded platforms, which limits its application in real-time monitoring.

[0005] Therefore, although the SOLOv2 network has strong advantages in many object detection tasks, it still has certain limitations when dealing with complex environments, long-tail data, and limited computing resources. In response to these problems, combining the advantages of the traditional SOLOv2 network to improve the robustness, efficiency, and accuracy of the model has become an important direction of current research. Summary of the invention

[0006] The purpose of the present invention is to address the deficiencies in the above-mentioned prior art and to provide a transmission line insulator string defect location system and method based on a cascade detection strategy, so as to achieve efficient identification and defect detection of insulator strings on transmission lines, thereby solving the technical problems of poor accuracy and low speed of insulator string defect detection in the prior art.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions to achieve it.

[0008] The present invention provides a transmission line insulator string defect location system based on a cascade detection strategy, which includes: An insulator string candidate region positioning network is used to identify the insulator strings in the processed image and mark the bounding box to locate the insulator string candidate region; An instance segmentation network is used to perform defect recognition and instance segmentation on candidate areas of insulator strings; the instance segmentation network uses an improved SOLOv2 network, including a trunk subnetwork, a neck subnetwork and a prediction subnetwork; the trunk subnetwork is used to extract multi-layer features of different scales from input image features; the neck subnetwork is used to fuse multi-layer features of different scales; the neck subnetwork includes a first fusion unit and a second fusion unit; the first fusion unit performs upsampling and addition processing on each output feature of the trunk subnetwork in the direction of increasing scale to obtain a multi-layer fusion output of the first fusion unit; the second fusion unit performs aggregation and addition processing on each layer of fusion output of the first fusion unit in the direction of reducing scale based on a bidirectional attention mechanism to obtain a multi-layer fusion output of the second fusion unit; the prediction subnetwork is used to identify defects of insulator strings and perform instance segmentation based on the multi-layer fusion output of the second fusion unit.

[0009] In one possible implementation, the insulator string candidate region positioning network is used as a cascaded window before the improved SOLOv2 network to first mark the insulator string from the image to be processed. In the present invention, the insulator string candidate region positioning network uses a YOLOv9 network.

[0010] In one implementable manner, the backbone subnetwork of the instance segmentation network uses an improved ResNet network; the improved ResNet network includes a plurality of residual modules arranged in sequence; the residual modules have the same structure, including a plurality of convolution modules and a deformable convolution layer arranged in sequence, and the input features of the input residual modules are sequentially passed through the plurality of convolution modules and the variable convolution layer and then spliced ​​with the input features by channels to obtain the output features of the residual modules.

[0011] In one possible implementation, the backbone subnetwork further includes a CQSFM module disposed before the plurality of residual modules, the CQSFM module including four different-scale two-dimensional convolution layers, a maximum pooling layer, and a splicing unit; first, the candidate region of the insulator string in the input image is convolved with four different-scale two-dimensional convolution layers, and then reduced to half of its original size by the maximum pooling layer; then, the four convolution outputs are spliced ​​in a clockwise direction to form a new candidate region of the insulator string as the input of the residual module. By using convolution kernels of different sizes to extract features, the problem of the insulator string being too large or too small due to the change in the shooting position of the drone can be effectively addressed.

[0012] In an implementable manner, in the neck subnetwork of the instance segmentation network, the second fusion unit includes a plurality of BiELAN modules, and specifically, the number of BiELAN modules is 1 less than the number of fusion output feature layers of the first fusion unit. The BiELAN module is used to aggregate the input features based on the bidirectional attention mechanism, and the specific operation is as follows: according to the set step size, the input feature image is divided into a number of blocks, three different blocks are randomly selected to respectively construct a key matrix, a query matrix and a value matrix, firstly the key matrix and the query matrix are multiplied and then processed by the Softmax function, the calculated result is then multiplied by the value matrix, and the result is aggregated by the ELAN module; repeating this step for the input feature image obtains the corresponding aggregation result, that is, the output feature of the BiELAN module.

[0013] In one implementable manner, the prediction subnetwork includes a classification branch, a mask core branch and a mask feature branch; the classification branch is used to identify the category of the insulator string (such as normal or defective) based on the fusion result of the neck subnetwork; the mask core branch is used to generate dynamic convolution kernel weights for insulator string defect segmentation based on the fusion result of the neck subnetwork; the mask feature branch is used to generate an insulator string defect instance segmentation image based on the fusion result of the neck subnetwork; the mask feature branch is dynamically convolved with the outputs of the classification branch and the mask core branch to obtain the final insulator string defect area instance segmentation result; the classification branch, mask core branch, and mask feature branch all adopt the conventional structure disclosed by traditional SOLOv2.

[0014] The present invention also provides a method for locating defects in an insulator string of a transmission line based on a cascade detection strategy, comprising the following steps: S1. Construct a data set for instance segmentation of insulator strings for transmission lines; S2, training the transmission line insulator string defect location system using the transmission line insulator string instance segmentation dataset; S3. Input the collected transmission line image containing the insulator string into the trained transmission line insulator string defect location system to obtain the insulator string defect instance segmentation result.

[0015] The above step S1 includes the following sub-steps: S11, collecting a number of real image samples containing insulator strings, and performing instance segmentation and annotation on the insulator strings; S12, simulating different environmental conditions through virtual scene building software, performing simulation processing on real image samples containing insulator strings, and obtaining a number of virtual image samples containing insulator strings; S13, virtual image samples and real image samples constitute the transmission line insulator string instance segmentation dataset.

[0016] In the above step S11, a drone equipped with a high-resolution image acquisition device is used to regularly take aerial photos of the transmission line to obtain multiple images of the transmission line and the insulator strings on its installation accessories. The flexibility and efficiency of the drone ensure the comprehensiveness and real-time nature of data acquisition, thereby providing sufficient image data for subsequent defect detection.

[0017] Perform instance segmentation and annotation on the collected images containing insulator strings to obtain accurate instance segmentation datasets of insulator strings and their defects. And use manual or semi-automatic annotation tools to accurately annotate the boundaries of each insulator string and its defects to ensure the high quality of the dataset and provide a reliable foundation for model training.

[0018] In the above step S12, the overall structure of the transmission line is simulated and constructed using virtual scene construction software (such as Unreal Engine 4 (UE4)) to generate a large number of high-quality virtual data sets. Through simulation technology, the state of the insulator string under different environmental conditions, such as different weather, lighting, angles, etc., can be simulated.

[0019] The virtual data set can also be subjected to various enhancement processing, including rotation, scaling, color jittering and other operations, to expand the real data set and improve the model's ability to adapt to diverse environmental conditions.

[0020] The above step S2 includes the following sub-steps: S21, dividing the transmission line insulator string instance segmentation dataset into a training set and a test set; S22, using the training set to train the transmission line insulator string defect location system; the step S22 includes the following sub-steps: S221, inputting samples in the training set into the transmission line insulator string defect location system, firstly locating and marking the insulator string candidate area through the insulator string candidate area location network; then inputting the image of the marked insulator string candidate area into the instance segmentation network to obtain the insulator string defect instance segmentation image; S222, calculating the loss value based on the instance segmentation image of the insulator string defect predicted by each sample and the labeled instance segmentation image; S223, optimizing the parameters of the transmission line insulator string defect location system according to the loss value; Repeat the above steps S221-S223 until the transmission line insulator string defect location system converges; S23. Use the test set to test and verify the trained transmission line insulator string defect location system.

[0021] The above step S3 includes the following sub-steps: S31, inputting the collected transmission line image containing the insulator string into the insulator string candidate area positioning network, locating the insulator string candidate area and marking it; S32, inputting the image with the candidate area of ​​the insulator string located into the instance segmentation network to obtain the instance segmentation image of the insulator string defect.

[0022] The transmission line insulator string defect location system and method based on the cascade detection strategy provided by the present invention have the following beneficial effects: (1) The present invention first locates the insulator string through the insulator string candidate region positioning network, and then performs instance segmentation on the insulator string defects through the instance segmentation network, forming a two-stage cascade detection strategy, which can more accurately locate the insulator string defects; (2) The present invention sets an insulator string candidate area positioning network as a cascaded window before the improved SOLOv2 network, which fully combines the capabilities of fast target detection and the improved SOLOv2 network in high-precision instance segmentation; this combination not only significantly improves the detection accuracy of insulator string defects, especially in the case of complex backgrounds and small targets, but also greatly improves the overall detection speed, meeting the high efficiency requirements of real-time monitoring of transmission lines; moreover, by using the bounding box information of the cascaded window as the input of the improved SOLOv2 network, the system can accurately limit the specific area of ​​instance segmentation, avoid redundant calculations within the entire map, significantly reduce the waste of computing resources, and improve resource utilization; (3) The present invention introduces the BiELAN module (Bidirectional Enhanced Local Attention Network) into the backbone subnetwork of the improved SOLOv2 network. Through the dual-layer routing attention mechanism and the adaptive query mechanism, more efficient multi-scale feature fusion is achieved, thereby improving the recognition capability of insulator strings of different sizes. In addition, the deformable convolution (DCN) is integrated into the residual structure of the improved SOLOv2 network, so that the model can dynamically adjust the sampling position of the convolution kernel and flexibly adapt to various geometric transformations and deformations of the insulator strings in the image, thereby further improving the segmentation quality and accuracy. (4) The present invention introduces a clockwise four-scale fusion module in the backbone subnetwork of the improved SOLOv2 network. By using convolution kernels of different sizes to extract features, the problem of insulator strings being too large or too small due to changes in the shooting position of the drone can be effectively addressed; (5) The present invention uses virtual scene construction software to generate a large amount of high-quality virtual data, and combines it with a variety of data enhancement techniques (such as rotation, scaling, color jittering, etc.) to effectively expand the real data set; this method not only solves the problems of insufficient training data and insufficient diversity, and improves the adaptability of the model under different environmental conditions, but also enhances the robustness and reliability of the system in the face of various complex environments and different types of insulator string defects through a number of technical improvements (such as BiELAN, multi-step convolution, deformable convolution, etc.), ensuring stability and efficiency in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic diagram of the structure of a transmission line insulator string defect location system based on a cascade detection strategy provided in Example 1 of the present invention; Figure 2 This is the schematic diagram of the CQSFM module principle; Figure 3 Schematic diagram of the residual module structure; Figure 4 Schematic diagram of deformable convolution principle; Figure 5 This is the schematic diagram of the BiELAN module principle; Figure 6 This is a schematic diagram of the ELAN module structure; Figure 7 A schematic flow chart of a method for locating defects in an insulator string of a transmission line based on a cascade detection strategy provided in Embodiment 2 of the present invention; Figure 8 This is an example of a transmission line insulator string image collected in Embodiment 2 of the present invention; Fig. 9 for Figure 8 The output image of the transmission line insulator string image after detection. DETAILED DESCRIPTION

[0024] To make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described in detail below. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0025] Example 1 This embodiment provides a transmission line insulator string defect location system based on a cascade detection strategy, which includes an insulator string candidate region location network and an instance segmentation network. The insulator string candidate region location network is used to identify the insulator string in the processed image and mark the boundary box to locate the insulator string candidate region. The instance segmentation network is used to perform defect recognition and instance segmentation on the insulator string candidate region.

[0026] The insulator string candidate region positioning network is a cascaded window before the improved SOLOv2 network, and the insulator string is first calibrated from the image to be processed. In this embodiment, the insulator string candidate region positioning network uses the YOLOv9 network. The network structure is disclosed in Chien-Yao Wang et al., "YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information" (Chien-Yao Wang, I-Hau Yeh, and Hong-Yuan Mark Liao, "YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information", arXiv:2402.13616, February 29, 2024). Using the YOLOv9 network to perform target detection on the input image, the candidate region of the insulator string can be quickly located, and the bounding box coordinates of each candidate region can be obtained. With its efficient detection capability, the YOLOv9 network can provide accurate bounding box information while maintaining high speed. The bounding box coordinates output by the YOLOv9 network are used as the input of the improved SOLOv2 network to accurately define the area where the improved SOLOv2 network performs instance segmentation; and the insulator string to be segmented can be determined based on the area of ​​the insulator string to further improve the accuracy of instance segmentation of insulator string defects. In this way, the improved SOLOv2 network does not need to perform segmentation in the entire image, thereby improving the efficiency and accuracy of segmentation and reducing the waste of computing resources.

[0027] Therefore, the insulator string is first obtained through cascaded windowing, while combining the speed advantage of the YOLOv9 network; then the high-precision instance segmentation capability of the improved SOLOv2 network is introduced, and a two-stage cascade detection strategy is adopted to significantly improve the ability to capture small targets and detailed features. This strategy not only speeds up the detection speed, but also improves the accuracy of detection, especially in the detection of insulator string defects under complex backgrounds.

[0028] In this embodiment, the instance segmentation network uses an improved SOLOv2 network, including a backbone subnetwork, a neck subnetwork and a prediction subnetwork.

[0029] (1) Backbone subnetwork The backbone sub-network is used to extract multi-layer features of different scales from the input image features.

[0030] like Figure 1 As shown, in this embodiment, the backbone sub-network uses an improved ResNet50 network. The improved ResNet50 network includes a CQSFM module (Clockwise Quad-Scale Fusion Module) and four residual modules (Residual Module 1 to Residual Module 4) arranged in sequence.

[0031] like Figure 2 As shown in the figure, the CQSFM module includes four different scales of two-dimensional convolutional layers, a maximum pooling layer, and a splicing unit. First, the input image is convolved with four different scales of two-dimensional convolutional layers (defined as Conv1, Conv2, Conv3, and Conv4 in this embodiment, with convolution kernel sizes of 1×1, 3×3, 5×5, and 7×7, respectively), and then the image is reduced to half of its original size (both length and width are reduced to half of the original size) through a 2×2 maximum pooling layer (MaxPooling); then, the four convolution outputs are spliced ​​in a clockwise direction to form a new image as the input of the input layer. By using convolution kernels of different sizes to extract features, the problem of insulator strings being too large or too small due to changes in the shooting position of drones can be effectively addressed.

[0032] The residual module has the same structure, such as Figure 3 As shown in the figure, it includes a 1×1 convolution module, a 3×3 convolution module, a 1×1 convolution module and a deformable convolution layer arranged in sequence. Each convolution module includes a convolution layer, a batch normalization layer and a ReLU activation function. The input features of the input residual module are sequentially concatenated with the input features by channel after passing through each convolution module and the variable convolution layer to obtain the output features of the residual module. Through multi-step convolution operations, the spatial variation information of the input feature map is obtained. Moreover, multi-step convolution can capture richer spatial context information (i.e., spatial variations of different scales and directions), generate an offset for adjusting the deformable convolution sampling position, and provide a more accurate sampling position adjustment basis for subsequent deformable convolutions.

[0033] Deformable Convolution (DCN) is an innovative convolutional neural network structure that allows the convolution kernel to dynamically adjust its sampling position according to the content in the input feature map when processing an image. DCN controls the sampling point of the convolution kernel by introducing offsets, making the model more flexible to adapt to changes in various shapes and sizes. Figure 4As shown, when a deformable convolution operation is performed, the original convolution kernel first determines an initial sampling grid on the input feature map; then, the position of this grid is adjusted according to the learned offset to obtain the corresponding offset area. In this way, even in the face of complex deformations such as rotation, scaling or distortion, the convolution kernel can effectively capture the key features of the target object. This mechanism not only improves the quality and accuracy of instance segmentation, but also enhances the model's adaptability to target objects of different shapes and angles. In this embodiment, a deformable convolution is integrated in the residual module of SOLOv2, the feature map generated in the previous step is used as input, the offset is determined according to the extracted spatial variation information, and the sampling position of the convolution kernel on the feature map is controlled by the offset, so as to realize the deformable convolution by dynamically adjusting the sampling position of the convolution kernel. Due to the existence of the offset, the output feature map can better reflect the true shape and boundary of the target object in the input image, which is crucial to improving the performance of semantic segmentation and instance segmentation tasks; especially when processing insulator strings of different shapes and angles, it can maintain a high level of segmentation quality and accuracy; improve the model's adaptability to complex deformations.

[0034] In this embodiment, the output features C2-C5 of residual module 1-residual module 4 are used as the output of the backbone sub-network.

[0035] (2) Neck subnetwork The neck subnetwork is used to fuse multi-layer features of different scales. The neck subnetwork includes a first fusion unit and a second fusion unit.

[0036] The first fusion unit upsamples and adds each output feature of the backbone subnetwork in the direction of increasing scale to obtain the multi-layer fusion output of the first fusion unit. The first fusion unit includes three upsampling modules, namely upsampling module 1, upsampling module 2 and upsampling module 3. The first fusion unit takes the output features C2-C5 of the backbone subnetwork as input, and obtains the fusion output features M2-M5 of the first fusion unit through each sampling module and horizontal connection. Specifically, feature C5 is processed by 1×1 convolution (Conv1×1) to obtain feature M5, feature M5 is processed by upsampling module 1 and feature C4 is processed by 1×1 convolution to obtain feature M4 (here refers to element-wise addition), feature M4 is processed by upsampling module 2 and feature C3 is processed by 1×1 convolution to obtain feature M3, and feature M3 is processed by upsampling module 3 and feature C2 is processed by 1×1 convolution to obtain feature M2.

[0037] The second fusion unit aggregates and adds the fusion outputs of each layer of the first fusion unit based on the bidirectional attention mechanism in the direction of scale reduction to obtain the multi-layer fusion output of the second fusion unit. The second fusion unit includes three BiELAN modules, namely BiELAN module 1, BiELAN module 2 and BiELAN module 3. The second fusion unit takes the output features M2-M5 of the first fusion unit as input, and obtains the fusion output features P2-P5 of the second fusion unit through each BiELAN module and horizontal connection. Specifically, feature M2 is processed by 3×3 convolution (Conv3×3) to obtain feature P2, feature P2 is processed by BiELAN module 1 and feature M3 is processed by 3×3 convolution to obtain feature P3, feature P3 is processed by BiELAN module 2 and feature M4 is processed by 3×3 convolution to obtain feature P4, and feature P4 is processed by BiELAN module 3 and feature M5 is processed by 3×3 convolution to obtain feature P5.

[0038] In this embodiment, the BiELAN (Bidirectional Level Efficient Layer Aggregation Networks) module is introduced into the improved SOLOv2 network to enhance the fusion effect of multi-scale features. The BiELAN module aggregates the input features based on the bidirectional attention mechanism. Through the bidirectional information flow and attention mechanism, it can more effectively integrate feature information of different scales and improve the model's detection ability for multi-scale targets.

[0039] like Figure 5 As shown, the specific operation of the BiELAN module is: according to the set step size (for example s =2), the input feature image is divided into several blocks, and three different blocks are randomly selected to construct the key matrix K , query matrix Q Sum Matrix V , that is, stack the image blocks with a step size of 1; for the key matrix K Sum Matrix V , and also given a coefficient k (in this embodiment, k = 2, i.e., stacking the image blocks twice), so the key matrix K Sum Matrix V The dimension is , the dimension of the query matrix is , W , H represents the width and height of the input feature image, C Indicates the number of channels. First, use the key matrix K and the query matrix QThe multiplication is then processed by the Softmax function (i.e. mm&Softmax in the figure, mm represents matrix multiplication), and the calculation result is then multiplied by the value matrix V Multiply them, and the result is aggregated by the ELAN module to obtain the aggregated result of the three image blocks. Repeat this step for all image blocks of the input feature image to obtain the aggregated result of the input feature image (each image block is not repeatedly selected), which is also the output feature of the BiELAN module.

[0040] The above ELAN (Efficient Layer Aggregation Networks) module structure is as follows Figure 6 As shown, it includes two branches, the first branch includes a 1×1 convolution module, and the second branch includes a 1×1 convolution module and four 3×3 convolution modules set in sequence; the input features of the ELAN module are processed by the two branches respectively, and the output features of the 1×1 convolution module in the second branch and the output feature residual of the second 3×3 convolution module are connected to the output of the fourth 3×3 convolution module, and then added to the output features obtained by the first branch and processed by the 1×1 convolution module to obtain the output result of the ELAN module.

[0041] The BiELAN module uses a two-layer routing attention mechanism to filter out the least relevant key-value pairs from the coarse-grained extracted insulator string features; this mechanism dynamically adjusts the attention weights to highlight key information and suppress irrelevant information, thereby improving the effectiveness and robustness of feature representation. And through the adaptive query mechanism, content perception in sparse mode is achieved; this mechanism dynamically adjusts the query strategy according to the content of the input features, allowing the model to capture key features more flexibly and improve the intelligence level of feature fusion.

[0042] Moreover, the BiELAN module used in this embodiment avoids relying on computationally intensive sparse matrix algorithms that merge memory operations when applying token-to-token attention, and instead uses hardware-friendly dense matrix algorithms to collect key-value tokens. This optimization significantly reduces computational complexity and improves the execution efficiency of the attention mechanism, making the model more real-time while maintaining high performance.

[0043] (3) Prediction subnetwork The prediction subnetwork is used to identify defects of the insulator string and perform instance segmentation based on the multi-layer fusion output of the second fusion unit.

[0044] The prediction subnetwork includes a classification branch, a mask kernel branch and a mask feature branch; the classification branch is used to identify the insulator string category (such as normal or defective) based on the fusion result of the neck subnetwork; the mask kernel branch is used to generate dynamic convolution kernel weights for insulator string defect segmentation based on the fusion result of the neck subnetwork; the mask feature branch is used to generate an insulator string defect instance segmentation image based on the fusion result of the neck subnetwork; the mask feature branch is dynamically convolved with the output of the classification branch and the mask kernel branch to obtain the final insulator string defect area instance segmentation result. The classification branch, mask kernel branch, and mask feature branch all adopt the conventional structure disclosed by traditional SOLOv2. For details, please refer to "SOLOv2: Dynamic and Fast Instance Segmentation" published in the 34th Conference on Neural Information Processing Systems, the authors are Xinlong Wang, Rufeng Zhang, Tao Kong, Lei Li, and Chunhua Shen (Xinlong Wang, Rufeng Zhang, Tao Kong, Lei Li, Chunhua Shen, "SOLOv2:Dynamic and Fast Instance Segmentation", 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada).

[0045] Example 2 This embodiment provides a method for locating defects in an insulator string of a transmission line based on a cascade detection strategy. Figure 7 As shown, the following steps are included: S1. Construct a transmission line insulator string instance segmentation dataset.

[0046] This step is to construct a transmission line insulator string instance segmentation dataset for training the transmission line insulator string defect location system based on the cascade detection strategy provided in Example 1.

[0047] This step includes the following sub-steps: S11. Collect a number of real image samples containing insulator strings, and perform instance segmentation and annotation on the insulator strings.

[0048] In this step, a DJI Matrice 300 RTK drone equipped with a DJI Zenmuse P1 high-resolution camera is used for image acquisition. Figure 8As shown. The drone conducts regular aerial photography of the target transmission line once a week, and the flight altitude is set to 120 meters to ensure that the acquired images have sufficient resolution and coverage. The images generated by each aerial photography cover the insulator strings on the transmission line and its installation accessories, ensuring the comprehensiveness and real-time nature of the data. During the image acquisition process, GPS tags and drone flight control systems are used to record the geographic location information of each image to facilitate subsequent data processing and model training. In this embodiment, the number of image samples collected containing insulator strings is 1,000.

[0049] The collected insulator string images are annotated by instance segmentation using the LabelMe annotation tool. The boundaries of the insulator strings in each image are annotated using precise polygon tools to generate high-quality instance segmentation masks. During the annotation process, the boundaries of each insulator string are ensured to be clear and accurate to avoid overlaps and omissions. After the annotation is completed, all annotated data is quality checked to ensure the high accuracy and consistency of the data set, providing a reliable real data foundation for subsequent model training.

[0050] S12. Using virtual scene building software to simulate different environmental conditions, simulate the real image samples containing the insulator string to obtain a number of virtual image samples containing the insulator string.

[0051] In this embodiment, Unreal Tournament 4 (UE4) is used to build a virtual transmission line environment based on the three-dimensional model of the real transmission line. In the virtual environment, images of insulator strings under various conditions including different weather (such as sunny, rainy, foggy), illumination (changes between day and night) and / or angles (different shooting angles) are generated. Based on the 1,000 real images obtained above, the number of virtual images generated by Unreal Engine 4 reaches 5,000.

[0052] This step also uses the built-in script automation of Unreal Engine 4 to perform various enhancement processing, including random rotation (±30 degrees), scaling (0.8 times to 1.2 times), color jitter (brightness, contrast, saturation changes), adding noise and other operations to perform data enhancement processing on the generated virtual image.

[0053] S13, virtual image samples and real image samples constitute the transmission line insulator string instance segmentation dataset.

[0054] The enhanced virtual image samples are combined with real image samples to form a transmission line insulator string instance segmentation dataset, which further expands the dataset size and improves the generalization ability and environmental adaptability of the model.

[0055] In this embodiment, real image samples and virtual image samples are combined to form a transmission line insulator string instance segmentation dataset of a total of 6,000 images.

[0056] S2. Training the transmission line insulator string defect location system using a transmission line insulator string instance segmentation dataset.

[0057] This step S2 includes the following sub-steps: S21. Divide the transmission line insulator string instance segmentation dataset into a training set and a test set.

[0058] In this step, the transmission line insulator string instance segmentation dataset is randomly divided into a training set (4800 images) and a test set (1200 images) in a ratio of 8:2. The training set contains 4800 images and their corresponding instance segmentation masks, and the test set contains 1200 images and their corresponding instance segmentation masks. This division method ensures that the model can fully learn different types and diverse data features during the training process, and can effectively evaluate the performance of the model in the test phase.

[0059] S22, using the training set to train the transmission line insulator string defect location system. This step includes the following sub-steps: S221. Input the samples in the training set into the transmission line insulator string defect location system. First, the insulator string candidate area is located and labeled through the insulator string candidate area location network. Then, the image of the labeled insulator string candidate area is input into the instance segmentation network to obtain the insulator string defect instance segmentation image.

[0060] This embodiment uses the YOLOv9 network pre-trained with a public dataset as the insulator string candidate region localization network. Object detection is performed on each image in the training set and the test set. The YOLOv9 network is configured with an input resolution of 1024×1024 pixels and efficient object detection is achieved through GPU acceleration (NVIDIA RTX 3090). YOLOv9 outputs the bounding box coordinates of each insulator string ( x , y , width, height) and their confidence scores. The detection results are stored in JSON format for subsequent processing.

[0061] The bounding box coordinates detected by the YOLOv9 network are used as the input area of ​​the instance segmentation network (improved SOLOv2 network). The improved SOLOv2 network only performs instance segmentation operations in these candidate areas to avoid redundant calculations in the entire image. In specific implementation, a Python script is used to convert the bounding box coordinates output by the YOLOv9 network into the input format required by SOLOv2, and pass it to the improved SOLOv2 network for accurate segmentation to obtain the instance segmentation image of the insulator string defect.

[0062] S222. Calculate the loss value based on the insulator string defect instance segmentation image predicted by each sample and the labeled instance segmentation image.

[0063] In this step, for the instance segmentation network part, the overall loss function L ins Mainly include: Classification branch loss L cate : Determine the category (such as "defect" or "normal") of the current instance (insulator string / defect); Mask branch loss L mask : Generate a binary segmentation mask of the insulator string (or defect area); Therefore, this part can usually be expressed as: (1); in, , Represent weighting factors respectively.

[0064] (2); in, p t For positive samples (i.e., the network predicts the correct category), p , for negative samples (i.e., the network predicts the wrong category), it is 1- p , p Represents the category probability obtained by the network; γ It is called the focusing parameter, which can suppress easy-to-classify samples and enhance the gradient weight of difficult samples. α t Used for weighting when the classes are imbalanced.

[0065] (3); in, Representation basis X and Y The calculated mask branch loss; X represents the image mask matrix generated by the network, x i ∈[0,1] represents the mask pixel value generated by the network; Y represents the real image mask matrix, y i ∈[0,1] represents the true mask pixel value.

[0066] According to the insulator string defect instance segmentation image predicted by each sample and the labeled instance segmentation image, the corresponding loss value is calculated by the above-mentioned overall loss function.

[0067] S223. Optimize the transmission line insulator string defect location system parameters based on the loss value.

[0068] According to the loss value calculated in step S222, the parameters of the improved SOLOv2 network are optimized using the gradient descent optimization algorithm Adam.

[0069] Repeat the above steps S221-S223 until the transmission line insulator string defect location system converges. When the loss value is less than the set threshold, it can be considered that the improved SOLOv2 network converges and a trained transmission line insulator string defect location system is obtained.

[0070] S23. Use the test set to test and verify the trained transmission line insulator string defect location system.

[0071] This step uses the test set to test the trained transmission line insulator string defect location system and evaluate the test results, such as evaluating the system through indicators such as accuracy, intersection over union (IoU), false positive rate (FPR), etc. If the evaluation effect does not meet the set requirements, it is necessary to adjust the system network parameters and retrain the system using the training set until the set requirements are met.

[0072] S3. Input the collected transmission line image containing the insulator string into the trained transmission line insulator string defect location system to obtain the insulator string defect instance segmentation result.

[0073] This step S3 includes the following sub-steps: S31, inputting the collected transmission line image containing the insulator string into the insulator string candidate area positioning network, locating the insulator string candidate area and marking it.

[0074] S32, inputting the image with the candidate area of ​​the insulator string located into the instance segmentation network to obtain the instance segmentation image of the insulator string defect.

[0075] Through the above steps S31 and S32, images of insulator strings and their defects highlighted in different colors are generated, such as Fig. 9 As shown; the red box in the figure marks the identified insulator string; and the normal area and defective area of ​​the specified insulator string in the figure are effectively instance segmented and distinguished by different colors.

[0076] In summary, the present invention adopts a two-stage cascade detection strategy. First, the insulator string candidate area positioning network quickly locates the insulator string candidate area, and then the instance segmentation network performs high-precision instance segmentation in a limited area. This strategy not only speeds up the overall detection speed, but also improves the ability to capture small targets and detailed features, especially in the insulator string defect detection under complex backgrounds. Test results show that this strategy increases the detection speed by 30% and the detection accuracy by 15%.

[0077] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A transmission line insulator string defect location system based on cascade detection strategy, characterized in that: include: An insulator string candidate region positioning network is used to identify the insulator strings in the processed image and mark the bounding box to locate the insulator string candidate region; An instance segmentation network is used to perform defect recognition and instance segmentation on candidate areas of insulator strings; the instance segmentation network uses an improved SOLOv2 network, including a trunk subnetwork, a neck subnetwork, and a prediction subnetwork; the trunk subnetwork is used to extract multi-layer features of different scales from input image features; The neck subnetwork is used to fuse multi-layer features of different scales; the neck subnetwork includes a first fusion unit and a second fusion unit; the first fusion unit performs upsampling and addition processing on each output feature of the trunk subnetwork in the direction of increasing scale to obtain a multi-layer fusion output of the first fusion unit; the second fusion unit performs aggregation and addition processing based on a bidirectional attention mechanism on each layer of fusion output of the first fusion unit in the direction of reducing scale to obtain a multi-layer fusion output of the second fusion unit; the prediction subnetwork is used to identify defects of the insulator string and perform instance segmentation based on the multi-layer fusion output of the second fusion unit.

2. The transmission line insulator string defect location system based on cascade detection strategy according to claim 1 is characterized in that: The insulator string candidate area positioning network uses the YOLOv9 network.

3. The transmission line insulator string defect location system based on cascade detection strategy according to claim 1 is characterized in that: The backbone subnetwork of the instance segmentation network uses an improved ResNet network; the improved ResNet network includes a plurality of residual modules arranged in sequence; the residual modules have the same structure, including a plurality of convolution modules and a deformable convolution layer arranged in sequence, and the input features of the input residual modules are sequentially passed through the plurality of convolution modules and the variable convolution layer and then spliced ​​with the input features by channels to obtain the output features of the residual modules.

4. The transmission line insulator string defect location system based on cascade detection strategy according to claim 3 is characterized in that: The backbone subnetwork also includes a CQSFM module arranged before multiple residual modules, and the CQSFM module includes four two-dimensional convolution layers of different scales, a maximum pooling layer and a splicing unit; first, the insulator string candidate area in the input image is convolved with four two-dimensional convolution layers of different scales, and then passed through the maximum pooling layer to reduce the size to half of the original size; then, the four convolution outputs are spliced ​​in a clockwise direction to form a new insulator string candidate area as the input of the residual module.

5. The transmission line insulator string defect location system based on cascade detection strategy according to any one of claims 1 to 4, characterized in that: In the neck subnetwork of the instance segmentation network, the second fusion unit includes multiple BiELAN modules. The BiELAN module is used to aggregate the input features based on the bidirectional attention mechanism. The specific operation is: according to the set step size, the input feature image is divided into several blocks, and three different blocks are randomly selected to construct a key matrix, a query matrix and a value matrix respectively. First, the key matrix and the query matrix are multiplied and then processed by the Softmax function. The calculated result is multiplied with the value matrix, and the result is aggregated by the ELAN module; repeating this step for the input feature image obtains the corresponding aggregation result, that is, the output feature of the BiELAN module.

6. The transmission line insulator string defect location system based on cascade detection strategy according to claim 5, characterized in that: The prediction subnetwork includes a classification branch, a mask core branch and a mask feature branch; The classification branch is used to identify the insulator string category based on the fusion result of the neck sub-network; The mask kernel branch is used to generate dynamic convolution kernel weights for segmenting insulator string defects according to the fusion result of the neck sub-network; The mask feature branch is used to generate an insulator string instance segmentation image based on the fusion result of the neck sub-network; the mask feature branch is dynamically convolved with the output of the classification branch and the mask kernel branch to obtain the final insulator string defect area instance segmentation result.

7. A method for locating defects in insulator strings of power transmission lines based on a cascade detection strategy, characterized in that: The following steps are involved: S1. Construct a data set for instance segmentation of insulator strings for transmission lines; S2. Training the transmission line insulator string defect location system according to any one of claims 1 to 6 using a transmission line insulator string instance segmentation dataset; S3. Input the collected transmission line image containing the insulator string into the trained transmission line insulator string defect location system to obtain the insulator string defect instance segmentation result.

8. The method for locating defects in insulator strings of power transmission lines based on a cascade detection strategy according to claim 7, characterized in that: The step S1 comprises the following sub-steps: S11, collecting a number of real image samples containing insulator strings, and performing instance segmentation and annotation on the insulator strings; S12, simulating different environmental conditions through virtual scene building software, performing simulation processing on real image samples containing insulator strings, and obtaining a number of virtual image samples containing insulator strings; S13, virtual image samples and real image samples constitute the transmission line insulator string instance segmentation dataset.

9. The method for locating defects in insulator strings of power transmission lines based on a cascade detection strategy according to claim 7, characterized in that: The step S2 comprises the following sub-steps: S21, dividing the transmission line insulator string instance segmentation dataset into a training set and a test set; S22, using the training set to train the transmission line insulator string defect location system; the step S22 includes the following sub-steps: S221, inputting samples in the training set into the transmission line insulator string defect location system, firstly locating and marking the insulator string candidate area through the insulator string candidate area location network; then inputting the image of the marked insulator string candidate area into the instance segmentation network to obtain the insulator string defect instance segmentation image; S222, calculating the loss value based on the instance segmentation image of the insulator string defect predicted by each sample and the labeled instance segmentation image; S223, optimizing the parameters of the transmission line insulator string defect location system according to the loss value; Repeat the above steps S221-S223 until the transmission line insulator string defect location system converges; S23. Use the test set to test and verify the trained transmission line insulator string defect location system.

10. The method for locating defects in insulator strings of power transmission lines based on a cascade detection strategy according to claim 7, characterized in that: The step S3 comprises the following sub-steps: S31, inputting the collected transmission line image containing the insulator string into the insulator string candidate area positioning network, locating the insulator string candidate area and marking it; S32, inputting the image with the candidate area of ​​the insulator string located into the instance segmentation network to obtain the instance segmentation image of the insulator string defect.

Images