Power transmission line similarity defect identification method based on collaborative mixed distribution training

Abstract

The invention discloses a power transmission line similarity defect identification method based on collaborative mixed distribution training, and the method comprises the steps: 1, obtaining image data of a power transmission line, and carrying out the preprocessing of the image data; 2, performing feature extraction to generate a series of feature maps, and performing convolution and up-sampling operation on each feature map through a feature pyramid to extract multi-scale features; 3, inputting the feature map generated in the step 2 into an encoder, and generating context information of defect features in the image; 4, receiving the context information in the step 3 through a decoder, processing the multi-scale features in the step 2 in combination with a Co-DETR model, generating a plurality of candidate positive sample coordinates for each Ground Truth, integrating the multi-scale features and the positive sample information through the decoder, and outputting prediction types and position information of similarity defects; according to the method, the problem that in the prior art, in a complex scene, artificial missing detection or false detection is likely to happen when similarity defects are artificially recognized is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of power transmission line defect detection, and in particular relates to a power transmission line similarity defect recognition method based on collaborative hybrid allocation training. Background Art

[0002] Transmission lines are a vital component of the power system. During the power transportation process, various defects often occur in transmission lines due to external reasons. If these defects are not handled in a timely manner, they may cause large-scale power outages or safety accidents, posing great hidden dangers to the safe operation of the power system. Among the common defects, similarity defects are the most difficult to identify, such as broken and loose wires. Traditional methods usually collect transmission line image data and manually identify similar defects. The traditional method has a low recognition accuracy, especially in complex scenarios. Due to the similar appearance and dispersed locations of similar defects, it is easy to cause human omissions or false detections.

[0003] In recent years, target detection technology has made breakthrough progress in the field of computer vision and has gradually become a key means of intelligent defect recognition. Target detection technology can achieve efficient detection of defects by automatically locating and classifying targets in images. Therefore, how to apply target detection technology to the similarity defect recognition process of transmission lines is an urgent problem to be solved. Summary of the invention

[0004] The purpose of the present invention is to provide a method for identifying similarity defects of power transmission lines based on collaborative hybrid allocation training, which solves the problem in the prior art that artificial similarity defects are prone to human omission or misdetection in complex scenarios.

[0005] The technical solution adopted by the present invention comprises the following steps:

[0006] Step 1: Obtain image data of the transmission line inspection equipment and pre-process the image data;

[0007] Step 2: Extract features from the preprocessed image data to generate a series of feature maps, perform convolution and upsampling operations on each feature map through the feature pyramid to extract multi-scale features;

[0008] Step 3: Input the feature map generated in step 2 into the encoder to generate context information of the defect features in the image;

[0009] Step 4: Receive the context information of the encoder in step 3 through the decoder, and process the multi-scale features extracted in step 2 in combination with the multi-scale adapter in the Co-DETR model to generate multiple candidate positive sample coordinates for each Ground Truth. Finally, integrate the multi-scale features and positive sample information through the decoder to output the predicted type and location information of the transmission line similarity defect.

[0010] The present invention is also characterized in that

[0011] The specific process of step 1 is as follows:

[0012] Step 1.1, collecting defect data of the transmission line;

[0013] Step 1.2: In the data preprocessing phase, the image data is labeled.

[0014] The defect data in step 1.1 specifically include wire broken strand image data, wire loose strand image data; insulator damage data, insulator self-explosion data.

[0015] The content of the image data annotation in step 1.2 specifically includes the defect location, defect type, and coordinate information related to the defect.

[0016] The specific process of step 2 is as follows:

[0017] The image data preprocessed in step 1 is input into the backbone network of the Co DETR model to extract image features and generate a series of feature maps, specifically represented as feature maps {F1, F2, ..., F j}, and then the feature map {F1, F2, ..., F j}After batch normalization BN and ReLU operations, normalization and nonlinear activation processing are performed, and convolution operations are performed to generate feature maps {C1, C2, ..., C n}, feature map {C1, C2, ..., C n}Convolution is performed through Conv2d 1×1, and then each layer of convolution features is upsampled to merge it with the convolution result of the previous layer. The upsampled feature maps are subjected to a Conv2d 3×3 convolution operation for further feature extraction. Finally, the prediction feature layer {P1, P2, ..., P n}, prediction feature layer P n Generate the prediction feature layer P through the maxpool operation n+1 , output the results and complete the extraction of multi-scale features.

[0018] The specific process of step 4 is as follows:

[0019] Step 4.1, according to the bounding box position of the ground truth, extract the feature information of the area corresponding to the bounding box from the feature map output by the encoder;

[0020] Step 4.2: Generate multiple candidate positive sample coordinates through random transformation of the feature information extracted in step 4.1. The candidate positive sample coordinates are located in the valid area of ​​the feature map and partially overlap with the ground truth.

[0021] Step 4.3: Encode the candidate positive sample coordinates obtained in step 4.2 into a query vector through position embedding and input it to the decoder;

[0022] Step 4.4: Introduce a hybrid collaborative allocation mechanism to allocate positive and negative samples to multiple auxiliary detection heads, each of which is responsible for feature detection at different scales.

[0023] The specific process of step 4.4 is as follows:

[0024] A combination of one-to-one and one-to-many matching methods is used to assign positive and negative samples to multiple auxiliary detection heads. One-to-one means that each Ground Truth can only match one detection result, that is, the Co DETR model will select a prediction box that matches the Ground Truth from all prediction boxes and regard it as a positive sample, and the other prediction boxes are regarded as negative samples. One-to-many matching means that each Ground Truth is matched with multiple detection results at the same time, that is, multiple positive samples are assigned to one target.

[0025] Step 4.5: For each auxiliary detection head, define the corresponding loss function. The loss includes classification loss and regression loss. The regression loss is used to calculate the difference between the predicted bounding box and the actual defect position, and the classification loss is used to measure the model's prediction accuracy for the defect type. By combining the multi-task loss function and introducing a balance coefficient to weight the loss, the final loss function is:

[0026]

[0027] Among them, L cls is the classification loss, L reg is the bounding box regression loss, L aux To assist in detecting head loss, λ1 and λ2 are balancing coefficients.

[0028] Step 4.6: The decoder integrates the multi-scale features with the mixed and coordinated positive and negative sample information, and outputs the predicted type and location information of the transmission line similarity defect.

[0029] The beneficial effects of the present invention are:

[0030] The power transmission line similarity defect recognition method based on collaborative hybrid allocation training of the present invention can obtain rich contextual information at different scales by introducing a multi-scale feature fusion module, thereby enhancing the system's adaptability to small targets, occlusions and similarity defects, and still has an efficient detection effect in complex scenarios, overcoming the problems of complex target information missing and easy human missed detection or false detection in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a flowchart diagram of a transmission line similarity defect identification method based on collaborative hybrid allocation training;

[0032] Figure 2 It is a schematic diagram of the system structure of a transmission line similarity defect identification method based on collaborative hybrid allocation training;

[0033] Figure 3 This is a schematic diagram of the feature pyramid network structure in the transmission line similarity defect recognition method based on collaborative hybrid allocation training;

[0034] Figure 4 It is a schematic diagram of the structure of the ATSS auxiliary head in the transmission line similarity defect identification method of the present invention using collaborative hybrid distribution training;

[0035] Figure 5 A schematic diagram of the structure of the Faster RCNN auxiliary head in the transmission line similarity defect recognition method of the present invention using collaborative hybrid distribution training;

[0036] Figure 6 It is a diagram of the detection results in Example 1 of the present invention. DETAILED DESCRIPTION

[0037] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] The present invention is based on the transmission line similarity defect identification method of the coordinated hybrid allocation training, such as Figure 1 and Figure 2 As shown, the following steps are included:

[0039] Step 1: Obtain image data of the transmission line inspection equipment and perform data preprocessing;

[0040] The specific process of step 1 is as follows:

[0041] Step 1.1, collecting defect data of the transmission line;

[0042] The defect data in step 1.1 specifically include conductor broken strand image data, conductor loose strand image data; insulator damage data, insulator self-explosion data.

[0043] Step 1.2: In the data preprocessing phase, the image data is labeled.

[0044] The content of the image data annotation in step 1.2 specifically includes the defect location, defect type, and coordinate information of the marked defect.

[0045] Step 2: Perform multi-scale feature extraction on the preprocessed image data to generate a series of feature maps. Perform convolution and upsampling operations on each feature map through the feature pyramid to extract multi-scale features.

[0046] The specific process of step 2 is as follows:

[0047] The image data preprocessed in step 1 is input into the backbone network of the Co DETR model to extract features from the image and generate a series of feature maps {F1, F2, ..., F j},like Figure 3 As shown, the feature maps {F1, F2, ..., F j}After batch normalization (BN) and linear rectifier unit (ReLU) operations, normalization and nonlinear activation are performed, and convolution operations are performed to generate feature maps {C1, C2, ..., C n}, feature map {C1, C2, ..., C n}Convolution is performed through Conv2d 1×1, and then each layer of convolution features is upsampled to merge it with the convolution result of the previous layer. The upsampled feature maps are subjected to a Conv2d 3×3 convolution operation for further feature extraction. Finally, the prediction feature layer {P1, P2, ..., P n}, which are used for detecting features of different resolutions. Prediction feature layer P n Generate the prediction feature layer P through the maxpool operation n+1 , output the result, complete the extraction of multi-scale features, reduce the resolution of the feature map by taking the maximum value of the local area on the feature map, while retaining the most significant features. Then, the network structure realizes the extraction of multi-scale features through multi-layer convolution, upsampling and feature fusion, and performs detection and recognition through the prediction feature layer.

[0048] By extracting multi-scale features through the feature pyramid, rich contextual information can be obtained at different scales, which enhances the system's adaptability to small targets, occlusions and similarity defects. It still has efficient detection effects in complex scenes, overcoming the problem in the existing technology that manual identification of similarity defects in complex scenes is prone to human omissions or false detections.

[0049] Step 3: Input the feature map generated in step 2 into the encoder to generate context information of the defect features in the image. The encoder is responsible for processing the extracted context information. This process helps the Co-DETR model understand the relationship between features and provide richer feature information for subsequent target detection.

[0050] Encoder such as Figure 2 As shown in the figure, the structure mainly includes: multi-layer self-attention layer, hybrid feedforward layer, and overlapping image block embedding, where the self-attention mechanism is used to capture the relationship between any positions on the feature map. Through the multi-head mechanism, the encoder can perform attention operations on different feature subspaces, thereby comprehensively considering the dependencies in different dimensions. The hybrid feedforward layer is a fully connected layer, which is mainly used to further linearly transform the features after self-attention processing, so as to perform nonlinear combination and enhance its distinguishability and expressibility. Overlapping image block embedding divides the image feature map into overlapping image blocks, and each image block is embedded in a high-dimensional space to form a feature vector. By overlapping, boundary information and local context information can be better preserved, reducing information loss.

[0051] Step 4: The multi-scale features extracted in step 2 are processed by the multi-scale adapter in the Co-DETR model to generate multiple candidate positive sample coordinates for each ground truth. Then, the multi-scale features and positive sample information are integrated through the decoder to output the predicted type and location information of the transmission line similarity defect.

[0052] Step 4.1, according to the bounding box position of the ground truth, extract the feature information of the area corresponding to the bounding box from the feature map output by the encoder;

[0053] Step 4.2: Generate multiple candidate positive sample coordinates through random transformation of the feature information extracted in step 4.1 to enhance the diversity of samples. The candidate positive sample coordinates are located in the valid area of ​​the feature map and partially overlap with the ground truth, thereby ensuring the authenticity and validity of the samples.

[0054] Step 4.3: Encode the candidate positive sample coordinates obtained in step 4.2 into a query vector through position embedding and input it to the decoder;

[0055] Step 4.4: Introduce a hybrid collaborative allocation mechanism to allocate positive and negative samples to multiple auxiliary detection heads, each of which is responsible for feature detection at different scales.

[0056] The specific process of step 4.4 is as follows:

[0057] A combination of one-to-one and one-to-many matching methods is used to assign positive and negative samples to multiple auxiliary detection heads. One-to-one means that each Ground Truth can only match one detection result, that is, the Co DETR model will select a prediction box that matches the Ground Truth from all prediction boxes and regard it as a positive sample, and the other prediction boxes are regarded as negative samples. One-to-many matching means that each Ground Truth is matched with multiple detection results at the same time, that is, multiple positive samples are assigned to one target.

[0058] One-to-one matching is used for final accurate detection. Each target will only have one final accurate detection frame. This is to improve the detection accuracy of the Co DETR model and avoid generating multiple redundant detection frames for the same target. One-to-many matching is used for auxiliary head detection. Each target can generate multiple positive samples, which are assigned to multiple auxiliary detection heads to enrich the model's supervision signals and increase the diversity of model learning. This can help the model better handle targets of different scales and shapes.

[0059] The auxiliary detection head includes ATSS auxiliary head and Faster R-CNN auxiliary head. The ATSS auxiliary head structure is as follows: Figure 4 As shown in the figure, after the image is input, the feature pyramid module extracts information of different scales in the image and generates a feature map. The feature map after convolution is shared to different branches, each of which focuses on different target detection tasks, including classification output layer, center / IoU output layer and regression output layer. The classification output layer is used to classify the objects in the image, and the output dimension is H×W×C, where H, W and C represent the width, height and number of channels of the image respectively. The center / IoU output layer is used to predict the center or IoU value of the target, and then judge the positioning accuracy of the target. The regression output layer is used to predict the bounding box parameters of the target, such as the width, height, position, etc. of the target. The anchor box is used to represent the initial position and size of the target. These anchor boxes cover different scales and sizes, helping the network to have a preliminary bounding box for adjustment at each feature map position. The prediction box is the final prediction box generated by the model, marking the detected target object. The decoding process combines the prediction box, classification results and confidence to generate the final target detection result.

[0060] The ATSS auxiliary head dynamically assigns positive samples to the detection box through an adaptive positive sample selection strategy. It can handle specific similar defect detection tasks more effectively, especially when the target scale varies greatly or is unevenly distributed. Through the classification output layer and the regression output layer, it can accurately locate the defect area in the image and use the IoU layer to optimize the accuracy of the bounding box, thereby reducing the occurrence of false detection and missed detection.

[0061] Faster R-CNN auxiliary head structure, such as Figure 5 As shown in the figure, after the input image, it usually includes multiple layers of convolution operations to capture important information such as edges, textures, and shapes in the image. After the input image and the corresponding feature map are input, the pooling layer helps to reduce the size of the feature map and improve the compactness of the features. The feature map output from the pooling layer will enter the attention layer. The feature map generated by the attention layer contains the feature information that has been enhanced. Each output feature represents features at different levels and adapts to the detection needs of targets of different scales. After the Conv1×1 convolution operation, it enters the classification prediction layer, which is responsible for class prediction of the extracted features and determines the type of target by classification confidence. After the convolution operation, it will also enter the category regression layer to predict the location information of the target. This step adjusts the bounding box so that the detection result can more accurately locate the position of the target in the image. After output from the category regression layer, the generated category label represents the target category predicted by the model. Combining the information of the classification prediction layer and the category regression layer, the final detection result is generated, showing the target box and related category labels in the image.

[0062] The Faster R-CNN auxiliary head generates potential target regions through the classic region proposal network (RPN), and further refines and classifies them through the classification and regression layers. Its advantage lies in handling complex backgrounds and irregularly shaped targets, and can more accurately locate defective areas based on the generated feature maps. Its classification and regression layer can effectively handle defects with occlusion or deformation, improving the robustness of the overall model.

[0063] By combining the dynamic positive sample selection of ATSS and the regional proposal capability of Faster R-CNN, efficient and accurate recognition of similar defects in transmission lines is ensured, especially in complex scenarios. At the same time, the choice of auxiliary detection heads is not limited to the above two types of auxiliary detection heads, and the specific types and quantities can be selected according to actual needs.

[0064] By introducing a hybrid collaborative allocation mechanism and optimizing the allocation mechanism of positive samples, it is ensured that each layer of auxiliary detection heads can receive the most appropriate positive samples, thereby improving the training efficiency and accuracy of the model, enabling the model to converge faster, significantly improving the training efficiency and overall detection effect. At the same time, by designing multiple auxiliary detection heads and optimizing for different types of defects, the entire system can efficiently identify multiple types of defects in the same network structure, thereby achieving more efficient recognition results for similarity defects in transmission lines.

[0065] Step 4.5: For each auxiliary detection head, define the corresponding loss function. The loss includes classification loss and regression loss. The regression loss is used to calculate the difference between the predicted bounding box and the actual defect position. The classification loss is used to measure the model's prediction accuracy of the defect type. By combining the multi-task loss function and introducing a balance coefficient to weight the loss, the final loss function form is:

[0066]

[0067] Among them, L cls is the classification loss, L reg is the bounding box regression loss, L aux To assist in detecting head loss, λ1 and λ2 are balancing coefficients.

[0068] Step 4.6: The decoder integrates the multi-scale features with the mixed and co-assigned positive and negative sample information, and outputs the predicted type and location information of the defect.

[0069] Decoder such as Figure 2 As shown in the figure, its main structure includes: Maskede self-attention layer, hybrid feedforward layer, and overlapping image block embedding. The Maskede self-attention layer is used to capture the relationship and contextual information between targets. The self-attention mechanism determines the relationship between each input feature and other features by calculating the correlation between features, helping the model understand the dependency between features. The hybrid feedforward layer is a fully connected layer, which is mainly used to further linearly transform the features processed by self-attention, so as to perform nonlinear combination and enhance its distinguishability and expressibility. Overlapping image block embedding divides the image feature map into overlapping image blocks, and each image block is embedded in a high-dimensional space to form a feature vector. By overlapping, boundary information and local context information can be better preserved, and information loss can be reduced.

[0070] The present invention is based on the transmission line similarity defect identification method of the collaborative hybrid allocation training, and the specific embodiments are as follows:

[0071] Example 1

[0072] The image data of a transmission line is obtained by the line inspection equipment, and the image detection is performed by the transmission line similarity defect recognition method based on collaborative hybrid allocation training of the present invention, wherein the auxiliary detection head adopts Faster R-CNN auxiliary head and ATSS auxiliary head, and the detection results are obtained as follows Figure 6 As shown, Figure 6 The detection result visualization effect diagram outputted by the transmission line similarity defect identification method based on collaborative hybrid allocation training of the present invention has a detection accuracy of up to 95% for loose strands of conductors under complex targets.

[0073] Example 2

[0074] On the basis of the above-mentioned embodiment 1, image detection is performed by using the transmission line similarity defect recognition method based on collaborative hybrid allocation training of the present invention, wherein the auxiliary detection head adopts the ATSS auxiliary head, and the detection accuracy of loose conductors under complex targets is as high as 93%.

[0075] Example 3

[0076] On the basis of the above-mentioned embodiment 1, image detection is performed by using the transmission line similarity defect recognition method based on collaborative hybrid allocation training of the present invention, wherein the auxiliary detection head adopts the Faster R-CNN auxiliary head, and the detection accuracy of loose conductors under complex targets is as high as 94%.

[0077] It can be seen from the above embodiments 1-3 that the image detection performed by the transmission line similarity defect recognition method based on collaborative hybrid allocation training of the present invention still has a high efficiency detection effect in complex scenes, which overcomes the problem in the prior art that manual recognition of similarity defects in complex scenes is prone to human omissions or false detections.

Claims

1. A similarity defect identification method for power transmission lines based on collaborative hybrid allocation training, characterized in that: The following steps are involved: Step 1: Obtain image data of the transmission line inspection equipment and pre-process the image data; Step 2: Extract features from the preprocessed image data to generate a series of feature maps, perform convolution and upsampling operations on each feature map through the feature pyramid to extract multi-scale features; Step 3: Input the feature map generated in step 2 into the encoder to generate context information of the defect features in the image; Step 4: Receive the context information of the encoder in step 3 through the decoder, and process the multi-scale features extracted in step 2 in combination with the multi-scale adapter in the Co-DETR model to generate multiple candidate positive sample coordinates for each Ground Truth. Finally, integrate the multi-scale features and positive sample information through the decoder to output the predicted type and location information of the transmission line similarity defect.

2. The method for identifying similarity defects of power transmission lines based on collaborative hybrid allocation training according to claim 1 is characterized in that: The specific process of step 1 is as follows: Step 1.1, collecting defect data of the transmission line; Step 1.2: In the data preprocessing phase, the image data is labeled.

3. The method for identifying similarity defects of power transmission lines based on collaborative hybrid allocation training according to claim 2 is characterized in that: The defect data in step 1.1 includes wire broken strand image data, wire loose strand image data, insulator damage data, and insulator self-explosion data; The content of the image data annotation in step 1.2 specifically includes the defect location, defect type, and coordinate information related to the defect.

4. The method for identifying similarity defects of power transmission lines based on collaborative hybrid allocation training according to claim 1, characterized in that: The specific process of step 2 is as follows: The image data preprocessed in step 1 is input into the backbone network of the Co DETR model to extract image features and generate a series of feature maps, specifically represented as feature maps {F1, F2, ..., F j }, and then the feature map {F1, F2, ..., F j }After batch normalization BN and ReLU operations, normalization and nonlinear activation processing are performed, and convolution operations are performed to generate feature maps {C1, C2, ..., C n }, feature map {C1, C2, ..., C n }Convolution is performed through Conv2d 1×1, and then each layer of convolution features is upsampled to merge it with the convolution result of the previous layer. The upsampled feature maps are subjected to a Conv2d 3×3 convolution operation for further feature extraction. Finally, the prediction feature layer {P1, P2, ..., P n }, prediction feature layer P n Generate the prediction feature layer P through the maxpool operation n+1 , output the results and complete the extraction of multi-scale features.

5. The method for identifying similarity defects of power transmission lines based on collaborative hybrid allocation training according to claim 1, characterized in that: The specific process of step 4 is as follows: Step 4.1, according to the bounding box position of the ground truth, extract the feature information of the area corresponding to the bounding box from the feature map output by the encoder; Step 4.2, generating a plurality of candidate positive sample coordinates by randomly transforming the feature information extracted in step 4.1, wherein the candidate positive sample coordinates are located in the valid area of ​​the feature map and partially overlap with the ground truth; Step 4.3: Encode the candidate positive sample coordinates obtained in step 4.2 into a query vector through position embedding and input it to the decoder; Step 4.4: Introduce a hybrid collaborative allocation mechanism to allocate positive and negative samples to multiple auxiliary detection heads, each of which is responsible for feature detection at different scales. Step 4.5: For each auxiliary detection head, define the corresponding loss function. The loss includes classification loss and regression loss. The regression loss is used to calculate the difference between the predicted bounding box and the actual defect position, and the classification loss is used to measure the model's prediction accuracy for the defect type. By combining the multi-task loss function and introducing a balance coefficient to weight the loss, the final loss function is: Among them, L cls is the classification loss, L reg is the bounding box regression loss, L aux To assist in detecting head loss, λ1 and λ2 are balance coefficients; Step 4.6: The decoder integrates the multi-scale features with the mixed and coordinated positive and negative sample information, and outputs the predicted type and location information of the transmission line similarity defect.

6. The method for identifying similarity defects of power transmission lines based on collaborative hybrid allocation training according to claim 5, characterized in that: The specific process of step 4.4 is as follows: A combination of one-to-one and one-to-many matching methods is used to assign positive and negative samples to multiple auxiliary detection heads. One-to-one means that each Ground Truth can only match one detection result, that is, the Co DETR model will select a prediction box that matches the Ground Truth from all prediction boxes and regard it as a positive sample, and the other prediction boxes are regarded as negative samples. One-to-many matching means that each Ground Truth is matched with multiple detection results at the same time, that is, multiple positive samples are assigned to one target.

Images