A weak supervision detection method for missing bolts in transmission lines

Abstract

The present invention discloses a weakly supervised detection method for missing bolts in transmission lines. This method uses an image-level labeled training set to detect missing bolts in transmission lines. For difficult samples in the bolt dataset, an adaptive loss value adjustment factor is introduced into the cross-entropy loss function to form an adaptive weighted cross-entropy loss function. The model dynamically adjusts the loss value of easy and difficult samples based on their classification scores, increasing the proportion of difficult samples in the loss value and enhancing the model's learning of difficult samples, thereby improving the detection accuracy of missing bolts. The weakly supervised detection method for missing bolts in transmission lines provided by the present invention addresses the problem of difficult samples in the dataset and achieves better detection results while only providing image-level annotations.

Description

Technical Field

[0001] The present invention relates to the technical field of power transmission and transformation equipment detection, and in particular to a weak supervision detection method for missing pin bolts in a transmission line. Background Art

[0002] Transmission lines are a critical part of my country's energy infrastructure. Bolts are found in large numbers throughout these lines, and are susceptible to pin loss due to wind and rain. Therefore, timely detection of missing bolts is crucial to their safe operation.

[0003] Currently, due to the generalizability of abstract image features extracted by deep neural networks, the application of fully supervised object detection models based on deep learning to transmission line defect detection has become a hot topic of research. Fully supervised object detection models require training datasets with object-level annotations. However, images of transmission line bolts are characterized by small, densely packed objects and unclear visual features. This makes object-level annotation time-consuming and labor-intensive, and the cost of annotation can be substantial when the dataset is very large. Summary of the Invention

[0004] The purpose of the present invention is to provide a weakly supervised detection method for missing bolts in transmission lines, which solves the problem of difficult samples in the data set and achieves better detection effect under the premise of only providing image-level annotation.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A weak supervision detection method for missing bolts in a transmission line comprises the following steps:

[0007] S1) obtaining a transmission line bolt image, generating multiple candidate frames on the transmission line bolt image using a candidate frame generation algorithm, and simultaneously inputting the transmission line bolt image into a deep convolutional feature extraction network for feature extraction to generate a feature map of the entire image;

[0008] S2) inputting the feature maps of the plurality of candidate boxes and the entire image into a region of interest pooling layer, outputting a feature tensor of each candidate box, flattening the feature tensor, and then passing it through a fully connected layer to obtain a feature vector of each candidate box;

[0009] S3) feeding the feature vector into a multi-instance learning network, the multi-instance learning network including a classification stream and a detection stream, applying a normalized index to the candidate box using the classification stream, outputting the probability of each category in each region, and applying a normalized index function to multiple categories using the detection stream, outputting the probability of each category in all regions, then merging the output results of the classification stream and the output results of the detection stream using an inner product method, summing the regions, and converting them into image-level category prediction scores, training the multi-instance learning network using an adaptive weighted cross entropy loss function to obtain preliminary detection results; the adaptive weighted cross entropy loss function L SAW The calculation formula is:

[0010] L SAW =-α(1-p t ) γ log(p t )

[0011] Where: α represents the positive and negative sample balance coefficient, ranging from 0 to 1;

[0012] p t Represents the prediction score of a certain category, with a value range of 0 to 1, and the calculation formula is y represents the true category of the labeled data, p represents the output after the activation function, and its value range is 0 to 1;

[0013] γ represents the adaptive loss value adjustment factor, and the calculation formula is

[0014] S4) inputting the feature vector in step S2) into a multi-level instance classification and purification network, wherein the multi-level instance classification and purification network includes K instance classifiers with the same structure, using the output of the multi-instance learning network to supervise the training of the first-level instance classifier, and using the output of the first-level instance classifier as a pseudo label to train the next-level instance classifier, after K layers of classification refinement, assigning a more appropriate label to the candidate box;

[0015] S5) Using the non-maximum suppression algorithm to remove redundant detection frames, the final detection result is obtained.

[0016] Optionally, in step S2), the feature tensors of the multiple candidate boxes have the same size.

[0017] Optionally, the category prediction score in step S3) is set with a threshold β;

[0018] Among them, when a certain category prediction score p tWhen the value is greater than the threshold β, the sample is easy to classify. The loss value of the easy-to-classify sample is greatly reduced under the action of the adaptive loss value adjustment factor γ. The easier the sample is to classify, the larger the adaptive loss value adjustment factor γ is, and the greater the degree of loss reduction is.

[0019] When the prediction score of a certain category is less than the threshold β, the sample is a difficult sample, and the loss value of the difficult sample is slightly reduced under the action of the adaptive loss value adjustment factor. t The smaller the adaptive loss value adjustment factor γ is, the better the loss value for difficult samples is maintained.

[0020] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: the weakly supervised detection method for missing bolts in transmission lines provided by the present invention achieves better detection effect under the premise of only providing image-level annotation; in response to the problem of difficult samples in the bolt data set, an adaptive loss value adjustment factor is introduced into the cross-entropy loss function to form an adaptive weighted cross-entropy loss function. The model dynamically adjusts the loss value of difficult and easy samples based on the classification score of difficult and easy samples, increases the proportion of difficult samples in the loss value, enhances the model's learning degree of difficult samples, and thus improves the detection accuracy of missing bolts. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 Flowchart of a weak supervision detection method for missing bolts in a transmission line according to an embodiment of the present invention;

[0023] Figure 2 This is a flowchart for implementing the adaptive weighted cross entropy loss function in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0025] The purpose of the present invention is to provide a weakly supervised detection method for missing bolts in transmission lines, which solves the problem of difficult samples in the data set and achieves better detection effect under the premise of only providing image-level annotation.

[0026] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] like Figure 1 As shown, the weak supervision detection method for missing bolts in a transmission line provided by an embodiment of the present invention includes the following steps:

[0028] S1) obtaining an image of a transmission line bolt, generating approximately 2,000 candidate boxes on the image using a selective search algorithm, and inputting the image of the transmission line bolt into a VGG16 deep feature extraction network pre-trained on the ImageNet dataset to obtain a deep convolutional feature map with 512 channels;

[0029] S2) Input the candidate box and feature map into the region of interest pooling layer to obtain the feature map corresponding to the candidate box, and scale the feature map of the candidate box of different sizes to a convolutional feature map of a fixed size of 512×7×7. Then, a flattening operation is performed to convert the multi-dimensional feature tensor into a one-dimensional feature vector of dimension 512×7×7, and then pass through two layers of fully connected layers to output a feature vector of dimension 4096;

[0030] S3) The feature vector is fed into the multi-instance learning network, which is divided into two branches: classification flow and detection flow. The classification flow is used to apply the normalized index to the candidate box, and the probability of each category in each area is output. The detection flow is used to apply the normalized index function to multiple categories, and the probability of each category in all areas is output. The results of the two branches are then merged by the inner product method, and the sum of the regions is converted into image-level category prediction scores. The multi-instance learning network is trained using the adaptive weighted cross entropy loss function to obtain preliminary detection results; the adaptive weighted cross entropy loss function L SAW The calculation formula is:

[0031] L SAW =-α(1-p t ) γ log(p t )

[0032] Where: α represents the positive and negative sample balance coefficient, ranging from 0 to 1. By controlling the value of α, the weight of positive and negative samples on the total loss value is controlled;

[0033] p t Represents the prediction score of a certain category, with a value range of 0 to 1, and the calculation formula is y represents the true category of the labeled data, p represents the output after the activation function, and its value range is 0 to 1;

[0034] γ represents the adaptive loss value adjustment factor, and the calculation formula is

[0035] like Figure 2 As shown, when a certain category prediction score p t When the value is greater than the threshold β, the sample is an easy-to-classify sample. The loss value of the easy-to-classify sample is greatly reduced under the action of the adaptive loss value adjustment factor γ. When the easier the sample is to classify, the larger the adaptive loss value adjustment factor γ is, and the greater the degree of loss reduction is. When the prediction score of a certain category is less than the threshold β, the sample is a difficult sample. The loss value of the difficult sample is slightly reduced under the action of the adaptive loss value adjustment factor. When the sample prediction score p is greater than the threshold β, the sample is a difficult sample. The loss value of the difficult sample is slightly reduced under the action of the adaptive loss value adjustment factor. t The smaller the adaptive loss adjustment factor γ, the better the loss value for difficult samples is maintained. Through the above adaptive adjustment of the loss value of difficult and easy samples, the contribution of difficult samples to the total loss value is increased, the model's learning ability for difficult samples is enhanced, and the prediction accuracy of the multi-instance learning network for missing pin bolts is improved, which in turn plays a positive role in the training results of the multi-level instance classification and purification network in the next step.

[0036] S4) Inputting the feature vector from step S2) into a multi-level instance classification refinement network. The multi-level instance classification refinement network includes multiple instance classifiers with the same structure. The logarithmic loss function is used to guide the training of each instance classifier. These streams are one after another, that is, the output of the previous instance classifier is used as a pseudo label to train the next instance classifier, and the first instance classifier uses the output of the multi-instance learning network as a pseudo label. After K layers of classification refinement, the classifier can select a candidate box that is more consistent with the target as a positive instance.

[0037] S5) Using the non-maximum suppression algorithm to remove redundant detection frames, the final detection result is obtained.

[0038] The trained model was tested on the test set and training set. The test results are shown in Table 1:

[0039] Table 1 Test results

[0040]

[0041] As can be seen from Table 1, the improved model has greatly improved the detection accuracy of the original model on the test set, and the detection accuracy of missing pin bolts has increased by 12.0%. The improved model has also significantly improved the positioning accuracy on the training set, and the positioning accuracy of missing pin bolts has increased by 18.6%. Therefore, whether from the comparison results of detection accuracy or positioning accuracy, the effectiveness of the present invention in detecting missing pin bolts can be verified. The improved model is significantly better than the original model in detecting missing pin bolts on the test set. The detection result frame obtained by the original model is very rough and can only frame the approximate position of the bolt target. Compared with the original model, the detection frame obtained by the improved model has a good fit and can accurately frame the bolt target. Therefore, it can be seen that the present invention is superior in the detection performance of missing pin bolts.

[0042] The weakly supervised detection method for missing bolts in transmission lines provided by the present invention achieves better detection results while only providing image-level annotations. To address the problem of difficult samples in bolt datasets, an adaptive loss value adjustment factor is introduced into the cross-entropy loss function to form an adaptive weighted cross-entropy loss function. The model dynamically adjusts the loss value of difficult and easy samples based on the classification scores of difficult and easy samples, increases the proportion of difficult samples in the loss value, enhances the model's learning of difficult samples, and thus improves the detection accuracy of missing bolts.

[0043] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A weak supervision detection method for missing bolts in power transmission lines, characterized in that: The following steps are involved: S1) obtaining a transmission line bolt image, generating multiple candidate frames on the transmission line bolt image using a candidate frame generation algorithm, and simultaneously inputting the transmission line bolt image into a deep convolutional feature extraction network for feature extraction to generate a feature map of the entire image; S2) inputting the feature maps of the plurality of candidate boxes and the entire image into a region of interest pooling layer, outputting a feature tensor of each candidate box, flattening the feature tensor, and then passing it through a fully connected layer to obtain a feature vector of each candidate box; S3) feeding the feature vector into a multi-instance learning network, the multi-instance learning network including a classification stream and a detection stream, applying a normalized index to the candidate box using the classification stream, outputting the probability of each category in each region, and applying a normalized index function to multiple categories using the detection stream, outputting the probability of each category in all regions, then merging the output results of the classification stream and the output results of the detection stream using an inner product method, summing the regions, and converting them into image-level category prediction scores, training the multi-instance learning network using an adaptive weighted cross entropy loss function to obtain preliminary detection results; the adaptive weighted cross entropy loss function L SAW The calculation formula is: L SAW =-α(1-p t ) γ log(p t ) Where: α represents the positive and negative sample balance coefficient, ranging from 0 to 1; p t Represents the prediction score of a certain category, with a value range of 0 to 1, and the calculation formula is y represents the true category of the labeled data, p represents the output after the activation function, and its value range is 0 to 1; γ represents the adaptive loss value adjustment factor, and the calculation formula is S4) inputting the feature vector in step S2) into a multi-level instance classification and purification network, wherein the multi-level instance classification and purification network includes K instance classifiers with the same structure, using the output of the multi-instance learning network to supervise the training of the first-level instance classifier, and using the output of the first-level instance classifier as a pseudo label to train the next-level instance classifier, after K layers of classification refinement, assigning a more appropriate label to the candidate box; S5) using a non-maximum suppression algorithm to remove redundant detection frames to obtain the final detection result; The category prediction score in step S3) is set with a threshold β; Among them, when a certain category prediction score p t When the value is greater than the threshold β, the sample is easy to classify. The loss value of the easy-to-classify sample is greatly reduced under the action of the adaptive loss value adjustment factor γ. The easier the sample is to classify, the larger the adaptive loss value adjustment factor γ is, and the greater the degree of loss reduction is. When the prediction score of a certain category is less than the threshold β, the sample is a difficult sample, and the loss value of the difficult sample is slightly reduced under the action of the adaptive loss value adjustment factor. t The smaller the adaptive loss value adjustment factor γ is, the better the loss value for difficult samples is maintained.

2. The weak supervision detection method for missing bolts in power transmission lines according to claim 1 is characterized in that: In step S2), the feature tensors of the plurality of candidate boxes have the same size.

Images