A power transmission line fitting detection method based on self-supervised learning

Abstract

The application discloses a kind of based on self-supervised learning transmission line hardware detection method, select single-stage target detector as basic model architecture, use improved Swin Transformer model as backbone network, introduce light self-supervised learning method, and realize high-precision hardware detection in combination with high-precision detection head;Increase downsampling in the original self-attention calculation of Swin Transformer, reduce network calculation amount, and recover part of the information lost after downsampling by using upsampling branch, realize efficient feature extraction, solve the problem of large calculation amount of original Swin network, and the improved network is called E-Swin (efficient Swin);In order to effectively use the large amount of picture data generated by transmission line inspection, reduce the resources consumed by human labeling, introduce self-supervised learning method, and make light improvement, for unsupervised pre-training of E-Swin;Design the detector with additional branch, get more accurate hardware target positioning frame, further improve the accuracy of transmission line hardware detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image analysis, and in particular to a transmission line hardware detection method based on self-supervised learning. BACKGROUND

[0002] In recent years, with the implementation of the new planning of State Grid, the power grid construction in China is accelerating, and the coverage of the transmission line is gradually expanding. As the most important infrastructure in the power system, whether the transmission line is safe and stable operation has an important influence on the entire power system and people's daily life. Ensuring that the key components (hardware, etc.) in the transmission line are in a normal state is an important link to maintain the stability of the power system. Hardware is a metal accessory made of aluminum or iron, including protective hardware, connecting hardware, connecting hardware, strain clamp and suspension clamp, etc., mainly used for supporting, fixing, connecting bare conductors, conductors and insulators. Such components are exposed to the outdoor environment for years, facing complex environment, prone to displacement, skew, damage, etc., affecting the stability of the conductor and tower connection structure, and once not found in time will cause major circuit damage accidents. To evaluate the operating state of the hardware and realize fault diagnosis, first of all, the transmission line hardware target needs to be accurately positioned and identified. With the development of deep learning and unmanned aerial vehicle inspection technology, the traditional manual inspection method is gradually replaced. At present, the hardware image is mainly obtained by unmanned aerial vehicle aerial photography, and then combined with deep learning technology for automatic analysis, so the research on the transmission line hardware detection model based on deep learning has great significance to ensure the safe operation of the power grid. At present, the main method of hardware detection is to optimize the target detection model that performs well on public data sets and apply it to the power vision field.

[0003] However, using deep learning methods to detect hardware in the transmission line has the following problems:

[0004] 1. At present, most hardware detection models rely on supervised learning, that is, human data labeling is required before model training. However, with the development of unmanned aerial vehicle inspection, more and more transmission line component data is collected, and all human labeling requires a lot of resources. Using self-supervised learning can effectively utilize a large amount of unlabeled data, but since a large amount of data is needed to support the extraction of data feature representation, the main problem of self-supervised learning is that the training speed is slow and the time required is long, which needs to be optimized and improved.

[0005] 2. Most existing researches use target detection models based on convolutional neural networks, while in recent years, Transformer has been widely used in the field of computer vision due to its excellent performance. The main problem of this type of model is that the matrix multiplication calculation is large when self-attention is calculated, which leads to low model operation efficiency, and needs to be improved in terms of calculation.

[0006] 3. In the transmission line hardware samples, the scales of the various categories of targets vary greatly, and the target distribution is different from that of general target detection datasets. The detection accuracy and positioning accuracy are low, and further improvements are needed to the detector to improve accuracy.

[0007] Therefore, in the above context, combining the improved target detection model and self-supervised learning method to further solve the related problems of transmission line hardware detection is in line with the needs of industrial development. Summary of the Invention

[0008] The purpose of the present invention is to provide a method for detecting transmission line hardware based on self-supervised learning, so as to solve the problems of utilizing a large amount of inspection data of transmission lines and the low positioning accuracy of transmission line hardware detection. A new model is designed to learn feature representation using unlabeled data, and an improved high-precision backbone network and a detection head with additional branches are designed to further improve the accuracy of hardware detection.

[0009] To achieve the above object, the present invention provides the following solutions:

[0010] A method for detecting transmission line hardware based on self-supervised learning, comprising the following steps:

[0011] Construct a hardware dataset for self-supervised learning and a hardware dataset for fine-tuning training. The hardware dataset for self-supervised learning contains unlabeled hardware images, while the hardware dataset for fine-tuning training contains labeled hardware images. Perform data augmentation on the data in the hardware dataset.

[0012] The single-stage target detection model is selected as the basic architecture, the improved Swin Transformer is used as the backbone network, and the model is built by combining the feature fusion module and the detection head module;

[0013] A self-supervised learning method is introduced and improved for lightweight unsupervised pre-training of the backbone network. This effectively utilizes the large number of unlabeled hardware images generated by transmission line inspections, which serve as hardware datasets for self-supervised learning. This reduces the resources consumed by manual labeling and improves the model's feature extraction capabilities through data mining.

[0014] A detection head with additional branches is designed to obtain more accurate hardware target positioning frames. After self-supervised training is completed, the model is integrated and fine-tuned using the hardware dataset trained for fine-tuning to achieve high-precision detection.

[0015] The method further includes introducing a downsampling module into the backbone network to reduce the computational complexity of network feature extraction and improve feature extraction efficiency.

[0016] The improved Swin Transformer is used as a backbone network to extract the accessory target features, then the obtained features are passed through a feature fusion module PANet (path aggregation network) to realize feature interaction of different scales and improve the feature representation effect, and finally a basic single-stage accessory detection model is built in combination with a detection head module.

[0017] The improved Swin Transformer is used as a backbone network to extract the accessory target features, then the obtained features are passed through a feature fusion module PANet (path aggregation network) to realize feature interaction of different scales and improve the feature representation effect, and finally a basic single-stage accessory detection model is built in combination with a detection head module.

[0018] The improved Swin Transformer includes the following steps:

[0019] The improved Swin Transformer uses a non-overlapping window to improve the calculation efficiency, and still uses a standard Transformer self-attention calculation method in the window, that is, Query (Q, query mark), Key (K, key mark) and Value (V, weight mark) are used for calculation. In the local window, the key K and the weight V are down-sampled to effectively reduce the calculation amount of matrix multiplication, and the V is up-sampled to reconstruct the information to obtain the improved backbone network E-Swin.

[0020] The self-supervised learning method is introduced for lightweight improvement, which is used for unsupervised pre-training of the backbone network, so as to effectively utilize a large amount of unlabeled accessory image data generated by power line inspection, that is, the accessory data set of self-supervised learning, reduce the resources consumed by human labeling, and improve the feature extraction capability of the model through data mining, and specifically includes the following steps:

[0021] The self-supervised learning method is used to pre-train the improved E-Swin backbone network, and the self-supervised learning method is a lightweight and smooth self-supervised learning method LS-UM (lightweight smooth uniform masking). The LS-UM method improves the efficiency of self-supervised learning by designing a lightweight single-layer decoder and a smooth loss function. The unlabeled data is used for self-supervised training, the corresponding image feature representation can be learned by the backbone network as an encoder, the decoder is removed, the encoder is used as the backbone of the target detection model, and the model is constructed. After that, a small amount of labeled data is used for fine-tuning training, and the migration from self-supervised learning to downstream tasks can be completed.

[0022] The detection head with an additional branch is designed to obtain more accurate accessory target positioning frame. After the self-supervised training is completed, the model is integrated, and the fine-tuning training is performed on the accessory data set to realize high-precision detection, and specifically includes the following steps:

[0023] The design adds an additional branch for predicting the intersection over union of the real frame and the predicted frame in addition to the classification and regression branches, the intersection over union detection head is used to predict the intersection over union between each regression bounding box and the real frame, and is connected in parallel to the last layer of the original branch;

[0024] During the training process, the intersection over union prediction part and the classification and regression prediction part are jointly trained, and during the non-maximum suppression process of inference, the classification score and the predicted intersection over union of each detection frame are multiplied to obtain a confidence score and are sorted according to the confidence score; the detection confidence can thus combine the classification accuracy and the positioning accuracy, thereby improving the positioning accuracy of the bounding box; during the training process, L cls represents the classification loss of all positive and negative samples, where cls represents the class classification (class), and L reg represents the regression loss between the predicted frame and the real frame of all positive samples, where reg represents the regression, the intersection over union prediction branch alone uses a binary cross entropy loss function (BCE Loss) to calculate the loss L iou where iou represents the intersection over union (Intersection over Union), and is represented by the following formula:

[0025]

[0026] In the formula, N pos represents the number of all positive samples, IoU' i represents the predicted intersection over union of each bounding box and the real frame, IoU i represents the actual intersection over union of the frame and the real frame, and is trained using a binary cross entropy loss function; the regression branch is jointly trained using the intersection over union prediction loss and the regression loss, thereby improving the positioning accuracy of the detection; the total training loss L train is as follows, where train represents training:

[0027] L train = L cls + L reg + L iou

[0028] During the inference process, the classification score is multiplied by the predicted intersection over union of the corresponding detection frame to obtain a sorting score for non-maximum suppression; thus, the confidence score sorting can combine the classification score and the positioning accuracy, thereby improving the relevance of classification and regression and making the positioning of the predicted frame more accurate.

[0029] The application discloses the following technical effects: the application provides a power transmission line hardware detection method based on self-supervised learning, which comprises the following steps: a hardware dataset for self-supervised learning and fine-tuning training is constructed, a large number of unannotated inspection images and a small number of annotated hardware images are contained respectively, and data enhancement processing is performed; a single-stage target detection model is selected as a basic architecture, a down-sampling module is introduced into the Swin Transformer, the calculation amount of network feature extraction is reduced, the feature extraction efficiency is improved, and the Swin Transformer is used as an improved backbone network E-Swin, a model is built in combination with a feature fusion module and a detection head module; a self-supervised learning method is introduced and used for unsupervised pre-training of the backbone network, so that a large number of picture data generated by power transmission line inspection can be effectively utilized, resources consumed by human annotation can be reduced, and the feature extraction capability of the model can be improved through data mining; a detection head with an additional branch is designed to obtain a more accurate hardware target positioning frame, after the self-supervised training is completed, the model is integrated, fine-tuning is performed on a small amount of annotated data, and high-precision detection is realized. It can be seen that the self-supervised learning is applied to the hardware detection, and the precision of the power transmission line hardware detection is effectively improved by combining the self-supervised learning and designing the detection head. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0031] Figure 1 is a flowchart of a power transmission line hardware detection method based on self-supervised learning according to an embodiment of the present application;

[0032] Figure 2 is a schematic diagram of an improved E-Swin window attention calculation structure according to an embodiment of the present application;

[0033] Figure 3 is a schematic diagram of a high-precision detection head structure with an additional branch according to an embodiment of the present application;

[0034] Figure 4 is an effect diagram of power transmission line hardware target detection according to an embodiment of the present application.

[0035] Figure 5 is a comparative effect diagram of hardware target positioning according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of the present application.

[0037] The purpose of the present application is to provide a power line fitting detection method based on self-supervised learning, to solve the problems of large amount of power line inspection pictures, large amount of detection model calculation, and inaccurate power line fitting positioning, and to achieve the purpose of improving fitting detection accuracy while meeting industrialization needs.

[0038] The main idea of the present application is to select a single-stage target detector as a basic model architecture, use an improved Swin Transformer model as a backbone network, introduce a lightweight self-supervised learning method, and combine a high-precision detection head to achieve high-precision fitting detection; increase downsampling in the original self-attention calculation of Swin Transformer, reduce network calculation amount, and use an upsampling branch to restore part of the information lost after downsampling, to achieve efficient feature extraction and solve the problem of large amount of calculation of the original Swin network; the improved network is called E-Swin (efficient Swin); in order to effectively utilize the large amount of picture data generated by power line inspection and reduce the resources consumed by manual labeling, a self-supervised learning method is introduced and improved to be lightweight for unsupervised pre-training of E-Swin; a detector with an additional branch is designed to obtain more accurate fitting target positioning frame and further improve the accuracy of power line fitting detection.

[0039] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] As shown in Figure 1 The self-supervised learning power line fitting detection method provided by the present application comprises the following steps:

[0041] A fitting dataset for self-supervised learning and a fitting dataset for fine-tuning training are constructed, the fitting dataset for self-supervised learning comprises un-labeled inspection images, the fitting dataset for fine-tuning training comprises labeled fitting images, and data in the fitting dataset is subjected to data enhancement processing;

[0042] A single-stage target detection model is selected as a basic architecture, an improved Swin Transformer is used as a backbone network, and a model is built by combining a feature fusion module and a detection head module;

[0043] The self-supervised learning method is introduced, light-weight improvement is carried out, is used for unsupervised pre-training of the backbone network, so that the large amount of picture data generated by power line inspection, that is, the hardware data set of self-supervised learning, is effectively utilized, the resources consumed by artificial labeling are reduced, and the feature extraction capability of the model is improved through data mining;

[0044] In addition, in order to reduce the calculation amount of network feature extraction and improve the feature extraction efficiency, a down-sampling module is introduced into the backbone network, based on which the application further provides a power line hardware detection method based on self-supervised learning, which specifically comprises the following steps:

[0045] S1, a hardware data set for self-supervised learning and a hardware data set for fine-tuning training are constructed, the hardware data set for self-supervised learning contains a large number of unlabeled hardware images, the hardware data set for fine-tuning training contains a small number of labeled hardware images, and data enhancement processing is performed on the data in the hardware data set;

[0046] S2, a single-stage target detection model is selected as a basic architecture, an improved Swin Transformer is used as a backbone network, and a model is built by combining a feature fusion module and a detection head module;

[0047] S3, a down-sampling module is introduced into the backbone network, so as to reduce the calculation amount of network feature extraction and improve the feature extraction efficiency;

[0048] S4, a self-supervised learning method is introduced, light-weight improvement is carried out, and is used for unsupervised pre-training of the backbone network, so that the large amount of unlabeled hardware images generated by power line inspection are effectively utilized, the resources consumed by artificial labeling are reduced, and the feature extraction capability of the model is improved through data mining;

[0049] S5, a detection head with an additional branch is designed to obtain more accurate hardware target positioning frame, after the self-supervised training is completed, the model is integrated, the hardware data set for fine-tuning is used for fine-tuning, that is, a small amount of labeled hardware images are used for fine-tuning, and high-precision detection is realized.

[0050] Through self-supervised learning, a large number of data set image samples are needed to support data feature mining, most of the unlabeled images collected by unmanned aerial vehicle inspection need to be artificially cleaned. In addition, the samples containing hardware types need to be further screened and subjected to data enhancement such as cropping for self-supervised pre-training. During the supervised fine-tuning process after the self-supervised training, a small amount of pictures need to be artificially labeled and integrated into a suitable data set format. Therefore, in the step S1, the following steps are specifically included:

[0051] The power transmission line image shot by the unmanned aerial vehicle is preprocessed by manual data cleaning and clipping, and a data set is constructed by selecting target more and clear data samples, including unmarked data for self-supervised learning and labeled data for fine tuning. The unmarked data set contains a large number of unmarked pictures, which are directly used in self-supervised pre-training; the labeled data set contains fewer pictures, and the hardware targets and categories on each picture are manually labeled and integrated according to the MS-COCO data set format. In the supervised fine tuning training, the model performance is adjusted.

[0052] In the method, considering the efficiency and inference speed of model calculation, a single-stage target detector is used to construct the basic model. The single-stage detector mainly includes the backbone network, the feature fusion module and the detection head. Therefore, in the step S2, a single-stage target detection model is selected as the basic architecture, an improved SwinTransformer is used as the backbone network, and the model is built by combining the feature fusion module and the detection head module, specifically including:

[0053] Firstly, in order to simplify the calculation and improve the inference speed of the model, a single-stage target detection model is used as the basic architecture, which greatly improves the inference speed compared with the two-stage model. The improved Swin Transformer is used as the backbone network to extract the hardware target features; then the obtained features are passed through the feature fusion module PANet (path aggregation network) to realize the interaction of features of different scales and improve the effect of feature representation; finally, the basic single-stage hardware detection model is built by combining the detection head module, and the effect of hardware detection is improved on this basis.

[0054] The improved E-Swin window attention calculation structure in the application is shown in Figure 2 .

[0055] After building the basic model, in order to solve the problem of large calculation amount and low efficiency of the Swin backbone network, the Swin backbone network is improved, and the improved backbone network E-Swin is built. Therefore, in the step S3, a down-sampling module is introduced into the backbone network to reduce the calculation amount of network feature extraction and improve the feature extraction efficiency, specifically including:

[0056] Improve the original Swin Transformer backbone network to reduce the amount of self-attention calculation and improve efficiency. Swin Transformer uses non-overlapping windows to improve computational efficiency, but still uses the standard Transformer self-attention calculation method within the window, that is, using Query (Q, query tag), Key (K, key value tag) and Value (V, weight tag) for calculation. For images with more redundant information, the amount of calculation is still slightly larger. The amount of self-attention calculation mainly comes from a large number of matrix multiplication operations. To address this problem, in the local window, by downsampling the key value K and the weight V, the amount of matrix multiplication calculation is effectively reduced. At the same time, in order to reduce the amount of calculation without losing effective information, V is upsampled to reconstruct the information. First, similar to the original multi-head self-attention calculation, a set of linear projection layers are used to transform the input It is converted into query Q, while K and V are obtained in different ways. First, X is reshaped into a two-dimensional feature map, and then downsampled using an s*s average pooling layer to reduce the subsequent calculation amount and make the model focus on the effective information of the contour features in the feature map. The feature size after downsampling is reduced, and then converted through the linear projection layer to obtain K and V. Then, conventional matrix calculations are performed, and Q and K of different image blocks interact and are multiplied with the corresponding V. After the improvement, the computational complexity of the interaction between Q, K, and V in the local window is In the original calculation without downsampling, the computational complexity is When s > 1, downsampling can effectively reduce computational complexity. While downsampling significantly reduces computational cost, it also inevitably results in the loss of some feature information. To achieve efficient computation while preserving valid information, the V value is upsampled to its original size before the final linear layer in the self-attention calculation, and then added to the calculated result. This calculation utilizes a pixel shuffle, first increasing the number of channels and then evenly distributing the pixels to achieve the upsampling effect. The upsampling branch efficiently reconstructs information with minimal increase in computational complexity. This approach achieves improvements and optimization, building the improved backbone network E-Swin.

[0057] The E-Swin backbone network is pre-trained using self-supervised learning to process unlabeled data and mine data feature representations. In step S4, a self-supervised learning method is introduced to perform lightweight improvements for unsupervised pre-training of the backbone network, effectively utilizing the large number of unlabeled hardware images generated by transmission line inspections, reducing the resources consumed by manual labeling, and improving the model's feature extraction capabilities through data mining. Specifically, the following steps are performed:

[0058] In order to make full use of the large amount of unannotated hardware image data of power transmission line inspection, data features are mined from large-scale unannotated data, and feature representations that can be transferred to downstream tasks are learned. The improved E-Swin backbone network is pre-trained in a self-supervised learning manner. The previous MAE (denoising autoencoder) method designs an image reconstruction auxiliary task, randomly masks 75% of the image, and then inputs it into the encoder and decoder, and makes the output reconstruct the original image, realizing the self-supervised training of the ordinary ViT (visual Transformer) backbone network. In order to improve the efficiency of self-supervised learning, optimization is carried out from the decoder and the loss function, and a lightweight and smooth self-supervised learning method LS-UM (lightweight smooth uniform masking) is proposed to pretrain the improved Swin backbone network. First, uniform sampling is performed on the image, and the sampling rate is 25%, that is, 1 grid is sampled for every 4 grid unit of 2*2, and 75% of the image blocks will be masked and will not be input into the encoder, ensuring that the computational overhead of the encoder is small. At the same time, uniform sampling ensures that an equal amount of image blocks can be allocated to each local window of Swin, so that the shift window can process discrete image block sets. However, compared with random sampling, uniform sampling will reduce the difficulty of the auxiliary task, resulting in a decrease in learning quality. Therefore, on the basis of uniform sampling, secondary masking is performed, and a small part of the image blocks obtained by the first sampling are masked, and a shared mask token is used to represent them. The mask token generated by the second sampling will still be input into the encoder, so the second sampling will not change the number of image blocks input into the encoder, ensuring that the attention calculation of the local window will not be affected. The difficulty of the image reconstruction auxiliary task is restored by the second sampling, so that the network focuses on learning high-quality representations of images. After two samplings, 25% of the original image is input into the encoder. The encoder uses the E-Swin backbone network to extract the features of the image. After passing through the encoder, the feature representation of the image is obtained, and the size is restored by upsampling. Then, the 75% of the image blocks masked by the first sampling are represented by a learnable vector, and are input into the decoder together with the feature representation obtained by the encoder. The decoder uses a lightweight single-layer Transformer decoder, plus a linear prediction layer, to make the output reconstruct the original image. The image reconstruction method is used to train the feature extraction capability of the encoder. The loss of self-supervised training is calculated using the predicted pixel value and the original pixel value, and the smooth loss function is as follows:

[0059]

[0060] In the formula, x represents the difference between the predicted value and the true value. The Smooth L1 smooth loss combines the mean absolute error and the mean square error loss. The mean absolute error loss is not smooth when the difference between the predicted value and the true value is small in the later training period, and the loss function fluctuates around a stable value, making it difficult for the model to converge. The mean square error has a large gradient when the difference between the predicted value and the true value is large in the early training period, and the gradient of some outliers explodes, making the training unstable. The smooth loss function divides the interval, and the combination of the two loss functions helps to stabilize the training. The LS-UM method improves the efficiency of self-supervised learning by designing a lightweight single-layer decoder and a smooth loss function. Using unlabeled data for self-supervised training, the backbone network as the encoder can learn the corresponding image feature representation. Remove the decoder, use the encoder as the backbone of the target detection model, and then use a small amount of labeled data to fine-tune the training to complete the transfer of self-supervised learning to the downstream task.

[0061] The structure diagram of the high-precision detection head with additional branches in the application is shown in Figure 3

[0062] After the self-supervised pre-training is completed, the backbone network is embedded into the single-stage target detection model that has been constructed, and a detection head module is designed to solve the problem of inaccurate positioning of the fittings. In step S5, a detection head with additional branches is designed to obtain more accurate fitting target positioning frames. After the self-supervised training is completed, the model is integrated and fine-tuned with the fitting data set for fine-tuning, that is, a small amount of labeled fitting image data is used for fine-tuning to achieve high-precision detection. Specifically, it includes:

[0063] ​After the feature fusion module, the obtained information is input into the last predictor for classification and regression prediction to obtain the prediction results of the target category and the bounding box in the picture. However, for transmission line fittings of different scales, the quality of the bounding box is poor. Since the classification branch and the regression branch in the predictor lack correlation, the bounding box with good quality may be suppressed. In the non-maximum suppression process, only one set of classification results and bounding boxes will be output for a prediction target, all predictions for the target will be arranged in descending order of classification scores, and the prediction box with the highest classification score will suppress other prediction boxes with a higher overlap degree than a certain threshold. However, this defaults to using the prediction box with the highest classification score as the bounding box, without considering the samples with slightly lower classification scores but closer prediction boxes to the real box. An additional branch is designed outside the classification and regression branch to predict the intersection over union of the real box and the prediction box. The intersection over union detection head is used to predict the intersection over union between each regression bounding box and the real box, and is connected in parallel to the last layer of the original branch. During the training process, the intersection over union prediction part and the classification and regression prediction part are jointly trained. During the inference non-maximum suppression process, the classification score and the predicted intersection over union of each detection box are multiplied to obtain a confidence score, and the detection confidence is sorted according to the confidence score. The detection confidence can combine the classification accuracy and the positioning accuracy, thereby improving the positioning accuracy of the bounding box. The training process uses L cls to represent the classification loss of all positive and negative samples, where cls represents the class classification (class), and L reg to represent the regression loss between the prediction box and the real box of all positive samples, where reg represents the regression, and the intersection over union prediction branch alone uses a binary cross entropy loss function (BCE Loss) to calculate the loss L iou between the predicted intersection over union and the real intersection over union, where iou represents the intersection over union, and is represented by the following formula:

[0064]

[0065] where N pos represents the number of all positive samples, IoU′ i represents the predicted intersection over union of each bounding box and the real box, IoU i represents the actual intersection over union of the box and the real box, and is trained using a binary cross entropy loss function. The regression branch is jointly trained using the intersection over union prediction loss and the regression loss to improve the positioning accuracy of the detection. The total training loss L train is shown in the following formula, where train represents training:

[0066] L train = L sls+L reg +L iou

[0067] In the reasoning process, the classification score is multiplied by the predicted intersection over union of the corresponding detection frame to obtain a ranking score for non-maximum suppression. Thus, the confidence ranking can combine the classification score and the positioning accuracy, improve the correlation of classification and regression, and make the positioning of the prediction frame more accurate.

[0068] The present application integrates the methods in steps S3, S4 and S5 into the base network constructed in step S2, and finally forms a complete transmission line fitting detection method based on self-supervised learning. The effect diagram of the transmission line fitting target detection of the method of the present application is shown in Figure 4 .

[0069] The fitting target positioning comparison effect of the method of the present application is shown in Figure 5 . On the basis of the single-stage target detection model, the improved Swin Transformer is used as the backbone network, the original network is improved in lightness by using the down-sampling module, the improved backbone network E-Swin is obtained, and the problem of large calculation amount and slow training speed of the original Swin network is solved; the self-supervised learning method is introduced to pre-train E-Swin, so that it learns the feature representation from the unlabeled inspection fitting image data, and provides a reference for using a large amount of unlabeled inspection fitting image data collected in the transmission line inspection; the detection head with an additional branch is designed and added, and the intersection over union prediction method is used to improve the fitting detection positioning accuracy. It can be seen that the method of the present application solves the problem of utilization of inspection data in the transmission line, ensures high detection accuracy, and meets the actual industrial demand.

[0070] The principles and implementation modes of the present application are described by applying specific examples in this paper, and the above description of the examples is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed. In view of the above, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A power transmission line fitting detection method based on self-supervised learning, characterized in that, The method comprises the following steps: constructing a fitting data set for self-supervised learning and a fitting data set for fine-tuning training, the fitting data set for self-supervised learning containing unlabeled fitting images, and the fitting data set for fine-tuning training containing labeled fitting images, and performing data enhancement processing on data in the fitting data set; selecting a single-stage target detection model as a basic architecture, adopting an improved Swin Transformer as a backbone network, and combining a feature fusion module and a detection head module to build a model; introducing a self-supervised learning method, performing lightweight improvement, and using the method for unsupervised pre-training of the backbone network, so as to effectively utilize a large number of unlabeled fitting images, i.e., the fitting data set for self-supervised learning, generated by power line inspection, reduce resources consumed by human labeling, and improve feature extraction capability of the model through data mining; designing a detection head with an additional branch to obtain a more accurate fitting target positioning frame, integrating the model after self-supervised training is completed, and fine-tuning the model with the fitting data set for fine-tuning training to realize high-precision detection. The improved Swin Transformer specifically comprises: The improved Swin Transformer adopts a non-overlapping window to improve computational efficiency, still adopts a standard Transformer self-attention calculation method in the window, i.e., uses Query (Q, query mark), Key (K, key value mark), and Value (V, weight value mark) for calculation, in a local window, effectively reduces computational amount of matrix multiplication by downsampling Key (K) and Value (V), and reconstructs information by upsampling V to obtain an improved backbone network E-Swin.

2. The power transmission line fitting detection method based on self-supervised learning according to claim 1, characterized in that, Further comprising introducing a downsampling module in the backbone network to reduce computational amount of network feature extraction and improve feature extraction efficiency. 3.The power transmission line fitting detection method based on self-supervised learning of claim 1, wherein, The improved Swin Transformer is adopted as the backbone network, combined with the feature fusion module and the detection head module to build the model, specifically comprising: The improved Swin Transformer is adopted as the backbone network to extract fitting target features, then the obtained features are transmitted through the feature fusion module PANet (path aggregation network) to realize feature interaction of different scales and improve feature representation effect, and finally the detection head module is combined to build a basic single-stage fitting detection model.

4. The power transmission line fitting detection method based on self-supervised learning according to claim 3, characterized in that, The self-supervised learning method is introduced, the lightweight improvement is performed, and the method is used for unsupervised pre-training of the backbone network, so as to effectively utilize a large number of unlabeled fitting images, i.e., the fitting data set for self-supervised learning, generated by power line inspection, reduce resources consumed by human labeling, and improve feature extraction capability of the model through data mining, specifically comprising: The improved E-Swin backbone network is pre-trained by using a self-supervised learning method, which is a light and smooth self-supervised learning method LS-UM. The LS-UM method improves the efficiency of self-supervised learning by designing a light single-layer decoder and a smooth loss function. The unlabelled data is used for self-supervised training, and the backbone network serving as an encoder can learn the corresponding image feature representation. The decoder is removed, the encoder is used as the backbone of the target detection model, and after the model is constructed, a small amount of labelled data is used for fine-tuning training, so that the transfer of self-supervised learning to the downstream task can be completed.

5. The power transmission line fitting detection method based on self-supervised learning according to claim 4, characterized in that, The detection head with an additional branch is designed to obtain a more accurate fitting target positioning box. After the self-supervised training is completed, the model is integrated, and the fitting data set is fine-tuned to realize high-precision detection. Specifically, the detection head with an additional branch is designed to predict the intersection over union of the real box and the predicted box in addition to the classification and regression branch. The intersection over union detection head is used to predict the intersection over union between each regression bounding box and the real box, and is connected in parallel to the last layer of the original branch. In the inference process, the classification score is multiplied by the predicted intersection over union of the corresponding detection box to obtain the ranking score for non-maximum suppression. In this way, the confidence ranking can combine the classification score and the positioning accuracy, improve the correlation of classification and regression, and make the positioning of the predicted box more accurate. In the training process, the intersection over union prediction part and the classification and regression prediction part are jointly trained, in the non-maximum suppression process of inference, the classification score and the predicted intersection over union of each detection box are multiplied to obtain a confidence score and are sorted according to the confidence score, and the detection confidence can thus combine the classification accuracy and the positioning accuracy, so that the positioning accuracy of the boundary box is improved, and the L cls classification loss of all positive and negative samples is represented, wherein cls represents the category classification, the L reg regression loss between the predicted box and the real box of all positive samples is represented, wherein reg represents the box regression, the intersection over union prediction branch alone adopts a binary cross-entropy loss function, which is used to calculate the loss L iou of the predicted intersection over union and the real intersection over union, wherein iou represents the intersection over union, and is represented by the following formula: In the formula, N pos represents the number of all positive samples, IoU i ′ represents the predicted intersection over union of each bounding box and the real box, IoU i represents the actual intersection over union of the box and the real box, is trained by using a binary cross-entropy loss function, and the regression branch is trained by using the intersection over union prediction loss and the regression loss together, so as to improve the positioning accuracy of detection, and the total training loss L train As shown in the following formula, wherein train refers to training: L train = L cls + L reg + L iou ​

Images