Catenary Insulator Breakage Detection Method and Device Based on Few-Shot Transfer Learning

Abstract

The present invention discloses a method and device for detecting damage of catenary insulators based on few-shot transfer learning. The method includes inputting an image of a catenary insulator to be detected into a two-stage contrastive proposal encoding network; the encoder performs feature encoding and feature extraction on the insulator image to extract the feature representation of the image; the adaptive feature fusion module obtains the optimal fusion weight through adaptive weight learning; the hierarchical feature extraction module receives the output of the adaptive feature fusion module and extracts the local position information of small objects, the global information of large objects in the deep network, and the intermediate semantic information between local and global in the intermediate network layer in the shallow network of the encoder; the inter-class and intra-class contrastive proposal encoding module receives the output of the hierarchical feature extraction module, calculates the semantic similarity between different categories, and realizes the damage detection of catenary insulators in complex environments and high-speed motion scenarios. The present invention improves the accuracy and efficiency of catenary insulator damage detection.

Description

Technical Field

[0001] The present invention relates to the technical field of catenary insulator breakage detection, and particularly to a catenary insulator breakage detection method and device based on few-shot transfer learning. Background Art

[0002] Catenary insulators play an important role in insulating and supporting wires in the electrified railway system, ensuring the stability and safety of power supply. Any damage to the insulators in the catenary, such as cracks or breaks, poses serious safety hazards, such as electrical faults or wire detachment, which may not only interrupt railway traffic but also cause safety accidents. Currently, the detection of catenary insulators mostly adopts manual inspection methods, and staff need to go to the site for visual inspection. This method is not only inefficient, relying on the experience and condition of inspectors, but also difficult to operate, with a long cycle and high maintenance costs.

[0003] In recent years, insulator breakage detection methods based on deep learning have provided a technical innovation solution to this problem. By using convolutional neural networks (CNNs) or other forms of machine vision algorithms, the breakage conditions of insulators in the catenary can be automatically identified and classified. Although these technologies perform excellently on large-scale datasets, they still face two major challenges at present: First, when the training samples are few, the target localization accuracy of damaged insulators is usually low; second, the subtle differences within classes lead to poor fine-classification performance of damaged insulators, making it difficult to distinguish subtle damages of insulators. Summary of the Invention

[0004] In view of the above technical problems, the present invention designs a catenary damaged insulator detection method based on improved few-shot transfer learning. This method can train the model with a small number of samples, and adopts an adaptive feature fusion and hierarchical feature extraction method to extract and enhance the features of catenary damaged insulators, improving the localization accuracy of defect targets; at the same time, using an inter-class and intra-class contrast proposal coding module to improve the inter-class and intra-class division accuracy of insulators and damaged insulators. The application of this technology makes the detection of various insulator damages in the catenary more accurate and efficient in complex environments and high-speed motion scenarios, greatly improving the safety and operation efficiency of the railway system.

[0005] The present invention discloses a catenary insulator breakage detection method based on few-shot transfer learning, which includes:

[0006] Inputting the image of the catenary insulator to be detected into a two-stage contrast proposal coding network; the two-stage contrast proposal coding network includes an encoder, an adaptive feature fusion module, a hierarchical feature extraction module, and an inter-class and intra-class contrast proposal coding module;

[0007] The encoder performs feature encoding and feature extraction on the input catenary insulator image to be detected, and extracts the feature representation of the image.

[0008] The adaptive feature fusion module receives the output of the encoder. By adding learnable parameter weights, the feature fusion weights can be adaptively learned during the backpropagation stage to obtain the optimal fusion weights. The hierarchical feature extraction module receives the output of the adaptive feature fusion module and extracts the local position information of small objects in the shallow network of the encoder, the global information of large objects in the deep network, and the intermediate semantic information between local and global in the intermediate network layer.

[0009] The inter-class and intra-class contrast proposal encoding module receives the output of the hierarchical feature extraction module, calculates the semantic similarity between different classes, and realizes the detection of damaged insulators in the catenary insulator image.

[0010] Furthermore, the inter-class and intra-class contrast proposal encoding module includes an inter-class contrast proposal encoding module and an intra-class contrast proposal encoding module.

[0011] The inter-class contrast proposal encoding module calculates the semantic similarity between the classification boxes of large-category insulators CLASS JYZ and the classification boxes of other large-category devices CLASS other to guide the two-level contrast proposal encoding network to perform large-category-level semantic learning on the insulator category.

[0012] The intra-class contrast proposal encoding module calculates the semantic similarity between the classification boxes of damaged insulators CLASS JYZ-ps and the classification boxes of other fine-grained categories to guide the two-level contrast proposal encoding network to perform small-category-level semantic learning on the damaged insulator category; other fine-grained categories include normal insulators CLASS JYZ-zc , dirty insulators CLASS JYZ-zw .

[0013] Furthermore, the inter-class contrast proposal encoding module calculates the semantic similarity between the classification boxes of large-category insulators CLASS JYZ and the classification boxes of other large-category devices CLASS other , including:

[0014] Calculating the cosine similarity between the classification boxes of large-category insulators CLASS JYZ and the classification boxes of other large-category devices CLASS other ;

[0015] The intra-class contrast proposal encoding module calculates the damaged insulatorsCLASS JYZ-ps The semantic similarity between the classification box of

[0016] broken insulators CLASS JYZ-ps and the classification boxes of other fine-grained categories, including:

[0017] Furthermore, the training process of the two-stage contrastive proposal encoding network is as follows:

[0018] Construct a base class dataset and a new class dataset; build a two-stage contrastive proposal encoding network; construct the loss function of the two-stage contrastive proposal encoding network; the base class dataset is a dataset of rigid and flexible catenary insulators including a large number of labeled samples; the new class dataset is a dataset of broken insulators with a small number of labels;

[0019] Divide the base class dataset into a training set, a validation set and a test set, train the two-stage contrastive proposal encoding network on the base class dataset, and use the optimal weights of the two-stage contrastive proposal encoding network on the test set as the optimal weights of the two-stage contrastive proposal encoding network; the validation set and the test set are used to verify and test the trained two-stage contrastive proposal encoding;

[0020] Initialize and fix the weights of the two-stage contrastive proposal encoding network using the optimal weights. After fixing, the weights of the network layer do not participate in the backpropagation gradient calculation;

[0021] According to the number of the new class dataset, perform balanced sampling on the base class dataset, and use the sampled base class dataset to perform supervised fine-tuning on the two-stage contrastive proposal encoding network. Use the new class dataset to test the trained two-stage contrastive proposal encoding network, obtain the optimal weights of the two-stage contrastive proposal encoding network, and use this optimal weight as the optimal weight of the trained two-stage contrastive proposal encoding network.

[0022] Furthermore, the base class dataset is a dataset of rigid and flexible catenary insulators including a large number of labeled samples; the new class dataset is a dataset of broken insulators with a small number of labels.

[0023] Furthermore, the loss function defined in the backpropagation of constructing the two-stage contrastive proposal encoding network is:

[0024]

[0025] where is the loss function, is the binary cross-entropy loss of the insulator feed-forward proposal box, is the cross-entropy loss of the insulator classification box, is the inter-class contrastive proposal encoding loss of the insulator, For the coding loss of comparison within the insulator category, and are hyperparameters.

[0026] Furthermore, the coding loss of comparison between insulator categories is defined as:

[0027]

[0028]

[0029] Among them, is the coding loss of comparison between insulator categories, is the number of prediction boxes with the predicted category of , is a hyperparameter, , is the norm value of the i-th ground truth bounding box, is the cosine similarity between the i-th prediction box and the -th prediction box on the projected hypersphere, is the -th prediction box, is the -th prediction box, is the -th prediction box, is the intersection over union score between the i-th prediction box and the label box, N is the number of prediction boxes, k is a temporary variable, is the -th predicted category, is the -th predicted category.

[0030] Furthermore, in the decoding stage, the adaptive feature fusion module determines the optimal weights for the fusion of features with different network depths:

[0031] is obtained by upsampling by a factor of two, , and are defined as:

[0032]

[0033] Among them, , , , are learnable parameters during the backpropagation process; , , are the input features of the adaptive feature fusion module; , , are the output features of the adaptive feature fusion module.

[0034] Furthermore, the processing flow of the hierarchical feature extraction module is as follows:

[0035] The hierarchical feature extraction module receives the multi-scale outputs of the adaptive feature fusion module. The multi-scale outputs include shallow features , intermediate features and deep features , all with dimensions of C×H×W;

[0036] The hierarchical feature extraction module determines the shallow feature mask , the intermediate feature mask , and the deep feature mask for feature filtering according to the hyperparameters . Among them, is a class mask composed of C 1×H×W, , , are respectively the mask sets that meet the conditions in the corresponding ;

[0037] According to the feature masks , , , the input features are processed. Among them, the shallow features are processed by the shallow feature mask to obtain the local position information of small objects in the shallow network ; the deep features are processed by the deep feature mask to obtain the global information of large objects in the deep network ; similarly, the intermediate layer network is used to extract the intermediate information between small objects and large objects .

[0038] Furthermore, the local position information in the shallow network, the global information in the deep network, and the intermediate information in the intermediate layer network are respectively expressed as:

[0039]

[0040] Among them, , , are the outputs of the hierarchical feature module; S、 、D are respectively the feature sets that meet the shallow, intermediate, and deep mask conditions, is the proportion of class i in D.

[0041] Furthermore, the expression of the class mask is as follows:

[0042]

[0043] Wherein, is the decoding feature maximum index matrix, i is the class, n is the row index, and m is the column index.

[0044] The present invention also discloses a catenary insulator breakage detection device based on few-shot transfer learning, which implements the above-mentioned catenary insulator breakage detection method based on few-shot transfer learning, and includes:

[0045] An image acquisition module, configured to input an image of a catenary insulator to be detected into a two-stage contrast proposal encoding network; the two-stage contrast proposal encoding network includes an encoder, an adaptive feature fusion module, a hierarchical feature extraction module, and an inter-class and intra-class contrast proposal encoding module;

[0046] The encoder is configured to perform feature encoding and feature extraction on the input image of the catenary insulator to be detected, and extract the feature representation of the image;

[0047] The adaptive feature fusion module is configured to receive the output of the encoder, and by adding learnable parameter weights, enable the feature fusion weights to perform adaptive learning in the backpropagation stage to obtain the optimal fusion weights;

[0048] The hierarchical feature extraction module is configured to receive the output of the adaptive feature fusion module, and extract the local position information of small objects in the shallow network of the encoder, the global information of large objects in the deep network, and the intermediate semantic information between local and global in the intermediate network layer;

[0049] The inter-class and intra-class contrast proposal encoding module is configured to receive the output of the hierarchical feature extraction module, calculate the semantic similarity between different classes, and implement the detection of broken insulators in the catenary insulator image.

[0050] Due to the adoption of the above technical solutions, the present invention has the following advantages:

[0051] 1. Based on the few-shot transfer learning network, the present invention proposes a hierarchical feature extraction and adaptive feature fusion method, which realizes the enhancement and efficient extraction of catenary insulator features, and realizes the precise positioning of broken catenary insulators in complex environments and high-speed motion scenarios.

[0052] 2. The present invention proposes an inter-class and intra-class contrast recommendation coding method. The inter-class contrast recommendation coding method is used to guide the detection network to perform large-class-level semantic learning on the insulator category; the intra-class contrast recommendation coding method is used to guide the network to perform small-class-level semantic learning on the damaged insulator category. It realizes the accurate classification of the damaged insulators on the catenary under complex environments and high-speed motion scenarios. Description of the Drawings

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0054] Figure 1 Schematic diagram of the two-level contrast recommendation coding network structure for the embodiments of the present invention;

[0055] Figure 2 Schematic diagram of the structure of the adaptive feature fusion module for the embodiments of the present invention;

[0056] Figure 3 Schematic diagram of the network structure of the hierarchical feature extraction module for the embodiments of the present invention;

[0057] Figure 4 Schematic diagram of the detailed training process of the two-level contrast recommendation coding network for the embodiments of the present invention;

[0058] Figure 5 Schematic diagram of the simplified training process of the two-level contrast recommendation coding network for the embodiments of the present invention;

[0059] Figure 6(a) shows the recognition effect diagram of the damage defect of the flat wrist arm insulator for the embodiments of the present invention;

[0060] Figure 6(b) shows the recognition effect diagram of the damage defect of the flat wrist arm insulator for the embodiments of the present invention;

[0061] Figure 6(c) shows the recognition effect diagram of the damage defect of the inclined wrist arm insulator for the embodiments of the present invention;

[0062] Figure 6(d) shows the recognition effect diagram of the damage defect of the rigid insulator for the embodiments of the present invention. Detailed Embodiments

[0063] The present invention will be further described in conjunction with the drawings and embodiments. The described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art shall fall within the scope protected by the embodiments of the present invention.

[0064] The main difficulties existing in the current detection tasks are as follows: (1) Sample imbalance, with few defective samples. In the detection of catenary insulator breakage, there is a huge imbalance between the number of normal samples and broken samples, which often leads to poor learning effects of machine learning models and makes it difficult to effectively identify defective samples of the minority class. (2) The operating environment of the catenary is complex, and there are significant differences in the structures of rigid catenary and flexible catenary. The operating environment of the catenary includes different catenary structures (such as rigid and flexible catenaries), and the differences in their forms and materials may limit the generality and accuracy of the detection algorithm. An embodiment of a method for detecting catenary insulator breakage based on few-shot transfer learning is provided in the present invention, aiming to accurately detect broken insulators among a large number of insulators.

[0065] See Figure 4 and Figure 5 , this embodiment includes the following steps:

[0066] Input the image of the catenary insulator to be detected into a two-stage contrastive proposal encoding network (Dul-FUSE-Net); the two-stage contrastive proposal encoding network includes an encoder (Detector, Det.), an adaptive feature fusion module (Adaptive Fusion Module, AFM), a hierarchical feature extraction module (Hierarchical Module, H.M), and an inter-class and intra-class contrastive proposal encoding module (Contrastive Proposal Encoding, CPE), where the encoder can be a Faster R-CNN encoder with a standard structure;

[0067] The encoder performs feature encoding and feature extraction on the input image of the catenary insulator to be detected, and extracts the feature representation of the image;

[0068] The adaptive feature fusion module receives the output of the encoder, and by adding learnable parameter weights, enables the feature fusion weights to be adaptively learned during the backpropagation stage to obtain the optimal fusion weights; the hierarchical feature extraction module receives the output of the adaptive feature fusion module, and extracts the local position information of small objects in the shallow network of the encoder, the global information of large objects in the deep network, and the intermediate semantic information between local and global in the intermediate network layer;

[0069] The inter-class and intra-class contrastive proposal encoding module receives the output of the hierarchical feature extraction module, calculates the semantic similarity between different classes, and realizes the detection of broken insulators in the catenary insulator image.

[0070] Optionally, the inter-class and intra-class contrastive proposal encoding module includes an inter-class contrastive proposal encoding module and an intra-class contrastive proposal encoding module;

[0071] The inter-class contrast recommendation encoding module calculates the semantic similarity between the classification boxes of large-class insulators CLASS JYZ and those of other large-class devices CLASS other to guide the two-level contrast recommendation encoding network in performing large-class-level semantic learning on the insulator category;

[0072] The intra-class contrast recommendation encoding module calculates the semantic similarity between the classification boxes of damaged insulators CLASS JYZ-ps and those of other fine-grained categories to guide the two-level contrast recommendation encoding network in performing small-class-level semantic learning on the damaged insulator category; other fine-grained categories include normal insulators CLASS JYZ-zc , dirty insulators CLASS JYZ-zw .

[0073] Specifically, the inter-class contrast recommendation encoding module is used to calculate the semantic similarity between categories, and the intra-class contrast recommendation encoding module is used to calculate the semantic similarity within a category; the calculation of the semantic similarity between categories is completed based on the cosine similarity of the classification boxes. The inter-class semantic similarity between category i and the category weight matrix and the intra-class semantic similarity between category i and the category weight matrix are calculated by the following formulas respectively:

[0074]

[0075]

[0076] where is the inter-class semantic similarity between category i and the category weight matrix , is the intra-class semantic similarity between category i and the category weight matrix , is the inter-class scale factor, is the intra-class scale factor, empirically taken as , ; is the hierarchical extraction feature of category i.

[0077] Optionally, the inter-class contrast recommendation encoding module calculates the semantic similarity between the classification boxes of large-class insulators CLASS JYZ and those of other large-class devices CLASS other , including:

[0078] Calculating the semantic similarity between the classification boxes of large-class insulators CLASS JYZ and those of other large-class devicesCLASS other The cosine similarity between the classification boxes;

[0079] The intra-class contrast suggestion encoding module calculates the CLASS JYZ-ps semantic similarity between the classification box of the damaged insulator and the classification boxes of other fine-grained categories, including:

[0080] Calculating the CLASS JYZ-ps cosine similarity between the classification box of the damaged insulator and the classification boxes of other fine-grained categories.

[0081] Optionally, the training process of the two-level contrast suggestion encoding network is as follows:

[0082] Construct a base class dataset and a new class dataset; build a two-level contrast suggestion encoding network; construct a loss function for the two-level contrast suggestion encoding network; the base class dataset is a dataset of rigid and flexible catenary insulators including a large number of labeled samples; the new class dataset is a dataset of a small number (e.g., 10) of labeled damaged insulators;

[0083] Divide the base class dataset into a training set (70%), a validation set (15%), and a test set (15%). Train the two-level contrast suggestion encoding network on the base class dataset (supervised training can be performed for 200 rounds), and use the optimal weight corresponding to the maximum value of the mean intersection over union (MIOU) metric of the predicted boxes of the two-level contrast suggestion encoding network on the test set as the optimal weight of the two-level contrast suggestion encoding network; the validation set and the test set are used to verify and test the trained two-level contrast suggestion encoding.

[0084] Initialize and fix the weights of the encoder, hierarchical feature extraction module, and adaptive feature fusion module in the two-level contrast suggestion encoding network using the optimal weight. After fixing, the weights of the network layer do not participate in the backpropagation gradient calculation;

[0085] According to the number of the new class dataset, perform balanced sampling on the base class dataset, and then use the sampled base class dataset to perform supervised fine-tuning on the two-level contrast suggestion encoding network. Use the new class dataset to test the trained two-level contrast suggestion encoding network to obtain the optimal weight of the two-level contrast suggestion encoding network, and use this optimal weight as the optimal weight value of the trained two-level contrast suggestion encoding network.

[0086] Optionally, the base class dataset is a dataset of rigid and flexible catenary insulators including a large number of labeled samples; the new class dataset is a dataset of a small number of labeled damaged insulators.

[0087] Optionally, the loss function defined in the backpropagation of the two-level contrast suggestion encoding network is:

[0088]

[0089] Among them, is the loss function, is the binary cross-entropy loss of the insulator feed-forward proposal box, is the cross-entropy loss of the insulator classification box, is the inter-class contrast proposal encoding loss of the insulator, is the intra-class contrast proposal encoding loss of the insulator, , are hyperparameters.

[0090] Optionally, in the network backpropagation, the inter-class contrast proposal encoding loss of the insulator is defined as:

[0091]

[0092]

[0093] Among them, is the inter-class contrast proposal encoding loss of the insulator, is the number of prediction boxes with the predicted class , is a hyperparameter, , is the modulus value of the i-th true target box, is the cosine similarity between the i-th prediction box and the -th prediction box on the projection hypersphere, is the -th prediction box, is the -th prediction box, is the -th prediction box, is the intersection over union ratio score between the i-th prediction box and the label box, N is the number of prediction boxes, k is a temporary variable, is the -th predicted class, is the -th predicted class. The loss function is used to guide the supervised learning of the network for the regression features and prediction box features of the damaged insulator classes.

[0094] Optionally, the structure of the adaptive feature fusion module is as Figure 2 shown. In the decoding stage, the adaptive feature fusion module determines the optimal weights for the feature fusion of different network depths:

[0095] is obtained by doubling the upsampling, , is defined as:

[0096]

[0097] Among them, , , , are learnable parameters during the forward propagation process; , , are the input features of the adaptive feature fusion module; , , are the output features of the adaptive feature fusion module.

[0098] Optionally, by adding the adaptive feature fusion weights , , , to perform a quantization expression for feature fusion during the backpropagation process, and use the output of the feature fusion module as the input of the hierarchical feature extraction module. The hierarchical feature extraction module ensures different-scale feature expressions for feature maps at different network depths by performing mask calculations based on class probabilities on the input feature maps, where the semantic information of the deep network predicts large objects and the semantic information of the shallow network is used to predict small objects.

[0099] Optionally, the structure of the hierarchical feature extraction module is as Figure 3 shown, and the processing flow of the hierarchical feature extraction module is as follows:

[0100] The hierarchical feature extraction module receives the multi-scale output of the adaptive feature fusion module. The multi-scale output includes shallow features , middle-level features and deep features , all with dimensions of C×H×W;

[0101] The hierarchical feature extraction module determines the shallow feature mask , for feature filtering, the middle-level feature mask , and the deep feature mask according to the hyperparameters , where is a class mask composed of C 1×H×W, and , , are the mask sets that meet the conditions corresponding to respectively;

[0102] Process the input features according to the feature masks , , , where the shallow features pass through the shallow feature mask Obtain the local position information of small objects in the shallow network (evaluating the size of an object based on the category proportion, where objects with a low proportion are small objects, and in the shallow network, the receptive field is small and the image position information, i.e., local information, is rich); the deep features are processed through a deep feature mask to obtain the global information of large objects in the deep network (in the deep features, the receptive field is large and the image global information is rich); similarly, the intermediate layer network is used to extract the intermediate information between small objects and large objects 。

[0103] Optionally, the local position information in the shallow network , the global information in the deep network and the intermediate information in the intermediate layer network are respectively expressed as:

[0104]

[0105] Among them, , , are the outputs of the hierarchical feature module; S、 、D are respectively the feature sets that meet the shallow, intermediate, and deep mask conditions, is the proportion of category i in D.

[0106] Optionally, the expression of the category mask is:

[0107]

[0108] Among them, is the decoding feature maximum value index matrix, i is the category, n is the row index, and m is the column index.

[0109] Common methods for solving the few-shot object detection problem include: meta-learning-based methods, transfer-learning-based methods, data-augmentation-based methods, and metric-learning-based methods. In the task of insulator breakage detection, the model design of meta-learning-based methods is difficult and not easy to converge; the model robustness of data-augmentation-based methods is not high; the computational cost of metric-learning-based methods is relatively large and the memory occupancy is high; therefore, a transfer-learning-based method is selected to transfer on the base class model and learn the deep semantic features of a small number of broken insulator samples to achieve accurate detection of broken insulators.

[0110] ​FIG. 6(a), FIG. 6(b), FIG. 6(c) and FIG. 6(d) are respectively the recognition effect diagrams of insulator breakage defects in the embodiments of the present invention, including the breakage detection results of flat / inclined wrist arm insulators in the flexible catenary and the breakage detection results of insulators in the rigid catenary.

[0111] The present invention also provides an embodiment of a catenary insulator breakage detection device based on few-shot transfer learning, which realizes the catenary insulator breakage detection method based on few-shot transfer learning described in the above embodiment, and includes:

[0112] An image acquisition module, configured to input an image of a catenary insulator to be detected into a two-stage contrast proposal encoding network; the two-stage contrast proposal encoding network includes an encoder, an adaptive feature fusion module, a hierarchical feature extraction module, and an inter-class and intra-class contrast proposal encoding module;

[0113] The encoder is configured to perform feature encoding and feature extraction on the input image of the catenary insulator to be detected, and extract the feature representation of the image;

[0114] The adaptive feature fusion module is configured to receive the output of the encoder, and by adding learnable parameter weights, enable the feature fusion weights to perform adaptive learning in the backpropagation stage to obtain the optimal fusion weights;

[0115] The hierarchical feature extraction module is configured to receive the output of the adaptive feature fusion module, and extract the local position information of small objects in the shallow network of the encoder, the global information of large objects in the deep network, and the intermediate semantic information between local and global in the intermediate network layer;

[0116] The inter-class and intra-class contrast proposal encoding module is configured to receive the output of the hierarchical feature extraction module, calculate the semantic similarity between different categories, and realize the detection of broken insulators in the catenary insulator image.

[0117] The encoder in this embodiment can adopt the standard Faster R-CNN encoding, see Figure 1 and Figure 4 , the standard Faster R-CNN encoder ( Detector, Det.)、 adaptive feature fusion module ( Adaptive Fusion Module, AFM)、 hierarchical feature extraction module ( Hierarchical Module, H.M)、 inter-class and intra-class contrast proposal encoding module ( Contrastive Proposal Encoding, CPE) ; wherein, the inter-class and intra-class contrast proposal encoding module includes an inter-class contrast encoding head, an intra-class contrast encoding head, position regression, and class regression; the functions of each module are shown in Table 1.

[0118] Table 1 Module Function Description

[0119]

[0120] Figure 1 In it, the base-class data set and the new-class data set are input into Dul-FUSE-Net in sequence, and feature extraction is carried out through the encoder, the adaptive feature fusion module and the hierarchical feature extraction module, and Dul-FUSE-Net is backpropagated according to the network position regression loss, the class regression loss, the insulator inter-class contrast proposal encoding loss and the insulator intra-class contrast proposal encoding loss.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: the specific implementation manners of the present invention can still be modified or equivalently replaced, and any modification or equivalent replacement without departing from the spirit and scope of the present invention shall be covered by the protection scope of the claims of the present invention.

Claims

1. A method for detecting damage of catenary insulators based on few-shot transfer learning, characterized in that, including: Input the catenary insulator image to be detected into a two-level contrast proposal encoding network; The two-level contrast proposal encoding network includes an encoder, an adaptive feature fusion module, a hierarchical feature extraction module, and an inter-class and intra-class contrast proposal encoding module; The encoder performs feature encoding and feature extraction on the input catenary insulator image to be detected, and extracts the feature representation of the image; The adaptive feature fusion module receives the output of the encoder, and by adding learnable parameter weights, enables the feature fusion weights to be adaptively learned during the backpropagation stage to obtain the optimal fusion weights; The hierarchical feature extraction module receives the output of the adaptive feature fusion module, and extracts the local position information of small objects in the shallow network of the encoder, the global information of large objects in the deep network, and the intermediate semantic information between local and global in the intermediate network layer; The inter-class and intra-class contrast proposal encoding module receives the output of the hierarchical feature extraction module, calculates the semantic similarity between different classes, and realizes the detection of damaged insulators in the catenary insulator image; The training process of the two-level contrast proposal encoding network is as follows: Construct a base class dataset and a new class dataset; build a two-level contrast proposal encoding network; construct the loss function of the two-level contrast proposal encoding network; the base class dataset is a dataset of rigid and flexible catenary insulators including a large number of labeled samples; the new class dataset is a dataset of damaged insulators with a small number of labels; Divide the base class dataset into a training set, a validation set, and a test set, train the two-level contrast proposal encoding network on the base class dataset, and use the optimal weights of the two-level contrast proposal encoding network on the test set as the optimal weights of the two-level contrast proposal encoding network; the validation set and the test set are used to verify and test the two-level contrast proposal encoding obtained by training; Use the optimal weights to initialize and solidify the weights of the two-level contrast proposal encoding network. After solidification, the weights of the network layer do not participate in the backpropagation gradient operation; According to the number of the new class dataset, perform balanced sampling on the base class dataset, and then use the sampled base class dataset to perform supervised fine-tuning on the two-level contrast proposal encoding network. Use the new class dataset to test the two-level contrast proposal encoding network obtained by training to obtain the optimal weights of the two-level contrast proposal encoding network, and use this optimal weight as the optimal weight of the trained two-level contrast proposal encoding network; The loss function defined in the backpropagation of constructing the two-level contrast proposal encoding network is: Among them, is the loss function, is the binary cross-entropy loss of the insulator feed-forward proposal box, is the cross-entropy loss of the insulator classification box, is the inter-class contrast proposal coding loss of the insulator, is the intra-class contrast proposal coding loss of the insulator, , are hyperparameters; The inter-class contrast proposal encoding loss of the insulator is defined as: Among them, is the coding loss for comparison between insulator classes, is the number of prediction boxes with the predicted class . is a hyperparameter, , is the modulus value of the i-th true target box, is the cosine similarity between the i-th prediction box and the -th prediction box on the projected hypersphere, is the -th prediction box, is the -th prediction box, is the -th prediction box, is the intersection over union score between the i-th prediction box and the label box, N is the number of prediction boxes, and k is a temporary variable. is the -th predicted class, is the -th predicted class.

2. The method for detecting the damage of catenary insulators based on few-shot transfer learning according to claim 1, characterized in that, The inter-class and intra-class contrast proposal encoding module includes an inter-class contrast proposal encoding module and an intra-class contrast proposal encoding module; The inter-class comparison suggestion encoding module calculates the semantic similarity between the classification boxes of large-class insulators CLASS JYZ and those of other large-class devices CLASS other to guide the two-level comparison suggestion encoding network to perform large-class level semantic learning on the insulator category; The intra-class comparison suggestion encoding module calculates the semantic similarity between the classification box of the damaged insulator CLASS JYZ-ps and the classification boxes of other fine-grained categories to guide the two-level comparison suggestion encoding network to perform small-class-level semantic learning on the damaged insulator category; the other fine-grained categories include normal insulators CLASS JYZ-zc , dirty insulators CLASS JYZ-zw .

3. The method for detecting the damage of catenary insulators based on few-shot transfer learning according to claim 2, wherein The inter-class comparison recommendation encoding module calculates the semantic similarity between the classification boxes of large-class insulators CLASS JYZ and those of other large-class devices CLASS other , including: Calculate the classification boxes of large-class insulators CLASS JYZ and the cosine similarity between the classification boxes of other large-class devices CLASS other ; The in-class comparison recommendation encoding module calculates the semantic similarity between the bounding box of the damaged insulator CLASS JYZ-ps and the bounding boxes of other fine-grained categories, including: Calculating the broken insulators CLASS JYZ-ps cosine similarity between the classification box and the classification boxes of other fine-grained categories 4. The method for detecting the damage of catenary insulators based on few-shot transfer learning according to claim 1, wherein The base class dataset is a dataset of rigid and flexible catenary insulators including a large number of labeled samples; the new class dataset is a dataset of damaged insulators with a small number of labels.

5. The method for detecting damage of catenary insulators based on few-shot transfer learning according to claim 1, wherein In the decoding stage, the adaptive feature fusion module determines the optimal weights for feature fusion at different network depths: Obtained by doubling upsampling, and is defined as: Among them, , , , are learnable parameters in the direction propagation process; , , are the input features of the adaptive feature fusion module; , , are the output features of the adaptive feature fusion module.

6. The method for detecting damage of catenary insulators based on few-shot transfer learning according to claim 1, characterized in that, The processing flow of the hierarchical feature extraction module is: The hierarchical feature extraction module receives the multi-scale outputs of the adaptive feature fusion module, and the multi-scale outputs include shallow features , intermediate features and deep features , all with dimensions of C×H×W; The hierarchical feature extraction module determines the shallow feature mask , for feature filtering, the middle-level feature mask , and the deep feature mask , where . Among them, is a class mask composed of C 1×H×W, , , are respectively the mask sets that meet the conditions corresponding to ; According to the feature mask , , process the input features, where the shallow features are processed by the shallow feature mask to obtain the local position information of small objects in the shallow network ; the deep features are processed by the deep feature mask to obtain the global information of large objects in the deep network ; similarly, the intermediate layer network is used to extract the intermediate information between small objects and large objects .

7. The method for detecting the damage of catenary insulators based on few-shot transfer learning according to claim 6, wherein, Local position information in the shallow network , global information in the deep network and intermediate information in the intermediate layer network are respectively represented as: Among them, , , are the outputs of the hierarchical feature module; S、 、D are the feature sets that meet the shallow, medium, and deep mask conditions respectively, is the proportion of class i in D.

8. The method for detecting the damage of catenary insulators based on few-shot transfer learning according to claim 6, characterized in that, The expression of the category mask is: Among them, is the index matrix of the maximum decoding feature value, i is the category, n is the row index, and m is the column index.

9. A catenary insulator breakage detection device based on few-shot transfer learning, which implements the catenary insulator breakage detection method based on few-shot transfer learning described in any one of claims 1-8, characterized in that, including: An image acquisition module for inputting the catenary insulator image to be detected into a two-stage contrast proposal encoding network; The two-stage contrast proposal encoding network includes an encoder, an adaptive feature fusion module, a hierarchical feature extraction module, and an inter-class and intra-class contrast proposal encoding module; The encoder is used for performing feature encoding and feature extraction on the input catenary insulator image to be detected, and extracting the feature representation of the image; The adaptive feature fusion module is used for receiving the output of the encoder, and by adding learnable parameter weights, enabling the feature fusion weights to perform adaptive learning in the backpropagation stage to obtain the optimal fusion weights; The hierarchical feature extraction module is used for receiving the output of the adaptive feature fusion module, and extracting the local position information of small objects in the shallow network of the encoder, the global information of large objects in the deep network, and the intermediate semantic information between local and global in the intermediate network layer; The inter-class and intra-class contrast proposal encoding module is used for receiving the output of the hierarchical feature extraction module, calculating the semantic similarity between different classes, and realizing the detection of damaged insulators in the catenary insulator image.

Images