Contact network insulator damage detection method and device based on few-sample transfer learning

Abstract

The invention discloses a contact network insulator damage detection method and device based on few-sample transfer learning. The method comprises the following steps: inputting a to-be-detected contact network insulator image into a two-stage comparison suggestion coding network; the encoder performs feature encoding and feature extraction on the insulator image, and extracts feature representation of the image; the adaptive feature fusion module obtains an optimal fusion weight through adaptive weight learning; a hierarchical feature extraction module receives output of the adaptive feature fusion module, and extracts local position information of small objects in a shallow network of the encoder, global information of large objects in a deep network and local and global intermediate semantic information in an intermediate network layer; the inter-class and intra-class comparison suggestion coding module receives the output of the hierarchical feature extraction module, calculates the semantic similarity between different classes, and achieves the damage detection of the contact network insulator in a complex environment and a high-speed motion scene. According to the invention, the accuracy and efficiency of contact network insulator damage detection are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of contact network insulator damage detection, and in particular to a contact network insulator damage detection method and device based on few-sample transfer learning. Background Art

[0002] Catenary insulators play an important role in insulating and supporting conductors in the electric railway system, ensuring stable and safe power supply. Damage to any insulator in the catenary, such as cracks or ruptures, poses serious safety hazards, such as electrical failures or conductor detachment, which may not only interrupt railway traffic but also cause safety accidents. At present, the inspection of catenary insulators mostly adopts manual inspection methods, and staff need to go to the site in person for visual inspection. This method is not only inefficient and dependent on the experience and condition of the inspectors, but also difficult to operate, has a long cycle and high maintenance costs.

[0003] In recent years, deep learning-based insulator damage detection methods have provided a technological innovation solution to this problem. By using convolutional neural networks (CNN) or other forms of machine vision algorithms, the damage of insulators in the contact network can be automatically identified and classified. Although these technologies perform well on large-scale data sets, they still face two major challenges: first, when there are fewer training samples, the target positioning accuracy of broken insulators is usually low; second, the subtle differences between classes lead to poor performance in the fine classification of broken insulators, making it difficult to distinguish subtle damage to insulators. Summary of the invention

[0004] In view of the above technical difficulties, the present invention designs a method for detecting damaged insulators in the contact network based on improved small sample transfer learning. This method can realize model training through a small number of samples, and adopts adaptive feature fusion and hierarchical feature extraction methods to realize the extraction and enhancement of damaged insulator features in the contact network, thereby improving the positioning accuracy of defective targets; at the same time, the coding module is recommended by using inter-class and intra-class comparison to improve the accuracy of inter-class and intra-class division of insulators and damaged insulators. The application of this technology makes the detection of damaged insulators of various types in the contact network more accurate and efficient in complex environments and high-speed motion scenarios, greatly improving the safety and operation efficiency of the railway system.

[0005] The present invention discloses a method for detecting damage of contact network insulators based on few-sample transfer learning, which comprises: The image of the contact network insulator to be detected is input into a two-level contrast suggestion coding network; the two-level contrast suggestion coding network includes an encoder, an adaptive feature fusion module, a hierarchical feature extraction module, and an inter-class and intra-class contrast suggestion coding module; The encoder performs feature encoding and feature extraction on the input image of the contact network insulator to be detected, and extracts the feature representation of the image; The adaptive feature fusion module receives the encoder output and adds learnable parameter weights to enable the feature fusion weights to be adaptively learned in the back-propagation stage to obtain the optimal fusion weights. The hierarchical feature extraction module receives the output of the adaptive feature fusion module and extracts the local position information of small objects in the shallow network of the encoder, the global information of large objects in the deep network, and the intermediate semantic information between the local and the global in the intermediate network layer. The inter-class and intra-class comparison suggestion encoding module receives the output of the hierarchical feature extraction module, calculates the semantic similarity between different categories, and realizes the detection of broken insulators in the contact network insulator image.

[0006] Further, the inter-class and intra-class comparison suggestion encoding module includes an inter-class comparison suggestion encoding module and an intra-class comparison suggestion encoding module; Inter-class comparison suggests coding module to calculate the major class of insulators CLASS JYZ Classification boxes and other major categories of equipment CLASS other The semantic similarity between the classification boxes is used to guide the two-level contrastive proposal encoding network to learn the semantics of the insulation sub-category at the large category level; Intra-class comparison suggests coding module to calculate damaged insulators CLASS JYZ-ps The semantic similarity between the classification box of and the classification boxes of other fine-grained categories is used to guide the two-level contrastive proposal encoding network to perform small-category semantic learning on the broken insulator category; other fine-grained categories include normal insulators CLASS JYZ-zc , Dirty Insulators CLASS JYZ-zw .

[0007] Furthermore, the inter-class comparison suggests that the coding module calculates the large class of insulators CLASS JYZ Classification boxes and other major categories of equipment CLASS other The semantic similarity between the classification boxes includes: Calculation of major types of insulators CLASS JYZ Classification boxes and other major categories of equipment CLASS other The cosine similarity between the classification boxes; The intra-class comparison suggests that the coding module calculates the damaged insulator CLASS JYZ-ps The semantic similarity between the classification box of and the classification boxes of other fine-grained categories, including: Calculation of damaged insulators CLASS JYZ-ps The cosine similarity between the classification box of and the classification boxes of other fine-grained categories.

[0008] Furthermore, the training process of the two-level contrast suggestion coding network is as follows: Construct base class data sets and new class data sets; build a two-level comparison suggestion coding network; construct the loss function of the two-level comparison suggestion coding network; the base class data set is a rigid and flexible contact network insulator data set including a large number of labeled samples; the new class data set is a damaged insulator data set with a small number of annotations; The base class data set is divided into a training set, a validation set and a test set, the two-level contrast suggestion encoding network is trained on the base class data set, and the optimal weight of the two-level contrast suggestion encoding network on the test set is used as the optimal weight of the two-level contrast suggestion encoding network; the validation set and the test set are used to verify and test the trained two-level contrast suggestion encoding; Use the optimal weights to initialize and solidify the two-level comparison suggestion encoding network. After solidification, the weights of the network layer do not participate in the back-propagation gradient operation. According to the number of new class data sets, the base class data sets are evenly sampled, and the sampled base class data sets are used to perform supervised fine-tuning on the two-level contrast suggestion encoding network. The new class data sets are used to test the trained two-level contrast suggestion encoding network to obtain the optimal weight of the two-level contrast suggestion encoding network, and the optimal weight is used as the optimal weight of the trained two-level contrast suggestion encoding network.

[0009] Furthermore, the base class data set includes a large number of labeled samples of rigid and flexible contact network insulator data sets; the new class data set is a small number of labeled damaged insulator data sets.

[0010] Furthermore, the loss function in the back propagation of the two-level contrast suggestion encoding network is defined as:

[0011] in, is the loss function, The binary cross entropy loss for the insulator feedforward proposal box, is the cross entropy loss of the insulator classification box, Suggested coding loss for comparison between insulator classes, Suggested coding loss for intra-class comparison of insulators, , is a hyperparameter.

[0012] Furthermore, the inter-class comparison suggests that the encoding loss is defined as:

[0013]

[0014] in, Suggested coding loss for comparison between insulator classes, For the predicted category The number of prediction boxes, is a hyperparameter, , is the modulus of the i-th true target box, is the relationship between the i-th prediction box and the The cosine similarity of the predicted boxes on the projected hypersphere, For the prediction boxes, For the prediction boxes, For the prediction boxes, is the intersection-over-union score of the i-th prediction box and the label box, N is the number of prediction boxes, k is a temporary variable, For the prediction categories, For the prediction categories.

[0015] Furthermore, the adaptive feature fusion module determines the optimal weights for fusion of different network depth features during the decoding phase: Depend on Double upsampling gives, , Defined as:

[0016] in, , , , is the learnable parameter in the direction propagation process; , , Input features to the adaptive feature fusion module; , , Output features for the adaptive feature fusion module.

[0017] Furthermore, the processing flow of the hierarchical feature extraction module is: The hierarchical feature extraction module receives the multi-scale output of the adaptive feature fusion module, which includes shallow features , middle-level features and deep features , the dimensions are C×H×W; The hierarchical feature extraction module is based on the hyperparameters , Determine shallow feature masks for feature filtering , mid-level feature mask , deep feature mask ,in is a category mask consisting of C 1×H×W, , , Corresponding to The set of masks that meet the conditions; Based on feature mask , , Process the input features, where shallow features After shallow feature mask Processing to obtain local position information of small objects in shallow networks ; Deep Features After deep feature masking Processing to obtain global information of large objects in deep networks ; Similarly, the middle layer network is used to extract the intermediate information between small objects and large objects .

[0018] Furthermore, local location information in shallow networks , Global Information in Deep Networks and intermediate information in the middle layer network Respectively expressed as:

[0019] in, , , Output of hierarchical feature module; S、 、D are the feature sets that satisfy the shallow, medium, and deep mask conditions, respectively. is the proportion of category i in D.

[0020] Furthermore, the expression of the category mask is:

[0021] in, is the decoding feature maximum index matrix, i is the category, n is the row index, and m is the column index.

[0022] The present invention also discloses a contact network insulator damage detection device based on few-sample transfer learning, which implements the above-mentioned contact network insulator damage detection method based on few-sample transfer learning, and comprises: An image acquisition module is used to input the image of the contact network insulator to be detected into a two-level comparison suggestion coding network; the two-level comparison suggestion coding network includes an encoder, an adaptive feature fusion module, a hierarchical feature extraction module, and an inter-class and intra-class comparison suggestion coding module; An encoder is used for performing feature encoding and feature extraction on the input image of the contact network insulator to be detected, and extracting feature representation of the image; The adaptive feature fusion module is used to receive the encoder output and add learnable parameter weights so that the feature fusion weights can be adaptively learned in the back-propagation stage to obtain the optimal fusion weights. A hierarchical feature extraction module is used to receive the output of the adaptive feature fusion module and extract the local position information of small objects in the shallow network of the encoder, the global information of large objects in the deep network, and the intermediate semantic information between local and global in the intermediate network layer; The inter-class and intra-class comparison suggestion encoding module is used to receive the output of the hierarchical feature extraction module, calculate the semantic similarity between different categories, and realize the detection of broken insulators in the contact network insulator image.

[0023] Due to the adoption of the above technical solution, the present invention has the following advantages: 1. Based on a small sample transfer learning network, the present invention proposes a hierarchical feature extraction and adaptive feature fusion method to enhance and efficiently extract the features of contact network insulators, and to accurately locate damaged contact network insulators in complex environments and high-speed motion scenarios.

[0024] 2. The present invention proposes inter-class and intra-class comparison suggestion coding methods. The inter-class comparison suggestion coding method is used to guide the detection network to perform large-class semantic learning of insulator categories; the intra-class comparison suggestion coding method is used to guide the network to perform small-class semantic learning of damaged insulator categories. Accurate classification of damaged insulators of contact networks in complex environments and high-speed motion scenarios is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0026] Figure 1 A schematic diagram of a two-stage comparison suggestion coding network structure according to an embodiment of the present invention; Figure 2 It is a schematic diagram of the structure of an adaptive feature fusion module according to an embodiment of the present invention; Figure 3 A schematic diagram of the network structure of a hierarchical feature extraction module according to an embodiment of the present invention; Figure 4 A schematic diagram of a detailed flow chart of a two-level comparison suggestion coding network training according to an embodiment of the present invention; Figure 5 A schematic diagram of a simplified process flow of two-level comparison suggestion coding network training according to an embodiment of the present invention; FIG6( a ) is a diagram showing the effect of identifying damage defects of a flat arm insulator according to an embodiment of the present invention; FIG6( b ) is a diagram showing the effect of identifying damage defects of a flat arm insulator according to an embodiment of the present invention; FIG6( c ) is a diagram showing the effect of identifying damage defects of an inclined arm insulator according to an embodiment of the present invention; FIG6( d ) is a diagram showing the effect of identifying damage defects of a rigid insulator according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention is further described in conjunction with the accompanying drawings and embodiments, and the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field should fall within the scope of protection of the embodiments of the present invention.

[0028] The main difficulties in current detection tasks are as follows: (1) Sample imbalance and few defective samples. In the detection of damaged contact insulators, there is a huge imbalance between the number of normal samples and damaged samples, which often leads to poor learning effects of machine learning models and difficulty in effectively identifying defective samples of minority classes. (2) The operating environment of the contact network is complex, and the structures of rigid contact networks and flexible contact networks are very different. The operating environment of the contact network includes different contact network structures (such as rigid and flexible contact networks). The differences in morphology and material of these structures may limit the versatility and accuracy of the detection algorithm. The present invention provides an embodiment of a method for detecting damaged contact insulators based on transfer learning of few samples, with the aim of accurately detecting damaged insulators among a large number of insulators.

[0029] See also Figure 4 and Figure 5 , this embodiment includes the following steps: Input the insulator image of the contact network to be detected into a two-level contrast proposal encoding network (Dul-FUSE-Net); the two-level contrast proposal encoding network includes an encoder (Detector, Det.), an adaptive feature fusion module (Adaptive Fusion Module, AFM), a hierarchical feature extraction module (Hierarchical Module, HM) and an inter-class and intra-class contrast proposal encoding module (Contrastive Proposal Encoding, CPE), wherein the encoder can be a FasterR-CNN encoder with a standard structure; The encoder performs feature encoding and feature extraction on the input image of the contact network insulator to be detected, and extracts the feature representation of the image; The adaptive feature fusion module receives the encoder output and adds learnable parameter weights to enable the feature fusion weights to be adaptively learned in the back-propagation stage to obtain the optimal fusion weights. The hierarchical feature extraction module receives the output of the adaptive feature fusion module and extracts the local position information of small objects in the shallow network of the encoder, the global information of large objects in the deep network, and the intermediate semantic information between the local and the global in the intermediate network layer. The inter-class and intra-class comparison suggestion encoding module receives the output of the hierarchical feature extraction module, calculates the semantic similarity between different categories, and realizes the detection of broken insulators in the contact network insulator image.

[0030] Optionally, the inter-class and intra-class comparison suggestion encoding module includes an inter-class comparison suggestion encoding module and an intra-class comparison suggestion encoding module; Inter-class comparison suggests coding module to calculate the major class of insulators CLASS JYZ Classification boxes and other major categories of equipment CLASS other The semantic similarity between the classification boxes is used to guide the two-level contrastive proposal encoding network to learn the large-category semantics of the insulation subcategory; Intra-class comparison suggests coding module to calculate damaged insulators CLASS JYZ-ps The semantic similarity between the classification box of and the classification boxes of other fine-grained categories is used to guide the two-level contrastive proposal encoding network to perform small-category semantic learning on the broken insulator category; other fine-grained categories include normal insulators CLASS JYZ-zc , Dirty Insulators CLASS JYZ-zw .

[0031] Specifically, the inter-class comparison suggestion encoding module is used to calculate the semantic similarity between categories, and the intra-class comparison suggestion encoding module is used to calculate the semantic similarity within a category; the semantic similarity between categories is calculated based on the cosine similarity of the classification box, and the category i is related to the category weight matrix The inter-class semantic similarity between categories, category i and category weight matrix The calculation formulas for the intra-class semantic similarity between are:

[0032]

[0033] in, is the matrix of category i and category weight The inter-class semantic similarity between is the matrix of category i and category weight The intra-class semantic similarity between is the inter-class scaling factor, is the intra-class scale factor, empirically taken , ; Extract features for the layer i of category i.

[0034] Optionally, the inter-class comparison suggests that the encoding module calculates the insulators of the major classes CLASS JYZ Classification boxes and other major categories of equipment CLASS other The semantic similarity between the classification boxes includes: Calculation of major types of insulators CLASS JYZ Classification boxes and other major categories of equipment CLASS other The cosine similarity between the classification boxes; Intra-class comparison suggests coding module to calculate damaged insulators CLASS JYZ-ps The semantic similarity between the classification box of and the classification boxes of other fine-grained categories, including: Calculate damaged insulators CLASS JYZ-ps The cosine similarity between the classification box of and the classification boxes of other fine-grained categories.

[0035] Optionally, the training process of the two-level contrast suggestion encoding network is: Construct a base class dataset and a new class dataset; build a two-level comparison suggestion coding network; construct a loss function for the two-level comparison suggestion coding network; the base class dataset is a rigid and flexible contact network insulator dataset including a large number of labeled samples; the new class dataset is a damaged insulator dataset with a small number (for example, 10) of annotations; The base class dataset is divided into a training set (70%), a validation set (15%), and a test set (15%). The two-level contrast suggestion encoding network is trained on the base class dataset (200 rounds of supervised training can be performed), and the optimal weight corresponding to the maximum value of the mean intersection over union (MIOU) indicator of the two-level contrast suggestion encoding network predicted on the test set is used as the optimal weight of the two-level contrast suggestion encoding network; the validation set and the test set are used to verify and test the trained two-level contrast suggestion encoding; Use the optimal weights to initialize and solidify the encoder, hierarchical feature extraction module, and adaptive feature fusion module in the two-level contrast suggestion encoding network. After solidification, the network layer weights do not participate in the back-propagation gradient operation. According to the number of new class data sets, the base class data sets are evenly sampled, and the sampled base class data sets are used to perform supervised fine-tuning on the two-level contrast suggestion encoding network. The new class data sets are used to test the trained two-level contrast suggestion encoding network to obtain the optimal weight of the two-level contrast suggestion encoding network, and the optimal weight is used as the optimal weight of the trained two-level contrast suggestion encoding network.

[0036] Optionally, the base class dataset includes a dataset of rigid and flexible contact network insulators with a large number of labeled samples; the new class dataset is a dataset of damaged insulators with a small number of labeled samples.

[0037] Optionally, the loss function in the back propagation of the two-level contrastive proposal encoding network is defined as:

[0038] in, is the loss function, The binary cross entropy loss for the insulator feedforward proposal box, is the cross entropy loss of the insulator classification box, Suggested coding loss for comparison between insulator classes, Suggested coding loss for intra-class comparison of insulators, , is a hyperparameter.

[0039] Optionally, during network back propagation, the inter-class contrast suggestion encoding loss is defined as:

[0040]

[0041] in, Suggested coding loss for comparison between insulator classes, For the predicted category The number of prediction boxes, is a hyperparameter, , is the modulus of the i-th true target box, is the relationship between the i-th prediction box and the The cosine similarity of the predicted boxes on the projected hypersphere, For the prediction boxes, For the prediction boxes, For the prediction boxes, is the intersection-over-union score of the i-th prediction box and the label box, N is the number of prediction boxes, k is a temporary variable, For the prediction categories, For the The loss function is used to guide the supervised learning of the network for the broken insulator category regression features and prediction box features.

[0042] Optionally, the adaptive feature fusion module structure is as follows Figure 2 As shown in the figure, the adaptive feature fusion module determines the optimal weights for fusion of different network depth features during the decoding stage: Depend on Double upsampling gives, , Defined as:

[0043] in, , , , is the learnable parameter in the direction propagation process; , , Input features to the adaptive feature fusion module; , , Output features for the adaptive feature fusion module.

[0044] Optionally, by adding adaptive feature fusion weights , , , To quantify the feature fusion in the back-propagation process, and use the output of the feature fusion module as the input of the hierarchical feature extraction module. The hierarchical feature extraction module ensures that feature maps of different network depths express features of different scales by performing mask calculation based on category probability on the input feature map, where the deep network semantic information predicts large objects and the shallow network semantic information is used to predict small objects.

[0045] Optionally, the hierarchical feature extraction module structure is as follows Figure 3 As shown in Figure 2, the processing flow of the hierarchical feature extraction module is as follows: The hierarchical feature extraction module receives the multi-scale output of the adaptive feature fusion module, which includes shallow features , middle-level features and deep features , the dimensions are C×H×W; The hierarchical feature extraction module is based on the hyperparameters , Determine shallow feature masks for feature filtering , mid-level feature mask , deep feature mask ,in is a category mask consisting of C 1×H×W, , , Corresponding to The set of masks that meet the conditions; Based on feature mask , , Process the input features, where shallow features After shallow feature mask Processing to obtain local position information of small objects in shallow networks (By category proportion To evaluate the size of an object, the smaller the proportion, the smaller the object. In the shallow network, the receptive field is small, and the image position information, i.e., local information, is rich); deep features After deep feature masking Processing to obtain global information of large objects in deep networks (The deep features have a large receptive field and rich global image information); Similarly, the intermediate layer network is used to extract intermediate information between small objects and large objects .

[0046] Optionally, local position information in shallow networks , Global Information in Deep Networks and intermediate information in the middle layer network Respectively expressed as:

[0047] in, , , Output of hierarchical feature module; S、 、D are the feature sets that satisfy the shallow, medium, and deep mask conditions, respectively. is the proportion of category i in D.

[0048] Optionally, the expression for the category mask is:

[0049] in, is the decoding feature maximum index matrix, i is the category, n is the row index, and m is the column index.

[0050] Common methods for solving the few-sample target detection problem include: meta-learning, transfer learning, data enhancement, and metric learning. In the task of insulator damage detection, the meta-learning method is difficult to design and not easy to converge; the data enhancement method model is not very robust; the metric learning method has a large amount of calculation and high memory usage; therefore, the transfer learning method is selected to learn the deep semantic features of a small number of damaged insulator samples by migrating on the base class model, so as to achieve accurate detection of damaged insulators.

[0051] Fig. 6(a), Fig. 6(b), Fig. 6(c) and Fig. 6(d) are respectively diagrams showing the effect of identifying insulator damage defects according to an embodiment of the present invention, including damage detection results of flat / inclined arm insulators in a flexible contact network and damage detection results of insulators in a rigid contact network.

[0052] The present invention also provides an embodiment of a contact network insulator damage detection device based on few-sample transfer learning, which implements the contact network insulator damage detection method based on few-sample transfer learning described in the above embodiment, and includes: An image acquisition module is used to input the image of the contact network insulator to be detected into a two-level comparison suggestion coding network; the two-level comparison suggestion coding network includes an encoder, an adaptive feature fusion module, a hierarchical feature extraction module, and an inter-class and intra-class comparison suggestion coding module; An encoder is used for performing feature encoding and feature extraction on the input image of the contact network insulator to be detected, and extracting feature representation of the image; The adaptive feature fusion module is used to receive the encoder output and add learnable parameter weights so that the feature fusion weights can be adaptively learned in the back-propagation stage to obtain the optimal fusion weights. A hierarchical feature extraction module is used to receive the output of the adaptive feature fusion module and extract the local position information of small objects in the shallow network of the encoder, the global information of large objects in the deep network, and the intermediate semantic information between local and global in the intermediate network layer; The inter-class and intra-class comparison suggestion encoding module is used to receive the output of the hierarchical feature extraction module, calculate the semantic similarity between different categories, and realize the detection of broken insulators in the contact network insulator image.

[0053] The encoder of this embodiment can use standard Faster R-CNN encoding, see Figure 1 and Figure 4 , standard Faster R-CNN encoder ( Detector, Det.)、 Adaptive feature fusion module ( Adaptive Fusion Module, AFM)、 Hierarchical feature extraction module ( Hierarchical Module, H.M)、 Inter-class and intra-class comparisons suggest encoding modules ( Contrastive Proposal Encoding, CPE); Among them, the inter-class and intra-class comparison suggestion encoding module includes the inter-class comparison encoding head, the intra-class comparison encoding head, the position regression and the category regression; the functions of each module are shown in Table 1.

[0054] Table 1 Module function description

[0055] Figure 1 In the paper, the base class dataset and the new class dataset are input into Dul-FUSE-Net in turn, and feature extraction is performed through the encoder, adaptive feature fusion module and hierarchical feature extraction module. Dul-FUSE-Net is back-propagated according to the network position regression loss, category regression loss, insulator inter-class contrast proposal coding loss and insulator intra-class contrast proposal coding loss.

[0056] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting damage of contact network insulators based on few-sample transfer learning, characterized in that: include: Input the image of the contact network insulator to be inspected into a two-level comparison suggestion coding network; The two-level contrastive proposal encoding network includes an encoder, an adaptive feature fusion module, a hierarchical feature extraction module, and inter-class and intra-class contrastive proposal encoding modules; The encoder performs feature encoding and feature extraction on the input image of the contact network insulator to be detected, and extracts the feature representation of the image; The adaptive feature fusion module receives the encoder output and adds learnable parameter weights so that the feature fusion weights can be adaptively learned in the back-propagation stage to obtain the optimal fusion weights. The hierarchical feature extraction module receives the output of the adaptive feature fusion module and extracts the local position information of small objects in the shallow network of the encoder, the global information of large objects in the deep network, and the intermediate semantic information between local and global in the intermediate network layer; The inter-class and intra-class comparison suggestion encoding module receives the output of the hierarchical feature extraction module, calculates the semantic similarity between different categories, and realizes the detection of broken insulators in the contact network insulator image.

2. The method for detecting damage of contact network insulators based on few-sample transfer learning according to claim 1 is characterized in that: The inter-class and intra-class comparison suggestion coding module includes an inter-class comparison suggestion coding module and an intra-class comparison suggestion coding module; Inter-class comparison suggests coding module to calculate the major class of insulators CLASS JYZ Classification boxes and other major categories of equipment CLASS other The semantic similarity between the classification boxes is used to guide the two-level contrastive proposal encoding network to learn the semantics of the insulation sub-category at the large category level; Intra-class comparison suggests coding module to calculate damaged insulators CLASS JYZ-ps The semantic similarity between the classification box of and the classification boxes of other fine-grained categories is used to guide the two-level contrastive proposal encoding network to perform small-category semantic learning on the broken insulator category; other fine-grained categories include normal insulators CLASS JYZ-zc , Dirty Insulators CLASS JYZ-zw .

3. The contact network insulator damage detection method based on few-sample transfer learning according to claim 2 is characterized in that: The inter-class comparison suggests that the coding module calculates the major class insulators CLASS JYZ Classification boxes and other major categories of equipment CLASS other The semantic similarity between the classification boxes includes: Calculation of major types of insulators CLASS JYZ Classification boxes and other major categories of equipment CLASS other The cosine similarity between the classification boxes; The intra-class comparison suggests that the coding module calculates the damaged insulator CLASS JYZ-ps The semantic similarity between the classification box of and the classification boxes of other fine-grained categories, including: Calculation of damaged insulators CLASS JYZ-ps The cosine similarity between the classification box of and the classification boxes of other fine-grained categories.

4. The method for detecting damage of contact network insulators based on few-sample transfer learning according to claim 1, characterized in that: The training process of the two-level contrast suggestion encoding network is as follows: Construct base class data sets and new class data sets; build a two-level comparison suggestion coding network; construct the loss function of the two-level comparison suggestion coding network; the base class data set is a rigid and flexible contact network insulator data set including a large number of labeled samples; the new class data set is a damaged insulator data set with a small number of annotations; The base class data set is divided into a training set, a validation set and a test set, the two-level contrast suggestion encoding network is trained on the base class data set, and the optimal weight of the two-level contrast suggestion encoding network on the test set is used as the optimal weight of the two-level contrast suggestion encoding network; the validation set and the test set are used to verify and test the trained two-level contrast suggestion encoding; Use the optimal weights to initialize and solidify the two-level comparison suggestion encoding network. After solidification, the weights of the network layer do not participate in the back-propagation gradient operation. According to the number of new class data sets, the base class data sets are evenly sampled, and the sampled base class data sets are used to perform supervised fine-tuning on the two-level contrast suggestion encoding network. The new class data sets are used to test the trained two-level contrast suggestion encoding network to obtain the optimal weight of the two-level contrast suggestion encoding network, and the optimal weight is used as the optimal weight of the trained two-level contrast suggestion encoding network.

5. The method for detecting damage of contact network insulators based on few-sample transfer learning according to claim 4, characterized in that: The base class dataset includes a large number of labeled samples of rigid and flexible contact network insulator datasets; the new class dataset is a small number of labeled damaged insulator datasets.

6. The method for detecting damage of contact network insulators based on few-sample transfer learning according to claim 4, characterized in that: The loss function in the back propagation of the two-level contrast suggestion encoding network is defined as: in, is the loss function, The binary cross entropy loss for the insulator feedforward proposal box, is the cross entropy loss of the insulator classification box, Suggested coding loss for comparison between insulator classes, Suggested coding loss for intra-class comparison of insulators, , is a hyperparameter.

7. The method for detecting damage of contact network insulators based on few-sample transfer learning according to claim 6, characterized in that: The inter-class comparison suggests that the encoding loss is defined as: in, Suggested coding loss for comparison between insulator classes, For the predicted category The number of prediction boxes, is a hyperparameter, , is the modulus of the i-th true target box, is the relationship between the i-th prediction box and the The cosine similarity of the predicted boxes on the projected hypersphere, For the prediction boxes, For the prediction boxes, For the prediction boxes, is the intersection-over-union score of the i-th prediction box and the label box, N is the number of prediction boxes, k is a temporary variable, For the prediction categories, For the prediction categories.

8. The method for detecting damage of contact network insulators based on few-sample transfer learning according to claim 1, characterized in that: The adaptive feature fusion module determines the optimal weights for fusion of different network depth features during the decoding phase: Depend on Double upsampling gives, , Defined as: in, , , , is a learnable parameter in the process of directional propagation; , , Input features to the adaptive feature fusion module; , , Output features for the adaptive feature fusion module.

9. The method for detecting damage of contact network insulators based on few-sample transfer learning according to claim 1, characterized in that: The processing flow of the hierarchical feature extraction module is: The hierarchical feature extraction module receives the multi-scale output of the adaptive feature fusion module, which includes shallow features , middle-level features and deep features , the dimensions are C×H×W; The hierarchical feature extraction module is based on the hyperparameters , Determine shallow feature masks for feature filtering , mid-level feature mask , deep feature mask ,in is a category mask consisting of C 1×H×W, , , Corresponding to The set of masks that meet the conditions; According to the feature mask , , Process the input features, where shallow features After shallow feature mask Processing to obtain local position information of small objects in shallow networks ; Deep Features After deep feature masking Processing to obtain global information of large objects in deep networks ; Similarly, the middle layer network is used to extract the intermediate information between small objects and large objects .

10. The method for detecting damage of contact network insulators based on few-sample transfer learning according to claim 9, characterized in that: Local location information in shallow networks , Global Information in Deep Networks and intermediate information in the middle layer network Respectively expressed as: in, , , Output of hierarchical feature module; S、 、D are the feature sets that satisfy the shallow, medium, and deep mask conditions, respectively. is the proportion of category i in D.

11. The method for detecting damage of contact network insulators based on few-sample transfer learning according to claim 9, characterized in that: The expression of the category mask is: in, is the decoding feature maximum index matrix, i is the category, n is the row index, and m is the column index.

12. A contact network insulator damage detection device based on few-sample transfer learning, which implements the contact network insulator damage detection method based on few-sample transfer learning as described in any one of claims 1 to 11, characterized in that: include: An image acquisition module is used to input the image of the contact network insulator to be detected into a two-level comparison suggestion coding network; The two-level contrastive proposal encoding network includes an encoder, an adaptive feature fusion module, a hierarchical feature extraction module, and inter-class and intra-class contrastive proposal encoding modules; An encoder is used for performing feature encoding and feature extraction on the input image of the contact network insulator to be detected, and extracting feature representation of the image; The adaptive feature fusion module is used to receive the encoder output and add learnable parameter weights so that the feature fusion weights can be adaptively learned in the back-propagation stage to obtain the optimal fusion weights. A hierarchical feature extraction module is used to receive the output of the adaptive feature fusion module and extract the local position information of small objects in the shallow network of the encoder, the global information of large objects in the deep network, and the intermediate semantic information between local and global in the intermediate network layer; The inter-class and intra-class comparison suggestion encoding module is used to receive the output of the hierarchical feature extraction module, calculate the semantic similarity between different categories, and realize the detection of broken insulators in the contact network insulator image.

Images