Insulator device state intelligent identification method and system

Abstract

The invention discloses an insulator device state intelligent identification method and system, relates to the technical field of artificial intelligence, and is used for effectively detecting the state of an insulator device. According to the method, the insulator feature analysis sub-network is used for extracting insulator features and performing feature enhancement and classification, the expert knowledge analysis sub-network is used for positioning, feature extraction, feature enhancement and classification of a damaged area, and the two enhanced features form a feedback loop for enhancing the insulator image features by the expert knowledge features. According to the method, the insulator image features are enhanced in the training process, so that the attention on the damage area is improved in the classification task, and the effectiveness and the accuracy of insulator device state classification are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an insulator device state intelligent identification method and system. Background Art

[0002] As the scale of my country's power transmission lines continues to expand, the number of power grid equipment built in complex environments is also increasing day by day. The safe, stable and reliable operation of the power grid is of vital importance. As an important component on the transmission line, insulators play a key role in suspending conductors and electrical insulation. Due to long-term work in open-air environments, insulator devices are prone to different types of faults, such as breakage and flashover, which seriously affect the safe operation of transmission lines. Therefore, regular inspections of insulators and intelligent analysis of insulators are of great significance to ensure the safe and stable operation of the power grid.

[0003] Traditionally, power transmission line inspection engineers usually conduct on-site inspections or use climbing robots to inspect insulator components. The above inspection methods are inefficient, have uneven inspection results, and pose safety risks, especially when inspecting in complex environments (such as uneven lighting, background interference from buildings, obstruction from other power equipment, and bad weather).

[0004] With the continuous development of image recognition technology, intelligent recognition methods based on pattern recognition technology have been applied to a certain extent in this field. For example, the paper entitled "Insulator detection and defect classification using rotation invariant local directional pattern" (TASKEED J, TANVEER A) proposed an insulator detection and defect classification algorithm, which used the local directional features and rotation invariant local directional features of the image of a single insulator device to train the support vector machine, and then used it for detection. The papers entitled "Composite insulator defect identification and quantitative method based on random houghtransform ellipse detection" (XIANG WZ) and "Research on UAV-assisted and insulator defect image recognition" (Wang Miao, Du Yi, Zhang Zhongrui) proposed a composite insulator defect recognition method, which used elliptic curves to fit the insulator edges extracted by the Canny operator, and effectively identified the insulator defect area by the difference between the edge curves before and after fitting. The paper titled "Insulator fault detection based on spatial morphological features of aerial images" (ZHAI Y, LI XF, YANG Q) identifies the target insulator area based on the color features and spatial morphological features of glass insulators and ceramic insulators, performs morphological processing on the target area to highlight the fault location, and realizes fault identification based on the spatial features of the insulators with or without faults. These pattern recognition methods detect and identify insulators based on artificially designed features such as contour features, color features, and texture features. Such methods are only applicable to scenes with little background interference, specific sizes, or specific lighting conditions, and cannot effectively complete the task of detecting and identifying insulator devices under complex backgrounds.

[0005] In recent years, intelligent recognition methods based on deep learning have been widely used in the field of computer vision. In particular, convolutional neural networks learn high-order features of images by repeatedly optimizing their own parameters in a large amount of data, effectively improving the accuracy of image recognition, making it possible to apply deep learning algorithms to the field of transmission line detection. For example, the paper "Multi-target detection of transmission lines based on improved CascadeR-CNN" (Li Xin, Liu Shuannan, Yang Zhen) designed a 6-layer new feature pyramid network to integrate with it, improving the model's ability to detect small target insulator faults; the paper "Insulator defect detection based on multi-scale feature fusion" (Li Bin, Qu Luyao, Zhu Xinshan) uses the SSD model as the basic network, integrates the residual attention module into the backbone feature extraction network to extract the multi-scale features of insulator defects, and uses Focal loss and Gaussian non-maximum suppression methods to improve the detection accuracy of insulator defects. Deep contrast learning shows excellent ability to process image data and has achieved success in image analysis tasks such as target recognition and detection.

[0006] Most of the above deep learning algorithms are based on semi-supervised learning methods, which require a large number of samples as data support, and the samples used for training need to be labeled with damaged areas. When only a small amount of labeled data is available, contrastive learning is more robust than semi-supervised learning. Moreover, it is also robust to class imbalanced data. Therefore, contrastive learning provides a promising direction for realizing intelligent analysis of insulators.

[0007] However, at present, there are few studies on applying contrastive learning to the field of insulator detection, and it is difficult. The biggest difficulty in using contrastive learning in insulator intelligent analysis is that most existing contrastive learning frameworks rely on maximizing the similarity between the two "feature views" of contrast, namely the anchor and its positive sample, which are usually generated by applying random data augmentation methods to the same image. This data augmentation strategy is not easy to directly apply to the field of insulator intelligent analysis, indicating that the current contrastive learning architecture is difficult to achieve an ideal recognition accuracy in insulator detection. Summary of the invention

[0008] The object of the present invention is to provide an intelligent identification method and system for the state of an insulator component to effectively detect the state of the insulator component in order to solve all or part of the above-mentioned problems.

[0009] The technical solution adopted by the present invention is as follows:

[0010] An intelligent identification method for an insulator device state, comprising:

[0011] The process of training the detection model:

[0012] Extracting a first characterizing feature of an insulator device image in a sample set, wherein all insulator device images in the sample set are annotated with a damaged area and an insulator device state;

[0013] Performing a first feature enhancement on the first characterization feature to obtain a first feature view;

[0014] Performing a first classification on the insulator device state according to the first characterization feature;

[0015] Extracting a second characterization feature of a damaged area of ​​the insulator device image;

[0016] Performing a second feature enhancement on the second characterization feature to obtain a second feature view;

[0017] Performing a second classification on the insulator device state according to the second characterization feature;

[0018] The results of the first classification and the second classification are integrated to obtain a final classification of the insulator device status;

[0019] Training the detection model with the goal of maximizing the consistency between the second feature view and the first feature view and minimizing the error between the first classification and the second classification;

[0020] Testing process:

[0021] The trained detection model is used to identify the image of the insulator device to be detected and the state of the insulator device is obtained.

[0022] The present invention also provides an insulator component state intelligent identification system, which is configured with a processor and a computer-readable storage medium, and the processor runs a computer program in the computer-readable storage medium to execute the above-mentioned insulator component state intelligent identification method.

[0023] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0024] The present invention adopts supervised classification learning and weakly supervised contrastive learning methods. To address the problem of imbalanced defect categories in a sample set, the present invention specifically adopts cross entropy loss to classify the states of insulator devices, and specifically designs a weakly supervised deep contrastive learning method to make the model pay more attention to the damaged area. The latter constitutes a feedback loop for enhancing the insulator features (i.e., the first characterization features) by expert knowledge features (i.e., the second characterization features). During the training process, the features of the insulator damaged area are enhanced, so that the feature extraction process of the insulator device state classification pays more attention to the damaged area, thereby improving the accuracy of the model in detecting the insulator device state. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The present invention will now be described by way of example with reference to the accompanying drawings, in which:

[0026] Figure 1 It is a data flow diagram of an intelligent identification method for insulator device status.

[0027] Figure 2 It is the structural diagram of the insulator feature analysis sub-network IFANet.

[0028] Figure 3 This is the IFEi submodule structure diagram.

[0029] Figure 4 This is the FEi unit structure diagram.

[0030] Figure 5 It is a structural diagram of a preferred embodiment of a feature expression subunit.

[0031] Figure 6 It is a structural diagram of a preferred embodiment of a feature update subunit.

[0032] Figure 7 It is a structural diagram of a preferred embodiment of the residual connection submodule.

[0033] Figure 8 It is the structural diagram of the feature extraction network of the expert knowledge feature extraction module EKFEM.

[0034] Fig. 9 This is the structural diagram of the KFEr submodule.

[0035] Fig.10 It is a performance comparison diagram of the present invention and four typical methods for classifying the status of insulator components.

[0036] Fig.11 It is a performance comparison chart of the insulator device state classification ablation experiment. DETAILED DESCRIPTION

[0037] All features disclosed in this specification, or steps in all methods or processes disclosed, except mutually exclusive features and / or steps, can be combined in any manner.

[0038] Any feature disclosed in this specification (including any additional claims and abstract), unless otherwise stated, may be replaced by other equivalent or alternative features having similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

[0039] The method for intelligent identification of the state of insulator devices proposed in the present invention improves the ability to distinguish the state of insulator devices through expert knowledge features, uses expert knowledge features as auxiliary features to enhance the comparative learning effect with insulator features, and designs a feedback loop that enables expert knowledge features to enhance insulator image features, thereby making the detection model pay more attention to the damaged area during the training process, and effectively completing the intelligent identification of the state of insulator devices.

[0040] Example 1

[0041] An insulator device state intelligent recognition method includes a process of training a detection model and a process of using the trained detection model to recognize the insulator device state on an insulator device image.

[0042] The process of training the detection model:

[0043] The insulator features (i.e., the first characterization features) of the insulator device images in the sample set are extracted, and all the insulator device images in the sample set are annotated with damaged areas and insulator states; the insulator features extracted here are the features of the insulator device in the entire insulator device image, and the damaged area is not paid attention to.

[0044] The first feature enhancement is performed on the insulator feature to obtain an insulator image feature view (ie, a first feature view).

[0045] The insulator device state (ie, defect category) is first classified according to the insulator image feature view. The insulator device state generally includes three states: normal, defective, and flashover.

[0046] The expert knowledge features (ie, the second characterization features) of the damaged area of ​​the insulator device image are extracted; the expert knowledge features here are only the features of the damaged area.

[0047] The expert knowledge feature is enhanced by a second feature to obtain an expert knowledge feature view (ie, a second feature view).

[0048] The insulator device status is secondly classified according to the expert knowledge characteristics.

[0049] The results of the first classification and the second classification are combined to obtain the final classification of the insulator device status.

[0050] The detection model is trained with the goal of maximizing the consistency between the expert knowledge feature view and the insulator image feature view and minimizing the error between the first classification and the second classification.

[0051] The above detection model involves the process of extracting, enhancing and classifying insulator features and expert knowledge features respectively. Figure 1As shown in the figure, the detection model is designed to include an insulator feature analysis subnetwork IFANet, an expert knowledge analysis subnetwork EKANet, and an identification result fusion unit IRFU (identification result fusion unit). The former two extract, enhance and classify insulator features and expert knowledge features respectively, and the latter fuses the two classification results and outputs them.

[0052] (1) Insulator feature analysis subnetwork IFANet: It is composed of the insulator feature extraction module IFEM (insulator feature extraction module), the insulator feature mapping module IFMM (insulator feature mapping module) (composed of two multi-layer perceptrons MPL with ReLU) and the insulator classification unit ICU (insulator classification unit).

[0053] (2) Expert knowledge analysis subnetwork EKANet: It consists of an expert knowledge feature extraction module EKFEM (expert knowledge feature extraction module), an expert knowledge feature mapping module EKFMM (expert knowledge feature mapping module) (composed of a multi-layer perceptron MLP with ReLU) and an expert knowledge feature classification unit EKFCU (expert knowledge feature classification unit).

[0054] (3) Identification result fusion unit (IRFU): The output results (probability values ​​of each category) of the insulator classification unit ICU and the expert knowledge feature classification unit EKFCU are weightedly fused according to the category probability values, and the category with the largest category probability value after fusion is taken as the final classification of the insulator device status, which includes three categories: normal insulator, insulator defect, and insulator flashover.

[0055] The insulator feature extraction module IFEM is used to extract the insulator features of the insulator device image; the insulator feature mapping module IFMM performs the first feature enhancement on the insulator features to obtain the insulator image feature view. The insulator classification unit ICU classifies the insulator status into normal, damaged or flashover according to the insulator features.

[0056] The expert knowledge feature extraction module EKFEM is used to extract the expert knowledge features of the damaged area of ​​the insulator device image, and the expert knowledge feature mapping module EKFMM performs a second feature enhancement on the expert knowledge features.

[0057] In the process of training the detection model, for a group of insulator device images, the detection model uses the above two sub-networks to extract two features and construct two feature views: the insulator feature analysis sub-network IFANet extracts insulator features and constructs the insulator image feature view IFView (insulator feature view), and the expert knowledge feature is extracted by the expert knowledge analysis sub-network EKANet, and the expert knowledge feature view EKFView (expert knowledge feature view) is constructed.

[0058] Insulator image feature view IFView, first, the insulator feature extraction module IFEM outputs the image representation IR (image representation) of the insulator device image image, that is, the insulator feature: IR = IFE (image), IFE (·) represents the internal processing operation of the insulator feature extraction module IFEM. Then, the insulator feature mapping module IFMM performs the first feature enhancement on the image representation IR and outputs the image mapping IP (image projection), that is, the insulator image feature view: IP = IFMM (IR).

[0059] Expert knowledge feature view EKFView: First, the expert knowledge feature extraction module EKFEM is used to obtain the expert knowledge feature EKR (expert knowledge representation), EKR = EKFEM (image), EKFEM (·) represents the internal processing operation of the expert knowledge feature extraction module EKFEM. Then the expert knowledge feature mapping module EKFMM outputs the expert knowledge feature view EKP: EKP = EKFMM (EKR), EKFMM (·) represents the internal processing operation of the expert knowledge feature mapping module EKFMM.

[0060] The insulator feature classification unit ICU performs the first classification based on the insulator feature IR. The result of the first classification will be different from the insulator device state marked by the sample, which is called the error loss of the first classification. The expert knowledge feature classification unit EKFCU performs the second classification based on the expert knowledge feature EKR. The result of the second classification will also be different from the insulator device state marked by the sample, which is called the error loss of the second classification.

[0061] Ideally, the insulator image feature view IFView and the expert knowledge feature view EKFView should be highly similar, that is, the insulator image feature view IFView should pay close attention to the damaged area, and the expert knowledge feature view EKFView is the feature view of the damaged area. Therefore, the ideal state of the final trained detection model is to maximize the consistency between the insulator image feature view IFView and the expert knowledge feature view EKFView, that is, to maximize the consistency between IP and EKP; in addition, minimize the classification errors of IR and EKR.

[0062] In this embodiment, the expert knowledge feature is to extract the features of the damaged area in the insulator device image, so only the features of the marked damaged area are extracted, and the area outside is not concerned. The expert knowledge feature view can also reflect the degree of attention paid by the insulator image feature view to the damaged area by comparing the consistency with the insulator image feature view, that is, the expert knowledge feature view can guide the insulator image feature view to pay more attention to the damaged area, so as to enhance the insulator features during the training process, and finally extract the features of the enhanced damaged area.

[0063] The detection model performs a multi-task learning process, including two supervised learning tasks of insulator device state recognition and one comparative learning task of dual-view mutual enhancement.

[0064] The total loss of the detection model is expressed as L, which includes three parts: weakly supervised contrast loss L1, and two-way supervised classification losses L2 and L3. L1 represents the consistency loss between the second feature view and the first feature view, L2 represents the error loss of the first classification, and L3 represents the error loss of the second classification.

[0065] L=β1*L1+β2*L2+β3*L3 (1)

[0066] β1, β2 and β3 are the weights set for loss L1, loss L2 and loss L3 respectively. Their value ranges are all from 0 to 1, and the default values ​​are all 1 / 3.

[0067] 1) Weakly supervised contrast loss L1.

[0068] The weakly supervised contrast loss function utilizes two modal information (insulator image feature view IFView and expert knowledge feature view EKFView). The construction process is as follows: a batch of samples (N) are randomly selected, and a contrast prediction task is defined on the paired augmented samples derived from the batch of samples. Assume that in a batch of samples (N), IFView dp It is the image of the defect area defect and the position pos in the insulator device. Positive sample (IFView dp ,EKFViewdp ) Defined as:

[0069]

[0070] Among them, L2||·|| represents the L2 distance, η dp is a non-zero constant, N is the sample batch size, IFView(v dp ) represents the image feature view corresponding to the damaged area p and the insulator part d, EKFView(v' dp ) refers to the expert knowledge feature view corresponding to the damaged area p and the insulator position d, EKFView(v' kp ) refers to the expert knowledge feature view corresponding to the damage area p and the insulator part k, and M represents the total number of insulator parts in an insulator device image. The weakly supervised contrast loss L1 is calculated on all the damage area-insulator device images in a batch of samples.

[0071]

[0072] 2) Supervised classification losses L2 and L3. The output IR of the insulator feature extraction module IFEM is sent to the insulator classification unit ICU (simple linear classifier) ​​to obtain the supervised classification loss L2; ​​the output EKR of the expert knowledge feature extraction module EKFEM is sent to the expert knowledge feature classification unit EKFCU to obtain the supervised classification loss L3. The supervised classification loss is defined as:

[0073]

[0074] state 1 and state 0 They represent the weights configured for defective samples and normal samples, y' represents the predicted insulator state label, y represents the insulator state label annotated for the sample, y = 1 represents the insulator state of the sample is defective, y = 0 represents the insulator state of the sample is normal, and hard represents the difficulty learning weight configured for samples of different insulator states to drive the model to learn more information from difficult samples. η2 is a small constant used to ensure that L2 and L3 are not zero.

[0075] The present embodiment uses contrastive learning for the following reasons: Contrastive learning is robust in classification tasks with imbalanced datasets. In practice, most insulator device image datasets have an imbalanced number of defect categories. In classical contrastive learning, the two feature perspectives often come from the same field. The difference is that the feature perspectives of the positive samples (defective insulators, including defects and flashovers) of the method of the present embodiment come from knowledge in different fields (existing studies have shown that the feature perspectives from multi-field knowledge should also be consistent), while the feature perspectives of the negative samples (normal insulators) come from knowledge in the same field.

[0076] 1. Positive sampling: In order to obtain a pair of positive sampling feature views, this embodiment randomly selects a labeled image with a specific defect and generates two feature views for it. The first view is its insulator image feature view, and the other view is its expert knowledge feature view (obtained by the expert knowledge feature extraction module EKFEM and enhanced by the expert knowledge feature mapping module EKFMM).

[0077] 2. Negative sampling: In order to identify negative difficult samples, unlike randomly selected images, this embodiment further uses "similar difficult sample" images. Therefore, normal insulator components or images of insulator components of the same type but with different defects are used as negative samples and are called "similar difficult sample" images.

[0078] For the test process after training, only the insulator feature extraction module IFEM, the insulator feature classification unit ICU, the expert knowledge feature extraction module EKFEM, the expert knowledge feature classification unit EKFCU, and the recognition result fusion unit IRFU are retained to act on the input insulator device image.

[0079] like Figure 2 As shown, in some embodiments, the insulator feature extraction module IFEM includes three IFEMs (insulator feature extraction modules) connected in sequence; each feature extraction module IFEMs is composed of two insulator feature extraction sub-modules IFEi connected together.

[0080] like Figure 3 As shown, the IFEi submodule includes three FEi units, and the first FEi unit is connected to the second FEi unit through a residual connection and then connected to the third FEi unit.

[0081] like Figure 4As shown, the FEi unit includes two groups of feature expression subunits FESU and two groups of feature update subunits FUSU. Specifically, the FEi unit includes the first convolution module x, the first feature expression subunit FESU, the first feature update subunit FUSU, the second feature update subunit FUSU, the second feature expression subunit FESU and the second convolution module x in sequence. The first convolution module x is also connected to the first feature update subunit FUSU, the second feature update subunit FUSU and the second feature expression subunit FESU respectively, the first feature expression subunit FESU is connected to the second feature update subunit FUSU, and the second feature update subunit FUSU is connected to the second convolution module x.

[0082] The feature expression subunit FESU consists of 4 residual connection submodules, 7 convolutional layers and 1 ReLU activation layer. Figure 5 As shown, in some embodiments, the feature expression subunit FESU includes a first residual connection submodule (Residual Block), a first convolutional layer (Conv Layer), a second convolutional layer, a second residual connection submodule and a third convolutional layer connected in sequence, and between the first convolutional layer and the second convolutional layer, there are also a third residual connection submodule, a fourth convolutional layer, a fifth convolutional layer, a first Relu activation layer (Relu Layer), a sixth convolutional layer, a seventh convolutional layer and a fourth residual connection submodule connected in sequence, and the fourth convolutional layer is also connected to the seventh convolutional layer.

[0083] The feature update subunit FUSU consists of 2 residual connection submodules, 4 convolutional layers and 1 ReLU activation layer. Figure 6 As shown, in some embodiments, the feature update subunit FUSU includes a fifth residual connection submodule, an eighth convolutional layer, a ninth convolutional layer, a second Relu activation layer, a sixth residual connection submodule, a tenth convolutional layer and an eleventh convolutional layer connected in sequence, the fifth residual connection submodule is also connected to the tenth convolutional layer, and the sixth residual connection submodule is also connected to the eleventh convolutional layer.

[0084] like Figure 7 The figure shows the hierarchical design of a preferred embodiment of the residual connection submodule, which consists of 3 convolutional layers and two Relu activation layers, and also includes two cross-layer shortcut connections. The twelfth convolutional layer, the third Relu activation layer, and the thirteenth convolutional layer are connected in sequence, the fourteenth convolutional layer, the fourth Relu activation layer, and the fifteenth convolutional layer are connected in sequence, and the output of the thirteenth convolutional layer is connected to the input of the residual connection submodule across layers, and then output to the fourteenth convolutional layer in one way, and the output of the fifteenth convolutional layer is connected to the output of the convolutional layer across layers in another way.

[0085] like Figure 8 The figure shows the structure of the expert knowledge feature extraction module EKFEM. Figure 8 As shown in FIG. 1 , the expert knowledge feature extraction module EKFEM includes three KFEMs (knowledge feature extration modules) connected in sequence. The structure of the KFEMs module is the same as that of the above-mentioned IFEMs module. Specifically, Figure 8 As shown, the KFEMs module is composed of two KFEr submodules connected together.

[0086] like Fig. 9 As shown in the figure, the KFEr submodule consists of three knowledge feature extraction units FE k The first FEk unit is connected to the second FEk unit by residual connection and then connected to the third FEk unit. The structure of the FEk unit is as follows Figure 4 shown.

[0087] The core component of the expert knowledge analysis subnetwork EKANet is the expert knowledge feature extraction module EKFEM, which is responsible for extracting the expert knowledge features of the damaged area of ​​the insulator device image.

[0088] The expert knowledge features used are composed of the following two categories of features:

[0089] 1) First-order statistical characteristics: used to measure the distribution of pixel intensity in the damaged area, including energy (measurement of pixel value intensity), entropy (measurement of pixel value uncertainty), and the maximum, average, and median values ​​of pixel grayscale in the area.

[0090] 2) Grayscale features. These include grayscale co-occurrence matrix features, grayscale size region matrix features, grayscale run matrix features, and neighborhood grayscale difference matrix features.

[0091] Example 2

[0092] This embodiment conducts a verification experiment on the method of Embodiment 1 by constructing an insulator device image sample set, and verifies the performance of the method in defect identification.

[0093] The data set used in this experiment contains 20,000 insulator device images. There are three categories of insulator device status labels in the data set, one of which is "Normal", and two types of defect labels, namely Damage and Flashover. There is an obvious imbalance in the number of insulator device images in different states: the number of normal insulator samples (17,000) is far more than the number of defective samples (3,000); the number of different defect categories in the defective samples is also quite different, including 2,250 insulator defect images and 750 insulator flashover images. The hardware configuration for the insulator device status recognition experiment is shown in Table 1.

[0094] Table 1 Hardware configuration of the insulator device defect intelligent detection experiment

[0095]

[0096] The sample set also includes 3,000 damage area bounding boxes (i.e., damage area annotation information) of two defect types (damage and flashover) annotated by insulator inspection engineers for 3,000 defective insulator device images. In the experiment of this embodiment, the insulator device image data set is randomly divided into three subsets: 70% for training, 10% for verification, and 20% for testing, where the normal, defect, and flashover data are divided according to the above ratios, and there is no overlap of insulator devices between all subsets.

[0097] The batch size is set to 8, and the detection model is trained for 150 epochs. The model is optimized by the Adam method, and the initial learning rate is decayed by 10% (starting from 0.001%) every 15 epochs. The weights β1, β2, and β3 of the damage function are all set to 1 / 3. In addition, the relevant models in the proposed method are all implemented based on the PyTorch framework, and four TITAN RTX GPUs are used to train the model.

[0098] Intelligent detection of insulator component defects includes two subtasks:

[0099] (1) Accurately locate the insulator component from the insulator component image;

[0100] (2) Identify the status of insulator components.

[0101] For subtask 1, the YOLO v8 model is used, and its positioning accuracy exceeds 99%. This example focuses on the performance evaluation of subtask 2, and compares it with four existing typical methods, and uses the area under curve (AUC) of the receiver operating characteristic curve (ROC) to evaluate the recognition performance of different methods.

[0102] Table 2 shows the ROC AUC scores of the proposed method (denoted by PROPOSED) for specific defect recognition and the average ROC AUC scores of two defect recognitions, and compares them with four existing typical recognition methods (denoted by C-1 to C-4, respectively). Fig.10 A performance comparison diagram of the method of Example 1 and four typical methods for classifying the status of insulator components is shown.

[0103] Table 2 AUC and average AUC of the comparative experiment of insulator device status recognition

[0104]

[0105] Comparison of existing insulator device status recognition algorithms:

[0106] [C-1]: "Composite insulator defect identification and quantitative method based on random hough transform ellipse detection" proposed by XIANG WZ;

[0107] [C-2]: "A new approach to optimize SVM for insulator state identification based on improved PSO algorithm" proposed by SONG L, LIANG Q, CHEN H and others;

[0108] [C-3]: “Insulator defect detection based on multi-scale feature fusion” proposed by Li Bin, Qu Luyao, Zhu Xinshan, etc.

[0109] [C-4]: “Insulator defect detection algorithm based on improved YOLOv5” proposed by Tang Liang, Yu Minghui, Wu Minghu, etc.

[0110] The experimental results show that the proposed method obtains better ROC AUC scores in the insulator device state recognition experiment, and its overall recognition performance is significantly improved, including the recognition of two defect states: "insulator defect" and "insulator flashover". It shows that by introducing a cross-modal contrast learning branch and a network structure carefully designed for the image features and expert knowledge features of the insulator, effective features can be obtained, and the recognition effect of different types of insulator states has good robustness. The above conclusions show the superiority of the proposed method over the four existing typical methods in the task of insulator device state recognition.

[0111] In addition, this embodiment also carries out an ablation experiment of insulator device state recognition to verify the proposed weighted cross entropy classification loss functions L2 and L3 and knowledge-enhanced contrastive learning to improve the analysis performance of insulator device image data sets with category imbalance, and to verify the rationality and effectiveness of each step in the proposed method.

[0112] Table 3 Performance comparison of ablation experiments for insulator device state identification

[0113]

[0114] This embodiment compares the performance of the basic model (without using the weighted cross entropy classification loss function and without using knowledge enhanced contrastive learning, marked as "P.BASE") with the case where only the weighted cross entropy classification loss functions L2 and L3 (marked as "P.LOSS") or only the knowledge enhanced contrastive learning (marked as "P.KECL") are used. The experimental results are shown in Table 3. Fig.11 The average recognition performance (ROC AUC score) of the ablation experiment is compared (for defects and flashovers at the same time). The ablation experiment results show that the use of the proposed classification losses L2 and L3 and the introduction of knowledge-enhanced contrastive learning significantly improve the performance of the insulator device state recognition task, and the contribution of knowledge-enhanced contrastive learning is greater. Because knowledge-enhanced contrastive learning tends to generate more robust expert knowledge features, it further improves the attention of the insulator feature extraction module IFEM to the defect area.

[0115] Example 3

[0116] This embodiment proposes an insulator device state intelligent identification system, which is configured with a processor and a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and running the program can execute the method of embodiment 1. The processor runs the computer program in the computer-readable storage medium to execute the insulator device state intelligent identification method described in embodiment 1.

[0117] The present invention is not limited to the above-mentioned specific embodiments, but extends to any new features or any new combination disclosed in this specification, as well as any new method or process steps or any new combination disclosed.

Claims

1. An intelligent identification method for insulator device status, characterized in that: include: The process of training the detection model: Extracting a first characterizing feature of an insulator device image in a sample set, wherein all insulator device images in the sample set are annotated with a damaged area and an insulator device state; Performing a first feature enhancement on the first characterization feature to obtain a first feature view; Performing a first classification on the insulator device state according to the first characterization feature; Extracting a second characterization feature of a damaged area of ​​the insulator device image; Performing a second feature enhancement on the second characterization feature to obtain a second feature view; Performing a second classification on the insulator device state according to the second characterization feature; The results of the first classification and the second classification are integrated to obtain a final classification of the insulator device status; Training the detection model with the goal of maximizing the consistency between the second feature view and the first feature view and minimizing the error between the first classification and the second classification; Testing process: The trained detection model is used to identify the image of the insulator device to be detected and the state of the insulator device is obtained.

2. The method for intelligently identifying the state of an insulator device according to claim 1, characterized in that: The fusion of the results of the first classification and the second classification to obtain the final classification of the insulator device state includes: The class probability values ​​of the first classification and the second classification are weightedly fused, and the class with the largest class probability value after fusion is used as the final classification of the insulator device state.

3. The method for intelligently identifying the state of an insulator device according to claim 1, characterized in that: The second characterization features include: first-order statistical features and grayscale features.

4. The method for intelligently identifying the state of an insulator device according to claim 1, characterized in that: The step of training the detection model with the goal of maximizing the consistency between the second feature view and the first feature view and minimizing the error between the first classification and the second classification comprises: The detection model is trained by minimizing the loss of the detection model, where the loss of the detection model is calculated as: L=β1*L1+β2*L2+β3*L3, Wherein, L represents the loss of the detection model, L1 represents the consistency loss between the second feature view and the first feature view, L2 represents the error loss of the first classification, L3 represents the error loss of the second classification, β1, β2 and β3 are weights set for loss L1, loss L2 and loss L3 respectively; there are: In the formula, L2||·|| represents the L2 distance, η dp is a non-zero constant, N is the sample batch size, IFView(v dp ) represents the image feature view corresponding to the damaged area p and the insulator part d, EKFView(v' dp ) refers to the expert knowledge feature view corresponding to the damaged area p and the insulator position d, EKFView(v' kp ) refers to the expert knowledge feature view corresponding to the damaged area p and the insulator part k, and M represents the total number of insulator parts in an insulator device image; state 1 and state 0 They represent the weights configured for defective samples and normal samples respectively, y' represents the predicted insulator state label, y represents the insulator state label annotated for the sample, y = 1 indicates that the insulator state of the sample is defective, y = 0 indicates that the insulator state of the sample is normal, hard is the difficulty learning weight configured for samples with different insulator states, and η2 is a set non-zero constant.

5. The method for intelligently identifying the state of an insulator device according to claim 1, characterized in that: An insulator feature extraction module is used to extract the first characterization feature of the insulator device image, and the insulator feature extraction module includes three IFEMs modules connected in sequence; each of the IFEMs modules is composed of two IFEi sub-modules connected; each IFEi sub-module includes three FEi units, and the first FEi unit is connected to the third FEi unit after the first FEi unit is residually connected to the second FEi unit; each FEi unit includes a first convolution module, a first feature expression subunit, a first feature update subunit, a second feature update subunit, a second feature expression subunit and a second convolution module in sequence, wherein the first convolution module is also connected to the first feature update subunit, the second feature update subunit and the second feature expression subunit respectively, the first feature expression subunit is connected to the second feature update subunit, and the second feature update subunit is connected to the second convolution module.

6. The method for intelligently identifying the state of an insulator device according to claim 5, characterized in that: The feature expression subunit includes a first residual connection submodule, a first convolutional layer, a second convolutional layer, a second residual connection submodule and a third convolutional layer connected in sequence; between the first convolutional layer and the second convolutional layer, the third residual connection submodule, the fourth convolutional layer, the fifth convolutional layer, the first Relu activation layer, the sixth convolutional layer, the seventh convolutional layer and the fourth residual connection submodule are connected in sequence; the fourth convolutional layer is also connected to the seventh convolutional layer.

7. The method for intelligently identifying the state of an insulator device according to claim 5, characterized in that: The feature update subunit includes a fifth residual connection submodule, an eighth convolutional layer, a ninth convolutional layer, a second Relu activation layer, a sixth residual connection submodule, a tenth convolutional layer and an eleventh convolutional layer connected in sequence, the fifth residual connection submodule is also connected to the tenth convolutional layer, and the sixth residual connection submodule is also connected to the eleventh convolutional layer.

8. The method for intelligently identifying the state of an insulator component according to any one of claims 5 to 7, characterized in that: The second characterization feature of the damaged area of ​​the insulator device image is extracted using an expert knowledge feature extraction module, and the expert knowledge feature extraction module includes three KFEMs modules connected in sequence; the structure of the KFEMs module is the same as that of the IFEMs module.

9. An intelligent identification system for insulator device status, the system is configured with a processor and a computer-readable storage medium, characterized in that: The processor runs the computer program in the computer-readable storage medium to execute the method for intelligently identifying the state of an insulator component according to any one of claims 1 to 8.

Images