Power equipment defect detection and autonomous naming method based on unsupervised learning

Abstract

The invention discloses a power equipment defect detection and autonomous naming method based on unsupervised learning, and belongs to the technical field of computer vision. Aiming at the problems that a traditional method neglects local density deviation in a feature space, identification and positioning of defects of power equipment cannot be effectively realized, and meanwhile, a complete transformer substation image data autonomous naming method cannot be formed for collected mass transformer substation image data, the invention provides a synthetic defect module; constructing self-supervised learning by synthesizing defect images to improve the characterization capability of the pre-training network; designing a local sensing feature extraction module to obtain feature representation of a positive sample; a multi-factor KNN-based detection method is provided, and meanwhile, feature space distance and density factors are considered, so that defect detection is realized. The newly collected image data is associated to the established multi-modal transformer substation sample knowledge base, and automatic image data naming is realized, so that the image data processing speed is accelerated, and the accuracy and consistency of image data naming are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a method for detecting and autonomously naming defects of electric power equipment based on unsupervised learning. Background Art

[0002] Defects and hidden dangers in substation equipment will directly threaten the stability and safe operation of the entire transmission line. In a complex external environment, its surface is susceptible to damage, deformation, stains and other faults caused by the natural environment. If it is not detected and measures are not taken in time, it may lead to large-scale power outages and cause serious economic losses. However, due to the relatively small number of defect samples of power equipment and the difficulty in obtaining them, it is difficult to detect surface defects in power equipment. With the rapid development of computer vision, unsupervised learning has become an excellent choice to solve the problem of insufficient defect samples. Taking normal samples as learning targets enables the network to have a strong distribution and reconstruction ability of normal samples, making it possible to carry out defect detection.

[0003] At the same time, in a complex substation environment, the amount of substation equipment image data to be collected is huge, and the data content covers multiple independent power equipment and various types of defects caused by the external environment. Effective management and maintenance of massive substation image data is the key to improving substation operation and maintenance management. However, due to subjective differences and physical limitations, the traditional manual naming method leads to problems such as inconsistent image data naming and high error rate. Introducing AI technology to autonomously analyze the characteristics of equipment and defects in images and realize intelligent naming and classification can effectively reduce the manual burden, effectively improve the speed and accuracy of data processing, reduce manual intervention, and reduce error rates. Summary of the invention

[0004] In order to solve the problem that traditional methods ignore the local density deviation in the feature space, cannot effectively realize the identification and positioning of power equipment defects, and cannot form a complete substation image data autonomous naming method for the collected massive substation image data, the present invention provides a power equipment defect detection and autonomous naming method based on unsupervised learning, which adopts unsupervised learning to realize the detection of surface defects of power equipment.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for detecting defects in power equipment based on unsupervised learning, the method comprising the following steps:

[0007] Step 1: Establish a power equipment dataset; construct a power equipment dataset according to the MVTecAD dataset format. The dataset includes three categories: damage, deformation, and stains. Each category is divided into a training set and a test set. The training set contains only normal images, and the test set contains normal images and abnormal images.

[0008] Step 2: Build an unsupervised learning deep learning model for surface defect detection of power equipment;

[0009] The deep learning model is mainly divided into three stages: self-supervised learning, feature learning, and anomaly detection based on multi-factor KNN;

[0010] In the self-supervised learning stage, the model pre-trained on ImageNet is fine-tuned using normal power equipment images and synthetic power equipment defect images to reduce domain bias;

[0011] In the feature learning stage, unlabeled normal power equipment pictures are used as training images to obtain the feature representation of positive samples;

[0012] In the anomaly detection stage based on multi-factor KNN, the multi-factor KNN detection method is used to obtain the normalized anomaly score for defect detection.

[0013] Furthermore, the specific operations of step 2 are:

[0014] 2.1 Construct a synthetic defect module for self-supervised learning; In order to improve the pre-trained network’s ability to extract deep features of power equipment, the proposed synthetic defect module is used to generate a large number of synthetic defect samples, such as Figure 3 As shown in the figure, the main contour of the power equipment is firstly extracted using the saliency model, and then the defect shape is generated using the B-spline curve. Then two filling methods with different distributions are generated, and finally the shape and filling are fused to generate a synthetic defect image by using the extracted power equipment contour constraints to synthesize the defect area, thereby constructing a self-supervised learning task fine-tuning pre-trained network.

[0015] Furthermore, the specific operation of step 2.1 is:

[0016] Step 2.1.1: In order to constrain the synthetic defect area, the extremely downsampled salient target detection model (EDN) is used as a salient extractor to extract the outline of the power equipment;

[0017] Step 2.1.2: Since the conventional shape is too simple and obviously different from the actual shape, it cannot effectively improve the network's feature extraction ability for actual samples and has poor generalization. Therefore, in order to better simulate natural defects, a B-spline curve with local controllability and smoothness is used to generate defect shapes;

[0018] Step 2.1.3: In order to fill the defect shape, two filling methods with different distributions are used: random noise filling is generated by controlling the mean and variance characteristics of the noise distribution, and a cut filling is obtained by randomly cutting an area from another image in the same data set; then the synthetic defect is directly pasted into the target area by a paste fusion method and the defect is fused with the background by a weighted combination method to enhance the diversity of the synthetic defect samples;

[0019] Step 2.1.4: Synthetic defects with different shapes and fillings are regarded as different categories. A multi-class classification training strategy is adopted to make full use of various types of synthetic defects and improve the representation ability of the model. The loss function is calculated using the cross entropy loss function:

[0020] L=y i *log(p(y i |x i ,Φ))+(1-y i )*log(1-p(y i |DC(x i ),Φ)) (1)

[0021] In the formula, x i represents a normal image, y i represents the label, DC() represents the synthetic defect algorithm, and Φ represents the CNN feature extractor.

[0022] 2.2 Construct a local perception feature extraction module for feature learning; input the image into the network obtained by 2.1, extract the features corresponding to the two adjacent layers in the middle, match the receptive fields of different sizes with the extracted maximum size through aggregation, and obtain the feature representation of the sample;

[0023] The specific operations of step 2.2 are:

[0024] Step 2.2.1: For sample x i Adopting the second level and the third level The middle-layer patch-level features of are used as feature representation;

[0025] When X N represents the normal image used in the training process, where Represents y x = 0 means image x is normal, y x =1 means image x is abnormal;

[0026] At the same time, when X T Represents the sample set provided during testing, where For any image x i ∈{XN , X T}, is x i In the feature representation of the jth layer, C j , H j and W j , represent feature channel, height and width respectively;

[0027] Step 2.2.2: In order to maximize the use of multi-level features, a feature aggregation module is introduced to aggregate multi-level features. First, and Perform local average pooling to obtain a larger receptive field, and then Perform upsampling to make it have the same same resolution; finally, and Aggregate into an integrated feature in the channel dimension, expressed as:

[0028]

[0029] Among them, C * , H * and W * Equal to C 2 +C 3 , H 2 and W 2 , Represents a concatenation operation.

[0030] Step 2.2.3: For a feature map It is the feature corresponding to a patch in the image, and its local perception patch feature set for:

[0031]

[0032] Step 2.2.4: Construct a feature record library M as the feature representation of normal power equipment, and collect the features of normal images into the feature record library M:

[0033]

[0034] 2.3 Core set subsampling: In order to reduce the size of the record library while retaining as much information as possible, the core set subsampling method is used to improve the efficiency of anomaly detection. The purpose of core set selection is to find a subset The solution to the problem on E can be approximated to the solution to the problem on O more quickly. Specifically, the core set is selected using the minimax facility location, and a greedy approximation strategy is used to select a point in the feature record library M each time, and then a farthest point is found within the nearest range of the point, so that any instance in the original feature record library M is consistent with the core set M. C Minimize the maximum distance to the nearest instance in the core set M, ensuring that the core set M is closer to the original feature record library M. C The coverage in the patch-level feature space is roughly similar, so the final core set M C As a feature record library, it is represented as:

[0035]

[0036] In the formula, m represents the feature points in the original feature record library, and n represents the feature points in the core set.

[0037] 2.4 Anomaly detection based on multi-factor KNN method: Aiming at the differences in feature distance and density in the feature record library, a multi-factor KNN algorithm is proposed for anomaly detection. It mainly includes three parts: neighbor distance calculation, local density learning and normalized anomaly detection.

[0038] The specific operations of step 2.4 are:

[0039] Step 2.4.1: In view of the differences in different distance measurement methods, a weighted multi-distance calculation method is proposed. Different weights are set for features of different distances, and Euclidean distance and Manhattan distance are used to calculate the neighboring distance d. i Perform the calculation:

[0040]

[0041] Where, X i , Y i Respectively represent different characteristics;

[0042] The weight coefficient ω of the i-th eigenvalue i for:

[0043]

[0044] In the formula, ε is a small positive number to prevent the denominator from being 0; k represents the number of nearest neighbors;

[0045] The anomaly score sd is calculated using weighted average:

[0046]

[0047] Step 2.4.2: For feature p, among the nearest neighbors to feature p, the distance between the kth nearest feature and feature p is called the k-neighbor distance of feature p, denoted as k-distance(p); the distance to feature p that is less than or equal to k-distance(p) is called the k-nearest-neighbor of feature p, denoted as Nk(p); when the parameter k is given, the reachable distance from feature p to o is reach-dist k (p,o) is the maximum value of the k-neighbor distance of o and the direct distance between feature p and o. The k-th reachable distance from feature p to o is defined as:

[0048] reach-distk(p,o)=max{k-distance(o),d(p,o)} (9)

[0049] In the formula, k-distance(o) represents the distance between the kth nearest neighbor of feature o and feature o, and d(p,o) represents the direct distance between feature p and feature o;

[0050] The local density of each feature is the inverse of the average reachable distance of its k nearest neighbors, denoted as ld mi :

[0051]

[0052] Where Nk(m i ) represents x i The k nearest neighbors of i Represents any feature in the feature record library; m ik Indicates m i Features in k-nearest-neighbor;

[0053] Step 2.4.3: Use the local anomaly factor to represent the local relative density to measure the degree of anomaly of a feature point. The advantage of this is that even if the data distribution is uneven and the density is different, it can still achieve better results. Specifically expressed as:

[0054]

[0055] Where, ld mi Represents the mean of the local density of all features in the k-distance neighborhood of the feature point, Indicates the local density of the point; the lower the density, the smaller the local density, the greater the relative density, that is, the greater the local anomaly factor, the more likely it is an anomaly.

[0056] Step 2.4.4: For the patch feature f to be detected j , search for its nearest neighbors and calculate the Euclidean distance as the patch-level anomaly score:

[0057]

[0058] In the formula, α is the distance hyperparameter, β is the density coefficient, which indicates the degree of density deviation between dense and sparse areas and is a hyperparameter; m j Indicates f in the feature record library M j The nearest neighbor of

[0059] Step 2.4.5: The maximum value of the patch-level anomaly score is the image-level anomaly score

[0060]

[0061] in, Represents x i A collection of patch-level anomaly scores.

[0062] Step 3: Parameter setting and model training; the image size input to the network during training is 256×256 pixels; the batch size is set to 32; the model is initially set to train 300 rounds, and the Early Stopping method is used to save the network parameters during training.

[0063] Step 4: Analysis of power equipment surface defect detection results.

[0064] A method for autonomously naming defects of power equipment based on unsupervised learning, the method comprising the following steps:

[0065] Step 1: Create a power equipment dataset;

[0066] The power equipment dataset is constructed according to the MVTecAD dataset format. The dataset includes three categories: damage, deformation, and stains. Each category is divided into two parts: training set and test set. The training set contains only normal images, and the test set contains normal images and abnormal images.

[0067] Step 2: Build an unsupervised learning deep learning model for surface defect detection of power equipment;

[0068] The deep learning model is mainly divided into three stages: self-supervised learning, feature learning, and anomaly detection based on multi-factor KNN;

[0069] In the self-supervised learning stage, the model pre-trained on ImageNet is fine-tuned using normal power equipment images and synthetic power equipment defect images to reduce domain bias;

[0070] In the feature learning stage, unlabeled normal power equipment pictures are used as training images to obtain the feature representation of positive samples;

[0071] In the anomaly detection stage based on multi-factor KNN, the multi-factor KNN detection method is used to obtain the normalized anomaly score for defect detection;

[0072] The step 2: constructing an unsupervised learning deep learning model for power equipment surface defect detection; using the established power equipment defect data set to train the model, the synthetic defect module generates a large number of synthetic defect sample data sets for fine-tuning the learning effect of the model; the synthetic defect samples are self-supervised learning to enable the model to learn multiple features of the image; the specific operations of inputting the newly collected substation image into the model for testing are:

[0073] 2.1 Construct a synthetic defect module for self-supervised learning; first, use the saliency model to extract the main contour of the power equipment, then use the B-spline curve to generate the defect shape, and then generate two different distribution filling methods. Finally, use the extracted power equipment contour to constrain the synthetic defect area to fuse the shape and filling to generate a synthetic defect image, thereby constructing a self-supervised learning task to fine-tune the pre-trained network;

[0074] 2.2 Construct a local perception feature extraction module for feature learning; input the image into the network obtained by 2.1, extract the features corresponding to the two adjacent layers in the middle, match the receptive fields of different sizes with the extracted maximum size through aggregation, and obtain the feature representation of the sample;

[0075] 2.3 Core set sampling;

[0076] 2.4 Anomaly detection based on multi-factor KNN method; mainly includes three parts: neighbor distance calculation, local density learning and normalized anomaly detection;

[0077] The above 2.1 constructs a synthetic defect module for self-supervised learning; first, the main contour of the power equipment is extracted using a saliency model, then the defect shape is generated using a B-spline curve, and then two filling methods with different distributions are generated. Finally, the shape and filling are fused to generate a synthetic defect image by using the extracted power equipment contour to constrain the synthetic defect area, thereby constructing a self-supervised learning task. The specific operations of fine-tuning the pre-trained network are as follows:

[0078] Step 2.1.1: Use the extreme downsampling salient object detection model as a salient extractor to extract the outline of the power equipment;

[0079] Step 2.1.2: Generate defect shape using B-spline curve with local controllability and smoothness;

[0080] Step 2.1.3: Generate random noise filling by controlling the mean and variance characteristics of the noise distribution, and obtain the cut filling by randomly cutting a region from another image in the same data set; then use two methods to fuse the synthetic defects into the image, namely, a paste fusion method that directly pastes the synthetic defects into the target area and a method that fuses the defects with the background in a weighted combination, to enhance the diversity of the synthetic defect samples;

[0081] Step 2.1.4: Synthetic defects with different shapes and fillings are regarded as different categories. A multi-class classification training strategy is adopted to make full use of various types of synthetic defects and improve the representation ability of the model. The loss function is calculated using the cross entropy loss function:

[0082] L=y i *log(p(y i |x i ,Φ))+(1-y i )*log(1-p(y i |DC(x i ),Φ)) (1)

[0083] In the formula, x i represents a normal image, y i represents the label, DC() represents the synthetic defect algorithm, and Φ represents the CNN feature extractor;

[0084] The 2.2 constructs a local perception feature extraction module for feature learning; the image is input into the network obtained by 2.1, the features corresponding to the two adjacent layers in the middle are extracted, and the receptive fields of different sizes are matched with the extracted maximum size by aggregation to obtain the specific operation of the feature representation of the sample:

[0085] Step 2.2.1: For sample x i Adopting the second level and the third level The middle-layer patch-level features of are used as feature representation;

[0086] When X N represents the normal image used in the training process, where y x ∈{0,1} is represented by y x = 0 means image x is normal, y x =1 means image x is abnormal;

[0087] At the same time, when X T Represents the sample set provided during testing, where For any image x i ∈{X N , X T}, is x i In the feature representation of the jth layer, C j , H j and W j , represent feature channel, height and width respectively;

[0088] Step 2.2.2: Introduce the feature aggregation module to aggregate multi-level features. First, and Perform local average pooling to obtain a larger receptive field, and then Perform upsampling to make it have the same same resolution; finally, and Aggregate into an integrated feature in the channel dimension, expressed as:

[0089]

[0090] Among them, C * , H * and W * Equal to C 2 +C 3 , H 2 and W 2 , Represents a splicing operation;

[0091] Step 2.2.3: For a feature map It is the feature corresponding to a patch in the image, and its local perception patch feature set for:

[0092]

[0093] Step 2.2.4: Construct a feature record library M as the feature representation of normal power equipment, and collect the features of normal images into the feature record library M:

[0094]

[0095] The specific operation of the 2.3 core set subsampling is:

[0096] The core set is selected using the maximum and minimum facility locations, and a greedy approximation strategy is used to select a point in the feature record library M each time, and then a farthest point is found within the nearest range of the point, so that any feature point in the feature record library M is consistent with the core set M. C Minimize the maximum distance of the nearest feature point in the core set M compared to the original feature record library M CThe coverage in the patch-level feature space is roughly similar, so the final core set M C As a feature record library, it is represented as:

[0097]

[0098] In the formula, m represents the feature points in the original feature record library, and n represents the feature points in the core set;

[0099] The above 2.4 performs anomaly detection based on the multi-factor KNN method; it mainly includes three parts: proximity distance calculation, local density learning and normalized anomaly detection. The specific operations are:

[0100] Step 2.4.1: In view of the differences in different distance measurement methods, a weighted multi-distance calculation method is proposed. Different weights are set for features of different distances, and Euclidean distance and Manhattan distance are used to calculate the neighboring distance d. i Perform the calculation:

[0101]

[0102] Where, X i , Y i Respectively represent different characteristics;

[0103] The weight coefficient ω of the i-th eigenvalue i for:

[0104]

[0105] In the formula, ε is a small positive number to prevent the denominator from being 0; k represents the number of nearest neighbors;

[0106] The anomaly score sd is calculated using weighted average:

[0107]

[0108] Step 2.4.2: For feature p, among the nearest neighbors to feature p, the distance between the kth nearest feature and feature p is called the k-neighbor distance of feature p, denoted as k-distance(p); the distance to feature p that is less than or equal to k-distance(p) is called the k-nearest-neighbor of feature p, denoted as Nk(p); when the parameter k is given, the reachable distance from feature p to o is reach-dist k (p,o) is the maximum value of the k-neighbor distance of o and the direct distance between feature p and o. The k-th reachable distance from feature p to o is defined as:

[0109] reach-distk(p,o)=max{k-distance(o),d(p,o)} (9)

[0110] In the formula, k-distance(o) represents the distance between the kth nearest neighbor of feature o and feature o, and d(p,o) represents the direct distance between feature p and feature o;

[0111] The local density of each feature is the inverse of the average reachable distance of its k nearest neighbors, denoted as ld mi :

[0112]

[0113] Where Nk(m i ) represents x i The k nearest neighbors of i Represents any feature in the feature record library; m ik Indicates m i Features in k-nearest-neighbor;

[0114] Step 2.4.3: Use the local anomaly factor to represent the local relative density, specifically expressed as:

[0115]

[0116] Where, ld mi Represents the mean of the local density of all features in the k-distance neighborhood of the feature point, represents the local density of the point;

[0117] Step 2.4.4: For the patch feature f to be detected j , search for its nearest neighbors and calculate the Euclidean distance as the patch-level anomaly score:

[0118]

[0119] In the formula, α is the distance hyperparameter, β is the density coefficient, which indicates the degree of density deviation between dense and sparse areas and is a hyperparameter; m j Indicates f in the feature record library M j The nearest neighbor of

[0120] Step 2.4.5: The maximum value of the patch-level anomaly score is the image-level anomaly score

[0121]

[0122] in, Represents x i The set of patch-level anomaly scores;

[0123] Step 3: Parameter setting and model training; the image size input to the network during training is 256×256 pixels; the batch size is set to 32; the model is initially set to train 300 rounds, and the Early Stopping method is used to save the network parameters during training.

[0124] Step 4: Analysis and self-naming of surface defect detection results of power equipment;

[0125] Self-naming includes the following steps:

[0126] Step 4.1: Establish a multimodal power equipment sample knowledge base; the knowledge base includes text descriptions and corresponding image data;

[0127] Step 4.2: Collect the power equipment defect data set and establish the power equipment defect data set; each type of defect data set is divided into a training set and a test set. The training set contains only normal images, and the test set contains normal images and abnormal images;

[0128] Step 4.3: Construct an unsupervised learning deep learning model for power equipment surface defect detection; use the established power equipment defect dataset to train the model, and the synthetic defect module generates a large number of synthetic defect sample datasets for fine-tuning the learning effect of the model; the synthetic defect samples are self-supervised to enable the model to learn multiple features such as texture, structure, and edge of the image; input the newly collected substation image into the model for testing;

[0129] Step 4.4: Associating with the multimodal power equipment sample knowledge base; Associating the test results with the multimodal power equipment sample knowledge base, and comparing with the image data features therein, so that the newly collected power equipment image is associated with the multimodal power equipment sample knowledge base; after associating with the multimodal power equipment sample knowledge base, the image features are decoded, and the decoding results are consistent with the corresponding image data of the multimodal power equipment sample knowledge base, so as to obtain the corresponding image text information as the basis for autonomous naming of the image.

[0130] Compared with the prior art, the present invention has the following advantages:

[0131] The present invention discloses a method for defect detection and autonomous naming of power equipment based on unsupervised learning. Firstly, the self-supervised learning task is constructed by simulating and generating defect images to reduce domain deviation and improve the representation ability of the pre-trained model; secondly, the patch features of the intermediate adjacent layers are aggregated to obtain the feature representation of the positive sample. At the same time, the core set sampling method is introduced to shorten the abnormal reasoning time; finally, the influence of distance and density in the feature space is comprehensively considered through the detection method based on multi-factor KNN to obtain better abnormality detection performance.

[0132] Compared with the current defect detection methods based on supervised learning and image reconstruction, the advantages of the present invention are mainly reflected in the following four aspects:

[0133] (1) High detection accuracy. Through experiments on a self-built power equipment dataset, it can be seen from Tables 2 and 3 that the method proposed in the present invention has the highest average Im.AUROC and Pi.AUROC in all categories. Im.AUROC is 4.3% higher than SimpleNet, and Pi.AUROC is 4.8% higher than PaDiM, which has higher defect detection accuracy.

[0134] (2) High detection performance: The method of the present invention introduces a core set sampling method to shorten the abnormal reasoning time, that is, it can maintain a high reasoning speed while maintaining high detection accuracy.

[0135] (3) Only a small number of normal samples are needed for training. Typical supervised learning methods require a large number of normal samples and defective samples for training, and data labeling requires a lot of manpower and is prone to human bias. The present invention does not require defective samples during the training process. During the inference process, the difference between the feature distribution of normal samples and defective samples in the feature space is used to determine whether the current area is a defective area.

[0136] (4) Unknown abnormal patterns can be discovered. The present invention does not require pre-labeled abnormal samples, but identifies abnormalities by learning the characteristic distribution of normal samples, so unknown abnormal patterns can be identified, and these patterns can be called abnormalities or defects.

[0137] (5) An autonomous naming method for power equipment defects based on unsupervised learning associates the newly collected image data with the established multimodal substation sample knowledge base to achieve automated image data naming, which not only speeds up image data processing but also improves the accuracy and consistency of image data naming. BRIEF DESCRIPTION OF THE DRAWINGS

[0138] Figure 1 It is a flow chart of a method for detecting defects in power equipment based on unsupervised learning;

[0139] Figure 2 This is the overall framework diagram of the model;

[0140] Figure 3 It is a flow chart of synthetic defects;

[0141] Figure 4 It is a schematic diagram of the local perception feature extraction module;

[0142] Figure 5 This is a flow chart of an autonomous naming method for power equipment defects based on unsupervised learning. DETAILED DESCRIPTION

[0143] In order to gain a deeper understanding of the present invention, we will provide a comprehensive and detailed description of the present invention. However, the present invention has multiple implementations and is not limited to the specific examples listed herein. The presentation of these examples is intended to deepen the comprehensive understanding of the disclosure of the present invention.

[0144] A method for detecting defects in power equipment based on unsupervised learning, the method comprising the following steps:

[0145] Step 1: Establish a power equipment dataset; construct a power equipment dataset according to the MVTec AD dataset format. The dataset includes three categories: damage, deformation, and stains, including 1209 normal images and 163 defective images. Each category is divided into two parts: training set and test set. The training set contains only normal images, and the test set contains normal images and abnormal images. The sample distribution of the power equipment dataset is shown in Table 1.

[0146] Table 1 Distribution of samples of power equipment dataset

[0147]

[0148] Step 2: Construct an unsupervised learning deep learning model for surface defect detection of power equipment; the overall framework is as follows Figure 2 shown.

[0149] The deep learning model is mainly divided into three stages: self-supervised learning, feature learning, and anomaly detection based on multi-factor KNN;

[0150] In the self-supervised learning stage, the model pre-trained on ImageNet is fine-tuned using normal power equipment images and synthetic power equipment defect images to reduce domain bias;

[0151] In the feature learning stage, unlabeled normal power equipment pictures are used as training images to obtain the feature representation of positive samples;

[0152] In the anomaly detection stage based on multi-factor KNN, the multi-factor KNN detection method is used to obtain the normalized anomaly score for defect detection.

[0153] Furthermore, the specific operations of step 2 are:

[0154] 2.1 Construct a synthetic defect module for self-supervised learning; first, use the saliency model to extract the main contour of the power equipment, then use the B-spline curve to generate the defect shape, and then generate two different distribution filling methods. Finally, use the extracted power equipment contour to constrain the synthetic defect area to fuse the shape and filling to generate a synthetic defect image, thereby constructing a self-supervised learning task to fine-tune the pre-trained network;

[0155] Furthermore, the specific operation of step 2.1 is:

[0156] Step 2.1.1: In order to constrain the synthetic defect area, the extremely downsampled salient target detection model (EDN) is used as a salient extractor to extract the outline of the power equipment;

[0157] Step 2.1.2: Since the conventional shape is too simple and significantly different from the actual shape, it cannot effectively improve the network's ability to extract features from actual samples and has poor generalization. Therefore, in order to better simulate natural defects, a B-spline curve with local controllability and smoothness is used to generate defect shapes;

[0158] Step 2.1.3: In order to fill the defect shape, two filling methods with different distributions are used: random noise filling is generated by controlling the mean and variance characteristics of the noise distribution, and a cut filling is obtained by randomly cutting an area from another image in the same data set; then the synthetic defect is directly pasted into the target area by a paste fusion method and the defect is fused with the background by a weighted combination method to enhance the diversity of the synthetic defect samples;

[0159] Step 2.1.4: Synthetic defects with different shapes and fillings are regarded as different categories. A multi-class classification training strategy is adopted to make full use of various types of synthetic defects and improve the representation ability of the model. The loss function is calculated using the cross entropy loss function:

[0160] L=y i *log(p(y i |x i ,Φ))+(1-y i )*log(1-p(y i |DC(x i ),Φ)) (14)

[0161] In the formula, x i represents a normal image, y i represents the label, DC() represents the synthetic defect algorithm, and Φ represents the CNN feature extractor.

[0162] 2.2 Construct a local perception feature extraction module for feature learning; input the image into the network obtained by 2.1, extract the features corresponding to the two adjacent layers in the middle, match the receptive fields of different sizes with the extracted maximum size through aggregation, and obtain the feature representation of the sample;

[0163] The specific operations of step 2.2 are:

[0164] Step 2.2.1: For sample x i Adopting the second level and the third level The middle-layer patch-level features of are used as feature representation; the patch feature extraction process is as follows Figure 4 shown.

[0165] When X N represents the normal image used in the training process, where y x ∈{0,1} represents y x = 0 means image x is normal, y x =1 means image x is abnormal;

[0166] At the same time, when X T Represents the sample set provided during testing, where For any image x i ∈{X N , X T}, is x i In the feature representation of the jth layer, C j , H j and W j , represent feature channel, height and width respectively;

[0167] Step 2.2.2: In order to maximize the use of multi-level features, a feature aggregation module is introduced to aggregate multi-level features. First, and Perform local average pooling to obtain a larger receptive field, and then Perform upsampling to make it have the same same resolution; finally, and Aggregate into an integrated feature in the channel dimension, expressed as:

[0168]

[0169] Among them, C * , H * and W * Equal to C 2 +C 3 , H 2 and W 2 , Represents a concatenation operation.

[0170] Step 2.2.3: For a feature map It is the feature corresponding to a patch in the image, and its local perception patch feature set for:

[0171]

[0172] Step 2.2.4: Construct a feature record library M as the feature representation of normal power equipment, and collect the features of normal images into the feature record library M:

[0173]

[0174] 2.3 Core set subsampling: In order to reduce the size of the record library while retaining as much information as possible, the core set subsampling method is used to improve the efficiency of anomaly detection. The purpose of core set selection is to find a subset The solution to the problem on E can be approximated to the solution to the problem on O more quickly. Specifically, the core set is selected using the minimax facility location, and a greedy approximation strategy is used to select a point in the feature record library M each time, and then a farthest point is found within the nearest range of the point, so that any instance in the original feature record library M is consistent with the core set M. C Minimize the maximum distance to the nearest instance in the core set M, ensuring that the core set M is closer to the original feature record library M. C The coverage in the patch-level feature space is roughly similar, so the final core set M C As a feature record library, it is represented as:

[0175]

[0176] In the formula, m represents the feature points in the original feature record library, and n represents the feature points in the core set.

[0177] 2.4 Anomaly detection based on multi-factor KNN method: Aiming at the differences in feature distance and density in the feature record library, a multi-factor KNN algorithm is proposed for anomaly detection. It mainly includes three parts: neighbor distance calculation, local density learning and normalized anomaly detection.

[0178] The specific operations of step 2.4 are:

[0179] Step 2.4.1: In view of the differences in different distance measurement methods, a weighted multi-distance calculation method is proposed. Different weights are set for features of different distances, and Euclidean distance and Manhattan distance are used to calculate the neighboring distance d. i Perform the calculation:

[0180]

[0181] Where, X i , Y iRespectively represent different characteristics;

[0182] The weight coefficient ω of the i-th eigenvalue i for:

[0183]

[0184] In the formula, ε is a small positive number to prevent the denominator from being 0; k represents the number of nearest neighbors;

[0185] The anomaly score sd is calculated using weighted average:

[0186]

[0187] Step 2.4.2: For feature p, among the nearest neighbors to feature p, the distance between the kth nearest feature and feature p is called the k-neighbor distance of feature p, denoted as k-distance(p); the distance to feature p that is less than or equal to k-distance(p) is called the k-nearest-neighbor of feature p, denoted as Nk(p); when the parameter k is given, the reachable distance from feature p to o is reach-dist k (p,o) is the maximum value of the k-neighbor distance of o and the direct distance between feature p and o. The k-th reachable distance from feature p to o is defined as:

[0188] reach-distk(p,o)=max{k-distance(o),d(p,o)} (22)

[0189] In the formula, k-distance(o) represents the distance between the kth nearest neighbor of feature o and feature o, and d(p,o) represents the direct distance between feature p and feature o;

[0190] The local density of each feature is the inverse of the average reachable distance of its k nearest neighbors, denoted as ld mi :

[0191]

[0192] Where Nk(m i ) represents x i The k nearest neighbors of i Represents any feature in the feature record library; m ik Indicates m i Features in k-nearest-neighbor;

[0193] Step 2.4.3: Use the local anomaly factor to represent the local relative density to measure the degree of anomaly of a feature point. The advantage of this is that even if the data distribution is uneven and the density is different, it can still achieve better results. Specifically expressed as:

[0194]

[0195] Where, ld mi Represents the mean of the local density of all features in the k-distance neighborhood of the feature point, Indicates the local density of the point; the lower the density, the smaller the local density, the greater the relative density, that is, the greater the local anomaly factor, the more likely it is an anomaly.

[0196] Step 2.4.4: For the patch feature f to be detected j , search for its nearest neighbors and calculate the Euclidean distance as the patch-level anomaly score:

[0197]

[0198] In the formula, α is the distance hyperparameter, β is the density coefficient, which indicates the degree of density deviation between dense and sparse areas and is a hyperparameter; m j Indicates f in the feature record library M j The nearest neighbor of

[0199] Step 2.45: The maximum value of the patch-level anomaly score is the image-level anomaly score

[0200]

[0201] in, Represents x i A collection of patch-level anomaly scores.

[0202] Step 3: Parameter setting and model training; the image size input to the network during training is 256×256 pixels; the batch size is set to 32; the model is initially set to train 300 rounds, and the Early Stopping method is used to save the network parameters during training.

[0203] Step 4: Analysis of power equipment surface defect detection results.

[0204] A method for autonomously naming defects of power equipment based on unsupervised learning, the method comprising the following steps:

[0205] Step 1: Establish a power equipment dataset; construct a power equipment dataset according to the MVTecAD dataset format. The dataset includes three categories: damage, deformation, and stains. Each category is divided into a training set and a test set. The training set contains only normal images, and the test set contains normal images and abnormal images.

[0206] Step 2: Build an unsupervised learning deep learning model for surface defect detection of power equipment;

[0207] The deep learning model is mainly divided into three stages: self-supervised learning, feature learning, and anomaly detection based on multi-factor KNN. In the self-supervised learning stage, the model pre-trained on ImageNet is fine-tuned using normal power equipment images and synthetic power equipment defect images to reduce domain bias. In the feature learning stage, unlabeled normal power equipment images are used as training images to obtain feature representations of positive samples. In the anomaly detection stage based on multi-factor KNN, a multi-factor KNN detection method is used to obtain a normalized anomaly score for defect detection.

[0208] Step 3: Parameter setting and model training;

[0209] Step 4: Analysis of the detection results of surface defects of power equipment and self-naming; self-naming includes the following steps:

[0210] Step 4.1: Establish a multimodal power equipment sample knowledge base; the knowledge base includes text descriptions and corresponding image data;

[0211] Step 4.2: Collect the power equipment defect data set and establish the power equipment defect data set; each type of defect data set is divided into a training set and a test set. The training set contains only normal images, and the test set contains normal images and abnormal images;

[0212] Step 4.3: Construct an unsupervised learning deep learning model for power equipment surface defect detection; use the established power equipment defect dataset to train the model, and the synthetic defect module generates a large number of synthetic defect sample datasets for fine-tuning the learning effect of the model; the synthetic defect samples are self-supervised to enable the model to learn multiple features such as texture, structure, and edge of the image; input the newly collected substation image into the model for testing;

[0213] Step 4.4: Associating with the multimodal power equipment sample knowledge base; Associating the test results with the multimodal power equipment sample knowledge base, and comparing with the image data features therein, so that the newly collected power equipment image is associated with the multimodal power equipment sample knowledge base; after associating with the multimodal power equipment sample knowledge base, the image features are decoded, and the decoding results are consistent with the corresponding image data of the multimodal power equipment sample knowledge base, so as to obtain the corresponding image text information as the basis for autonomous naming of the image.

[0214] The detection results are compared by measuring the Im.AUROC and Pi.AUROC of different defect detection methods and the method proposed in the present invention on the self-built power equipment data set, as shown in Table 2 and Table 3. The experimental results show that the average Im.AUROC and Pi.AUROC of the method proposed in the present invention are the highest in all categories, with the average values ​​reaching 94.1% and 96.6%, showing better performance in the surface defect detection of power equipment.

[0215] Table 2 Im.AUROC% of different methods on power equipment dataset

[0216]

[0217] Table 3 Pi.AUROC% of different methods on the power equipment dataset

[0218]

[0219] In order to verify the influence of different feature representation methods on anomaly detection results, experiments were conducted on a self-built power equipment dataset, and the experimental results are shown in Table 4. It can be seen from the results that the proposed method simulates the generation of real defects and increases the diversity of synthetic defects through random noise and cut-and-fill, so that the backbone network obtains better feature extraction capabilities and improves the detection performance.

[0220] Table 4 Defect detection results under different feature representation methods %

[0221]

[0222] In addition to the traditional KNN, the effects of other KNN variants on the detection results are also compared, as shown in Table 5. It can be seen that the multi-factor based KNN method proposed by the method of the present invention comprehensively considers the feature distance and relative density factors, and well balances the influence of the two, thereby achieving better detection results.

[0223] Table 5 Detection results under different KNN methods %

[0224]

[0225] In order to verify that core set subsampling can reduce the anomaly detection reasoning time, different sampling ratios of the feature record library (n% represents the percentage of the original feature record library sampled) and SPADE and PaDiM are used to experiment on the self-built power equipment data set. The detection results are shown in Table 6. From the results, it can be seen that as the subsampling ratio decreases, the detection time is shortened, but the detection performance also decreases slightly. Therefore, using a 10% subsampling ratio can maintain a shorter detection time while maintaining a high detection accuracy.

[0226] Table 6 Average detection time per image in the power equipment dataset

[0227]

[0228] The contents not described in detail in the specification of the present invention belong to the prior art known to the professional and technical personnel in the field. Although the illustrative specific embodiments of the present invention are described above to facilitate the understanding of the present invention by the technical personnel in the field, it should be clear that the present invention is not limited to the scope of the specific embodiments. For the ordinary technical personnel in the field, as long as various changes are within the spirit and scope of the present invention defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A method for detecting defects in power equipment based on unsupervised learning, characterized in that: The method comprises the following steps: Step 1: Create a power equipment dataset; The power equipment dataset is constructed according to the MVTecAD dataset format. The dataset includes three categories: damage, deformation, and stains. Each category is divided into two parts: training set and test set. The training set contains only normal images, and the test set contains normal images and abnormal images. Step 2: Build an unsupervised learning deep learning model for surface defect detection of power equipment; The deep learning model is mainly divided into three stages: self-supervised learning, feature learning, and anomaly detection based on multi-factor KNN; In the self-supervised learning stage, the model pre-trained on ImageNet is fine-tuned using normal power equipment images and synthetic power equipment defect images to reduce domain bias; In the feature learning stage, unlabeled normal power equipment pictures are used as training images to obtain the feature representation of positive samples; In the anomaly detection stage based on multi-factor KNN, the multi-factor KNN detection method is used to obtain the normalized anomaly score for defect detection; Step 3: Parameter setting and model training; Step 4: Analysis of power equipment surface defect detection results.

2. The method for detecting defects in electric power equipment based on unsupervised learning according to claim 1, characterized in that: The specific operation of step 2: constructing an unsupervised learning deep learning model for surface defect detection of power equipment is as follows: 2.1 Construct a synthetic defect module for self-supervised learning; first, use the saliency model to extract the main contour of the power equipment, then use the B-spline curve to generate the defect shape, and then generate two different distribution filling methods. Finally, use the extracted power equipment contour to constrain the synthetic defect area to fuse the shape and filling to generate a synthetic defect image, thereby constructing a self-supervised learning task to fine-tune the pre-trained network; 2.2 Construct a local perception feature extraction module for feature learning; input the image into the network obtained by 2.1, extract the features corresponding to the two adjacent layers in the middle, match the receptive fields of different sizes with the extracted maximum size through aggregation, and obtain the feature representation of the sample; 2.3 Core set sampling; 2.4 Anomaly detection based on multi-factor KNN method; mainly includes three parts: proximity distance calculation, local density learning and normalized anomaly detection.

3. The method for detecting defects in electric power equipment based on unsupervised learning according to claim 2, characterized in that: The above 2.1 constructs a synthetic defect module for self-supervised learning; first, the main contour of the power equipment is extracted using a saliency model, then the defect shape is generated using a B-spline curve, and then two filling methods with different distributions are generated. Finally, the shape and filling are fused to generate a synthetic defect image by using the extracted power equipment contour to constrain the synthetic defect area, thereby constructing a self-supervised learning task. The specific operations of fine-tuning the pre-trained network are as follows: Step 2.1.1: Use the extreme downsampling salient object detection model as a salient extractor to extract the outline of the power equipment; Step 2.1.2: Generate defect shape using B-spline curve with local controllability and smoothness; Step 2.1.3: Generate random noise filling by controlling the mean and variance characteristics of the noise distribution, and obtain the cut filling by randomly cutting a region from another image in the same data set; then use two methods to fuse the synthetic defects into the image, namely, a paste fusion method that directly pastes the synthetic defects into the target area and a method that fuses the defects with the background in a weighted combination, to enhance the diversity of the synthetic defect samples; Step 2.1.4: Synthetic defects with different shapes and fillings are regarded as different categories. A multi-class classification training strategy is adopted to make full use of various types of synthetic defects and improve the representation ability of the model. The loss function is calculated using the cross entropy loss function: Ly i *log(p(y i |x i ,Φ))+(1-y i )*log(1-p(y i |DC(x i ),Φ)) (1) In the formula, x i represents a normal image, y i represents the label, DC() represents the synthetic defect algorithm, and Φ represents the CNN feature extractor.

4. The method for detecting defects in electric power equipment based on unsupervised learning according to claim 3, characterized in that: The 2.2 constructs a local perception feature extraction module for feature learning; the image is input into the network obtained by 2.1, the features corresponding to the two adjacent layers in the middle are extracted, and the receptive fields of different sizes are matched with the extracted maximum size by aggregation to obtain the specific operation of the feature representation of the sample: Step 2.2.1: For sample x i Adopt the second level and the third level The middle-layer patch-level features of are used as feature representation; When X N represents the normal image used in the training process, where y x ∈{0,1} is represented by y x = 0 means image x is normal, y x =1 means image x is abnormal; At the same time, when X T Represents the sample set provided during testing, where For any image x i ∈{X N , X T }, is x i In the feature representation of the jth layer, C j , H j and W j , represent feature channel, height and width respectively; Step 2.2.2: Introduce the feature aggregation module to aggregate multi-level features. First, and Perform local average pooling to obtain a larger receptive field, and then Perform upsampling to make it have the same same resolution; finally, and Aggregate into an integrated feature in the channel dimension, expressed as: Among them, C * , H * and W * Equal to C 2 +C 3 , H 2 and W 2 , Represents a splicing operation; Step 2.2.3: For a feature map It is the feature corresponding to a patch in the image, and its local perception patch feature set for: Step 2.2.4: Construct a feature record library M as the feature representation of normal power equipment, and collect the features of normal images into the feature record library M:

5. The method for detecting defects in electric power equipment based on unsupervised learning according to claim 4, characterized in that: The specific operation of the 2.3 core set subsampling is: Use the maximum and minimum facility locations to select the core set, use the greedy approximation strategy to select a point in the feature record library M each time, and then find the farthest point in the nearest range of the point, so that any feature point in the feature record library M and the core set M C Minimize the maximum distance of the nearest feature point in the core set M compared to the original feature record library M C The coverage in the patch-level feature space is roughly similar, so the final core set M C As a feature record library, it is represented as: In the formula, m represents the feature points in the original feature record library, and n represents the feature points in the core set.

6. The method for detecting defects in electric power equipment based on unsupervised learning according to claim 5, characterized in that: The above 2.4 performs anomaly detection based on the multi-factor KNN method; it mainly includes three parts: proximity distance calculation, local density learning and normalized anomaly detection. The specific operations are: Step 2.4.1: In view of the differences in different distance measurement methods, a weighted multi-distance calculation method is proposed. Different weights are set for features of different distances, and Euclidean distance and Manhattan distance are used to calculate the neighboring distance d. i Perform the calculation: In the formula, X i , Y i Respectively represent different characteristics; The weight coefficient ω of the i-th eigenvalue i for: In the formula, ε is a small positive number to prevent the denominator from being 0; k represents the number of nearest neighbors; The anomaly score sd is calculated using weighted average: Step 2.4.2: For feature p, among the nearest neighbors to feature p, the distance between the kth nearest feature and feature p is called the k-neighbor distance of feature p, denoted as k-distance(p); the distance to feature p that is less than or equal to k-distance(p) is called the k-nearest-neighbor of feature p, denoted as Nk(p); when the parameter k is given, the reachable distance from feature p to o is reach-dist k (p,o) is the maximum value of the k-neighbor distance of o and the direct distance between feature p and o. The k-th reachable distance from feature p to o is defined as: reach-distk(p,o)=max{k-distance(o),d(p,o)} (9) In the formula, k-distance(o) represents the distance between the kth nearest neighbor of feature o and feature o, and d(p,o) represents the direct distance between feature p and feature o; The local density of each feature is the inverse of the average reachable distance of its k nearest neighbors, denoted as ld mi : Where Nk(m i ) represents x i The k nearest neighbors of i Represents any feature in the feature record library; m ik Indicates m i Features in k-nearest-neighbor; Step 2.4.3: Use the local anomaly factor to represent the local relative density, specifically expressed as: Where, ld mi Represents the mean of the local density of all features in the k-distance neighborhood of the feature point, represents the local density of the point; Step 2.4.4: For the patch feature f to be detected j , search for its nearest neighbors and calculate the Euclidean distance as the patch-level anomaly score: In the formula, α is the distance hyperparameter, β is the density coefficient, which indicates the degree of density deviation between dense and sparse areas and is a hyperparameter; m j Indicates f in the feature record library M j The nearest neighbor of Step 2.4.5: The maximum value of the patch-level anomaly score is the image-level anomaly score in, Represents x i A collection of patch-level anomaly scores.

7. The method for detecting defects in electric power equipment based on unsupervised learning according to claim 1, characterized in that: The specific operations of step 3: parameter setting and model training are as follows: the image size of the input network during training is 256×256 pixels; the batch size is set to 32; the model is initially set to train 300 rounds, and the Early Stopping method is used to save the network parameters during the training process.

8. A method for autonomously naming defects of power equipment based on unsupervised learning, characterized in that: The method comprises the following steps: Step 1: Create a power equipment dataset; The power equipment dataset is constructed according to the MVTecAD dataset format. The dataset includes three categories: damage, deformation, and stains. Each category is divided into two parts: training set and test set. The training set contains only normal images, and the test set contains normal images and abnormal images. Step 2: Build an unsupervised learning deep learning model for surface defect detection of power equipment; The deep learning model is mainly divided into three stages: self-supervised learning, feature learning, and anomaly detection based on multi-factor KNN; In the self-supervised learning stage, the model pre-trained on ImageNet is fine-tuned using normal power equipment images and synthetic power equipment defect images to reduce domain bias; In the feature learning stage, unlabeled normal power equipment pictures are used as training images to obtain the feature representation of positive samples; In the anomaly detection stage based on multi-factor KNN, the multi-factor KNN detection method is used to obtain the normalized anomaly score for defect detection; Step 3: Parameter setting and model training; Step 4: Analyze and name the surface defect detection results of power equipment.

9. The method for autonomously naming defects of power equipment based on unsupervised learning according to claim 8 is characterized in that: The step 4 of analyzing the detection results of the surface defects of the power equipment and naming them autonomously, wherein the autonomous naming includes the following steps: Step 4.1: Establish a multimodal power equipment sample knowledge base; the knowledge base includes text descriptions and corresponding image data; Step 4.2: Collect the power equipment defect data set and establish the power equipment defect data set; each type of defect data set is divided into a training set and a test set. The training set contains only normal images, and the test set contains normal images and abnormal images; Step 4.3: Construct an unsupervised learning deep learning model for power equipment surface defect detection; use the established power equipment defect dataset to train the model, and the synthetic defect module generates a large number of synthetic defect sample datasets for fine-tuning the learning effect of the model; the synthetic defect samples are self-supervised to enable the model to learn multiple features of the image; the newly collected substation images are input into the model for testing; Step 4.4: Associating with the multimodal power equipment sample knowledge base; Associating the test results with the multimodal power equipment sample knowledge base, and comparing with the image data features therein, so that the newly collected power equipment image is associated with the multimodal power equipment sample knowledge base; after associating with the multimodal power equipment sample knowledge base, the image features are decoded, and the decoding results are consistent with the corresponding image data of the multimodal power equipment sample knowledge base, so as to obtain the corresponding image text information as the basis for autonomous naming of the image.

10. The method for autonomously naming defects of power equipment based on unsupervised learning according to claim 9 is characterized in that: The step 2: constructing an unsupervised learning deep learning model for surface defect detection of power equipment; The model is trained using the established power equipment defect dataset. The synthetic defect module generates a large number of synthetic defect sample datasets for fine-tuning the learning effect of the model. The synthetic defect samples are used for self-supervised learning to enable the model to learn multiple features of the image. The specific operations of inputting the newly collected substation image into the model for testing are as follows: 2.1 Construct a synthetic defect module for self-supervised learning; first, use the saliency model to extract the main contour of the power equipment, then use the B-spline curve to generate the defect shape, and then generate two different distribution filling methods. Finally, use the extracted power equipment contour to constrain the synthetic defect area to fuse the shape and filling to generate a synthetic defect image, thereby constructing a self-supervised learning task to fine-tune the pre-trained network; 2.2 Construct a local perception feature extraction module for feature learning; input the image into the network obtained by 2.1, extract the features corresponding to the two adjacent layers in the middle, match the receptive fields of different sizes with the extracted maximum size through aggregation, and obtain the feature representation of the sample; 2.3 Core set sampling; 2.4 Anomaly detection based on multi-factor KNN method; mainly includes three parts: neighbor distance calculation, local density learning and normalized anomaly detection; The above 2.1 constructs a synthetic defect module for self-supervised learning; first, the main contour of the power equipment is extracted using a saliency model, then the defect shape is generated using a B-spline curve, and then two filling methods with different distributions are generated. Finally, the shape and filling are fused to generate a synthetic defect image by using the extracted power equipment contour to constrain the synthetic defect area, thereby constructing a self-supervised learning task. The specific operations of fine-tuning the pre-trained network are as follows: Step 2.1.1: Use the extreme downsampling salient object detection model as a salient extractor to extract the outline of the power equipment; Step 2.1.2: Generate defect shape using B-spline curve with local controllability and smoothness; Step 2.1.3: Generate random noise filling by controlling the mean and variance characteristics of the noise distribution, and obtain the cut filling by randomly cutting a region from another image in the same data set; then use two methods to fuse the synthetic defects into the image, namely, a paste fusion method that directly pastes the synthetic defects into the target area and a method that fuses the defects with the background in a weighted combination, to enhance the diversity of the synthetic defect samples; Step 2.1.4: Synthetic defects with different shapes and fillings are regarded as different categories. A multi-class classification training strategy is adopted to make full use of various types of synthetic defects and improve the representation ability of the model. The loss function is calculated using the cross entropy loss function: Ly i *log(p(y i |x i ,Φ))+(1-y i )*log(1-p(y i |DC(x i ),Φ)) (1) In the formula, x i represents a normal image, y i represents the label, DC() represents the synthetic defect algorithm, and Φ represents the CNN feature extractor; The 2.2 constructs a local perception feature extraction module for feature learning; the image is input into the network obtained by 2.1, the features corresponding to the two adjacent layers in the middle are extracted, and the receptive fields of different sizes are matched with the extracted maximum size by aggregation to obtain the specific operation of the feature representation of the sample: Step 2.2.1: For sample x i Adopt the second level and the third level The intermediate layer patch-level features of are used as feature representation; When X N represents the normal image used in the training process, where y x ∈{0,1} is represented by y x = 0 means image x is normal, y x =1 means image x is abnormal; At the same time, when X T Represents the sample set provided during testing, where For any image x i ∈{X N , X T }, is x i In the feature representation of the jth layer, C j , H j and W j , represent feature channel, height and width respectively; Step 2.2.2: Introduce the feature aggregation module to aggregate multi-level features. First, and Perform local average pooling to obtain a larger receptive field, and then Perform upsampling operation to make it have the same same resolution; finally, and Aggregate into an integrated feature in the channel dimension, expressed as: Among them, C * , H * and W * Equal to C 2 +C 3 , H 2 and W 2 , Represents a splicing operation; Step 2.2.3: For a feature map It is the feature corresponding to a patch in the image, and its local perception patch feature set for: Step 2.2.4: Construct a feature record library M as the feature representation of normal power equipment, and collect the features of normal images into the feature record library M: The specific operation of the 2.3 core set subsampling is: Use the maximum and minimum facility locations to select the core set, use the greedy approximation strategy to select a point in the feature record library M each time, and then find the farthest point in the nearest range of the point, so that any feature point in the feature record library M and the core set M C Minimize the maximum distance of the nearest feature point in the core set M compared to the original feature record library M C The coverage in the patch-level feature space is roughly similar, so the final core set M C As a feature record library, it is represented as: In the formula, m represents the feature points in the original feature record library, and n represents the feature points in the core set; The above 2.4 performs anomaly detection based on the multi-factor KNN method; it mainly includes three parts: proximity distance calculation, local density learning and normalized anomaly detection. The specific operations are: Step 2.4.1: In view of the differences in different distance measurement methods, a weighted multi-distance calculation method is proposed. Different weights are set for features of different distances, and Euclidean distance and Manhattan distance are used to calculate the neighboring distance d. i Perform the calculation: In the formula, X i , Y i Respectively represent different characteristics; The weight coefficient ω of the i-th eigenvalue i for: In the formula, ε is a small positive number to prevent the denominator from being 0; k represents the number of nearest neighbors; The anomaly score sd is calculated using weighted average: Step 2.4.2: For feature p, among the nearest neighbors to feature p, the distance between the kth nearest feature and feature p is called the k-neighbor distance of feature p, denoted as k-distance(p); the distance to feature p that is less than or equal to k-distance(p) is called the k-nearest-neighbor of feature p, denoted as Nk(p); when the parameter k is given, the reachable distance from feature p to o is reach-dist k (p,o) is the maximum value of the k-neighbor distance of o and the direct distance between feature p and o. The k-th reachable distance from feature p to o is defined as: reach-distk(p,o)=max{k-distance(o),d(p,o)} (9) In the formula, k-distance(o) represents the distance between the kth nearest neighbor of feature o and feature o, and d(p,o) represents the direct distance between feature p and feature o; The local density of each feature is the inverse of the average reachable distance of its k nearest neighbors, denoted as ld mi : Where Nk(m i ) represents x i The k nearest neighbors of i Represents any feature in the feature record library; m ik Indicates m i Features in k-nearest-neighbor; Step 2.4.3: Use the local anomaly factor to represent the local relative density, specifically expressed as: Where, ld mi Represents the mean of the local density of all features in the k-distance neighborhood of the feature point, represents the local density of the point; Step 2.4.4: For the patch feature f to be detected j , search for its nearest neighbors and calculate the Euclidean distance as the patch-level anomaly score: In the formula, α is the distance hyperparameter, β is the density coefficient, which indicates the degree of density deviation between dense and sparse areas and is a hyperparameter; m j Indicates f in the feature record library M j The nearest neighbor of Step 2.4.5: The maximum value of the patch-level anomaly score is the image-level anomaly score in, Represents x i The set of patch-level anomaly scores; The specific operations of step 3: parameter setting and model training are as follows: the image size of the input network during training is 256×256 pixels; the batch size is set to 32; the model is initially set to train 300 rounds, and the EarlyStopping method is used to save the network parameters during the training process.

Images