Power equipment defect classification method based on confidence enhancement and image reconstruction

Abstract

The invention proposes a power equipment defect classification method based on confidence enhancement and image reconstruction, and relates to the technical field of machine learning, and the method comprises the steps: dividing a collected image data sample into a support set and a query set, and inputting a feature extractor to obtain sample features; performing feature enhancement on sample features of the support set, then adding a mask and performing reconstruction, and calculating reconstruction loss; extracting a sample label, and encoding the sample label by using a maximum equiangular encoder to obtain label embedding; splicing the sample features of the query set and the sample enhancement features of the support set to obtain spliced feature embedding, splicing the spliced feature embedding and the label embedding, inputting the spliced feature embedding and the label embedding into a non-causal sequence to obtain new sample features of the query set, inputting the new sample features into a classifier to obtain a classification prediction result, and calculating classification loss; constructing a loss function model according to the reconstruction loss and the classification loss to realize classification; according to the method, the classification performance of the model is improved by fully utilizing the local features and global label information of the image.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and in particular to a method for classifying defects of electric power equipment based on confidence enhancement and image reconstruction. Background Art

[0002] At present, the power system has a great demand for equipment maintenance. Relying solely on manpower cannot guarantee comprehensive and reliable maintenance. Now the deep learning method is introduced to improve the efficiency of power equipment maintenance. However, power equipment is in normal operation in most cases, and the number of defect images that can be collected is relatively small. Traditional deep learning methods are mainly used to solve the problem of balancing the number of categories. They cannot be fully trained on small-scale defect image data and are prone to underfitting. Therefore, many existing studies are based on small sample methods to solve the problem of defect classification of power equipment. This not only reduces the cost and difficulty of collecting power equipment image data, but also improves the generalization ability and robustness of the model.

[0003] Although many small-sample learning methods have made some progress in the classification of power equipment defect images, especially the transfer learning-based methods, which use the general feature representation capabilities learned in the pre-training stage to generalize them to the target domain. However, the current small-sample power equipment defect classification methods based on transfer learning are all pre-trained on defect-free power data. Since defect-free power data has a single scene and lacks diversity, and the model is prone to over-reliance on specific representations to distinguish different categories when trained on small-sample data, it is impossible to learn the intrinsic structure and distribution of features, resulting in a decrease in classification performance. Summary of the invention

[0004] The purpose of the present invention is to provide a method for classifying defects in electric power equipment based on confidence enhancement and image reconstruction, which significantly improves the classification performance of the model by making full use of local features of the image and global label information.

[0005] The technical solution of the present invention is:

[0006] In a first aspect, the present application provides a method for classifying defects of power equipment based on confidence enhancement and image reconstruction, which comprises the following steps:

[0007] S1, obtain the original image data sample, and divide the image data sample into a support set and a query set;

[0008] S2, inputting the image data samples of the support set and the query set into the feature extractor to obtain the sample features of the support set and the query set, and performing feature enhancement on the sample features of the support set to obtain the sample enhanced features of the support set;

[0009] S3, add masks to the sample enhancement features of the support set and reconstruct them, and calculate the reconstruction loss;

[0010] S4, extracting sample labels according to the support set and the query set, and encoding the sample labels using a maximum equiangular encoder to obtain label embedding;

[0011] S5, concatenating the sample features of the query set with the sample enhancement features of the support set to obtain a concatenated feature embedding, and concatenating the concatenated feature embedding with the label embedding and inputting the concatenated feature embedding into the non-causal sequence to obtain a new sample feature of the query set;

[0012] S6. Input the new sample features of the query set into the classifier to obtain the classification prediction result and calculate the classification loss;

[0013] S7. Construct a loss function model based on reconstruction loss and classification loss to classify defects of power equipment.

[0014] Furthermore, in step S2, the image data samples of the support set and the query set are input into the feature extractor to obtain the expression of the sample features of the support set and the query set:

[0015] F S =f(X S )

[0016] F Q =f(X Q )

[0017] In the formula, F S is the sample feature of the support set, F Q is the sample feature of the query set, f(·) is the feature extraction network, X S is the image data sample of the support set, X Q is the image data sample of the query set.

[0018] Furthermore, in step S2, the process of enhancing the sample features of the support set adopts a feature enhancement algorithm based on AdaIN, and its calculation formula includes:

[0019]

[0020] p=g(F S )

[0021] α cls = -y log(p)

[0022] Where F' S is the sample enhancement feature of the support set, AdaIN(·) is the style transfer algorithm, σ target represents the variance that the image features of the support set need to achieve after enhancement, F S represents the image features of the support set, μ(F S) represents the mean of the image feature X of the support set, σ(F S ) represents the variance of the image feature X of the support set, μ target represents the mean value that the image features of the support set need to reach after enhancement, B is the batch size, C is the number of feature channels, P is the feature dimension, ∈ is the smoothing term, is the mean of the classification confidence, is the variance of the classification confidence, ε is the style transfer ratio, p is the predicted probability of the image data sample in the support set, g(·) is the classifier, α cls is the classification confidence, and y is the sample label of the support set.

[0023] Furthermore, in step S4, the calculation process expression of label embedding is:

[0024] X′ label =ELMES(X label ,X nor ,Q)

[0025]

[0026] Where X' label is the label embedding after equal length maximum angle encoding, X label is the one-hot encoded label embedding, ELMES(·) is the label encoder, X nor is the normalized matrix, Q is the orthogonalized matrix, n is the number of categories, E is the identity matrix, I is the all-1 matrix, U is the transformation matrix, and R is the upper triangular matrix.

[0027] Furthermore, in step S6, the calculation formula of the classification loss is:

[0028] L cls =-Y Q log(P Q )

[0029] Where, L cls is the classification loss, Y Q is the sample label of the query set, P Q The classification prediction results.

[0030] Furthermore, in step S7, the calculation formula for constructing the loss function model according to the reconstruction loss and the classification loss includes:

[0031] L=Lcls+α·Lre

[0032] In the formula, L is the loss function, L cls is the classification loss, α is the weight parameter of the reconstruction loss, L re is the reconstruction loss.

[0033] In a second aspect, the present application provides an electronic device, including:

[0034] A memory for storing one or more programs;

[0035] processor;

[0036] When the one or more programs are executed by the processor, a method for classifying defects of electric power equipment based on confidence enhancement and image reconstruction as described in any one of the first aspects is implemented.

[0037] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for classifying defects in power equipment based on confidence enhancement and image reconstruction as described in any one of the first aspects above.

[0038] Compared with the prior art, the present invention has at least the following advantages or beneficial effects:

[0039] (1) The present invention provides a power equipment defect classification method based on confidence enhancement and image reconstruction. By introducing a feature enhancement method of classification confidence, the migration of any style is realized. Without being restricted by the original style of the sample, the target style can be matched in real time, the diversity of the data is increased, and the problem of single scene of power equipment defect images is solved.

[0040] (2) The present invention enables the model to learn the intrinsic distribution of features through image reconstruction, thereby improving the robustness of the model;

[0041] (3) The present invention proposes an embedding method based on the fusion of image features and label encoding, which fully utilizes the local features of the image and the global label information, thereby significantly improving the classification performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 A step diagram of a method for classifying defects of electric power equipment based on confidence enhancement and image reconstruction according to the present invention;

[0044] Figure 2 A structural schematic diagram of a power equipment defect classification based on confidence enhancement and image reconstruction according to the present invention;

[0045] Figure 3The figure is a schematic structural block diagram of an electronic device according to an embodiment of the present invention.

[0046] Icon: 101, memory; 102, processor; 103, communication interface. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0048] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0049] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0050] It should be noted that, in this article, the term "include" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0051] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0052] Example 1

[0053] See also Figure 1 , Figure 1 Shown is a step diagram of a method for classifying defects of electric power equipment based on confidence enhancement and image reconstruction according to the present invention;

[0054] In a first aspect, the present application provides a method for classifying defects of power equipment based on confidence enhancement and image reconstruction, which comprises the following steps:

[0055] S1, obtain the original image data sample, and divide the image data sample into a support set and a query set;

[0056] S2, inputting the image data samples of the support set and the query set into the feature extractor to obtain the sample features of the support set and the query set, and performing feature enhancement on the sample features of the support set to obtain the sample enhanced features of the support set;

[0057] S3, add masks to the sample enhancement features of the support set and reconstruct them, and calculate the reconstruction loss;

[0058] S4, extracting sample labels according to the support set and the query set, and encoding the sample labels using a maximum equiangular encoder to obtain label embedding;

[0059] S5, concatenating the sample features of the query set with the sample enhancement features of the support set to obtain a concatenated feature embedding, and concatenating the concatenated feature embedding with the label embedding and inputting the concatenated feature embedding into the non-causal sequence to obtain a new sample feature of the query set;

[0060] S6. Input the new sample features of the query set into the classifier to obtain the classification prediction result and calculate the classification loss;

[0061] S7. Construct a loss function model based on reconstruction loss and classification loss to classify defects of power equipment.

[0062] As a preferred implementation, in step S2, the image data samples of the support set and the query set are input into the feature extractor to obtain the expression of the sample features of the support set and the query set:

[0063] F S =f(X S )

[0064] F Q =f(X Q )

[0065] In the formula, F S is the sample feature of the support set, F Q is the sample feature of the query set, f(·) is the feature extraction network, X S is the image data sample of the support set, X Q is the image data sample of the query set.

[0066] As a preferred implementation, in step S2, the process of enhancing the sample features of the support set adopts a feature enhancement algorithm based on AdaIN, and its calculation formula includes:

[0067]

[0068] p=g(F S )

[0069] α cls = -y log(p)

[0070] Where F' S is the sample enhancement feature of the support set, AdaIN(·) is the style transfer algorithm, σ target represents the variance that the image features of the support set need to achieve after enhancement, F S represents the image features of the support set, μ(F S ) represents the mean of the image feature X of the support set, σ(F S ) represents the variance of the image feature X of the support set, μ target represents the mean value that the image features of the support set need to reach after enhancement, B is the batch size, C is the number of feature channels, P is the feature dimension, ∈ is the smoothing term, is the mean of the classification confidence, is the variance of the classification confidence, ε is the style transfer ratio, p is the predicted probability of the image data sample in the support set, g(·) is the classifier, α cls is the classification confidence, and y is the sample label of the support set.

[0071] As a preferred implementation, in step S4, the calculation process expression of label embedding is:

[0072] X′ label =ELMES(X label ,X nor ,Q)

[0073]

[0074]

[0075] Where X' label is the label embedding after equal length maximum angle encoding, X label is the one-hot encoded label embedding, ELMES(·) is the label encoder, X nor is the normalized matrix, Q is the orthogonalized matrix, n is the number of categories, E is the identity matrix, I is the all-1 matrix, U is the transformation matrix, and R is the upper triangular matrix.

[0076] As a preferred implementation, in step S6, the calculation formula of the classification loss is:

[0077] L cls =-Y Q log(P Q )

[0078] Where, L cls is the classification loss, Y Qis the sample label of the query set, P Q The classification prediction results.

[0079] As a preferred implementation, in step S7, the calculation formula for constructing the loss function model according to the reconstruction loss and the classification loss includes:

[0080] L=L cls +α·L re

[0081] In the formula, L is the loss function, L cls is the classification loss, α is the weight parameter of the reconstruction loss, L re is the reconstruction loss.

[0082] Example 2

[0083] As a preferred embodiment, the present invention uses a power equipment defect image dataset for experiments, which contains 11 defect categories: grading ring tilt, insulator breakage, insulator covering, foreign objects hanging on the transmission line, insulator missing, insulator flashover, insulator foreign objects, upper and lower buckle corrosion, suspension corrosion, variable handle corrosion and lightning rod suspension corrosion; among them, the six defect categories of insulator breakage, insulator flashover, foreign objects hanging on the transmission line, variable handle corrosion, upper and lower buckle corrosion and lightning rod suspension corrosion are divided into training set and verification set in a ratio of 7:3, and the remaining defect categories are divided into test set, and their detailed information is shown in the following table:

[0084] Table 1 Defect types and quantities

[0085] Fault category Number of images Pressure equalizing ring tilted 270 Insulator damage 810 Insulator Cover 181 Foreign matter in power transmission lines 224 Insulator missing 221 Insulator flashover 812 Foreign matter on insulator 235 Corrosion of top and bottom buckles 763 Suspension corrosion 292 Variable Grip Corrosion 400 Lightning rod hanging out of corrosion 307

[0086] All algorithms perform 3-way K-shot and 5-way K-shot small sample tasks, and the evaluation is based on the average average accuracy of 200 randomly sampled small sample test tasks. For the feature extraction network, ViT is selected, and its structure is as follows Figure 2 As shown in the figure, the comparison models include ProtoNet, DSN, Feat, DeepBDC, CPEA, and CAML.

[0087] The experimental results are shown in Table 2:

[0088] Table 2 Average accuracy of different small sample methods

[0089]

[0090] In the table, each row corresponds to an algorithm, and each column corresponds to a small sample task setting. It can be seen from the average classification accuracy of each algorithm that on the power equipment defect image dataset, the method of the present invention has achieved the best results on both 3-way K-shot and 5-way K-shot.

[0091] Example 3

[0092] See also Figure 3 , Figure 3 A schematic structural block diagram of an electronic device provided in an embodiment of the present application.

[0093] An electronic device includes a memory 101, a processor 102 and a communication interface 103, wherein the memory 101, the processor 102 and the communication interface 103 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules, and the processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 can be used to communicate signaling or data with other node devices.

[0094] Among them, the memory 101 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0095] The processor 102 may be an integrated circuit chip with signal processing capability. The processor 102 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0096] It can be understood that the structure shown in the figure is only for illustration, and a method for classifying power equipment defects based on confidence enhancement and image reconstruction may also include more or fewer components than those shown in the figure, or have a different configuration than those shown in the figure. Each component shown in the figure may be implemented by hardware, software, or a combination thereof.

[0097] In the embodiments provided in the present application, it should be understood that the disclosed method can also be implemented in other ways. The embodiments described above are merely schematic, for example, the flowchart or block diagram in the accompanying drawings shows the possible implementation architecture, functions and operations of the method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0098] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0099] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0100] In summary, an embodiment of the present application provides a method for classifying defects in power equipment based on confidence enhancement and image reconstruction. The method obtains original image data, collects small sample tasks from it, and divides the collected image data into a support set and a query set; then the support set and query set images are input into a feature extractor to obtain image features, and the support set image features are enhanced using a style transfer method based on classification confidence; through a proxy method based on image reconstruction, a mask is added to the enhanced support set image features and reconstructed, and the reconstruction loss is calculated; an embedding method that fuses image features and label encoding is used to input the labels of the support set and query set into an equal-length maximum angle encoder, which is then concatenated with the image features of the support set and query set, and jointly input into a non-causal sequence model to output the encoded embedding; the query set image features are obtained from the embedding, and are input into a classifier to complete the prediction, the classification loss is calculated, and the gradient is updated together with the reconstruction loss to construct a loss function model to classify defects in power equipment.

[0101] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0102] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present application. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A method for classifying defects in power equipment based on confidence enhancement and image reconstruction, characterized in that: The following steps are involved: S1, obtain the original image data sample, and divide the image data sample into a support set and a query set; S2, inputting the image data samples of the support set and the query set into the feature extractor to obtain the sample features of the support set and the query set, and performing feature enhancement on the sample features of the support set to obtain the sample enhanced features of the support set; S3, add masks to the sample enhancement features of the support set and reconstruct them, and calculate the reconstruction loss; S4, extracting sample labels according to the support set and the query set, and encoding the sample labels using a maximum equiangular encoder to obtain label embedding; S5, concatenating the sample features of the query set with the sample enhancement features of the support set to obtain a concatenated feature embedding, and concatenating the concatenated feature embedding with the label embedding and inputting the concatenated feature embedding into a non-causal sequence to obtain a new sample feature of the query set; S6. Input the new sample features of the query set into the classifier to obtain the classification prediction result and calculate the classification loss; S7. Construct a loss function model based on reconstruction loss and classification loss to classify defects of power equipment.

2. The method for classifying defects of electric power equipment based on confidence enhancement and image reconstruction according to claim 1, characterized in that: In step S2, the image data samples of the support set and the query set are input into the feature extractor to obtain the expression of the sample features of the support set and the query set: F S =f(X S ) F Q =f(X Q ) In the formula, F S is the sample feature of the support set, F Q is the sample feature of the query set, f(·) is the feature extraction network, X S is the image data sample of the support set, X Q is the image data sample of the query set.

3. The method for classifying defects of electric power equipment based on confidence enhancement and image reconstruction according to claim 1, characterized in that: In step S2, the process of enhancing the sample features of the support set adopts a feature enhancement algorithm based on AdaIN, and its calculation formula includes: p=g(F S ) a cls =-ylog(p) In the formula, F′ S is the sample enhancement feature of the support set, AdaIN(·) is the style transfer algorithm, σ target represents the variance that the image features of the support set need to achieve after enhancement, F S represents the image features of the support set, μ(F S ) represents the mean of the image feature X of the support set, σ(F S ) represents the variance of the image feature X of the support set, μ target represents the mean value that the image features of the support set need to reach after enhancement, B is the batch size, C is the number of feature channels, P is the feature dimension, ∈ is the smoothing term, is the mean of the classification confidence, is the variance of the classification confidence, ε is the style transfer ratio, p is the predicted probability of the image data sample in the support set, g(·) is the classifier, α cls is the classification confidence, and y is the sample label of the support set.

4. The method for classifying defects of electric power equipment based on confidence enhancement and image reconstruction according to claim 1, characterized in that: In step S4, the calculation process expression of label embedding is: X′ label =ELMES(X label ,X nor ,Q) In the formula, X′ label is the label embedding after equal length maximum angle encoding, X label is the one-hot encoded label embedding, ELMES(·) is the label encoder, X nor is the normalized matrix, Q is the orthogonalized matrix, n is the number of categories, E is the identity matrix, I is the all-1 matrix, U is the transformation matrix, and R is the upper triangular matrix.

5. The method for classifying defects of electric power equipment based on confidence enhancement and image reconstruction according to claim 1, characterized in that: In step S6, the calculation formula of classification loss is: L cls =-Y Q log(P Q ) Where, L cls is the classification loss, Y Q is the sample label of the query set, P Q The classification prediction result.

6. The method for classifying defects of electric power equipment based on confidence enhancement and image reconstruction as claimed in claim 5, characterized in that: In step S7, the calculation formula of the loss function model is constructed based on the reconstruction loss and the classification loss. include: L=L cls +α·L re In the formula, L is the loss function, L cls is the classification loss, α is the weight parameter of the reconstruction loss, L re is the reconstruction loss.

7. An electronic device, characterized in that: include: A memory for storing one or more programs; processor; When the one or more programs are executed by the processor, a method for classifying defects of electric power equipment based on confidence enhancement and image reconstruction as described in any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for classifying defects of electric power equipment based on confidence enhancement and image reconstruction as described in any one of claims 1 to 6 is implemented.

Images