Power equipment image classification method, device, power equipment and storage medium

Abstract

The embodiment of the present invention discloses a method, device, power equipment and storage medium for power equipment image classification. The power equipment image classification method includes: constructing a training set based on sample power equipment images; using the training set to train a preset convolutional neural network model to obtain an image defect recognition model; identifying the image to be detected based on the image defect recognition model to determine the defect type of the image to be detected. The power equipment image classification method provided by the embodiment of the present invention solves the problem of low efficiency and easy errors in image classification by manpower in the prior art, and can automatically identify and classify batch images generated by inspections in the power system, providing a reliable basis for operation and maintenance personnel to identify faults of power equipment.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of electrical equipment operation and maintenance technology, and in particular, to a method and device for classifying images of electrical equipment, electrical equipment, and a storage medium. Background Art

[0002] In recent years, digital camera and robotics technologies have continued to develop. Newly built smart substations and some renovated unmanned substations have achieved efficient substation inspections through video surveillance or robots equipped with cameras, and transmitted on-site image data to the monitoring center for manual analysis and identification.

[0003] The existing image and video data are large in stock and growing rapidly, but there is relatively little useful information that can be discovered; manual judgment is subjective, ambiguous, incomplete, and inefficient. There are many types of specific defects in substation equipment during inspections, and the number of images generated by inspections that require manual identification and analysis is large. Manual image defect identification and classification alone is inefficient, and in the face of a large number of images, the staff's eyes become tired after working for a long time, resulting in a decrease in judgment accuracy. Summary of the invention

[0004] The embodiments of the present invention provide a method and device for classifying images of electric power equipment, electric power equipment and a storage medium, which automatically identify defects in batches of inspection pictures so as to classify the inspection pictures in batches.

[0005] In a first aspect, an embodiment of the present invention provides a method for classifying images of electric power equipment, comprising:

[0006] Construct a training set based on sample power equipment images;

[0007] Using the training set to train a preset convolutional neural network model to obtain an image defect recognition model;

[0008] The image to be detected is identified based on the image defect recognition model to determine the defect type of the image to be detected.

[0009] Optionally, after constructing the training set based on the sample power equipment pictures, the method further includes:

[0010] The sample power equipment images are labeled with defect types to assign category labels to the sample images, wherein the category labels are used as a basis for classifying the power equipment images.

[0011] Optionally, after using the training set to train a preset convolutional neural network model to obtain an image defect recognition model, the method further includes:

[0012] Using the image defect recognition model to perform classification prediction on the test images in a preset test set to obtain a predicted category label set, wherein each test image in the test set corresponds to a matching actual category label;

[0013] Determining the classification accuracy of the image defect recognition model based on a comparison result between each predicted category label in the predicted category label set and the corresponding actual category label;

[0014] If the classification accuracy of the image defect recognition model is less than a preset accuracy threshold, then increasing the number of the sample power pictures in the training set and / or increasing the number of training times of the sample power pictures in the training set, retraining the image defect recognition model, and when the classification accuracy of the image defect recognition model is greater than or equal to the accuracy threshold, fixing the parameters of the image defect recognition model, and determining the image defect recognition model after the parameter update as the target image defect recognition model;

[0015] Accordingly, the step of identifying the image to be detected based on the image defect recognition model to determine the defect type of the image to be detected includes:

[0016] The image to be detected is identified based on the target image defect recognition model to determine the defect type of the image to be detected.

[0017] Optionally, the defect type includes at least one of the following: component damage, respirator defect, status indication defect, foreign matter, oil leakage and untidy appearance.

[0018] Optionally, the preset convolutional neural network model is a VGG16 model.

[0019] Optionally, the identifying the image to be detected based on the image defect recognition model to determine the defect type of the image to be detected includes:

[0020] The convolution layer in the image defect recognition model extracts a preset number of feature blocks of the image to be detected;

[0021] The pooling layer in the image defect recognition model selects a target feature block from the preset number of feature blocks;

[0022] The fully connected layer in the image defect recognition model processes the feature block to obtain a target feature area;

[0023] The output layer in the image defect recognition model outputs the prediction result of the feature area to determine the defect type of the image to be detected.

[0024] In a second aspect, an embodiment of the present invention further provides a device for classifying images of electric power equipment, comprising:

[0025] A training set construction module, used to construct a training set based on sample power equipment images;

[0026] A picture defect recognition model determination module, used to train a preset convolutional neural network model using the training set to obtain a picture defect recognition model;

[0027] An image defect type determination module is used to identify the image to be detected based on the image defect recognition model to determine the defect type of the image to be detected;

[0028] A classification module is used to classify the image to be detected based on the defect type.

[0029] Optionally, the electric power equipment image classification device further includes:

[0030] The category label assignment module is used to mark the defect types of the sample power equipment images to assign category labels to the sample images.

[0031] In a third aspect, an embodiment of the present invention further provides an electric power device, including:

[0032] one or more processors;

[0033] A storage device for storing one or more programs;

[0034] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for classifying images of electric power equipment as described in any embodiment of the present invention.

[0035] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for classifying images of electric power equipment as described in any embodiment of the present invention.

[0036] The power equipment image classification method provided in the embodiment of the present invention constructs a training set to train a preset neural convolutional network model to obtain an image defect recognition model. The image defect recognition model automatically identifies the image to be detected to determine whether the image to be detected has defects, and automatically locates the defect type of the corresponding image when it is determined that the image to be detected has defects. This solves the problem of low efficiency and easy errors in image classification through manpower in the prior art, and can automatically identify and classify batch images generated by inspections in the power system, providing a reliable basis for operation and maintenance personnel to identify faults of power equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A flowchart of a method for classifying images of electric power equipment provided by an embodiment of the present invention;

[0038] Figure 2 The overall architecture diagram of the VGG16 model provided by the embodiment of the present invention;

[0039] Figure 3 A flowchart of another method for classifying electric power equipment images provided by an embodiment of the present invention;

[0040] Figure 4 A structural block diagram of a power equipment image classification device provided by an embodiment of the present invention;

[0041] Figure 5 A structural block diagram of an electric power device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0043] Figure 1 This is a flowchart of a method for classifying images of power equipment provided by an embodiment of the present invention. The method can be applied to the situation where images generated during inspections in a power system are automatically identified and the images are automatically classified based on the automatic identification results. Thus, operation and maintenance personnel can obtain defective equipment in a timely manner based on the image classification results. The method can be executed by power equipment, which can be, for example, a computer device equipped with an image classification algorithm. Reference Figure 1 , the power equipment image classification method includes the following steps:

[0044] S110. Construct a training set based on sample power equipment images.

[0045] The sample power equipment pictures are a set of power equipment pictures including defective pictures and normal pictures without defects, wherein the defective pictures include various defect types.

[0046] In one embodiment, the sample power equipment pictures include pictures of the following defect types: pictures of damaged parts, pictures of respirator defects, pictures of status indication defects, pictures of foreign matter, pictures of oil leakage, and pictures of untidy appearance.

[0047] For example, sample power equipment images may include 8,000 images and xml files corresponding to the 8,000 images, among which samples of rust defects in secondary junction boxes of current transformers, samples of oil leakage defects in transformers, samples of contamination defects in transformer bushings, and samples of paint peeling defects in transformers each have 2,000 images and 2,000 xml files.

[0048] S120: Use the training set to train the preset convolutional neural network model to obtain an image defect recognition model.

[0049] Among them, each sample power equipment image in the training set contains various types of defect images. Therefore, by training the convolutional neural network with the training set, a model of the correspondence between image defect features and image types can be obtained, which is an image defect recognition model.

[0050] In one embodiment, the convolutional neural network model is a VGG16 model. The convolution kernel size of the VGG16 model uses 3*3, which avoids the parameter explosion caused by the large convolution kernel. A large number of 3*3 convolutions are sufficient to capture horizontal, vertical and diagonal pixel changes, reduce spatial dimensions, increase depth, and effectively extract input features. Therefore, 3*3 convolutions are generally used in VGG16 to replace 5*5 and 7*7. VGG16 uses 3 fully connected layers, and finally a softmax output layer is connected, and the softmax output layer outputs the classification results for the image.

[0051] S130 , identifying the image to be detected based on the image defect recognition model to determine the defect type of the image to be detected.

[0052] Among them, the image to be detected is input into the image defect recognition model. Since the image defect recognition model establishes a corresponding relationship between the image defect type and the characteristic area in the image, the image defect recognition model can determine the defect type of the image to be detected by identifying the image to be detected, thereby fully utilizing the inspection images generated by the inspection equipment to realize fault or problem diagnosis of the power equipment, and further realize accurate positioning of the defect type of the power equipment.

[0053] It should be noted that when the recognition result indicates that the image has no defects, the image defect recognition model will also assign a preset category label to the image to indicate that the image is a complete image.

[0054] Optionally, based on the use of the VGG16 model to establish an image defect recognition model, the process of identifying the image to be detected through the image defect recognition model can be optimized as follows:

[0055] The convolutional layer in the image defect recognition model extracts a preset number of feature blocks of the image to be detected;

[0056] The pooling layer in the image defect recognition model selects the target feature block from a preset number of feature blocks;

[0057] The fully connected layer in the image defect recognition model processes the target feature block to obtain the target feature region;

[0058] The output layer in the image defect recognition model outputs the prediction results of the feature area to determine the defect type of the image to be detected.

[0059] Specifically, Figure 2 The overall architecture diagram of the VGG16 model provided in the embodiment of the present invention is composed of Figure 2 It can be seen that the VGG16 model includes convolutional layers, pooling layers, fully connected layers, and output layers. The convolutional layers and input layers are regularly interlaced. The convolutional layers perform feature extraction, and the pooling layers perform feature selection. Usually, multiple windows of the convolutional layer perform convolution operations on the input layer, and the pooling layer uses one window to perform pooling operations on the convolutional layer. The fully connected layer can integrate local information with category distinction and integrate all local features into global features. The output layer outputs the classification results.

[0060] Specifically in this embodiment, because the defect recognition model is trained by the VGG16 model, the image defect recognition model and the VGG16 model have the same network structure.

[0061] Specifically, the convolution layer in the image defect recognition model is composed of multiple feature planes, each of which is composed of multiple feature units. Each feature unit is connected to the local area of ​​the previous layer through the convolution kernel. Each feature unit in the convolution layer corresponds to a 3*3 or 5*5 small block in the previous layer input network, and acts on the features of the local small block in the previous layer to obtain more abstract features. The convolution kernel is a weight matrix, and each convolution kernel corresponds to a set of fixed weights. Different convolution kernels extract different features such as contours, textures, color depth, etc. Therefore, extracting different features of the image requires the action of multiple different convolution kernels.

[0062] It can be seen that each feature block extracted by the convolution layer is a partial feature of the power equipment. These features are scattered, so they need to be integrated to obtain the complete image features of the power equipment.

[0063] Before integrating the feature blocks, the pooling layer in the image defect recognition model processes the feature blocks to reduce the data dimension and reduce the amount of calculation. The pooling layer in the image defect recognition model does not change the depth of the image defect recognition model, but can reduce the size of the input matrix, thereby reducing the number of parameters of the image defect recognition model.

[0064] After several rounds of convolution and pooling, the shallow feature information of the image is abstracted into features with higher information content. Convolution and pooling operations are the process of image feature extraction. After several layers of convolution and pooling operations, the obtained feature maps are expanded row by row, connected into vectors, and input into the fully connected layer. The fully connected layer can integrate local information with category discrimination and integrate all local features into global features. Finally, the classification result of the image to be detected is output through the output layer.

[0065] The power equipment image classification method provided in the embodiment of the present invention constructs a training set to train a preset neural convolutional network model to obtain an image defect recognition model. The image defect recognition model automatically identifies the image to be detected to determine whether the image to be detected has defects, and automatically locates the defect type of the corresponding image when it is determined that the image to be detected has defects. This solves the problem of low efficiency and easy errors in image classification through manpower in the prior art, and can automatically identify and classify batch images generated by inspections in the power system, providing a reliable basis for operation and maintenance personnel to identify faults of power equipment.

[0066] Optional, Figure 3 This is a flowchart of another method for classifying power equipment images provided by an embodiment of the present invention. This embodiment is optimized based on the above embodiment. Figure 3 , the method specifically comprises:

[0067] S310: construct a training set based on sample power equipment images.

[0068] S320, labeling the defect types of the sample power equipment images to assign category labels to the sample images.

[0069] Among them, the category label is used as the basis for classifying the power equipment images.

[0070] For various visible equipment defects that appear during power system inspections, they can be divided into the following categories according to the nature of the defects: component damage, respirator defect, status indication defect, foreign matter, oil leakage, and untidy appearance. Each category is further subdivided, and the specific defect classification is shown in Table 1. For each defect subcategory, a category label name is further set for the equipment defect. The principle of formulating the category label name can be, for example, to use the first letter of the keyword of the defect category as a prefix, followed by the first letter of the specific defect name. The specific category label can be shown in Table 1.

[0071] Table 1 Classification and label assignment of power equipment defects

[0072]

[0073] This embodiment assigns category labels to sample power equipment images and establishes a correspondence between image features and category labels. Thus, an image defect recognition model obtained by training a neural convolutional network based on sample power equipment images has a correspondence between image features and category labels. Therefore, the image defect recognition model can automatically classify the images to be detected based on feature extraction of the images to be detected.

[0074] S330: Use the training set to train the preset convolutional neural network model to obtain an image defect recognition model.

[0075] S340: Use the image defect recognition model to perform classification prediction on the test images in the preset test set to obtain a predicted category label set.

[0076] Among them, each test image in the test set corresponds to a matching actual category label.

[0077] After the image defect recognition model classifies and predicts the test images, it assigns a predicted category label to each test image, so that the predicted category labels of all test images in the test set constitute a predicted category label set. The predicted category label is used to compare with the actual list label of the test image to detect the reliability of the image defect recognition model.

[0078] S350 , determining the classification accuracy of the image defect recognition model based on the comparison result between each predicted category label in the predicted category label set and the corresponding actual category label.

[0079] The image defect recognition model predicts a category label for each test image in the test set. The predicted category label corresponds to a type of image defect. The predicted category label is compared with the actual category label of the test image to determine whether the two are consistent. The comparison results of each predicted category and the corresponding actual category are further statistically analyzed to determine the accuracy of the image defect recognition model's prediction for the test image.

[0080] S360. If the classification accuracy of the image defect recognition model is less than the preset accuracy threshold, increase the number of sample power pictures in the training set and / or increase the number of training times of the sample power pictures in the training set, retrain the image defect recognition model, and when the classification accuracy of the image defect recognition model is greater than or equal to the accuracy threshold, fix the parameters of the image defect recognition model, and determine the image defect recognition model with updated parameters as the target image defect recognition model.

[0081] If the accuracy is low, the classification accuracy of the image defect recognition model is improved by increasing the number of sample power images in the training set and / or increasing the number of training times of the sample power images in the training set.

[0082] The following further introduces the training and parameter adjustment process of the image defect recognition model using balanced training samples and unbalanced training samples as examples.

[0083] In one embodiment, balanced training samples for transformer substation equipment defect classification are shown in Table 2. The total number of images under balanced training samples is 11,000, including 10,000 training samples and 1,000 test samples. Five image categories are set during training to establish a correspondence between category numbers and defect types. Category 0 corresponds to transformer oil leakage defects, category 1 corresponds to transformer bushing contamination defects, category 2 corresponds to transformer paint stripping defects, category 3 corresponds to current transformer secondary junction box rust defects, and category 4 corresponds to images under normal conditions. In addition, 200 images of four defects, transformer oil leakage, current transformer secondary junction box rust, transformer bushing contamination, transformer paint stripping, and normal operation, constitute 1,000 test samples, and the ratio of the number of training samples to the number of test samples is 10:1.

[0084] Table 2 Substation equipment defect classification and balance training samples

[0085]

[0086] 1000 test images were input into the trained model for classification, and the number of accurately classified images was 704, with an accuracy rate of 70.4%. By checking the excel table of test image categories and corresponding classification output categories, it was found that 83 of the 247 detection errors were due to the corrosion defects of the secondary junction box of the current transformer being judged as normal, 31 were due to the contamination defects of the transformer bushing being judged as normal, 45 were due to the oil leakage defects of the transformer being judged as normal, and 29 were due to the paint peeling of the transformer being judged as normal. After analysis, the classification errors were concentrated on the normal images and defective images of the same equipment. In order to further improve the classification accuracy, the number of normal images and defective images of the same equipment in the training samples was adjusted, and the classification of substation inspection images under unbalanced samples was added.

[0087] Optionally, in order to further improve the accuracy of substation inspection picture classification, a substation inspection picture dataset under unbalanced samples is produced. The unbalanced training samples for substation equipment defect classification are shown in Table 3. The training samples include 2000 pictures of transformer oil leakage defects, 2000 pictures of transformer bushing contamination defects, 2000 pictures of current transformer secondary junction box rust defects, 2000 pictures of transformer paint peeling defects, and 4000 normal pictures. The 4000 normal pictures include 1000 transformers without oil leakage, 1000 pictures of current transformer secondary junction boxes without rust, 1000 pictures of transformer bushing without contamination, and 1000 pictures of transformer without paint peeling. Compared with the balanced samples, the number of normal picture training samples corresponding to each defect increases by 500.

[0088] Table 3 Unbalanced training samples for substation equipment defect classification

[0089]

[0090] When classifying unbalanced training sample images, the training parameter settings are the same as those when classifying balanced training sample images. 1,000 test images are input into the model for detection, and the number of accurately detected images is 863, with a classification accuracy of 86.3%. It can be seen that by setting unbalanced training samples, increasing the number of normal class images in training can effectively improve the classification accuracy. In order to further study the impact of the number of training times on the classification accuracy, the accuracy of balanced training samples and unbalanced training samples when the number of training times is 2,000, 4,000, 6,000, 8,000 and 10,000 is statistically analyzed. The training time and classification accuracy under different training times are shown in Table 4.

[0091] Table 4 Classification accuracy of detection samples under different training times

[0092]

[0093] From the above analysis, we can see that when the training samples are fixed, the training batches double and the training time also doubles. As the number of times each image sample participates in the training increases, the classification accuracy gradually increases. After the training samples increase, as the number of times each image sample participates in the training increases, the classification accuracy gradually increases. In addition, even if the number of times each image participates in the training in the large training sample is less than the number of times each image participates in the training in the small sample, there will be a situation where the classification accuracy is high. This shows that increasing the number of training samples for substation equipment defects and increasing the number of defect sample training times can effectively improve the classification accuracy. Through these two means, the classification accuracy of substation equipment defects finally reached 99.7%.

[0094] S370: Identify the image to be detected based on the target image defect recognition model to determine the defect type of the image to be detected.

[0095] Among them, the target image defect recognition model is the above-mentioned image defect recognition model that has been verified to be qualified by the test set.

[0096] This embodiment optimizes the training process of the image defect recognition model, compares the predicted category of the test image with the actual category of the test image based on the image defect recognition model to determine whether the predicted category and the actual category are consistent, and determines the accuracy of the image defect recognition model by statistically comparing the comparison results of all predicted categories with the corresponding actual categories. When the accuracy of the image defect recognition model does not meet the requirements, the image defect recognition model is retrained by using the training set of balanced samples and unbalanced samples to adjust the parameters of the image defect recognition model and improve the classification accuracy of the image defect recognition model. Thus, the images to be detected are classified based on the optimized target image defect recognition model, further improving the classification accuracy of the images to be detected. This embodiment preprocesses, classifies and detects the defect images of the substation based on deep vision, which can not only determine whether the images taken for specific equipment of the substation during the inspection have faults, but also locate the defects of the substation equipment with a relatively high confidence, thereby improving the level of intelligent inspection of the substation, and has a good engineering application value for timely and effective discovery of substation equipment defects and ensuring the safe and stable operation of the equipment.

[0097] Optional, Figure 4 The present invention provides a structural block diagram of an electric power equipment image classification device, which includes: a training set construction module 410, an image defect recognition model determination module 420, an image defect type determination module 430 and a classification module 440, wherein:

[0098] A training set construction module 410 is used to construct a training set based on sample power equipment pictures;

[0099] The image defect recognition model determination module 420 is used to train a preset convolutional neural network model using a training set to obtain an image defect recognition model;

[0100] The image defect type determination module 430 is used to identify the image to be detected based on the image defect recognition model to determine the defect type of the image to be detected;

[0101] The classification module 440 is used to classify the image to be inspected based on the defect type.

[0102] Optionally, based on the above technical solution, the power equipment image classification device further includes:

[0103] The category label assignment module is used to annotate the defect types of sample power equipment images to assign category labels to the sample images, wherein the category labels are used as a basis for classifying the power equipment images.

[0104] Optionally, based on the above technical solution, the power equipment image classification device further includes:

[0105] A testing module is used to use the image defect recognition model to classify and predict the test images in a preset test set to obtain a predicted category label set, wherein each test image in the test set corresponds to a matching actual category label;

[0106] A classification accuracy determination module, used to determine the classification accuracy of the image defect recognition model based on the comparison result of each predicted category label in the predicted category label set and the corresponding actual category label;

[0107] The adjustment module is used to increase the number of sample power pictures in the training set and / or increase the number of training times of the sample power pictures in the training set if the classification accuracy of the picture defect recognition model is less than a preset accuracy threshold, retrain the picture defect recognition model, and fix the parameters of the picture defect recognition model when the classification accuracy of the picture defect recognition model is greater than or equal to the accuracy threshold, and determine the picture defect recognition model with updated parameters as the target picture defect recognition model.

[0108] Optionally, based on the above technical solution, the image defect type determination module 430 is further used to:

[0109] The image to be inspected is identified based on the target image defect recognition model to determine the defect type of the image to be inspected.

[0110] Optionally, based on the above technical solution, the defect types include at least one of the following: damaged parts, respirator defects, status indication defects, foreign matter, oil leakage and untidy appearance.

[0111] Optionally, based on the above technical solution, the preset convolutional neural network model is a VGG16 model.

[0112] Optionally, based on the above technical solution, the image defect type determination module 430 is further used to:

[0113] Extracting a preset number of feature blocks of the image to be detected through the convolutional layer in the image defect recognition model;

[0114] The target feature block is selected from a preset number of feature blocks through the pooling layer in the image defect recognition model;

[0115] The feature blocks are processed through the fully connected layer in the image defect recognition model to obtain the target feature area;

[0116] The output layer in the image defect recognition model outputs the prediction results of the feature area to determine the defect type of the image to be detected.

[0117] The power equipment image classification method provided in the embodiment of the present invention constructs a training set to train a preset neural convolutional network model to obtain an image defect recognition model. The image defect recognition model automatically identifies the image to be detected to determine whether the image to be detected has defects, and automatically locates the defect type of the corresponding image when it is determined that the image to be detected has defects. This solves the problem of low efficiency and easy errors in image classification through manpower in the prior art, and can automatically identify and classify batch images generated by inspections in the power system, providing a reliable basis for operation and maintenance personnel to identify faults of power equipment.

[0118] Optional, Figure 5 A structural block diagram of an electric power device provided in an embodiment of the present invention. Figure 5 A schematic diagram of the structure of an electric power device provided in an embodiment of the present invention. Figure 5 A block diagram of an exemplary power device 512 suitable for use in implementing embodiments of the present invention is shown. Figure 5 The electric device 512 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0119] like Figure 5 As shown, the power device 512 is in the form of a general purpose computing device. The components of the power device 512 may include, but are not limited to: one or more processors or processing units 516, a system memory 528, and a bus 518 connecting various system components (including the system memory 528 and the processing unit 516).

[0120] Bus 518 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0121] The power device 512 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the power device 512, including volatile and non-volatile media, removable and non-removable media.

[0122] The system memory 528 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 530 and / or cache memory 532. The power device 512 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 534 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical medium) may be provided. In these cases, each drive may be connected to bus 518 via one or more data medium interfaces. Memory 528 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of various embodiments of the present invention.

[0123] A program / utility 540 having a set (at least one) of program modules 542 may be stored, for example, in the memory 528, such program modules 542 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 542 generally perform the functions and / or methods of the embodiments described herein.

[0124] The power device 512 may also communicate with one or more external devices 514 (e.g., keyboard, pointing device, display 524, etc.), may also communicate with one or more devices that enable a user to interact with the power device 512, and / or communicate with any device that enables the power device 512 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed via an input / output (I / O) interface 522. In addition, the power device 512 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) via a network adapter 520. As shown, the network adapter 520 communicates with other modules of the power device 512 via a bus 518. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the power device 512, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0125] The processing unit 516 executes various functional applications and data processing by running the program stored in the system memory 528, for example, implementing the power equipment image classification method provided in the embodiment of the present invention, the method comprising:

[0126] Construct a training set based on sample power equipment images;

[0127] Use the training set to train the preset convolutional neural network model to obtain an image defect recognition model;

[0128] The image to be inspected is identified based on the image defect recognition model to determine the defect type of the image to be inspected.

[0129] Optionally, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for classifying images of electric power equipment provided in an embodiment of the present invention is implemented. The method includes:

[0130] Construct a training set based on sample power equipment images;

[0131] Use the training set to train the preset convolutional neural network model to obtain an image defect recognition model;

[0132] The image to be inspected is identified based on the image defect recognition model to determine the defect type of the image to be inspected.

[0133] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0134] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0135] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0136] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0137] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for classifying images of electric power equipment, characterized in that: include: Construct a training set based on sample power equipment images; Using the training set to train a preset convolutional neural network model to obtain an image defect recognition model; Identify the image to be detected based on the image defect recognition model to determine the defect type of the image to be detected; Wherein, the preset convolutional neural network model is a VGG16 model; The step of identifying the image to be detected based on the image defect recognition model to determine the defect type of the image to be detected includes: The convolution layer in the image defect recognition model extracts a preset number of feature blocks of the image to be detected; The pooling layer in the image defect recognition model selects a target feature block from the preset number of feature blocks; The fully connected layer in the image defect recognition model processes the feature block to obtain a target feature area; The output layer in the image defect recognition model outputs the prediction result of the feature area to determine the defect type of the image to be detected; The convolution kernel size of the VGG16 model is 3×3; After the preset convolutional neural network model is trained using the training set to obtain the image defect recognition model, the method further includes: Using the image defect recognition model to perform classification prediction on the test images in a preset test set to obtain a predicted category label set, wherein each test image in the test set corresponds to a matching actual category label; Determining the classification accuracy of the image defect recognition model based on a comparison result between each predicted category label in the predicted category label set and the corresponding actual category label; If the classification accuracy of the image defect recognition model is less than a preset accuracy threshold, the number of the sample power pictures in the training set is increased and / or the number of training times of the sample power pictures in the training set is increased, the image defect recognition model is retrained, and when the classification accuracy of the image defect recognition model is greater than or equal to the accuracy threshold, the parameters of the image defect recognition model are fixed, and the image defect recognition model with updated parameters is determined as the target image defect recognition model; wherein the image defect recognition model is retrained respectively by using a training set of balanced samples and a training set of unbalanced samples to adjust the parameters of the image defect recognition model; Accordingly, the step of identifying the image to be detected based on the image defect recognition model to determine the defect type of the image to be detected includes: The image to be detected is identified based on the target image defect recognition model to determine the defect type of the image to be detected.

2. The method for classifying electric power equipment images according to claim 1, characterized in that: After constructing the training set based on the sample power equipment pictures, the method further includes: The sample power equipment images are labeled with defect types to assign category labels to the sample images, wherein the category labels are used as a basis for classifying the power equipment images.

3. The method for classifying electric power equipment images according to claim 1, characterized in that: The defect type includes at least one of the following: component damage, respirator defect, status indication defect, foreign matter, oil leakage and untidy appearance.

4. A device for classifying images of electric power equipment, characterized in that: include: A training set construction module, used to construct a training set based on sample power equipment images; A picture defect recognition model determination module, used to train a preset convolutional neural network model using the training set to obtain a picture defect recognition model; An image defect type determination module is used to identify the image to be detected based on the image defect recognition model to determine the defect type of the image to be detected; A classification module, used for classifying the image to be detected based on the defect type; Wherein, the preset convolutional neural network model is a VGG16 model; The image defect type determination module is further used to extract a preset number of feature blocks of the image to be detected through the convolution layer in the image defect recognition model; select a target feature block from the preset number of feature blocks through the pooling layer in the image defect recognition model; process the feature block through the fully connected layer in the image defect recognition model to obtain a target feature area; output the prediction result of the feature area through the output layer in the image defect recognition model to determine the defect type of the image to be detected; The convolution kernel size of the VGG16 model is 3×3; Wherein, the device further comprises: A testing module, used to use the image defect recognition model to perform classification prediction on the test images in a preset test set to obtain a predicted category label set, wherein each test image in the test set corresponds to a matching actual category label; A classification accuracy determination module, used to determine the classification accuracy of the image defect recognition model based on a comparison result between each predicted category label in the predicted category label set and the corresponding actual category label; An adjustment module, for increasing the number of sample power pictures in the training set and / or increasing the number of training times of the sample power pictures in the training set if the classification accuracy of the picture defect recognition model is less than a preset accuracy threshold, retraining the picture defect recognition model, and fixing the parameters of the picture defect recognition model when the classification accuracy of the picture defect recognition model is greater than or equal to the accuracy threshold, and determining the picture defect recognition model after the parameter update as the target picture defect recognition model; wherein the picture defect recognition model is retrained respectively by using a training set of balanced samples and a training set of unbalanced samples to adjust the parameters of the picture defect recognition model; The image defect type determination module is further configured to identify the image to be detected based on the target image defect recognition model to determine the defect type of the image to be detected.

5. The power equipment image classification device according to claim 4, characterized in that: Also includes: The category label assignment module is used to mark the defect types of the sample power equipment images to assign category labels to the sample images.

6. An electric power device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the power equipment image classification method as described in any one of claims 1-3.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for classifying images of electric power equipment as described in any one of claims 1 to 3 is implemented.

Images