A deep learning-based rapid insulator defect detection method and system thereof

Abstract

The application provides a deep learning-based fast insulator defect detection method and system, and belongs to the technical field of computers. The method comprises the following steps: acquiring an image of an insulator to be detected; based on a pre-trained insulator segmentation model, the insulator in the image of the insulator to be detected is segmented, and an insulator segmentation effect diagram is output; the insulator segmentation effect diagram is input into a pre-trained insulator defect detection model, the insulator is subjected to defect detection, and an insulator defect detection result is output. The application effectively improves the insulator defect detection speed while ensuring the detection accuracy.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method and system for rapid insulator defect detection based on deep learning. Background Technology

[0002] As a crucial component of high-voltage transmission lines, insulator monitoring is vital for the safety and stability of the power system. Due to their role in electrical insulation and mechanical support, damage to insulator components can directly lead to transmission line outages, severely impacting their normal operation. Therefore, preventing insulator defects is a top priority for power companies. Furthermore, modern power grids typically have more complex interconnected topologies, making it essential to detect potential equipment faults at an early stage to prevent permanent failures.

[0003] In existing research, traditional insulator defect detection methods employ techniques such as texture features and detection features. For example, a method has been proposed to extract insulator texture features by constructing a contour model using semi-local operators (QGWu and JBAn, "An active contour model based on texture distribution for extracting inhomogeneous insulators from aerial images," IEEE Transactions on Geoscience and Remote Sensing, vol. 52, no. 6, pp. 3613-3626, Jun. 2014). While this method achieves satisfactory segmentation results, its adaptive selection ability for weight parameters is poor. Furthermore, a method using the K-Means algorithm to cluster images (H. Zou and FZHuang, "A novel intelligent fault diagnosis method for electrical equipment using infrared thermography," Infrared Physics and Technology, vol. 73, pp. 29-35, Nov. 2015) has been proposed to obtain statistical features of the images, which are then used for detection using support vector machines. However, in the detection process, accuracy and speed cannot be simultaneously optimized. In addition, a Faster Regions with Convolutional Neural Network (Faster R-CNN) was used for insulator localization, and a deep multi-task neural network composed of a deep material classifier and a deep denoising autoencoder was used for insulator defect detection. However, the high complexity of this model resulted in a slow detection speed. Furthermore, an infrared insulator image detection model based on improved feature fusion SSD was proposed (HBZheng, YHSun, XHLiu, etc. Infrared Image Detection of Substation Insulators Using an Improved Fusion Single Shot Multibox Detector[J]. IEEE Transactions on PowerDelivery, 2021, 36(6):3351-3359.) to achieve automatic detection of substation insulators.Although this method improves the detection speed, the detection accuracy decreases, and the detection accuracy is extremely low when the contrast between the insulator and the surrounding environment is low.

[0004] In order to solve the problem of the imbalance between retrieval accuracy and retrieval speed, a series of semantic segmentation methods have been proposed in the field of computer vision, including Fully Convolutional Networks (FCN)

[11] , U-network (U-net) and PSPnet network. Semantic segmentation methods are applied to insulator condition detection in complex backgrounds. Fast R-CNN is used for insulator detection, and then the detected insulators are cropped and segmented using FCN. Finally, fault detection is performed. The Unet network is improved to segment insulator strings, and then Convolutional Neural Network (CNN) is used for defect detection. Mask Regions with Convolutional Neural Network (Mask R-CNN) is used to automatically extract multiple insulators in infrared images, and then the temperature distribution of each insulator is extracted by function fitting. Machine language is used to establish relevant standard rules to evaluate the insulator condition. Although the above techniques have all segmented the insulators better, the detection speed has not achieved good performance. Moreover, most of them only segmented the insulators and did not combine insulator segmentation with insulator defect detection.

[0005] Therefore, the urgent technical problem to be solved is: how to effectively improve the detection speed of insulator defects while ensuring detection accuracy. Summary of the Invention

[0006] The purpose of this application is to provide a rapid insulator defect detection method and system based on deep learning, which can effectively improve the detection speed of insulator defects while ensuring detection accuracy.

[0007] To achieve the above objectives, this application provides a rapid insulator defect detection method based on deep learning, the method comprising:

[0008] Acquire an image of the insulator to be inspected;

[0009] Based on a pre-trained insulator segmentation model, the insulators in the image of the insulator to be detected are segmented, and the segmentation result image of the insulator is output.

[0010] The insulator segmentation effect image is input into the pre-trained insulator defect detection model to perform defect detection on the insulator and output the insulator defect detection results.

[0011] As mentioned above, the training method for the insulator segmentation model includes the following steps:

[0012] The standard dataset is input into the pyramid scene parsing network model for pre-training to obtain the pre-trained model;

[0013] The labeled insulator images are input into the pre-trained pyramid scene parsing network model to further train the parameters and obtain the insulator segmentation model.

[0014] As mentioned above, the method of inputting the labeled insulator image into the pre-trained pyramid scene parsing network model to further train the parameters and obtain the insulator segmentation model includes:

[0015] Collect insulator images, annotate the insulator images with insulator contours, generate corresponding labels, obtain an insulator dataset, and divide the insulator dataset into training set, validation set and test set;

[0016] The training and validation sets are input into the pre-trained pyramid scene parsing network model, the parameters of the pyramid scene parsing network model are optimized and adjusted, the loss function is calculated, and the loss function and the weights of the pyramid scene parsing network model are output.

[0017] The output weights are introduced into the pyramid scene parsing network model as an insulator segmentation model, which is used to segment insulators in the insulator image to be detected.

[0018] As shown above, segmenting the insulators in the image of the insulator to be detected includes the following sub-steps:

[0019] A backbone residual network is used to extract features from the insulator images to be detected, thus obtaining the basic feature layer.

[0020] The pyramid pooling module is used to extract enhanced features from the basic feature layer to obtain the enhanced feature layer.

[0021] The feature layer will be enhanced by fusing contextual information from different regions in the image of the insulator to be detected, thereby generating a global feature layer.

[0022] The basic feature layer and the global feature layer are aggregated to obtain an insulator image feature layer containing rich semantic information;

[0023] Each pixel in the feature layer of the insulator image is classified to obtain the insulator segmentation result, and the insulator segmentation effect image is output.

[0024] As mentioned above, the method for insulator defect detection using the insulator defect detection model includes the following steps:

[0025] The insulator segmentation effect image is input into the defect detection backbone network for basic feature extraction to obtain the defect detection basic feature layer.

[0026] The prediction results are obtained from the basic feature layer of defect detection;

[0027] The prediction results are decoded and nonmaximum suppression (NMS) is applied to generate the final defect detection results.

[0028] As mentioned above, the methods for obtaining prediction results from the defect detection basic feature layer include:

[0029] Six feature layers are extracted from the basic feature layer of defect detection as effective feature layers;

[0030] A convolution operation of num_anchors×4 is performed on each extracted effective feature layer; where num_anchors represents the number of prior boxes for each feature point in the effective feature layer; 4 represents the adjustment of four parameters: x_offset, y_offset, h, and w; x_offset represents the x-axis offset of the ground truth box from the center of the prior box; y_offset represents the y-axis offset of the ground truth box from the center of the prior box; h represents the change in height of the ground truth box relative to the prior box; and w represents the change in width of the ground truth box relative to the prior box.

[0031] Predict the offset of each prior box at each grid point on the effective feature layer, and adjust the corresponding prior box according to the offset to obtain the predicted box;

[0032] Then, perform a convolution operation of num_anchors×num_classes on the effective feature layer to predict the defect type corresponding to each prediction box at each grid point on the effective feature layer.

[0033] Here, num_classes represents the total number of defect categories.

[0034] As mentioned above, the training method for the insulator defect detection model includes the following steps:

[0035] Defect annotations are performed on the insulator segmentation effect image to generate corresponding labels and form an insulator defect dataset. The insulator defect dataset is divided into an insulator defect training set, an insulator defect verification set, and an insulator defect test set.

[0036] The SSD model is pre-trained using the VOC public dataset to obtain the pre-trained SSD model;

[0037] Input the insulator defect training set and the insulator defect validation set into the pre-trained SSD model;

[0038] Adjust the training parameters of the SSD model and output the loss function and weights of the SSD model;

[0039] The weights of the output SSD model are introduced into the SSD model to serve as an insulator defect detection model for defect detection of insulator images.

[0040] This application also provides a rapid insulator defect detection system based on deep learning, the system comprising:

[0041] An image acquisition device for acquiring images of insulators to be inspected;

[0042] The insulator segmentation unit is used to segment insulators in the image of insulators to be detected based on a pre-trained insulator segmentation model, and output the insulator segmentation result image.

[0043] The insulator defect detection unit is used to connect with the insulator segmentation unit to perform defect detection on the insulators in the insulator segmentation effect diagram output by the insulator segmentation unit, and output the insulator defect detection results.

[0044] As shown above, the insulator segmentation unit includes the insulator segmentation model.

[0045] The insulator segmentation model includes a data layer, a training layer, and a testing layer;

[0046] The data layer is used to collect insulator images, annotate the insulator images to generate corresponding labels, and divide the generated dataset into training set, validation set and test set;

[0047] The validation set is used to verify whether the parameters of the Pyramid Scene Parsing Network (PSPNet) model are overfitted.

[0048] The training layer is used to determine the parameters of the pyramid scene parsing network model, which includes the learning rate, number of iterations, batch size, and network weights for transfer learning.

[0049] The test layer is used to test the segmentation accuracy of the trained pyramid scene parsing network model for insulators by calculating evaluation metrics.

[0050] As mentioned above, the insulator defect detection unit includes an insulator defect detection model.

[0051] The insulator defect detection model includes a data layer, a training layer, and a testing layer;

[0052] The data layer is used to annotate the insulator segmentation effect image to generate corresponding labels and form a dataset, which is divided into training set, validation set and test set.

[0053] The training layer is used to determine the parameters of the SSD model, including the learning rate, number of iterations, and batch size.

[0054] The validation set is used to verify whether the parameters of the SSD model are overfitted.

[0055] The testing layer is used to test the accuracy of the trained SSD model in detecting insulator defects by calculating evaluation metrics.

[0056] The beneficial effects achieved by this application are as follows:

[0057] (1) This application proposes to optimize the PSPNet model by using transfer learning, data augmentation technology and Dice loss, and then combine the optimized semantic segmentation network IPSPNet model and the target detection model SSD to generate an insulator defect detection algorithm, which simplifies the detection process and improves the detection speed at the same time.

[0058] (2) The performance of the algorithm proposed in this application is compared with that of insulator defect detection algorithms based on the Unet-SSD model, the Deeplab-SSD model, and the original PSPNet-SSD model. The test results show that the proposed fast insulator defect detection algorithm based on the IPSPNet-SSD model has improved detection accuracy and shortened detection time. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0060] Figure 1 This is a flowchart illustrating a rapid insulator defect detection method based on deep learning, as exemplified by this application.

[0061] Figure 2 This is a flowchart illustrating the training method for an insulator segmentation model according to an embodiment of this application.

[0062] Figure 3 This is a flowchart illustrating the method for obtaining an insulator segmentation model according to an embodiment of this application.

[0063] Figure 4 This is a flowchart illustrating a method for segmenting insulators in an image of an insulator to be detected, according to an embodiment of this application.

[0064] Figure 5 This is a flowchart illustrating a method for detecting defects in insulators using an insulator defect detection model, as described in an embodiment of this application.

[0065] Figure 6 This is a flowchart illustrating a method for obtaining prediction results from a defect detection basic feature layer according to an embodiment of this application.

[0066] Figure 7 This is a flowchart illustrating a method for decoding the prediction results and performing non-maximum suppression processing to generate the final defect detection results, as described in this application embodiment.

[0067] Figure 8 This is a flowchart illustrating a method for testing insulator segmentation according to an embodiment of this application.

[0068] Figure 9 This is a flowchart illustrating a method for detecting defects in insulators according to an embodiment of this application.

[0069] Figure 10 This is a flowchart illustrating the training method for the insulator defect detection model according to an embodiment of this application.

[0070] Figure 11 This is a schematic diagram of the IPSPNet network model according to an embodiment of this application.

[0071] Figure 12 This is a schematic diagram of a rapid insulator defect detection system based on deep learning, according to an embodiment of this application.

[0072] Reference numerals: 10-Image acquisition device; 20-Insulator segmentation unit; 30-Insulator defect detection unit; 100-Insulator defect detection system. Detailed Implementation

[0073] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0074] Example 1

[0075] like Figure 12 As shown, this application provides a rapid insulator defect detection system 100 based on deep learning, the system comprising:

[0076] Image acquisition device 10 is used to acquire images of the insulator to be inspected.

[0077] The insulator segmentation unit 20 is used to segment insulators in the image of insulators to be detected based on a pre-trained insulator segmentation model, and to calculate the average cross-interference-union ratio and average pixel accuracy of insulator segmentation, and output the insulator segmentation effect image, the average cross-interference-union ratio and the average pixel accuracy of insulator segmentation.

[0078] The insulator segmentation crossover ratio and the insulator segmentation pixel accuracy are used to evaluate the accuracy of the insulator segmentation effect diagram.

[0079] The insulator segmentation unit 20 includes an insulator segmentation model, which is mainly based on the IPSPNet model (an improved pyramid scene parsing network model) for insulator segmentation. Figure 11 The image shows the IPSPNet network model. The IPSPNet model consists of a feature extraction module and a pyramid pooling module. The feature extraction module primarily uses a ResNet network to extract features from insulators in the image, generating a basic feature layer. The pyramid pooling module includes pooling, convolution, upsampling, and layer resolution modules. The pyramid pooling module is mainly used to acquire contextual information from different levels and then fuse them to generate a global feature map, thereby improving the ability to acquire global information.

[0080] The insulator segmentation model comprises a data layer, a training layer, and a testing layer. The data layer collects and labels insulator images, dividing the dataset into training, validation, and testing sets. The training layer determines the parameters of IPSPNet, including the learning rate, number of iterations, batch size, and transfer learning network weights. The validation set is used to verify whether the parameters are overfitting. Finally, the testing layer calculates evaluation metrics to test the segmentation accuracy of the trained IPSPNet model for insulators.

[0081] The insulator defect detection unit 30 is connected to the insulator segmentation unit 20 to perform defect detection on the insulators in the insulator segmentation effect diagram output by the insulator segmentation unit and output the insulator defect detection results.

[0082] The insulator defect detection unit 30 includes an insulator defect detection model. This model is primarily based on SSD (Single-Stage Real-Time Detection Model) for insulator defect detection. SSD is a fast single-stage target detection model consisting of a feature extraction module and a prediction module. The feature extraction module mainly improves upon VGG for feature extraction of insulator defects to generate a basic feature layer. The prediction module obtains the detection results from the basic features and then performs NMS processing on the detection results to generate the final defect detection result.

[0083] The insulator defect detection model comprises a data layer, a training layer, and a testing layer. The data layer annotates the insulator segmentation images to generate corresponding labels, forming a dataset which is then divided into training, validation, and testing sets. The training layer determines the parameters of the SSD model, including the learning rate, number of iterations, and batch size. The validation set is used to verify whether the parameters are overfitting. Finally, the testing layer calculates evaluation metrics to test the accuracy of the trained SSD model in detecting insulator defects.

[0084] Example 2

[0085] like Figure 1 As shown, this application provides a fast insulator defect detection method based on deep learning, which includes the following steps:

[0086] Step S1: Obtain an image of the insulator to be detected.

[0087] The method for obtaining the image of the insulator to be detected is to acquire the image of the insulator through aerial photography.

[0088] Step S2: Based on the pre-trained insulator segmentation model, segment the insulators in the insulator image to be detected and output the insulator segmentation result image.

[0089] Specifically, based on the pre-trained insulator segmentation model, the insulators in the insulator image to be detected are segmented, and the average cross-interference-union ratio and average pixel accuracy of the insulator segmentation are calculated. The insulator segmentation result image, the average cross-interference-union ratio and the average pixel accuracy of the insulator segmentation are output.

[0090] like Figure 2 As shown, the training method for the insulator segmentation model includes the following steps:

[0091] Step T1: Input the PASCALVOC standard dataset into the pyramid scene parsing network model, fine-tune the parameters for pre-training, and obtain the pre-trained model.

[0092] The PASCALVOC dataset provides a standardized dataset for image recognition and classification, serving as a benchmark for measuring image classification and recognition capabilities, and is a public dataset for insulator image recognition. The PASCALVOC dataset is input into the IPSPNet model for fine-tuning and pre-training, resulting in a pre-trained IPSPNet model. This pre-trained IPSPNet model has learned a large number of features for insulator images with common geometric shapes.

[0093] Step T2 involves inputting the labeled insulator image into the pre-trained pyramid scene parsing network model to further train the parameters and obtain the insulator segmentation model.

[0094] Step T2 is executed based on step T1. The training process in step T2 is accelerated, and the convergence speed and accuracy of the insulator segmentation method are improved.

[0095] Among them, the labeled insulator image refers to the image that is drawn point by point along the outline of the insulator and labeled as insulator.

[0096] like Figure 3 As shown, step T2 includes the following sub-steps:

[0097] Step T210: Collect insulator images, annotate the insulator images with insulator outlines, generate corresponding labels, and obtain an insulator dataset.

[0098] The insulator dataset is divided into a training set, a validation set, and a test set.

[0099] The insulator dataset with insulator outline annotations was divided into training, validation, and test sets in a 9:1:1 ratio.

[0100] Step T220: Input the training set and validation set into the pre-trained IPSPNet model, optimize and adjust the parameters of the IPSPNet model, calculate the loss function, and output the loss function and the weights of the IPSPNet model.

[0101] The parameters of the IPSPNet model include the learning rate, number of iterations, batch size, and network weights for transfer learning.

[0102] The calculation of the loss function includes two parts: the cross-classification entropy loss function and the Dice loss function.

[0103] The first part, the cross-entropy classification loss function, is defined as follows:

[0104]

[0105] Where l(y,z) represents the cross-entropy classification loss function with y and z as independent variables; f(z) k ) represents the insulator segmentation detection value in the insulator image input to the pre-trained IPSPNet model, y k This represents the true segmentation value of the insulator in the input insulator image, where c represents the number of input insulator images, and k represents the k-th input insulator image. Specifically, the insulator segmentation detection value refers to the classification result of the detector on the insulator pixels in the input insulator image, and the classification result displays the insulator pixels in the input image; the true segmentation value refers to the result of manually drawing the insulator outline point by point in the insulator image, that is, the manually labeled insulator pixel value.

[0106] The cross-entropy classification loss function classifies insulators by comparing each pixel one by one to obtain the loss value. The loss of the entire image is the average of the losses of each pixel. It is mainly used for insulator classification.

[0107] Cross-entropy loss function is suitable for most semantic segmentation scenarios. However, when performing semantic segmentation on insulators in aerial images of insulators in this application, the number of insulator pixels is much smaller than the number of background pixels, resulting in a problem of class imbalance. In this case, background loss dominates, leading to poor network performance. Therefore, we propose to use Dice loss function to balance the number of insulator and background pixels, so as to achieve more accurate classification of insulator pixels.

[0108] The second part, the Dice loss function, is defined as follows:

[0109] Dice loss = 1 - s;

[0110] Wherein, Dice loss represents the Dice loss function; s represents the Dice coefficient, and the Dice coefficient takes values ​​in the range of [0, 1]. The larger the Dice coefficient, the greater the overlap between the detected value and the actual value of the insulator segmentation. Therefore, the larger the Dice coefficient, the better.

[0111] The Dice coefficient is defined as follows:

[0112]

[0113] Where X represents the detected insulator segmentation value, and Y represents the actual insulator segmentation value. Here, X is related to the aforementioned f(z). k ) are the same data; Y is the same as the above y k It's the same data.

[0114] Step T230: The weights of the output IPSPNet model are introduced into the IPSPNet model as an insulator segmentation model, which is used to segment the insulators in the image of the insulator to be detected.

[0115] Step T240: Test the insulator segmentation model.

[0116] Input the insulator image and corresponding label into the insulator segmentation model to segment the insulator and obtain MAP (Mean Intersection over Union) and MIoU (Mean Pixel Accuracy) values; output the MAP value, MIoU value and the insulator segmentation result image.

[0117] When an insulator image and its corresponding insulator label are input into the insulator segmentation model, the insulator segmentation model segments the insulators in the insulator image, calculates the MAP (Mean Intersection over Union) value and MIoU (Mean Pixel Accuracy) value, and outputs the MAP value, MIoU value and the insulator segmentation result image.

[0118] MAP and MIoU values ​​are used to comprehensively evaluate the performance of the insulator segmentation algorithm.

[0119] The pixel precision for a single class is the ratio of the intersection of the true value and the predicted value for that class to the true value. The MAP value is calculated as follows:

[0120]

[0121] Where P represents the predicted insulator segmentation value, which is the predicted set of insulator pixels in the insulator image; GT represents the set of insulator pixels in the manually annotated insulator image, which is the true insulator segmentation value.

[0122] The Intersection over Union (MIoU) ratio for a single class is the ratio of the intersection and union of the true values ​​and the predicted values ​​for that class. The MIoU value is calculated as follows:

[0123]

[0124] Wherein, TP represents a positive sample predicted as positive by the model (true positive); FN represents a positive sample predicted as negative by the model (false negative); and FP represents a negative sample predicted as positive by the model (false positive).

[0125] like Figure 4 As shown, segmenting the insulators in the image of the insulator to be detected includes the following sub-steps:

[0126] Step S210: Use the Residual Network (ResNet50) to extract features from the insulator in the image of the insulator to be detected, and obtain the basic feature layer.

[0127] The Residual Network (ResNet50) builds upon traditional CNNs by introducing the concept of residual structures. By stacking residual structures, the network becomes increasingly deeper, solving the gradient vanishing problem in deep neural networks while maintaining excellent accuracy. ResNet50 primarily consists of two types of stacked residual structures: convolutional blocks and ontology blocks. Convolutional blocks have different input and output dimensions, so they cannot be concatenated; this is used to change the network's dimensionality. Ontology blocks have the same input and output dimensions and can be concatenated to deepen the network.

[0128] A convolutional block is mainly composed of two superimposed parts. The first part is generated by performing a second convolution, batch normalization, and ReLU activation on the input image, followed by a single convolution and batch normalization. The second part is generated by performing a single convolution and batch normalization on the input image, producing a feature layer of the same size as the first part. The first and second feature layers are then superimposed. Finally, the superimposed feature layer is input into the ReLU activation function to generate the feature layer output by the convolutional block.

[0129] The ontology block also consists of two parts. The first part is generated by passing the input image through a second convolution, batch normalization, and ReLU activation operation, followed by a single convolution and batch normalization operation. Finally, the feature layer generated in the first part is superimposed on the input image and input into the ReLU activation function to generate the feature layer output by the ontology block.

[0130] The advantages of using the ReLU activation function are: it increases the non-linear relationship between the layers of the neural network. Otherwise, without the activation function, the relationship between layers is a simple linear one, and each layer is equivalent to matrix multiplication, which cannot complete the complex tasks that the neural network needs to perform. Therefore, after using the ReLU activation function, the computational cost of the entire feature extraction process is greatly reduced, enabling the training of deep networks. The ReLU activation function will cause some neurons to output 0, which will create the sparsity of the network and reduce the interdependence of parameters, thus alleviating the overfitting problem.

[0131] In the ResNet50 network architecture, the image input is first zero-padding, followed by single convolution, batch normalization, activation, and max pooling operations. Then, it is passed sequentially through single convolutional blocks and multiple ontology blocks, and finally average pooling and fully connected operations are performed to output the basic feature layer after deep feature extraction.

[0132] Step S220: Using the pyramid pooling module, perform enhanced feature extraction on the basic feature layer to obtain the enhanced feature layer.

[0133] Specifically, the basic feature layer is divided into regions of different sizes, and the contextual information within each region is integrated. This contextual information is the insulator feature information. First, the input basic feature layer is divided into regions and average pooled. Then, convolution operations are used to adjust the number of channels and increase the feature layer area. Next, 3×3 convolutions are used to integrate the features. Finally, 1×1 convolutions are used to adjust the number of channels to the number of classes, and the image size is resized so that the output layer's width and height are the same as the input image of the overall PSPNet network model.

[0134] Step S230 involves fusing contextual information from different regions in the image of the insulator to be detected into an enhanced feature layer to generate a global feature layer.

[0135] Step S240: Aggregate the basic feature layer and the global feature layer to obtain an insulator image feature layer containing rich semantic information.

[0136] Step S250: Classify each pixel in the feature layer of the insulator image to obtain the insulator segmentation result, and output the insulator segmentation effect image.

[0137] Step S3: Input the insulator segmentation effect image into the pre-trained insulator defect detection model to perform defect detection on the insulator and output the insulator defect detection results.

[0138] like Figure 5 As shown, the insulator defect detection model includes the following steps for defect detection in insulators:

[0139] Step S310: Input the insulator segmentation effect image into the defect detection backbone network for basic feature extraction to obtain the defect detection basic feature layer.

[0140] Specifically, the defect detection backbone network is an improved VGG network. The fully connected layers of VGG16 are converted into convolutional layers, all dropout layers are removed, and multiple convolutional layers are added after the VGG network. Finally, the insulator segmentation image is input into a network composed of several convolutional layers for basic feature extraction, with the aim of better learning defect features.

[0141] Step S320: Obtain the prediction results from the defect detection basic feature layer.

[0142] like Figure 6 As shown, step S320 includes the following sub-steps:

[0143] Step S321: Extract six feature layers from the defect detection basic feature layer as effective feature layers. The effective feature layers are used to obtain the prediction results.

[0144] Step S322: Perform a num_anchors×4 convolution operation on each extracted effective feature layer.

[0145] In this context, num_anchors×4 represents the number of prior boxes for each feature point in the effective feature layer; 4 represents the adjustment values ​​of the four parameters x_offset, y_offset, h, and w; x_offset represents the offset of the ground truth box from the center of the prior box along the x-axis; y_offset represents the offset of the ground truth box from the center of the prior box along the y-axis; h represents the change in the height of the ground truth box relative to the prior box; and w represents the change in the width of the ground truth box relative to the prior box. The ground truth box refers to the rectangular box that the staff pre-generated at the defect location using the labelmg software.

[0146] Step S323: Predict the offset of each prior box at each grid point on the effective feature layer, and adjust the corresponding prior box according to the offset to obtain the predicted box.

[0147] Step S324: Perform a convolution operation of num_anchors×num_classes on the effective feature layer to predict the defect type corresponding to each prediction box at each grid point on the effective feature layer.

[0148] Here, num_classes represents the total number of defect categories, that is, the total number of defect categories trained by the insulator defect detection model.

[0149] Step S330: Decode the prediction results and perform non-maximum suppression (NMS) processing to generate the final defect detection results.

[0150] like Figure 7 As shown, step S330 includes the following sub-steps:

[0151] Step S331: Add the corresponding offset distances (x_offset and y_offset) between the center point coordinates of the grid and the center of the prior box to generate the center point coordinates of the predicted box.

[0152] Step S332: Adjust the prior box using the relative values ​​h and w of the ground truth box to obtain the height and width of the predicted box, and draw the predicted box on the image.

[0153] Specifically, the defect detection results of each insulator are decoded to generate a series of detection boxes. The decoding process involves adding the deviation of the real box from the center of the prior box to the center of each grid. The result after adding is the center of the detection box. Then, the changes in the length and width of the prior box and the real box relative to the prior box are combined to calculate the length and width of the predicted box. In this way, the position of the entire predicted box can be obtained.

[0154] Step S333: Calculate the confidence score of each predicted box, extract the detection boxes with confidence scores greater than the confidence threshold, and generate detection boxes.

[0155] The confidence threshold is preset, and the confidence threshold of this application is set to 0.5. The confidence score refers to the probability score of belonging to a certain defect category.

[0156] The confidence score of each predicted box is calculated by the degree of overlap between the ground truth box and the prior box. The confidence scores of each predicted box are sorted. The detection boxes with confidence scores greater than a specified confidence threshold are extracted from the predicted boxes. Non-maximum suppression is performed using the detection boxes and confidence scores to remove duplicate detection boxes of the same target. The detection box with the highest confidence score and its corresponding confidence score are then output.

[0157] Step S334: Perform non-maximum suppression filtering on the position of the detection box and the probability score of its defect category to obtain a prediction result with only one defect detection box.

[0158] Among them, nonmaximum suppression screening selects the detection box with the highest confidence and outputs it.

[0159] like Figure 10 As shown, the training method for the insulator defect detection model includes the following steps:

[0160] Step Y1: Label the defects in the insulator segmentation image to generate corresponding labels and form an insulator defect dataset. Divide the insulator defect dataset into an insulator defect training set, an insulator defect verification set, and an insulator defect test set.

[0161] The insulator dataset with defect annotations was divided into an insulator defect training set, an insulator defect verification set, and an insulator defect test set in a ratio of 9:1:1.

[0162] Step Y2: Train the SSD model in advance using the VOC public dataset to obtain the pre-trained SSD model.

[0163] Step Y3: Input the insulator defect training set and the insulator defect validation set into the pre-trained SSD model.

[0164] Step Y4: Adjust the training parameters of the SSD model, output the loss function, and output the weights of the SSD model.

[0165] The loss functions used in the insulator defect detection model are the SmoothL1 regression loss function for generating insulator and defect prediction boxes, the cross-entropy classification loss function for predicting insulator and defect categories, and the cross-entropy classification loss function for predicting background categories.

[0166] The loss function used in the insulator defect detection model includes:

[0167] The first is the SmoothL1 regression loss function, defined as follows:

[0168]

[0169] Among them, Smooth l1 denoted as SmoothL1 regression loss function; x represents the difference between the predicted insulator defect result and the actual insulator defect result; else represents other cases.

[0170] The SmoothL1 regression loss function is used to calculate the defect detection box loss and output the insulator defect detection box. First, the loss is calculated by outputting the coordinates of the four corner points of the defect detection box, and then these are summed to obtain the final detection box loss value. The cross-entropy classification loss function classifies the insulator defect pixels by comparing each pixel individually, and is used for classifying the insulator defect pixels.

[0171] Second, the cross-entropy classification loss function:

[0172]

[0173] Among them, z k c1 represents the predicted insulator defect result (predicted insulator defect pixels); c1 represents the number of input insulator segmentation images; y1 k This represents the actual results of insulator defects (manually marked actual defect pixels).

[0174] Step Y5: The weights of the output SSD model are introduced into the SSD model as an insulator defect detection model, which is used to detect defects in insulator images.

[0175] Step Y6: Test the insulator defect detection model.

[0176] The segmented image of the insulator after defect labeling and the corresponding labels are input into the SSD model to detect insulator defects and calculate the MAP value; the MAP value and the insulator defect detection image are output.

[0177] Furthermore, this application proposes testing methods for two parts: insulator segmentation and insulator defect detection. To ensure testing efficiency and training speed, this application uses GPU devices for both training and testing. The testing environment is as follows: CPU: Intel(R) Core(TM) i7-10875H CPU@2.30GHz; Graphics card: NVIDIA GeForce RTX 2060; Memory: 32GB; Deep learning framework: Keras=2.1.5; Python version: Python 3.6; CUDA (a general-purpose parallel computing architecture launched by NVIDIA): CUDA 10.0; Cudnn: Cudnn 7.4.1.

[0178] like Figure 8 As shown, the method for testing insulator segmentation includes the following steps:

[0179] Step S410: Obtain the insulator segmentation dataset and expand the dataset using data augmentation techniques.

[0180] As a specific embodiment of the present invention, the expanded dataset contains a total of 889 images.

[0181] Step S420: Manually label the insulators on the insulator image using LabelMe software.

[0182] As a specific embodiment of the present invention, when labeling insulators, the insulator outline is first drawn point by point and its label is set as insulator. Then, the defect outline is drawn point by point and its label is set as defect. Other areas are the background. Then, a label JSON file is generated and used for training network weights.

[0183] Step S430: The data obtained from both the original insulator image (img.jpg) and the label image (label.png) is used as the final dataset.

[0184] The labeled image is a diagram that outlines the defects in the insulator.

[0185] As a specific embodiment of the present invention, the generated dataset is divided into a training set, a validation set, and a test set in a ratio of 9:1:1.

[0186] Step S440: Input the training set into the network model for training.

[0187] As a specific embodiment of the present invention, the trained weights are continuously validated using a validation set during the training process to obtain the weights with the best performance and output them. The performance of the trained network weight model is then tested using a test set.

[0188] Transfer learning and dynamic learning rate were introduced during the training of the IPSPNet segmentation method. First, the IPSPNet segmentation method was pre-trained using the VOC dataset to output pre-trained model weights. Then, based on the pre-trained weight model, a pre-made insulator segmentation dataset was input to extract insulator image features. Finally, the network model was continuously trained by comparing the loss function and convergence speed under dynamic learning rate changes, selecting the optimal learning rate. A training set of 727 images was used for each round of training, while a validation set of 80 images was used to avoid parameter overfitting and to verify network performance after each round of training. Early stopping was used to record the best validation set accuracy during training. The process was set such that if the validation loss did not decrease after every ten rounds of overall training, it indicated that the model had basically converged, and training was stopped.

[0189] The basic network parameters of the IPSPNet model are shown in the table below:

[0190]

[0191] Here, 5e-4 and 1e-4 represent the learning rate. One epoch (epoch) refers to training all the training data completely once, which means training the data once in a forward-backward direction.

[0192] The proposed method in this application is compared with insulator segmentation methods based on the Unet and Deeplab models to verify the performance of the proposed IPSPNet-based insulator segmentation method. The IPSPNet model was tested using 80 high-resolution insulator images, all uniformly scaled to 473×473 pixels. During the testing phase, Transfer Learning was first added to the original PSPNet model to generate a Transfer-PSPNet model. To further improve the accuracy of insulator segmentation, data augmentation techniques and Dice loss were added to generate the IPSPNet model, and insulator segmentation was performed under different models. The insulator segmentation methods based on the Unet model, Deeplab model, and PSPNet models with different optimization levels were tested and their performance compared. The insulator segmentation performance comparison is shown in the table below:

[0193]

[0194] As shown in the insulator segmentation performance comparison table, the optimized model in this application gradually improves the insulator segmentation accuracy. The IPSPNet model has the highest pixel accuracy (MPA) of 92.52%, and its Intersection over Union (MIoU) for insulator segmentation is 11.14% higher than that of the original PSPNet model. Among other methods, the Deeplab model has low insulator segmentation accuracy and poor performance, while the Unet model's insulator segmentation performance is lower than that of the optimized PSPNet model.

[0195] like Figure 9 As shown, the method for detecting defects in insulators includes the following steps:

[0196] Step S510: Use the image output by the insulator segmentation method as the input to the insulator defect detection algorithm.

[0197] The insulator defect detection dataset is divided into training, validation, and test sets in a 9:1:1 ratio.

[0198] Step S520: Combine the insulator segmentation image (img.jpg) with the generated label file (img.xml) to form a dataset.

[0199] As a specific embodiment of the present invention, manual annotation is performed using labelImg software. During annotation, rectangular boxes are used to mark defective parts of the insulator, and their labels are set to "defect," while other areas are left as background. After annotation, an XML file is generated, and the insulator segmentation image (img.jpg) and the label file (img.xml) together form a dataset.

[0200] Step S530: Incorporate transfer learning and dynamic learning rate into the training process of the insulator defect detection method.

[0201] First, the SSD model is pre-trained using the VOC dataset, and the pre-trained model weights are output. Then, based on the pre-trained model weights, transfer learning is used to train the SSD model to learn the image features of insulator defects. Finally, the model is trained by comparing the loss function and convergence speed while continuously adjusting the learning rate, and the optimal learning rate is selected for further training. Early stopping is used to record the best validation set accuracy during training; training is stopped if the validation loss does not decrease after ten training epochs.

[0202] The basic network parameters of the SSD model are shown in the table below:

[0203]

[0204] To better evaluate the performance of the insulator defect detection algorithm, this application assesses both the model's detection accuracy and computational speed. MAP (Magnetic Mapping Analysis) is used to evaluate detection accuracy, while the model's training time and the time spent detecting a single image are used to evaluate computational speed.

[0205] The proposed method is compared with three different insulator defect detection algorithms: Unet-SSD, Deeplab-SSD, and the original PSPNet-SSD. The comparison results are shown in the table below:

[0206]

[0207]

[0208] As shown in the table above, the insulator defect detection algorithm based on IPSPNet-SSD proposed in this application has the shortest training time of 167 minutes and the lowest detection speed of 48ms. Among the four defect detection algorithms, under the same equipment conditions, the insulator defect detection algorithm based on Unet-SSD has a relatively smaller number of training epochs, but takes the longest to train, and the average detection time per image is also relatively long. The Deeplab-SSD model has the fewest epochs and a relatively short training time, but the detection speed per image is the slowest. The original PSPNet-SSD model has the longest number of epochs and a long training time, but the detection speed per image is relatively fast. The algorithm proposed in this application introduces transfer learning, data augmentation techniques, and Dice loss on the basis of the original PSPNet-SSD model insulator defect detection algorithm. Compared with the original algorithm, it reduces the number of epochs and greatly shortens the training time. Compared with the Unet-SSD model and the Deeplab-SSD model, it greatly speeds up the detection speed, reducing the detection time per image to 48ms.

[0209] This application compares the Double-M2Det algorithm and the IPSPNet-SSD algorithm, and objectively analyzes the accuracy and speed of the algorithms, as shown in the table below:

[0210]

[0211] Analysis shows that the IPSPNet-SSD algorithm proposed in this application not only guarantees high average accuracy performance, but also shortens training time and improves model running speed, increasing the detection speed by 20ms / image. This is because the M2Det model is complex, and the insulator defect detection algorithm based on the Double-M2Det algorithm has a cumbersome detection process, resulting in long training time and decreased detection speed performance.

[0212] The beneficial effects achieved by this application are as follows:

[0213] (1) This application proposes to optimize the PSPNet model by using transfer learning, data augmentation technology and Dice loss, and then combine the optimized semantic segmentation network IPSPNet model and the target detection model SSD to generate an insulator defect detection algorithm, which simplifies the detection process and improves the detection speed at the same time.

[0214] (2) The performance of the algorithm proposed in this application is compared with that of insulator defect detection algorithms based on the Unet-SSD model, the Deeplab-SSD model, and the original PSPNet-SSD model. The test results show that the proposed fast insulator defect detection algorithm based on the IPSPNet-SSD model has improved detection accuracy and shortened detection time.

[0215] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A rapid insulator defect detection method based on deep learning, characterized in that, The method includes: Acquire an image of the insulator to be inspected; Based on a pre-trained insulator segmentation model, insulators in the image to be detected are segmented, and the segmentation result map is output. The insulator segmentation model is based on the improved pyramid scene parsing network IPSPNet. The IPSPNet model consists of a feature extraction module and a pyramid pooling module. The feature extraction module uses a ResNet network to extract features from the insulators in the image to generate a basic feature layer. The pyramid pooling module includes a pooling module, a convolution module, an upsampling module, and a layer parsing module. The pyramid pooling module is used to obtain contextual information at different levels and then fuse them to generate a global feature map. The insulator segmentation effect image is input into the pre-trained insulator defect detection model to perform defect detection on the insulator and output the insulator defect detection results. The training method for the insulator segmentation model includes the following steps: The PASCAL VOC standard dataset is input into the pyramid scene parsing network model for pre-training to obtain the pre-trained model. The labeled insulator images are input into the pre-trained pyramid scene parsing network model to further train the parameters and obtain the insulator segmentation model. One method for obtaining the insulator segmentation model by inputting the labeled insulator image into a pre-trained pyramid scene parsing network model to further train the parameters includes: Collect insulator images, annotate the insulator images with insulator contours, generate corresponding labels, obtain an insulator dataset, and divide the insulator dataset into training set, validation set and test set; The training and validation sets are input into the pre-trained pyramid scene parsing network model, the parameters of the pyramid scene parsing network model are optimized and adjusted, the loss function is calculated, and the loss function and the weights of the pyramid scene parsing network model are output. The output weights are introduced into the pyramid scene parsing network model as an insulator segmentation model, which is used to segment insulators in the insulator image to be detected. The segmentation of insulators in the image of insulators to be detected includes the following sub-steps: A backbone residual network is used to extract features from the insulator images to be detected, thus obtaining the basic feature layer. The pyramid pooling module is used to extract enhanced features from the basic feature layer to obtain the enhanced feature layer. The feature layer will be enhanced by fusing contextual information from different regions in the image of the insulator to be detected, thereby generating a global feature layer. The basic feature layer and the global feature layer are aggregated to obtain an insulator image feature layer containing rich semantic information; Each pixel in the feature layer of the insulator image is classified to obtain the insulator segmentation result, and the insulator segmentation effect image is output. The backbone residual network is composed of two stacked residual structures: convolutional blocks and bulk blocks. Convolutional blocks have different input and output dimensions, used to change the network dimensions; ontology blocks have the same input and output dimensions, used to deepen the network. The convolutional block consists of two superimposed parts. The first part is generated by passing the input image through double convolution, batch normalization, and ReLU activation operations, followed by single convolution and batch normalization operations. The second part is generated by passing the input image through only single convolution and batch normalization operations, generating a feature layer of the same size as the first part. The first and second feature layers are then superimposed. Finally, the superimposed feature layer is input into the ReLU activation function to generate the feature layer output by the convolutional block. The ontology block consists of two parts. The first part is to process the input image through a second convolution, batch normalization and ReLU activation operation, and then through a single convolution and batch normalization operation to generate the first part of the feature layer. Finally, the feature layer generated in the first part and the input image are superimposed and input into the ReLU activation function to generate the feature layer output by the ontology block. The calculation of the loss function includes two parts: the cross-classification entropy loss function and the Dice loss function; The cross-entropy classification loss function is defined as follows: ； in, Let represent the cross-entropy classification loss function with y and z as independent variables; This represents the insulator segmentation detection value in the insulator image input to the pre-trained IPSPNet model. This represents the true value of the insulator segmentation in the input insulator image, and c represents the number of input insulator images; Indicates the input number of the first... Image of an insulator; Among them, the insulator segmentation detection value refers to the classification result of the detector on the insulator pixels in the input insulator image, and the classification result shows the insulator pixels in the input image; the insulator segmentation true value refers to the result of manually drawing point by point along the insulator outline in the insulator image. The Dice loss function is defined as follows: ； in, Represents the Dice loss function; This represents the Dice coefficient, which takes values ​​in the range [0, 1]. The Dice coefficient is defined as follows: ； Where X represents the insulator segmentation detection value, and Y represents the insulator segmentation actual value.

2. The rapid insulator defect detection method based on deep learning according to claim 1, characterized in that, The insulator defect detection model for detecting defects in insulators includes the following steps: The insulator segmentation effect image is input into the defect detection backbone network for basic feature extraction to obtain the defect detection basic feature layer. The prediction results are obtained from the basic feature layer of defect detection; The prediction results are decoded and non-maximum suppression is applied to generate the final defect detection results.

3. The rapid insulator defect detection method based on deep learning according to claim 2, characterized in that, Methods for obtaining prediction results from the defect detection basic feature layer include: Six feature layers are extracted from the basic feature layer of defect detection as effective feature layers; Perform on each extracted effective feature layer The convolution operation; where, In This indicates the number of prior boxes that each feature point in the effective feature layer possesses; express , , and The adjustment status of the four parameters; This indicates the offset of the true bounding box from the center of the prior bounding box along the x-axis. This indicates the offset of the ground truth bounding box from the center of the prior bounding box along the y-axis. This indicates how the height of the actual bounding box changes relative to the prior bounding box. This indicates how the width of the actual bounding box changes relative to the prior bounding box. Predict the offset of each prior box at each grid point on the effective feature layer, and adjust the corresponding prior box according to the offset to obtain the predicted box; Then perform the effective feature layer The convolution operation predicts the type of defect corresponding to each prediction box at each grid point on the effective feature layer. in, This indicates the total number of insulator defect categories.

4. The rapid insulator defect detection method based on deep learning according to claim 1, characterized in that, The training method for the insulator defect detection model includes the following steps: Defect annotations are performed on the insulator segmentation effect image to generate corresponding labels and form an insulator defect dataset. The insulator defect dataset is divided into an insulator defect training set, an insulator defect verification set, and an insulator defect test set. The SSD model is pre-trained using the VOC public dataset to obtain the pre-trained SSD model; Input the insulator defect training set and the insulator defect validation set into the pre-trained SSD model; Adjust the training parameters of the SSD model and output the loss function and weights of the SSD model; The weights of the output SSD model are introduced into the SSD model to serve as an insulator defect detection model for defect detection of insulator images.

5. A rapid insulator defect detection system based on deep learning, used to execute the method of any one of claims 1-4, characterized in that, The system includes: An image acquisition device for acquiring images of insulators to be inspected; The insulator segmentation unit is used to segment insulators in the image of insulators to be detected based on a pre-trained insulator segmentation model, and output the insulator segmentation result image. The insulator defect detection unit is used to connect with the insulator segmentation unit to perform defect detection on the insulators in the insulator segmentation effect diagram output by the insulator segmentation unit, and output the insulator defect detection results.

6. The rapid insulator defect detection system based on deep learning according to claim 5, characterized in that, Insulator segmentation unit includes insulator segmentation model, The insulator segmentation model includes a data layer, a training layer, and a testing layer; The data layer is used to collect insulator images, annotate the insulator images to generate corresponding labels, and divide the generated dataset into training set, validation set and test set; The validation set is used to verify whether the parameters of the pyramid scene parsing network model are overfitted. The training layer is used to determine the parameters of the pyramid scene parsing network model, which includes the learning rate, number of iterations, batch size, and network weights for transfer learning. The test layer is used to test the segmentation accuracy of the trained pyramid scene parsing network model for insulators by calculating evaluation metrics.

7. The rapid insulator defect detection system based on deep learning according to claim 5, characterized in that, The insulator defect detection unit includes an insulator defect detection model. The insulator defect detection model includes a data layer, a training layer, and a testing layer; The data layer is used to annotate the insulator segmentation effect image to generate corresponding labels and form a dataset, which is divided into training set, validation set and test set. The training layer is used to determine the parameters of the SSD model, including the learning rate, number of iterations, and batch size. The validation set is used to verify whether the parameters of the SSD model are overfitted. The testing layer is used to test the accuracy of the trained SSD model in detecting insulator defects by calculating evaluation metrics.

Images