Lightweight power transmission line insulator defect detection method based on knowledge distillation

Abstract

The invention discloses a lightweight power transmission line insulator defect detection method based on knowledge distillation, and relates to the technical field of intelligent power grid operation and maintenance. The detection method comprises the following steps: constructing an insulator defect detection model based on knowledge distillation, and inputting insulator image data collected in real time into the insulator defect detection model based on knowledge distillation to obtain an insulator defect detection result; the insulator defect detection model based on knowledge distillation comprises a feature extraction backbone network, an RPN layer, an ROI pooling layer, a classification head and a bounding box regression head; the feature extraction backbone network comprises a Gaussian mask distillation module and a non-local distillation module, and knowledge can be adaptively transmitted to students according to specific prior in tasks. A small student network is trained by transferring knowledge from a large teacher model based on the knowledge distillation module, the insulator defect recognition accuracy is improved, meanwhile, the insulator defect recognition method can be deployed on equipment with low calculation performance, and high practicability is achieved.

Description

Technical Field

[0001] The present invention relates to the field of smart grid operation and maintenance technology, and in particular to a lightweight transmission line insulator defect detection method based on knowledge distillation. Background Art

[0002] The safe and stable operation of transmission lines is crucial to ensuring the reliability of the power system. However, due to the influence of various factors such as the environment, equipment aging, and external force damage, transmission line failures occur frequently, especially for insulators, which are important components in transmission lines. Their defect detection has become an important part of ensuring the safety of the power system.

[0003] Traditional methods for detecting defects in insulators of power transmission lines mainly rely on manual inspection, signal processing, pattern recognition and other technologies. Although these methods can detect defects in insulators to a certain extent, they have the following limitations: First, the detection efficiency is low: manual inspection requires a lot of manpower and material resources, and it is difficult to cover all transmission lines; second, the detection accuracy is low: methods based on signal processing and pattern recognition often have the problem of low detection accuracy when facing complex and changeable insulator defects. Third, the real-time performance is poor: traditional methods are difficult to achieve real-time monitoring and early warning of insulator defects.

[0004] In recent years, with the rapid development of deep learning technology, it has achieved remarkable results in the fields of image processing and target detection. Deep learning technology has powerful feature learning and classification capabilities, and can automatically extract the features of input data to achieve classification and recognition of complex data. Therefore, the transmission line insulator defect detection method based on deep learning has gradually become a research hotspot. However, the application of deep learning technology to transmission line insulator defect detection also faces some challenges: such as high model complexity, strong data dependence, and poor knowledge transferability.

[0005] In order to solve the above problems, the present invention introduces knowledge distillation technology into the field of transmission line insulator defect detection. Knowledge distillation is a model compression and acceleration technology that distills the knowledge of a complex but high-performance teacher model into a simple student model, thereby reducing the complexity and computational complexity of the model while maintaining high detection accuracy. Summary of the invention

[0006] In view of the above shortcomings, the present invention provides a lightweight transmission line insulator defect detection method based on knowledge distillation, which can reduce the complexity and calculation amount of the defect detection model while maintaining a high defect detection accuracy. The specific technical solution is as follows:

[0007] A lightweight transmission line insulator defect detection method based on knowledge distillation, the detection method comprising: constructing an insulator defect detection model based on knowledge distillation, inputting insulator image data collected in real time into the insulator defect detection model based on knowledge distillation, and obtaining an insulator defect detection result;

[0008] The insulator defect detection model based on knowledge distillation includes: a feature extraction backbone network, an RPN layer, a ROI pooling layer, a classification head and a bounding box regression head; the feature extraction backbone network includes a Gaussian mask distillation module and a non-local distillation module, which can adaptively transfer knowledge to students according to specific priors in the task;

[0009] Furthermore, the feature extraction backbone network can be any two-stage target detection network, such as Faster RCNN, RetinaNet, Cascade Mask R-CNN, etc. Its working principle is to extract features from the input image through a series of convolutional layers, activation functions and pooling layers to obtain the corresponding feature map. Its purpose is to extract key information from the image and provide a basis for subsequent candidate region generation and classification; in particular, the present invention introduces a two-dimensional Gaussian mask in the backbone network to further highlight the foreground pixels of the insulator and suppress the boundary area around the insulator;

[0010] The Gaussian mask distillation module introduces a two-dimensional Gaussian mask to highlight the foreground pixels of the insulator and suppress the boundary area around the insulator. Based on the Gaussian mask operation, the feature extraction backbone network extracts image features by minimizing the following loss function:

[0011] in, The i, j, and c in the figure are the subscripts of the mathematical summation, indicating the position of the summation term. W, H, and C correspond to the width, height, and number of channels of the feature map, respectively. and They represent the image features extracted from the student backbone and the teacher backbone respectively, where s represents the student and t represents the teacher.

[0012] Furthermore, the non-local distillation module captures the relationship information between global insulator objects, which is expressed as follows: Among them, r is the calculated relationship information between objects, i, j are the spatial indices of the output positions of the response to be calculated, i', j' are the spatial indices that enumerate all possible positions; f is a pairwise function used to calculate the relationship between two pixels, g is a unary function used to calculate the representation of a single pixel, W and H correspond to the width and height of the feature map respectively;

[0013] The non-local distillation module loses L NLD As the L2 loss between teacher-student relationship information, it is expressed as: LNLD =L2, where L2 is expressed as follows:

[0014]

[0015] Among them, y i is the target value, f(x i ,w) is the predicted value of the student model, and N is the number of samples.

[0016] Furthermore, the RPN layer is used to generate candidate boxes on the student side and feed them back to the teacher side. It slides a small window on the last layer feature map of the full convolutional network, that is, it first performs a 3×3 convolution, then a 1×1 convolution, and then performs a true or false value judgment and an offset for each point k anchor points, and obtains 2k scores and 4k offset coordinates. The 256-dimensional feature map of the convolutional layer is converted into 6k features. That is: generate a series of candidate boxes, and perform binary classification (foreground or background) and bounding box regression on each candidate box to obtain accurate candidate areas. This step greatly reduces the number of candidate areas and improves the running speed of the algorithm.

[0017] Furthermore, the ROI pooling layer maintains the integrity and original image information of the picture, transforms the proposal box output by the RPN from the scale coordinates to the coordinates of (M / 16)*(N / 16), regenerates a uniform-sized border, and performs a maximum pooling operation on each block therein.

[0018] Furthermore, the classification head calculates the category probability vector of each border by a fully connected layer and a softmax operation;

[0019] The classification head is based on the region proposal sharing mechanism; the classification head shares the student's RPN results with the teacher model, using the soft label output as prior knowledge for distillation. The loss of the classification head is defined as follows:

[0020]

[0021] Among them, L CE and L BCE They represent the cross entropy loss and binary cross entropy loss functions respectively; y is the Groud-truth information, y s is the predicted output of the student model, y t is the predicted output of the teacher model, β1 is the balance factor that controls the two loss functions; by imitating the soft labels, the student model can learn the hidden structural information of the object, and N corresponds to the total number of items (total times) of a summation.

[0022] Furthermore, the bounding box regression head uses the bounding box position regressed in the teacher model as an additional regression target of the student model;

[0023] The bounding box regression head is used to further correct the detection box, and the Softmax operation is as follows:

[0024]

[0025] The bounding box regression head does not directly use the regression results output by the teacher, but uses an adaptive regression distillation strategy to selectively use the teacher's regression results. The loss function of the bounding box regression head is as follows:

[0026]

[0027] Among them, L refers to the smooth L1 loss, β2 is a balancing factor, and L dist refers to adaptive distillation loss; r represents region, s, t represent student and teacher respectively, and gt represents ground-truth.

[0028] Furthermore, the method for constructing the insulator defect detection model based on knowledge distillation comprises the following steps:

[0029] S101, use image annotation tools to locate the defect positions of the insulators in the collected insulator images and annotate the defect types. After the data set is constructed, the data set is further expanded by using geometric data augmentation, adding noise, etc. Then the expanded data set is divided into training set, validation set, and test set in a ratio of 8:1:1;

[0030] S102, establishing an insulator defect detection model based on knowledge distillation;

[0031] S103, reading the training set in the experimental environment, performing multiple batches and multiple rounds of training using a corresponding deep learning development platform based on the objective function, and cross-using the validation set to evaluate the model performance during the training process, and taking the model with the best performance during the training process as the intermediate training model;

[0032] S104, using the test set to evaluate the intermediate training model of S103, and selecting the model with the best evaluation performance as the final model, which is the insulator defect detection model based on knowledge distillation;

[0033] S105: Input the transmission line images collected outside the data set constructed in step S101 into the final lightweight transmission line insulator defect detection model for detection, and obtain the position of the defective insulator relative to the input image and the category of the defect.

[0034] Furthermore, in step S103, the objective function is:

[0035] L total =λLbackbone +αL NLD +L class +L regression +L RPN ;

[0036] Among them, λ refers to the equilibrium parameter of the backbone distillation network, α refers to the equilibrium parameter of the non-local distillation network, and L RPN Refers to the RPN training loss in two-stage object detection.

[0037] Furthermore, the insulator defect detection model based on knowledge distillation can be deployed in low-computing-power devices to perform real-time reasoning on the insulator image data collected during the inspection process to generate reasoning results.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. The present invention is a lightweight transmission line insulator defect detection method based on knowledge distillation. By adding a Gaussian mask distillation module and a non-local distillation module to the feature extraction backbone network, the feature extraction capability of the backbone network is improved.

[0040] 2. The feature backbone network of the present invention includes a knowledge distillation module, so that knowledge is adaptively transferred to students based on specific priors in the task. A two-dimensional Gaussian mask is used to model the foreground area to highlight defect information while suppressing redundant background noise. The teacher model uses the area provided by the student to output soft labels from the classification head, providing structural information beyond the implicit ground-truth label, and can achieve better generalization during joint training. The bounding box position regressed in the teacher model is used as an additional regression target for the student model. In this way, the regression target gradually approaches the ground-truth. This makes the bounding box regression subtask more robust.

[0041] 3. The student side of the present invention generates a proposed region through RPN and feeds it back to the teacher side, which alleviates the negative feedback from the teacher side to the student side due to ROI operation, and further improves the classification accuracy and bounding box regression accuracy.

[0042] 4. The larger teacher-side network of the present invention only provides guidance for the student-side network and is not used as the final model component. The thin student-side network guided by the teacher greatly reduces the number of model parameters while maintaining accuracy, thereby reducing the amount of calculation and facilitating deployment on low-computing power devices. It has great application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below.

[0044] Figure 1 is a basic flow chart of the method of the embodiment of the present invention;

[0045] Figure 2 It is a structural diagram of the lightweight transmission line insulator defect detection method proposed by the present invention;

[0046] Figure 3 It is a structural diagram of the non-local distillation module in the present invention;

[0047] Figure 4 It is a structural diagram of the RPN module in the present invention. DETAILED DESCRIPTION

[0048] The specific embodiments of the present invention are described in detail below, but it should be understood that the protection scope of the present invention is not limited by the specific embodiments.

[0049] It should be noted that the feature maps, images, and images involved below are all matrices unless otherwise specified during the computer solution process, and the number of summation symbols represents the number of dimensions of the matrix.

[0050] Example 1

[0051] like Figure 1 As shown, the present invention proposes a lightweight transmission line insulator defect detection method based on knowledge distillation, comprising the following steps:

[0052] S101, building a transmission line insulator defect detection image dataset

[0053] It should be noted that the data set comes from the Chinese power transmission line insulator data set and some insulator image data collected by the inventors themselves, and the image annotation tool used is LabelMe. After the data set is constructed, the data set is further expanded by using geometric data augmentation, adding noise, etc. The expanded image size is 640×640×3 and normalized. Finally, the expanded data set is divided into training set, validation set, and test set in a ratio of 8:1:1.

[0054] S102, establish a lightweight transmission line insulator defect detection model based on knowledge distillation.

[0055] The flow of image data in the established model is shown in Figure 2 .

[0056] It should be noted that the established model includes a feature extraction backbone network, an RPN layer, a ROI pooling layer, a defect classification head, and a defect position bounding box regression head.

[0057] Furthermore, the feature extraction backbone network selected in this embodiment is Faster RCNN.

[0058] More preferably, the present invention introduces a two-dimensional Gaussian mask into the backbone network to further highlight the insulator foreground pixels and suppress the boundary area around the insulator. The two-dimensional Gaussian mask operation is described as follows:

[0059]

[0060] in, and are the attenuation factors in the x and y directions. For simplicity, these two attenuation factors are set to the same value, that is: The mask is only valid within the Ground truth bounding box and is set to 0 elsewhere. When several Gaussian masks overlap at a single pixel (x, y), the mask value of the mask is set to the maximum value.

[0061] Based on the Gaussian mask operation, the feature extraction backbone network extracts image features by minimizing the following loss function:

[0062]

[0063] in, W, H, and C correspond to the width, height, and number of channels of the feature map, respectively; and Represent the image features extracted from the student backbone and teacher backbone respectively; M ijc is a sub-item in the M matrix.

[0064] The introduced non-local distillation module captures the relationship information between global insulator objects. It is expressed as follows:

[0065]

[0066] Among them, r is the calculated relationship information between objects, i, j are the spatial indices of the output positions of the response to be calculated, and i', j' are the spatial indices that enumerate all possible positions. f is a pairwise function used to calculate the relationship between two pixels, and g is a unary function used to calculate the representation of a single pixel.

[0067] Constructing a non-local distillation module such as Figure 3 As shown, the loss L NLD As the L2 loss between teacher-student relationship information, it is expressed as: L NLD =L2, where L2 is expressed as follows:

[0068]

[0069] Among them, y i is the target value, f(x i ,w) is the predicted value of the model, and N is the number of samples.

[0070] RPN is constructed to generate candidate boxes on the student side and feed them back to the teacher side. It slides a small window on the last layer feature map of the full convolutional network, that is, it first performs a 3×3 convolution, then a 1×1 convolution, and then performs a true or false value judgment and offset for each point k anchor points, and obtains 2k scores and 4k offset coordinates. The 256-dimensional feature map of the convolution layer is converted into 6k features. That is: a series of candidate boxes are generated, and each candidate box is classified into two categories (foreground or background) and bounding box regression to obtain accurate candidate areas. This step greatly reduces the number of candidate areas and improves the running speed of the algorithm.

[0071] Constructing ROI pooling maintains the integrity of the image and the original image information, transforming the RPN output proposal from M×N scale coordinates to (M / 16)*(N / 16) coordinates, and then generating a uniform-sized border, and performing a maximum pooling operation on each block.

[0072] The classification head is constructed by a fully connected layer and a softmax operation to calculate the category probability vector of each bounding box.

[0073] Furthermore, N region proposals are given from the output of the student RPN layer. Calculate the number of positive samples N from the output result p , where C′ represents the number of object categories. The loss of the classification head is defined as follows:

[0074]

[0075] Among them, L CE and L BCE Denote the cross entropy loss and binary cross entropy loss functions respectively. s is the predicted output of the student model, and β1 is the balance factor that controls the two loss functions. By imitating the soft labels, the student model can learn the hidden structural information of the object.

[0076] The bounding box regression head is used to further correct the detection box. The Softmax operation is as follows:

[0077]

[0078] The bounding box regression head does not directly use the regression results output by the teacher, but instead utilizes an adaptive regression distillation strategy to selectively use the teacher's regression results.

[0079] Specifically, given N p Positive region proposals from the output of the student RPN layer, bounding box positions before regression: The regression results of the teacher: The regression results of the student side are: And the corresponding Ground truth: First calculate and between, and The Intersection-Over-Union (IoU) between This indicates that the teacher's regression output is reliable and can be used as a guide for the student. Otherwise, no distillation operation is performed on the student's regression head. In the present invention, the loss function of the bounding box regression head is as follows:

[0080]

[0081] Where L refers to the smoothed L1 loss. β2 is a balancing factor. L dist Refers to the adaptive distillation loss.

[0082]

[0083] S103, read the prepared training data set in the experimental environment, use the corresponding deep learning development platform to perform multiple batches and multiple rounds of training based on the following objective function, and cross-use the validation set to evaluate the model performance during the training process, and take the model with the best performance during the training process as the result model.

[0084] L total =λL backbone +αL NLD +L class +L regression +L RPN

[0085] Where λ refers to the equilibrium parameter of the backbone distillation network, α refers to the equilibrium parameter of the non-local distillation network, and L RPN Refers to the RPN training loss in two-stage object detection.

[0086] It should be noted that the present invention relies on the following conditions, see Table 1 for training experiments.

[0087] Table 1:

[0088] Conditional Items Implementation details CPU Intel Xeno Gold 6133 GPU Nvidia Tesla v100*4 operating system Ubuntu LTSC 20.04 programming language Python 3.10 Encoding Dependencies PyTorch

[0089] The details of the training parameter settings are shown in Table 2.

[0090] Table 2:

[0091]

[0092] Furthermore, in order to accelerate network convergence, the network optimizer used in the present invention is Adam, and its initial network learning rate is set to 1×103

[0093] S104, evaluate the intermediate model trained in S103, and select the model with the best evaluation performance as the final model. In order to accelerate network convergence, this example uses the cross-validation method during training to evaluate the intermediate model. Specifically, after every 200 training processes, the overall model is frozen, and 50 training processes are verified on the basis of freezing, and then the model is unfrozen, and this is repeated. Until 5000 training processes are trained. Since the image size in the data set is fixed, a large amount of graphics card memory is consumed. In this example, the batch size of each training process is set to 8.

[0094] S105, applying the final model selected in S104 to complex scenarios in practice. Specifically, the final model is regarded as a lightweight transmission line insulator defect detection method based on knowledge distillation proposed in the present invention. The model is deployed to low-computing power devices such as drones, and the insulator image data collected during the inspection process is inferred in real time to generate inference results. Assist technical experts to efficiently complete inspection tasks.

[0095] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.

Claims

1. A lightweight transmission line insulator defect detection method based on knowledge distillation, characterized in that: The detection method comprises: constructing an insulator defect detection model based on knowledge distillation, inputting the insulator image data collected in real time into the insulator defect detection model based on knowledge distillation, and obtaining an insulator defect detection result; The insulator defect detection model based on knowledge distillation includes: a feature extraction backbone network, an RPN layer, a ROI pooling layer, a classification head and a bounding box regression head; the feature extraction backbone network includes a Gaussian mask distillation module and a non-local distillation module.

2. A lightweight transmission line insulator defect detection method based on knowledge distillation according to claim 1, characterized in that: The Gaussian mask distillation module introduces a two-dimensional Gaussian mask to highlight the foreground pixels of the insulator and suppress the boundary area around the insulator. Based on the Gaussian mask operation, the feature extraction backbone network extracts image features by minimizing the following loss function: in, W, H, and C correspond to the width, height, and number of channels of the feature map, respectively; and They represent the image features extracted from the student backbone and the teacher backbone respectively.

3. The method for defect detection of lightweight transmission line insulators based on knowledge distillation according to claim 1 is characterized in that: The non-local distillation module captures the relationship information between global insulator objects, which is expressed as follows: Among them, r is the calculated relationship information between objects, i, j are the spatial indices of the output positions of the response to be calculated, i', j' are the spatial indices that enumerate all possible positions; f is a pairwise function used to calculate the relationship between two pixels, g is a unary function used to calculate the representation of a single pixel, W and H correspond to the width and height of the feature map respectively; The non-local distillation module loses L NLD As the L2 loss between teacher-student relationship information, it is expressed as: L NLD =L2, where L2 is expressed as follows: Among them, y i is the target value, f(x i ,w) is the predicted value of the student model, and N is the number of samples.

4. The method for detecting defects in lightweight transmission line insulators based on knowledge distillation according to claim 1 is characterized in that: The RPN layer first performs a 3×3 convolution, then a 1×1 convolution, and finally performs true or false value judgment and offset of k anchor points for each point, obtaining 2k scores and 4k offset coordinates. The 256-dimensional feature map of the convolution layer is converted into 6k features.

5. The method for defect detection of lightweight transmission line insulators based on knowledge distillation according to claim 1 is characterized in that: The ROI pooling layer maintains the integrity of the picture and the original image information, transforms the proposal box output by the RPN layer from the scale coordinates to the coordinates of (M / 16)*(N / 16), regenerates a uniform-sized border, and performs a maximum pooling operation on each block therein.

6. The method for detecting defects in lightweight transmission line insulators based on knowledge distillation according to claim 1 is characterized in that: The classification head is based on the region proposal sharing mechanism; the classification head shares the student's RPN results with the teacher model, using the soft label output as prior knowledge for distillation. The loss of the classification head is defined as follows: Among them, L CE and L BCE They represent the cross entropy loss and binary cross entropy loss functions respectively; y is the Groud-truth information, y s is the predicted output of the student model, y t is the predicted output of the teacher model, and β1 is the balance factor that controls the two loss functions; by imitating the soft labels, the student model can learn the hidden structural information of the object.

7. The method for defect detection of lightweight transmission line insulators based on knowledge distillation according to claim 1 is characterized in that: The bounding box regression head uses the bounding box position regressed in the teacher model as an additional regression target for the student model; The bounding box regression head is used to further correct the detection box, and the Softmax operation is as follows: The bounding box regression head does not directly use the regression results output by the teacher, but uses an adaptive regression distillation strategy to selectively use the teacher's regression results. The loss function of the bounding box regression head is as follows: Among them, L refers to the smooth L1 loss, β2 is a balancing factor, and L dist Refers to the adaptive distillation loss.

8. The method for defect detection of lightweight transmission line insulators based on knowledge distillation according to claim 1 is characterized in that: The method for constructing the insulator defect detection model based on knowledge distillation comprises the following steps: S101, constructing a transmission line insulator defect detection image dataset, further expanding the dataset, normalizing the expanded images, and dividing them into a training set, a validation set, and a test set in a ratio of 8:1:1; S102, establishing an insulator defect detection model based on knowledge distillation; S103, reading the training set in the experimental environment, performing multiple batches and multiple rounds of training using a corresponding deep learning development platform based on the objective function, and cross-using the validation set to evaluate the model performance during the training process, and taking the model with the best performance during the training process as the intermediate training model; S104, using the test set to evaluate the intermediate training model of S103, and selecting the model with the best evaluation performance as the final model, which is the insulator defect detection model based on knowledge distillation.

9. The method for detecting defects in lightweight transmission line insulators based on knowledge distillation according to claim 8, characterized in that: In step S103, the objective function is: L total =λL backbone +αL NLD +L class +L regression +L RPN Among them, λ refers to the equilibrium parameter of the backbone distillation network, α refers to the equilibrium parameter of the non-local distillation network, and L RPN Refers to the RPN training loss in two-stage object detection.

10. The method for defect detection of lightweight transmission line insulators based on knowledge distillation according to claim 1, characterized in that: The insulator defect detection model based on knowledge distillation can be deployed in low-computing-power devices to perform real-time reasoning on the insulator image data collected during the inspection process to generate reasoning results.

Images