A high-speed railway insulator defect detection method based on knowledge distillation

Abstract

The invention discloses a high-speed rail insulator defect detection method based on knowledge distillation, which relates to the technical field of computer vision image recognition, and comprises the following steps: S1, inputting a 4C grayscale image containing insulators; S2, extracting insulators in the grayscale image, and dicing the extracted insulator image to obtain small insulator samples; S3, establishing a knowledge distillation teacher network, and using normal insulator small block samples and background small block samples of the same size to classify and train the teacher network so that its parameter space contains characteristic information of the insulators; S4, establishing a knowledge distillation student network after channel pruning, and jointly training and testing the teacher network and the student network, judging the defective insulator small block samples according to the different intermediate layer outputs of the teacher network and the student network when facing abnormal samples during the test process; establishing a defect segmentation heat map according to the defect judgment result, and marking the insulator defect area in the heat map.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision image recognition, and more specifically to a high-speed rail insulator defect detection method based on knowledge distillation. Background Art

[0002] In the actual operation environment of high-speed rail, due to the long-term high-density operation of trains and the outdoor operation environment, it is very easy to cause the contact network support device to fail. Under the influence of the harsh and complex external environment, high-speed rail train vibration and other complex factors, the parts will have defects such as "loose, detached, broken, and cracked", resulting in the structural stability of the contact network support device being reduced and the electrical connection being damaged, which brings huge hidden dangers to the normal operation of the train. Through years of investigation and statistics, among the various equipment failures in the high-speed rail contact network system, the failure of the support device parts accounts for more than 70%, so it needs to get more attention.

[0003] In the field of image-based contact network defect detection, there are usually three approaches: 1) Artificially designing criteria based on prior knowledge and then using classifiers for classification; 2) Converting fault detection problems into classification problems through supervised learning algorithms in deep learning algorithms; 3) Converting fault detection problems into anomaly detection problems through unsupervised learning algorithms in deep learning algorithms.

[0004] The research on contact network defect detection should adopt unsupervised method, which is the third method among the above three approaches.

[0005] The first type of algorithm belongs to the traditional image processing algorithm. The traditional image processing algorithm generally adopts edge detection, threshold segmentation, region growing, feature clustering and other algorithms. Its steps are similar to target positioning. It also uses various features of the image (shape, texture, edge, color, etc.) to establish the feature descriptor of the defect, and then uses the classifier for classification. The second type of supervised learning algorithm based on deep learning, although compared with the traditional image processing algorithm, it can use convolutional neural network to extract component features, and its algorithm robustness has been greatly improved. The third type of unsupervised learning algorithm based on deep learning essentially converts the problem of contact network defect detection into anomaly detection problem. Use a small number of negative samples (or even no negative samples) to detect contact network defects. This fits the actual situation that there are only a small number of defect samples in the field of railway contact network component defect detection.

[0006] The unsupervised learning method solves the following two difficulties in contact network defect detection:

[0007] 1. The design features of traditional image algorithms are complex and have low robustness. The features need to be redesigned if the detection object changes slightly.

[0008] 2. The feature extraction work of the supervised learning algorithm in deep learning requires a large number of contact network defect samples.

[0009] In summary, defect detection of contact network components through unsupervised learning algorithms has important theoretical and practical significance.

[0010] In the field of railway contact network defect detection, supervised deep learning has certain applications in the presence of a large number of negative samples. Junwen Chen et al. designed a three-level cascaded deep convolutional neural network (DCNNs) framework to diagnose fastener defects. Xavier Giben et al. trained an end-to-end DCNNs to classify and semantically segment railway rail materials. Junping Zhong optimized the structure of the DCNNs variant network PVANET to better adapt to the positioning and detection of contact network cotter pins. Zhigang Liu et al. used improved convolutional neural networks and hole analysis methods to detect loose strand faults of equipotential lines.

[0011] In most cases, negative samples are scarce, and unsupervised learning methods are needed to complete defect detection. Gaoqiang Kang et al. used a deep denoising autoencoder for unsupervised learning and formed a multi-task learning network with a deep material classifier. No defective insulator samples are required for training and the performance is good. Junping Zhong et al. introduced a regression frame that can regress the deflection angle, and the tiltable positioning frame can achieve a more accurate positioning effect. GAN-2 was used to detect defects in insulator slices, and the reconstruction error of the patch was evaluated to analyze the defect situation. Xuemin Lu et al. combined the three loss functions of reconstruction loss, coding loss and adversarial loss, and used MS-SSIM to jointly evaluate the loss function to distinguish insulator defect samples. On the other hand, Dongkai Zhang et al. also tried to extract the features of normal insulators, input them into the encoder, add noise to the encoded vector, and decode to generate defect features. The defect detection classifier was trained using the generated defect features to achieve unsupervised learning of insulator defect detection. Experiments have shown that the detection effect is good.

[0012] Mohammadreza Salehi et al. extracted the outputs of each maxpool2D layer of the expert network and let the corresponding maxpool2D layer of the student network imitate its output, so that it has similar feature output and result output when facing normal sample input. When abnormal samples are input, due to the insufficient number of student network parameters and insufficient imitation ability, the feature output and result output of the two networks will be different. Using this incomplete imitation ability of the student network, anomaly detection of the anomaly detection dataset MVTecAD was completed, and the indicators reached the SOTA level. The latter also compared the indicators of the knowledge distillation network with common anomaly detection algorithms: AE, AnoGAN, VAE-grad, etc., and the detection results were better than the control group algorithm.

[0013] In the existing technology, it is difficult to detect defects in high-speed rail contact network insulators when defect samples are scarce. At present, the research on unsupervised learning defect detection of contact network insulators at home and abroad mainly uses algorithms based on GAN networks and AE networks, and the research using knowledge distillation algorithms has not yet appeared or been made public. Summary of the invention

[0014] In order to overcome the defects existing in the above-mentioned prior art, the present invention discloses a high-speed rail insulator defect detection method based on knowledge distillation. The purpose of the present invention is to solve the problem in the prior art that it is difficult to realize defect detection when there are few defect samples of high-speed rail contact network insulators. The method uses normal samples and background samples of the same size to classify and pre-train the VGG-16 teacher network so that it has insulator feature information. In the training stage, normal samples are used to forward propagate and back propagate the teacher network and the student network, so that the convolution layer features of the student network when facing normal input are similar to those of the teacher network. In the testing stage, the test set contains defective samples and normal samples, and the scores corresponding to the defective samples are high, and the scores corresponding to the normal samples are low. After the classification is completed, the gradient-based significance algorithm can also be used to obtain the insulator defect segmentation heat map. The core is to realize the determination of defective insulators by different intermediate layer outputs of the teacher network and the student network when facing abnormal samples.

[0015] In order to achieve the above objectives, the technical solution adopted by the present invention is:

[0016] A high-speed rail insulator defect detection method based on knowledge distillation comprises the following steps:

[0017] 1. Image Input

[0018] S1. Input a 4C grayscale image containing insulators.

[0019] In the present invention, in step S1, a 4C grayscale image containing an insulator is input, which is denoted as I.

[0020] 2. Image Preprocessing

[0021] S2. extract the insulators in the grayscale image, and cut the extracted insulator image into blocks to obtain small insulator samples.

[0022] Preferably, in the step S2, 180 angle category information is added to the prediction frame of the YOLOv5s network to form an R-YOLOv5s network, and then the insulator is extracted.

[0023] In the present invention, R-YOLOv5 is used to extract the insulator image: the improved YOLOv5s network is used to accurately extract the insulator. The improvement content is: 180 angle category information is added to the prediction frame so that YOLOv5s can obtain the angle category information. R-YOLOv5s realizes the precise positioning of the insulator and eliminates the interference of background information.

[0024] Preferably, in step S2, the extracted insulator image is cut into blocks, and the sample size of the insulator block is 128*128 pixels.

[0025] In the present invention, because insulator defects are usually too small, the knowledge distillation defect detection algorithm cannot accurately detect some small defects, so the insulator image is cut into blocks with a size of 128*128 pixels. The small block insulator images are subsequently used for training and testing, and finally spliced ​​according to the position information of the small blocks.

[0026] 3. Teacher Network Pre-training

[0027] S3. Establish a knowledge distillation teacher network, and use normal insulator small block samples and background small block samples of the same size to classify and train the teacher network so that its parameter space contains the characteristic information of the insulator.

[0028] Preferably, in the step S3, the VGG-16 backbone network is used, and its classification layer is removed as the teacher network of the knowledge distillation network. The above network is selected as the teacher network of the knowledge distillation network because the VGG-16 network has strong feature extraction capabilities and can meet the needs of multi-level and multi-scale feature extraction in the subsequent insulator defect detection process. At the same time, it has strong generalization capabilities, which can be different from the poor generalization capabilities of the student network in the subsequent process. This difference in generalization capabilities is the essential attribute for realizing unsupervised defect detection.

[0029] The teacher network needs to contain the characteristic information of the insulator first, and then it can pass the characteristic information of the insulator contained in it to the student network in the subsequent "knowledge distillation" process. The purpose of transferring "knowledge" is to make the student network have similar characteristic layer outputs as the teacher network when facing a normal insulator network, and when facing defective insulator samples, the student network has insufficient generalization ability and the intermediate characteristic layer output is different from that of the teacher network (although the teacher network also does not have the characteristics of defective insulators, due to its strong generalization ability, when facing defective sample input, the characteristic output of the intermediate layer does not change much).

[0030] 4. Defect Detection

[0031] S4. Establish a knowledge distillation student network after channel pruning, and jointly train and test the teacher network and the student network. According to the different intermediate layer outputs of the teacher network and the student network when facing abnormal samples during the test process, judge the small samples of defective insulators; establish a defect segmentation heat map based on the defect judgment results, and mark the insulator defect areas in the heat map.

[0032] The S4 step includes a joint training and testing step, a scoring and judging step, a small block heat map generation step, and a defect detection report generation step, which are specifically as follows:

[0033] 1. Joint training and testing

[0034] The joint training and testing step establishes a knowledge distillation student network after channel pruning, and uses normal insulator small block samples to jointly train the teacher network and the student network. After the training, each insulator small block sample is added to the teacher network and the student network for joint testing. During the testing process, the total loss value of the sample is obtained based on the different intermediate layer outputs of the teacher network and the student network when facing abnormal samples.

[0035] The joint training and testing step comprises the following steps:

[0036] (1) Joint training of the teacher network and the student network: A knowledge distillation student network is established after channel pruning, and the teacher network and the student network are jointly trained using normal insulator small block samples. During joint training, the student network is gradient updated, while the teacher network is not gradient updated;

[0037] Preferably, in the joint training of the teacher network and the student network, the channel pruned VGG-16 is used as the student network of the knowledge distillation algorithm, and the number of channels of the other convolutional layers except the last layer of each convolutional module is compressed to 16.

[0038] In the present invention, the student network is obtained by channel pruning of the teacher network (VGG-16 backbone network). Its repeated upsampling and downsampling steps will cause the feature information to be lost during the forward transmission process. This causes the generalization ability of the student network to be worse than that of the teacher network. As a result, when the student network faces the input of defective insulator images, the feature output of the middle layer is significantly different from the output of the teacher network. In this way, when a normal image is input, the output difference between the middle layers of the two networks is small, and when a defective image is input, the output difference between the middle layers of the two networks is large. After quantifying this gap and setting a reasonable threshold, the defective image can be successfully identified.

[0039] Preferably, when the teacher network and the student network are trained together, the loss function used in the training is L t ,

[0040]

[0041] L t =L v +αL dir

[0042] Among them, L t is the total loss function, L v is the distance loss function, which measures the Euclidean distance of the feature vector in the middle layer, L dir is the cosine similarity loss function, which measures the directional similarity of the feature vectors in the middle layer. α is a hyperparameter that adjusts the weights of the two loss functions. Its value will determine whether the network attaches more importance to the similarity of the vectors in distance or in direction, which often depends on the characteristics of the data set. α = 0.5, N c For L i The number of neurons in a layer, that is, the number of its channels, T n (i.e., the teacher network) in the Lth i The output features of the layer, For S n (i.e. student network) in the L i The output feature of the layer, T stands for Transpose, which is the transposition operation of the vector;

[0043] In the first formula, i represents traversing all the intermediate layers to be extracted, specifically the 9th, 16th, 23rd, and 30th layers of the teacher network (all maxpool2D layers, which are also the last layer of each convolution module), and the 11th, 20th, 29th, and 38th layers of the student network (all maxpool2D layers, which are also the last layer of each convolution module); j represents the number of all channels traversed in a layer;

[0044] The i in the second formula represents traversing all the intermediate layers to be extracted, specifically the 9th, 16th, 23rd, and 30th layers of the teacher network (all are maxpool2D layers, which are also the last layer of each convolution module), and the 11th, 20th, 29th, and 38th layers of the student network (all are maxpool2D layers, which are also the last layer of each convolution module).

[0045] In the present invention, during training, only the student network performs gradient updates, and the teacher network does not participate in gradient updates. This is because the teacher network has stored the image feature information of the insulator in pre-training. The purpose of training is to pass this feature information to the student network so that the student network also "feels familiar" with the normal insulator image, or in other words, when facing the normal insulator input, the intermediate layer output of the student network is similar to that of the teacher network. Therefore, the parameters of the teacher network must be fixed, which can achieve the purpose of knowledge transfer and save computing resources.

[0046] (2) Joint testing of the teacher network and the student network: After the teacher network and the student network are trained together, each small insulator sample is added to the teacher network and the student network for joint testing. During the test, the total loss value of the sample is obtained based on the different intermediate layer outputs of the teacher network and the student network when facing abnormal samples.

[0047] In the present invention, during testing, the teacher network and the student network participate together. When facing normal samples, the difference between the intermediate layer feature outputs of the two is not large, and the total loss value is low. Since the generalization ability of the student network deteriorates after pruning, when facing a small image of a defective insulator, the intermediate layer feature output is quite different from that of the teacher network, and the total loss value is large.

[0048] 2. Scoring

[0049] The scoring and judging step scores each tested insulator small block sample according to the total loss value of all samples, and outputs the insulator small block sample position information according to the scoring result; then, based on the scoring result and the sample position information, reports the information of the normal insulator small block sample and the defective insulator small block sample to obtain the insulator defect detection result.

[0050] The scoring and reporting step comprises the following steps:

[0051] (1) Scoring of test samples: Calculate the total loss value of all insulator block samples, and assign a defect score to each insulator block sample based on the total loss value of all insulator block samples.

[0052] Preferably, the defect score is:

[0053]

[0054] Among them, Si is the defect score of the i-th sample, is the total loss of the i-th sample, and the median function takes the median of the total loss values ​​of all samples.

[0055] (2) Whether the threshold is met: the defect score is compared with the threshold to determine whether the sample is a defective insulator patch sample or a normal insulator patch sample.

[0056] Preferably, in the threshold comparison, 1 is used as the threshold, and a defect score greater than 1 is judged as a defective insulator small block sample, and a defect score less than 1 is judged as a normal insulator small block sample.

[0057] (3) Outputting the location information of defective samples and normal samples: Outputting the location information of defective insulator small block samples and normal insulator small block samples according to the judgment results;

[0058] In the present invention, a square sliding window of 128 pixels is used for block segmentation, the step length in the row direction and the step length in the column direction are both 64 pixels, and the overlapping area between small blocks in the row direction and the column direction is 50%, thereby outputting the position information of each small block.

[0059] (4) Reporting normal insulators and defective insulators: Based on the scores of the insulator patch samples and the location information of the insulator patch samples, the information of the normal insulator patch samples and the defective insulator patch samples is reported to obtain the insulator defect detection results.

[0060] In the present invention, the insulator defect detection result is obtained based on the previously obtained insulator small block defect score and the insulator small block position information, that is, which part of the insulator the small block comes from.

[0061] 3. Small block heat map generation

[0062] The small block heat map generation step uses the integral gradient method and the smooth gradient method to perform gradient backpropagation on the teacher network and the student network and generate a small block heat map, and then suppresses noise on the heat map;

[0063] The small block heat map generation step includes the following steps:

[0064] (1) Gradient feedback between the teacher network and the student network: The integral gradient method and the smooth gradient method are used to perform gradient feedback between the teacher network and the student network;

[0065] In the present invention, a gradient-based saliency algorithm is used: smooth gradient and integrated gradient. The gradient of back propagation needs to be used to participate in the calculation of the next step of the heat map.

[0066] (2) Generate small-block heat maps: Generate small-block heat maps by using the gradient information stored in the reverse transmission of the teacher network and the student network.

[0067] Preferably, the small block thermal map is:

[0068]

[0069] Among them, Λ is a small block heat map, is the input image formed by doping the image x with a certain amount of Gaussian noise, and x' is the input image The integral reference image is taken as a pure black image, k is the number of path images, and N is the number of repetitions of the smooth gradient method;

[0070] The i-th path image is:

[0071]

[0072] The magnitude of Gaussian noise is:

[0073]

[0074] Wherein, p is a hyperparameter for adjusting the noise size, p=1, max(x) is the pixel value of the brightest pixel in the input image, and min(x) is the pixel value of the darkest pixel in the input image.

[0075] For each path image x i , we all use the original gradient method to calculate its heat map:

[0076]

[0077] Then, calculate all κ i The average value of κ is obtained, and then multiplied by the path length The thermal map of a single integrated gradient can be obtained. The above process is repeated N times to eliminate the influence of the randomness of random Gaussian noise and obtain a thermal map of a smooth gradient. In the present invention, k=25 and N=5.

[0078] (3) Heat map noise suppression: In the generated small block heat map, a threshold is set to suppress the noise;

[0079] Preferably, in the step of suppressing the noise of the heat map, in the obtained small-block heat map, a threshold T=5e-4 is set, pixels with pixel values ​​less than this value are suppressed to 0, and then the pixel values ​​of the heat map are restored to [0, 255] to obtain the final heat map.

[0080] In the present invention, because the behavior of the neural network has a certain degree of randomness, it is necessary to set a threshold for noise suppression. In the obtained small block heat map, the threshold T=5e-4 is set, and the pixels with pixel values ​​less than this value are suppressed to 0, and then the pixel values ​​of the heat map are restored to [0,255] to obtain the final heat map.

[0081] 4. Defect detection report generation

[0082] The defect detection report generation step uses the position information of the small block samples of the insulator and the small block heat map after noise suppression, combines the small block heat maps into a complete heat map, and marks the insulator defect area in the heat map, and then generates an insulator defect detection report based on the complete heat map and the insulator defect detection results.

[0083] The defect detection report generation step comprises the following steps:

[0084] (1) Combining into a complete thermal map: Using the location information of the small block samples of the insulator and the small block thermal map after noise suppression, the small block thermal map is pieced back into the original image to form a complete thermal map of the insulator, and the highlighted pixel area is taken to segment the insulator defects at the pixel level.

[0085] In the present invention, the position information of the previous insulator small block and the small block heat map are used to piece the small block heat map back into the original image, and the highlighted pixel area is taken to achieve pixel-level segmentation of the insulator defects.

[0086] (2) Inspection report: Generate an insulator defect inspection report based on the complete thermal map of the insulator and the insulator defect inspection results.

[0087] In the present invention, an insulator defect detection report is formed based on the previously obtained complete thermal map of the insulator and the insulator defect detection results.

[0088] Beneficial effects of the present invention:

[0089] The high-speed rail insulator defect detection method provided by the present invention uses normal samples and background samples of the same size to classify and pre-train the VGG-16 teacher network so that it has insulator feature information. In the training stage, normal samples are used to forward and backward propagate the teacher network and the student network, so that the convolutional layer characteristics of the student network when facing normal input are similar to those of the teacher network. In the testing stage, the test set contains defective samples and normal samples. The intermediate layer outputs of the teacher network and the student network are different when facing abnormal samples. The scores corresponding to defective samples are high and the scores corresponding to normal samples are low, so as to distinguish between defective samples and normal samples. After the classification is completed, the gradient-based significance algorithm can also be used to obtain the insulator defect segmentation heat map.

[0090] The high-speed rail insulator defect detection method provided by the present invention establishes a teacher-student network structure through a knowledge distillation method, and can realize defective insulator abnormality detection by learning the characteristics of normal insulator samples. The defective samples are not used in the training process, and the defect detection function can be realized without relying on the defective samples. Therefore, it can solve the problem in the prior art that it is difficult to realize defect detection when the defective samples of high-speed rail contact network insulators are scarce.

[0091] The high-speed railway insulator defect detection method provided by the present invention is suitable for deep defect detection of contact network insulators, and provides support for the inspection and maintenance of high-speed railway contact network insulators.

[0092] The high-speed rail insulator defect detection method provided by the present invention can realize unsupervised defect detection of high-speed rail insulators, which is suitable for the current situation of lack of defect samples at the operation and maintenance site. The method can alleviate the pressure of manual inspection of railway operation and maintenance and improve the efficiency of operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 Insulator defect detection algorithm flow for knowledge distillation;

[0094] Figure 2 Positioning results for insulators;

[0095] Figure 3 It is a small piece of normal insulator;

[0096] Figure 4 It is a small piece of defective insulator;

[0097] Figure 5 It is the network structure of the knowledge distillation algorithm;

[0098] Figure 6 Score probability distribution for defective samples and normal samples;

[0099] Figure 7 The thermal map corresponding to the small sample of insulator;

[0100] Figure 8 This is a thermal diagram of a defective insulator after splicing;

[0101] Fig. 9 This is a thermal diagram of a defective insulator after splicing;

[0102] Fig.10 This is a thermal diagram of a defective insulator after splicing;

[0103] Fig.11 This is a thermal diagram of a defective insulator after splicing;

[0104] Fig.12 This is a thermal diagram of a defective insulator after splicing;

[0105] Fig.13 This is a thermal diagram of a defective insulator after splicing. DETAILED DESCRIPTION

[0106] The concept, specific structure and technical effects of the present invention will be clearly and completely described below in conjunction with the embodiments and drawings to fully understand the purpose, characteristics and effects of the present invention.

[0107] Example 1

[0108] A high-speed railway insulator defect detection method based on knowledge distillation, such as Figure 1 As shown, the following steps are included:

[0109] S1, input 4C grayscale image containing insulators;

[0110] S2, extracting the insulators in the grayscale image, and cutting the extracted insulator image into blocks to obtain small insulator block samples;

[0111] S3. Establish a knowledge distillation teacher network, use normal insulator small block samples and background small block samples of the same size to classify and train the teacher network, so that its parameter space contains the characteristic information of the insulator;

[0112] S4. Establish a knowledge distillation student network after channel pruning, and jointly train and test the teacher network and the student network. According to the different intermediate layer outputs of the teacher network and the student network when facing abnormal samples during the test process, judge the small samples of defective insulators; establish a defect segmentation heat map based on the defect judgment results, and mark the insulator defect areas in the heat map.

[0113] Example 2

[0114] This embodiment is further described on the basis of embodiment 1. Figure 1 As shown, step S1 is an image input step, and the input is a 4C grayscale image containing an insulator, denoted as I.

[0115] Step S2 is an image preprocessing step, which includes the following steps:

[0116] (1) R-YOLOv5s extracts insulator images: The improved YOLOv5s network: R-YOLOv5s accurately extracts insulators. The improvements are: 180 angle category information is added to the prediction box so that YOLOv5s can obtain angle category information. R-YOLOv5s achieves accurate positioning of insulators and eliminates the interference of background information. Figure 2 This is the positioning result of R-YOLOv5s.

[0117] (2) Image segmentation: Since insulator defects are usually too small, the knowledge distillation defect detection algorithm cannot accurately detect some small defects. Therefore, the insulator image is segmented into small blocks of 128*128 pixels. Small blocks of insulator images are used for training and testing. Finally, the small blocks are stitched together based on their position information. Figure 3 and Figure 4 is a schematic diagram of insulator cutting, where: Figure 3 For normal insulator small pieces, Figure 4 It is a small piece of defective insulator.

[0118] Step S3 is the teacher network pre-training step. The VGG-16 backbone network is used, and its classification layer is removed as the teacher network of the knowledge distillation network. The teacher network is trained for classification using normal insulator patches and background patches of the same size, so that its parameter space contains the characteristic information of the insulator.

[0119] Example 3

[0120] This embodiment is further described on the basis of embodiment 2. Figure 1 As shown, the S4 step includes the following steps:

[0121] (1) Joint training of the teacher network and the student network: Use the pruned VGG-16 as the student network of the knowledge distillation algorithm, such as Figure 5 As shown in the network below, except for the last layer of each convolution module, the number of channels of other convolution layers is compressed to 16. The teacher network and the student network are jointly trained using normal insulator patch images.

[0122] In the joint training of the teacher network and the student network, the loss function used in the training is L t ,

[0123]

[0124] L t =L v +αL dir

[0125] Among them, L t is the total loss function, L v is the distance loss function, which measures the Euclidean distance of the feature vector in the middle layer, L dir is the cosine similarity loss function, which measures the directional similarity of the feature vectors in the middle layer. α is a hyperparameter that adjusts the weights of the two loss functions. Its value will determine whether the network attaches more importance to the similarity of the vectors in distance or in direction, which often depends on the characteristics of the data set. α = 0.5, N c For L i The number of neurons in a layer, that is, the number of its channels, T n (i.e., the teacher network) in the Lth i The output features of the layer, For S n (i.e. student network) in the L i The output feature of the layer, T stands for Transpose, which is the transposition operation of the vector;

[0126] In the first formula, i represents traversing all the intermediate layers to be extracted, specifically the 9th, 16th, 23rd, and 30th layers of the teacher network (all maxpool2D layers, which are also the last layer of each convolution module), and the 11th, 20th, 29th, and 38th layers of the student network (all maxpool2D layers, which are also the last layer of each convolution module); j represents the number of all channels traversed in a layer;

[0127] The i in the second formula represents traversing all the intermediate layers to be extracted, specifically the 9th, 16th, 23rd, and 30th layers of the teacher network (all are maxpool2D layers, which are also the last layer of each convolution module), and the 11th, 20th, 29th, and 38th layers of the student network (all are maxpool2D layers, which are also the last layer of each convolution module).

[0128] (2) Student network gradient update: During training, only the student network performs gradient updates, and the teacher network does not participate in gradient updates.

[0129] (3) Joint testing of the teacher network and the student network: During the test, the teacher network and the student network jointly participate. When facing normal samples, the difference between the intermediate layer feature outputs of the two is not large, and the total loss value is low. Since the generalization ability of the student network deteriorates after pruning, when facing small images of defective insulators, the intermediate layer feature outputs are quite different from those of the teacher network, and the total loss value is large.

[0130] (4) Scoring the test samples: According to the total loss value of all samples, each sample is scored for defects. The defect score is:

[0131]

[0132] Among them, S i is the defect score of the i-th sample, is the total loss of the i-th sample, and the median function takes the median of the total loss values ​​of all samples.

[0133] The score probability distribution diagram of defective samples and normal samples is as follows: Figure 6 shown.

[0134] (5) Whether the threshold is met: Taking 1 as the threshold, samples with a defect score greater than 1 are judged as defective samples, and samples with a defect score less than 1 are judged as normal samples.

[0135] (6) Output the location information of defective samples and normal samples: When segmenting small blocks, a square sliding window of 128 pixels is used, the step length in the row direction and the step length in the column direction are both 64 pixels, and the overlapping area between small blocks in the row direction and the column direction is 50%. In this way, the location information of each small block can be output.

[0136] (7) Gradient backpropagation between the teacher network and the student network: Gradient-based saliency algorithms are used: smooth gradient and integral gradient. The backpropagated gradient needs to be used to participate in the calculation of the next step of the heat map.

[0137] (8) Generate a small block heat map: Use the gradient information stored in the reverse transmission to perform the following calculation:

[0138]

[0139] Among them, Λ is a small block heat map, is the input image formed by doping the image x with a certain amount of Gaussian noise, and x' is the input image The integral reference image is taken as a pure black image, k is the number of path images, and N is the number of repetitions of the smooth gradient method;

[0140] The i-th path image is:

[0141]

[0142] The magnitude of Gaussian noise is:

[0143]

[0144] Wherein, p is a hyperparameter for adjusting the noise size, p=1, max(x) is the pixel value of the brightest pixel in the input image, and min(x) is the pixel value of the darkest pixel in the input image.

[0145] For each path image x i , we all use the original gradient method to calculate its heat map:

[0146]

[0147] Then, calculate all κ i The average value of κ is obtained, and then multiplied by the path length The thermal map of a single integrated gradient can be obtained. The above process is repeated N times to eliminate the influence of the randomness of random Gaussian noise and obtain a thermal map of a smooth gradient. In the present invention, k=25 and N=5.

[0148] (9) Heat map noise suppression: Because the behavior of the neural network has a certain degree of randomness, it is necessary to set a threshold for noise suppression. In the obtained small block heat map, the threshold T = 5e-4 is set, and the pixels with values ​​less than this value are suppressed to 0. Then the pixel values ​​of the heat map are restored to [0, 255] to obtain the final heat map. The small block heat map after suppression is shown in the figure. Figure 7 shown.

[0149] (10) Combined into an insulator heat map: Using the previous position information of the insulator small block and the small block heat map, the small block heat map is pieced back into the original image, and the highlighted pixel area is taken to achieve pixel-level segmentation of the insulator defects. The combined insulator heat map is as follows: Figure 8-13 shown.

[0150] (11) Reporting normal insulators and defective insulators: Based on the previously obtained insulator patch defect score and the insulator patch location information, that is, which insulator the patch comes from and where, the insulator defect detection result is obtained. The test result is F1 = 0.9792.

[0151] (12) Inspection report: Based on the previously obtained complete thermal map of the insulator and the insulator defect inspection results, an insulator defect inspection report is generated.

[0152] In this implementation, the insulator image size is adjusted to 1024×512 pixels. The insulator image is segmented using a sliding window of size 128 pixels and step size 64 pixels to obtain insulator block samples. The teacher network is classified and pre-trained using normal insulator block samples and background block samples so that it has insulator feature information. In the training stage, the teacher network and the student network are trained simultaneously using normal insulator block samples so that the intermediate layer feature output of the student network is similar to that of the teacher network. In the testing stage, the test set contains defective insulator block samples and normal insulator block samples. The defect scores corresponding to the defective insulator block samples are high, and the defect scores corresponding to the normal insulator block samples are low. After completing the defect detection, the integral gradient method and the smooth gradient method can be used to obtain the defect segmentation heat map and accurately mark the insulator defect area.

[0153] The above is a specific description of the implementation mode of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention, and these equivalents or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A high-speed rail insulator defect detection method based on knowledge distillation, characterized in that: The following steps are involved: S1, input 4C grayscale image containing insulator; S2, extracting the insulators in the grayscale image, and cutting the extracted insulator image into blocks to obtain small insulator block samples; S3. Establish a knowledge distillation teacher network, use normal insulator small block samples and background small block samples of the same size to classify and train the teacher network, so that its parameter space contains the characteristic information of the insulator; S4. Establish a knowledge distillation student network after channel pruning, and jointly train and test the teacher network and the student network. According to the different intermediate layer outputs of the teacher network and the student network when facing abnormal samples during the test process, judge the defective insulator small block samples; establish a defect segmentation heat map based on the defect judgment results, and mark the insulator defect area in the heat map; The S4 step includes a joint training and testing step, a scoring and judging step, a small block heat map generating step, and a defect detection report generating step; The joint training and testing step establishes a knowledge distillation student network after channel pruning, and uses normal insulator small block samples to jointly train the teacher network and the student network. After the training, each insulator small block sample is added to the teacher network and the student network for joint testing. During the testing process, the total loss value of the sample is obtained according to the different intermediate layer outputs of the teacher network and the student network when facing abnormal samples; The scoring and judging step scores each tested insulator small block sample according to the total loss value of all samples, and outputs the insulator small block sample position information according to the scoring result; Then, according to the scoring results and the sample location information, the information of the normal insulator small block samples and the defective insulator small block samples are reported to obtain the insulator defect detection results; The small block heat map generation step uses the integral gradient method and the smooth gradient method to perform gradient backpropagation on the teacher network and the student network and generate a small block heat map, and then suppresses noise on the heat map; The defect detection report generation step uses the position information of the small block samples of the insulator and the small block heat map after noise suppression, combines the small block heat maps into a complete heat map, and marks the insulator defect area in the heat map, and then generates an insulator defect detection report based on the complete heat map and the insulator defect detection results.

2. The high-speed rail insulator defect detection method according to claim 1, characterized in that: In the step S2, the insulator is extracted using the R-YOLOv5s network, and its prediction frame contains 180 angle category information; the insulator small block sample size is 128*128 pixels.

3. The high-speed rail insulator defect detection method according to claim 1, characterized in that: In the S3 step, the VGG-16 backbone network is used, and its classification layer is removed as the teacher network of the knowledge distillation network.

4. The high-speed rail insulator defect detection method according to claim 1, characterized in that: In the joint training and testing step, during joint training, the student network is gradient updated, but the teacher network is not gradient updated; Use the pruned VGG-16 as the student network of the knowledge distillation algorithm. Except for the last layer of each convolution module, the number of channels of other convolution layers is compressed to 16. The loss function used in training is L t , L t =L v +αL dir Among them, L t is the total loss function, L v is the distance loss function, L dir is the cosine similarity loss function, α is the hyperparameter for adjusting the weights of the two loss functions, N c For L i The number of neurons in the layer, T n In the L i The output features of the layer, For S n In the L i The output features of the layer, T stands for Transpose, which is the transpose operation of the vector, i represents traversing all the intermediate layers to be extracted, and j represents the number of channels traversed in a layer.

5. The high-speed rail insulator defect detection method according to claim 1, characterized in that: The scoring and judging step comprises the following steps: Calculate the total loss value of all insulator small block samples, and assign defect scores to each insulator small block sample according to the total loss value of all insulator small block samples; Compare the defect score with the threshold value to determine whether the sample is a defective insulator patch sample or a normal insulator patch sample; According to the judgment result, the position information of the defective insulator small block sample and the normal insulator small block sample is output; According to the scores of the insulator small block samples and the position information of the insulator small block samples, the information of the normal insulator small block samples and the defective insulator small block samples is reported to obtain the insulator defect detection result.

6. The high-speed rail insulator defect detection method according to claim 5, characterized in that: In the scoring and judging step, the defect score is: Among them, S i is the defect score of the i-th sample, is the total loss of the i-th sample, and the median function is the median of the total loss values ​​of all samples; In the threshold comparison, 1 is taken as the threshold, and the defect scores greater than 1 are judged as defective insulator small block samples, and the defect scores less than 1 are judged as normal insulator small block samples.

7. The high-speed rail insulator defect detection method according to claim 1, characterized in that: The small block heat map generation step includes the following steps: The integral gradient method and smooth gradient method are used to perform gradient backpropagation on the teacher network and the student network. Generate a small heat map using the gradient information stored in the reverse transmission of the teacher network and the student network gradient feedback; In the generated small block heat map, a threshold is set to suppress noise.

8. The high-speed rail insulator defect detection method according to claim 7, characterized in that: In the step of generating a small block heat map, the small block heat map is: Among them, Λ is a small block heat map, is the input image formed by doping the image x with a certain amount of Gaussian noise, and x' is the input image The integral reference image is taken as a pure black image, k is the number of path images, and N is the number of repetitions of the smooth gradient method; The i-th path image is: The magnitude of Gaussian noise is: Where p is a hyperparameter for adjusting the noise size, p=1, max(x) is the pixel value of the brightest pixel in the input image, and min(x) is the pixel value of the darkest pixel in the input image; In noise suppression, in the obtained small block heat map, the threshold T=5e-4 is set, and the pixels with pixel values ​​less than this value are suppressed to 0, and then the pixel values ​​of the heat map are restored to [0,255] to obtain the final heat map.

9. The high-speed rail insulator defect detection method according to claim 1, characterized in that: The defect detection report generation step comprises the following steps: Using the location information of the small block samples of the insulator and the small block heat map after noise suppression, the small block heat map is pieced back into the original image to form a complete insulator heat map, and the highlighted pixel area is taken to segment the insulator defects at the pixel level. Generate an insulator defect detection report based on the complete insulator thermal map and insulator defect detection results.

Images