High-voltage cable terminal thermogram defect identification method based on deep learning

Abstract

The invention discloses a high-voltage cable terminal thermogram defect detection method based on deep learning. The method comprises the following steps: S1, acquiring an initial cable terminal thermogram and preprocessing the initial cable terminal thermogram by using a BM3D image denoising algorithm based on image segmentation; s2, expanding the volume of the image set through a data enhancement technology; s3, marking the expanded thermogram to form a data set, and dividing the data set into a test set and a training set; s4, improving and modeling a multi-scale convolutional neural network model, adding the attention mechanism module CBAM into a backbone feature extraction layer of the multi-scale convolutional neural network model by taking VGG-16 as an infrastructure to optimize the network, and replacing the conventional convolution Conv of the VGG-16 network model with a lightweight ghosting shuffling convolution module GSConv in the aspect of the model; and S5, carrying out training and performance evaluation on the improved network model. Compared with an original convolution VGG-16 network model, the cable terminal thermal image feature extraction method can better realize feature extraction of the cable terminal thermal image.

Description

Technical Field

[0001] The present invention relates to the fields of image processing and power equipment defect detection, and particularly to a method for identifying defects in thermal images of high-voltage cable terminals based on deep learning. Background Art

[0002] As one of the most important components in a transmission line, various defect information will emerge when high-voltage cable terminals are applied in various harsh environmental conditions for a long time. How to accurately and quickly detect cable terminal defects is of great significance for avoiding large-scale power outages and reducing power grid losses. The infrared thermal image recognition method is currently the recommended method for detecting defects during the operation and maintenance of cable terminals, and deep learning algorithms have also been widely used in the recognition and processing of thermal images of high-voltage cable terminal defects.

[0003] For composite sleeve or porcelain sleeve oil-filled terminals of high-voltage cables, typical defects that often occur during operation, such as oil leakage, heating of the upper conductor fitting, heating of the lower tail pipe, and fouling on the surface of the insulating sleeve, etc., will all form distinguishable heating phenomena. During night inspections, the heating information of the defects can be detected by an infrared thermal imager, forming a thermal image with defect characteristics. Since a large number of thermal imaging pictures will be generated during inspections, the accuracy and efficiency of picture processing have become a difficult point and a research hotspot of this technology. There are various artificial intelligence object detection algorithms applied to the image processing of power cable terminals. Among them, the VGG series has relatively superior advantages. Currently, a VGG-16 method based on deep learning is a relatively new algorithm, but it still needs to be improved to meet the scene requirements where it is difficult to distinguish the defect characteristics of cable terminal thermal images.

[0004] Therefore, the present invention provides a method for identifying defects in thermal images of cable terminals based on deep learning. Its core technology is to use an improved VGG-16 algorithm to enhance the fusion ability of the attention mechanism of the detection module for thermal image photos after image preprocessing, so as to realize the detection of weak feature defects such as cable terminal fouling in the transmission line, improve the accuracy and efficiency of image processing, and provide a new idea for the intelligent inspection of transmission lines.

[0005] Patent CN 117788928A discloses a method and system for automatically diagnosing and warning heating defects of cable oil-filled terminals. This method uses a VGG-11 feature network to optimize the PIAFuson network model. By improving the PIA Fusion algorithm, the fusion effect of infrared and visible light images is improved, and indirectly, the target detection recognition rate of Yolo is enhanced. The object of this invention is the processing of infrared and visible light images.

[0006] Patent CN 113433167B discloses a method and system for monitoring the status of power cable terminals based on infrared thermal images. This method determines whether there are defects in the cable terminals based on the temperature difference between the average temperature of any foreground sub-region and the average temperature of the background region in the infrared thermal image of the cable terminal. This method requires calculating the average temperature of all foreground sub-regions in the infrared thermal image, and then transmitting the data to a computer to perform fault diagnosis by comparing the temperature differences. It can provide accurate and timely discrimination of the working temperature status of the target power cable terminal equipment, and has the advantages of high detection accuracy, long-distance detection, safety and reliability. However, the amount of repeated calculation is relatively large, consuming computing resources. Summary of the Invention

[0007] The present invention proposes a method for detecting defects in cable terminal thermal images based on a multi-scale network integrating an attention mechanism, realizing rapid diagnosis of cable terminal thermal image faults, improving the speed and accuracy of operations such as image recognition and feature extraction, and providing guidance for subsequent transmission line maintenance. To achieve the above invention purpose, the technical solution adopted by the present invention is as follows:

[0008] The present invention provides a method for identifying defects in cable terminal thermal images based on deep learning, including the following steps:

[0009] S1. Obtain the initial cable terminal thermal image and perform image processing using the BM3D image denoising algorithm based on image segmentation;

[0010] S2. Supplement the cable terminal thermal image dataset through data augmentation techniques;

[0011] S3. Label the cable terminal thermal images to form a dataset and divide it into a test set and a training set;

[0012] S4. Improve and build a multi-scale convolutional neural network model;

[0013] S5. Train and evaluate the performance of the improved network model to obtain defect detection results.

[0014] Specifically, in step S1, the cable terminal thermal image is collected by using an infrared thermal imager, and the initial dataset required by the present invention is formed by combining various online dataset public resources.

[0017] Specifically, in step S1, based on the BM3D image denoising algorithm, by combining the advantages of the SFCM clustering algorithm and adaptive edge detection, a BM3D image denoising algorithm based on image segmentation is proposed. It can improve the accuracy of block matching and retain the details such as the edge texture of the image to the greatest extent.

[0018] The BM3D image denoising algorithm based on image segmentation is divided into a basic estimation stage and a final estimation stage. In the basic estimation stage, the noisy image is first roughly denoised, and then image segmentation is performed to divide the image into a homogeneous region and an edge region. If the reference block contains edge information, it is made to match along the edge direction in the edge region. If the reference block does not contain edge information, it is made to match along a fixed direction in the homogeneous region.

[0015] Specifically, in step S2, based on traditional data augmentation techniques, the dataset is mainly expanded by operations such as mirror flipping, contrast adjustment, and random cropping of the image. These methods are simple to operate and easy to implement; the Mosaic online augmentation technique is used to generalize and expand the sample data. This data augmentation technique can stitch four different images in the training set together to form a new image. The cable terminal thermal image dataset is supplemented by combining these two data augmentation techniques of online augmentation and offline augmentation.

[0016] Specifically, in step S3, the annotation tool Labelimg is used to annotate the defective cable terminal thermal imaging.PNG file to obtain an.xml file, and then it is converted into a.txt file adapted to the VGG (Visual Geometry Group) network. After annotating the image data, the required dataset is formed and divided into a test set and a training set according to a ratio.

[0019] Specifically, in step S4, the multi-scale network is divided into three branches. Taking VGG-16 as the basic architecture, the attention mechanism module CBAM (Convolutional Block Attention Module) is added to the backbone feature extraction layer of the multi-scale convolutional neural network model to optimize the network, and at the same time, new detection layers are added. The idea of the attention mechanism module CBAM is to decompose a large convolutional kernel into three parts: 1 depth convolution, 1 depth dilated convolution, and 1 convolution. The specific process expression is: k attention =Conv 1×1 (DW_D_Conv(DW_Conv(F)))

[0020] where F is the input feature of this layer; DW_Conv, DW_D_Conv, and Conv 1×1 represent depth convolution, depth dilated convolution, and convolution respectively; k attention is the calculated attention parameter kernel, and each parameter represents the importance of each feature of F.

[0022] Specifically, in step S4, the lightweight Ghost-shuffle convolution module GSConv (Ghost-shuffle convolution) is used to replace the conventional convolution Conv in the original VGG-16 network model.

[0023] The lightweight Ghost-shuffle convolution module (GSConv) simplifies some traditional convolution processes and realizes feature extraction through linear transformation. The specific process includes: first, extracting basic features from the input feature map using standard convolution; then, applying linear transformation to each channel to generate GS feature maps with the same number of channels as the input; then, splicing these GS feature maps with the original basic feature map; finally, further enhancing the generalization ability of the model through channel shuffle to obtain the final output feature map. This design not only reduces the computational complexity and the number of parameters, but also can effectively maintain and improve the network performance, and is especially suitable for scenarios that require efficient processing of a large amount of data. In addition, GSConv can also be flexibly combined with other deep learning technologies to adapt to different task requirements and computational resource limitations.

[0024] Compared with the prior art, the beneficial effects of the present invention are:

[0025] A method for detecting cable terminal thermal image defects based on a multi-scale network with a fusion attention mechanism provided by the present invention collects a large number of cable terminal thermal images, supplements the cable terminal thermal image data set through data augmentation technology and divides it into a test set and a training set, and then uses the multi-scale network model with a fusion attention mechanism to detect the cable terminal thermal images. Based on the BM3D image denoising algorithm, combining the advantages of the SFCM clustering algorithm and adaptive edge detection, the initial collected images are processed, and weighted fusion is performed with the VGG-16 as the basic architecture in the multi-scale convolutional neural network and the CBAM attention mechanism. In order to reduce the computational cost of cable terminal fault recognition, the lightweight Ghost-shuffle convolution module GSConv is used to replace the standard convolution.

[0026] Finally, the invented defect detection method is used for defect detection of cable terminal thermal images. The experimental results show that the present invention realizes the rapid diagnosis of cable terminal thermal image faults, and can provide a reference for subsequent high-voltage cable line and terminal maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] To make the technical solutions of the embodiments of the present invention clearer, the drawings involved in the description will be briefly described below. Obviously, these drawings only show the preferred embodiments of the present invention, and those skilled in the art can obtain other drawings without creative work accordingly.

[0028] Figure 1 It is a flowchart of the present invention.

[0029] Figure 2 It is a structural schematic diagram of the BM3D image denoising algorithm for image segmentation in the present invention.

[0030] Figure 3 It is a multi-scale network structure diagram of the present invention. (Where A, B, and C are three branches of the multi-scale network; C1...C5 are 5 convolutional modules of branch A based on the VGG16 architecture; branch B uses the second and fourth convolutional modules in the VGG 16 architecture; branch C uses the second convolutional module in the VGG16 architecture).

[0031] Figure 4 It is a process diagram of the channel domain weighted fusion of the attention mechanism in the present invention. (Where f sq (*) Scale compression operation; f ex (I 0 , w) is the importance operation of each channel's information; f sc (*) operation weights the importance of each channel to the input image I0).

[0031] Figure 5 It is a structural diagram of the lightweight ghost shuffle convolution module GSConv introduced in the present invention. Where 4-1: Input; 4-2: Standard convolution; 4-3: Depth convolution; 4-4: Concatenation operation; 4-5: Channel shuffle; 4-6: Output).

[0032] Figure 6 It is a structural schematic diagram of the multi-scale network model and module integrating the attention mechanism of the present invention. (Where C0 is the original image after feature extraction; A, B, and C are three branches of the multi-scale network; C1...C5 are 5 convolutional modules of the multi-scale convolutional neural network; C1 * ...C5 * represents the convolutional module after weighted fusion by the attention module) Detailed implementation manner

[0033] Combined with Figure 1 , a schematic diagram of the cable terminal thermal image defect detection method based on deep learning provided by the present invention, the method includes the following steps:

[0034] Step S1, obtain the initial cable terminal thermal image;

[0035] See Figure 2, the BM3D denoising algorithm based on image segmentation mainly consists of two stages: basic estimation and final estimation. In the basic estimation stage, first, a preliminary denoising process is performed on the noisy image, and then image segmentation is carried out to distinguish homogeneous regions and edge regions. For the reference block, if it contains edge information, it is matched along the edge direction within the edge region; if it does not contain edge information, it is matched in a fixed direction within the homogeneous region. The present invention uses an infrared thermal imager to collect infrared images of cable terminals and constructs the required initial dataset by combining various network dataset resources.

[0036] Step S2: Adopt a data augmentation technology that combines online augmentation and offline augmentation to expand the cable terminal thermal image dataset.

[0037] Based on Step S1, the initial dataset is further enriched using data augmentation technology, which includes two core strategies: online augmentation and offline augmentation. By performing diverse operations such as flipping, brightness fine-tuning, random cropping, and rotation on the images in the initial dataset, the diversity and quantity of samples are significantly improved. Before the data is input into the network model, various preprocessing steps such as flipping, brightness adjustment, Mosaic processing, and random cropping are performed on the samples. These two augmentation technologies cooperate with each other, not only effectively expanding the sample scale of the dataset but also enhancing the generalization ability of the network, thereby improving the robustness and performance of the model.

[0038] Step S3: Annotate the cable terminal thermal images to form a dataset and divide it into a test set and a training set;

[0039] For the normal cable terminals and defective cable terminals in the thermal images, we use the annotation software LabelImg for accurate annotation. The thermal images of the defective cable terminals are saved in the.PNG format and the corresponding.xml files are generated. Subsequently, these.xml files are converted into.txt format files dedicated to the multi-scale convolutional neural network. After the annotation work is completed, the required dataset is obtained and divided into a test set and a training set according to a certain ratio, laying a foundation for subsequent model training and evaluation. The ratio is 1:9 to 2:8, and the scale of the dataset is greater than 800 images.

[0040] Step S4: Improve and model the multi-scale network model;

[0041] The present invention selects the VGG16 network model as the basic architecture of each branch of the multi-scale network, optimizes the backbone feature extraction layer of its model, introduces the attention mechanism module CBAM to enhance the model's feature extraction ability for small targets, which helps the model better locate and identify cable terminal targets; introduces the lightweight ghost shuffle convolution module GSConv to reduce the number of model parameters and balance the accuracy and speed of the model.

[0042] In this example, the attention mechanism module CBAM is introduced into the backbone feature extraction layer of the multi-scale network model. The specific position is the last layer of the backbone network, that is, the thirty-first layer of the entire initial VGG16 network model.

[0043] See Figure 3 , the multi-scale network uses VGG 16 as the basic architecture of branch A of the multi-scale network. All 3x3 convolutional kernels and 2x2 pooling kernels are used in VGGNet. Compared with AlexNet, although VGGNet has more model layers and longer training time per round, due to the implicit regularization results brought by deeper networks and smaller convolutional kernels, VGGNet converges in fewer epochs.

[0047] The network has a total of 3 branches. Branch A is based on VGG 16 and consists of 5 convolutional modules. The first convolutional module consists of 64 3x3 convolutional kernels and a 2x2 max pooling kernel; the second convolutional module consists of 128 3x3 convolutional kernels and a 2x2 max pooling kernel; the third convolutional module consists of two layers of 3x3x256 convolutional kernels and a 2x2 max pooling kernel; the fourth and fifth convolutional modules are both composed of two layers of 3x3x512 convolutional kernels and a 2x2 max pooling kernel; Branch B uses the second and fourth convolutional modules in the VGG16 architecture; Branch C uses the second convolutional module in the VGG16 architecture.

[0047] See Figure 5 , the lightweight Ghost Shuffle Convolution module (GSConv) simplifies part of the traditional convolution process and realizes feature extraction through linear transformation. The specific process includes: first, use standard convolution to extract basic features from the input feature map; then, apply linear transformation to each channel to generate GS feature maps with the same number of input channels; then, splice these GS feature maps with the original basic feature map; finally, further enhance the generalization ability of the model through channel shuffle to obtain the final output feature map. This design reduces the computational complexity and the number of parameters, can effectively maintain and improve the network performance, and is especially suitable for scenarios that need to efficiently process a large amount of data.

[0048] In this example, the lightweight Ghost Shuffle Convolution module (GSConv) is used to replace the traditional convolutional layer to optimize the model performance. While reducing the consumption of computing resources, GSConv effectively maintains the correlation between channels and ensures the high accuracy of the model. Due to the introduction of GSConv, although the number of network layers has increased, considering that the backbone feature extraction layer has approached the limit in some dimensions, the application of GSConv can instead improve the overall inference speed and achieve double optimization of accuracy and speed. On the basis of maintaining similar effects to ordinary convolution, the model significantly reduces the computational cost while improving the inference efficiency and accuracy.

[0049] See Figure 4 , the channel-domain weighted fusion process is divided into the following steps: (1) The original image I0 is subjected to f sq (*) operation for scale compression, where h and b are the length and width of the image respectively, c is the number of channels of the image, and the calculation formula of f sq (*) is as follows: (2) The compressed 1*1*c image passes through adaptive learning f ex (I 0 , w), and the importance of each channel information is obtained as: f ex (I 0 , w) = (w 1 , w 2 , w 3 ,..., w c ) In the formula, w 1 ...w c are the importance of each channel after adaptive learning. (3) Through f sc (*) operation, the importance of each channel is weighted to the input image I 0 , and the image I 1 after channel information weighted fusion is obtained. I 1 = f sc (I 0 ) = (f 1 w 1 , f 2 w 2 , f 3 w 3 ,..., f c w c ) In the formula, f 1 …f c are the images of each channel on the input image.

[0050] See Figure 6 , the multi-scale network and the attention mechanism are fused, and C1 * , C2 * , C3 * , C4 * , C5 * represent the convolutional modules after weighted fusion by the attention module.

[0050] Step S5: Train and evaluate the performance of the improved network model to obtain the defect detection result;

[0051] In this example, experiments were carried out for verification according to the above process. For the multi-scale network integrating the attention mechanism in the present invention, multi-dimensional comparisons were made with other mainstream object detection models. The comparison dimensions cover precision (P), recall (R), mean Average Precision (mAP@0.5) when the IoU value is 0.5, and an index representing the frame processing speed per second of the model.

[0052] The calculation formulas for precision P and recall R are as follows:

[0053] TP (True Positive) represents the correctly predicted positive sample, that is, the sample is detected and the prediction result is correct; FP (False Positive) represents the incorrectly predicted positive sample, that is, the sample is detected but the prediction result is incorrect, also known as false detection; FN (False Negative) represents the incorrectly predicted negative sample, that is, the sample is not detected, and there should be a sample at this position in fact, also known as missed detection.

[0054] IoU is generally used to compare the overlapping degree between the predicted box and the ground truth box. In some object detection tasks, a threshold will be preset for it, and the calculation formula is as follows:

[0055] The mean Average Precision (mAP@0.5) is usually calculated by presetting a threshold and sorting the prediction results in descending order according to the IoU value. By adjusting the threshold and repeating this process, the P-R curve and the AP value can be obtained, and the calculation formulas are as follows:

[0056] In some alternative embodiments, comparative experiments were conducted using other mainstream object detection models, the YOLOv5 network model, and the multi-scale convolutional neural network model integrating the attention mechanism provided by the present invention, and the experimental results are shown in Table 1: Table 1 Comparison table of performance results of each object detection model model P(％) R(％) mAP@0.5 Detection speed (FPS) CNN 81.2～81.5 76.1～76.6 83.6～83.8 42.3～42.6 FasterR-CNN 83.1～83.4 82.5～82.8 84.3～84.5 48.5～48.7 SSD 85.6～85.8 85.4～85.7 85.3～85.5 51.7～51.9 YOLOv5 89.1～89.4 88.4～88.6 87.0～87.2 62.2～62.5 this model 91.5～91.8 90.3～90.5 90.6～90.9 55.6～55.8

[0057] In Table 1, compared with the CNN, Faster R-CNN, SSD, and YOLOv5 algorithm models, the detection mean Average Precision of the network model of the present invention has been greatly improved, further verifying the effectiveness of the method of the present invention.

[0058] The preferred embodiments of the present invention have been described above. It should be understood that the present invention is not limited to these specific embodiments, and the devices and structures not described in detail should be implemented by conventional methods in the art. Those skilled in the art can make various possible adjustments and modifications to the above methods and contents, or equivalent replacements, which are all equivalent change embodiments without departing from the technical solution of the present invention. Therefore, as long as it does not deviate from the technical solution of the present invention, any simple changes, equivalent replacements and modifications made to the embodiments according to its technical essence still fall within the protection scope of the present invention.

Claims

1. A cable terminal thermal image defect detection method based on deep learning, characterized in that: The following steps are involved: S1, obtaining the required initial cable terminal thermal image and performing image processing using the BM3D image denoising algorithm based on image segmentation; S2, supplement the cable terminal thermal image dataset through the data enhancement technology combining online enhancement and offline enhancement; S3, annotating the thermal images of the cable terminals to form a data set and dividing it into a test set and a training set; S4, introducing the attention mechanism CBAM and the lightweight ghost shuffle convolution module GSConv into the multi-scale convolutional neural network model; obtaining the improved multi-scale convolutional neural network model integrating the attention mechanism; S5. Use the improved network model to perform defect detection and performance evaluation on the image data set to be detected to obtain detection results.

2. The cable terminal thermal imaging defect detection method based on deep learning according to claim 1 is characterized in that: The method is to process the obtained initial cable terminal thermal image using the BM3D image denoising algorithm based on image segmentation, and introduce the attention mechanism module CBAM and the lightweight ghost shuffle convolution module GSConv into the multi-scale convolutional neural network model; The initial cable terminal thermal image is processed using the BM3D image denoising algorithm based on image segmentation; the CBAM attention mechanism is added to the backbone feature extraction layer (backbone) network of the multi-scale convolutional neural network model; the lightweight ghost shuffle convolution module GSConv replaces the conventional convolution Conv in the VGG16 network model.

3. The cable terminal thermal imaging defect detection method based on deep learning according to claim 1 is characterized in that The cable terminal thermal image is subjected to data preprocessing, including: The cable terminal thermal imaging image data set is expanded by using data enhancement technology, and the cable terminal thermal imaging image is annotated by using annotation tools to obtain the required file. After the annotated image data is completed, the required data set is formed and divided into a test set and a training set according to the proportion.

4. The data enhancement technique according to claim 3, characterized in that Based on traditional data enhancement technology, the dataset is expanded mainly by mirror flipping, adjusting contrast, random cropping and other operations; Mosaic online enhancement technology is used to generalize and expand the sample data. The cable terminal thermal image dataset is supplemented by the data enhancement technology combining the online enhancement and the offline enhancement.

5. The cable terminal thermal imaging defect detection method based on deep learning according to claim 1 is characterized in that: After training, the performance evaluation of the multi-scale convolutional neural network model with fusion attention mechanism is tested. The model evaluation indicators include precision P (Precision), recall R (Recall), average precision (mAP@0.5, meanAverage Precision) when the IoU value is 0.5, and the frame processing speed of the representative model per second as the performance evaluation indicators of the multi-scale convolutional neural network model with fusion attention mechanism.

Images