An insulator defect detection method based on an improved CenterNet network
By improving the CenterNet network and combining convolutional block attention modules and pyramid pooling modules, an insulator defect dataset was constructed and the model was trained, which solved the problems of low efficiency and high false positive rate of manual inspection and achieved efficient and accurate insulator defect detection.
Patent Information
- Application Number
- CN202210924238.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-02
- Publication Date
- 2026-01-13
- Estimated Expiration
- 2042-08-02
AI Technical Summary
In the current technology, manual inspection of insulators is inefficient and has a high error rate, which makes it difficult to meet the growing monitoring needs of insulators. In addition, drone inspection requires manual image interpretation, which is costly and time-consuming.
An improved CenterNet network is adopted, which deepens the feature extraction network, introduces the convolutional block attention module (CBAM) and pyramid pooling module, constructs an insulator defect dataset, and performs training and parameter tuning to generate the optimal network model for detection.
It improves the accuracy of insulator testing, reduces missed and false detections, enhances testing efficiency and accuracy, and solves the problem of insulator defect identification in complex environments.
Smart Images
Figure CN115239681B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of power system transmission line detection technology, and particularly relates to an insulator defect detection method based on an improved CenterNet network. Background Technology
[0002] Insulators are one of the most important infrastructure components in a power system, ensuring the safe and stable operation of transmission lines and substations. However, insulators are primarily exposed to harsh outdoor environments, making them highly susceptible to defects such as spontaneous explosion and contamination. These defects can affect the safety and stability of the entire power system, leading to socio-economic losses and safety hazards. Therefore, regular inspections of the working status and fault conditions of insulators are crucial for ensuring the stable operation of the power system. The primary challenge in this process is the accurate detection and identification of insulators.
[0003] Manual inspection of insulators is highly inefficient, and field inspections are inherently dangerous. Furthermore, the results obtained are not very accurate, failing to meet the ever-increasing monitoring demands for insulators. Current technologies utilize drones in conjunction with manual inspections. While the continuous development of drone technology has brought new possibilities to power line inspections, manual interpretation of insulator images is still necessary. The limitations of manual interpretation are twofold: firstly, to avoid misidentification or omission of insulators, inspectors require extensive experience; secondly, manually interpreting a large number of aerial insulator images is not only slow and inefficient but also costly. Therefore, it is necessary to propose an insulator defect detection method based on an improved CenterNet network. Summary of the Invention
[0004] The purpose of this invention is to provide an insulator defect detection method based on an improved CenterNet network, which aims to solve the problems mentioned in the background art.
[0005] This invention is implemented as follows: an insulator defect detection method based on an improved CenterNet network, the method comprising the following steps:
[0006] Establish an insulator defect dataset;
[0007] Establish an image label library for the insulator defect dataset. Specifically, this includes: labeling all images in the insulator defect dataset using the Labellmg tool, and organizing the label files and images in the insulator defect dataset into the PASCAL VOC2007 dataset format to generate the complete insulator defect dataset.
[0008] The entire insulator defect dataset is divided into a training set and a test set;
[0009] Constructing a network based on the improved CenterNet, specifically including:
[0010] The original backbone network of GenterNet, resnet18, is deepened into resnet50, and a convolutional block attention module (CBAM) is introduced.
[0011] Embed CBAM before connecting the original block and the residual structure, and then build the pyramid pooling module;
[0012] The feature map output by ResNet50-CBAM is used as the input to the pyramid pooling module, and then the feature map output by the pyramid pooling module is subjected to three deconvolution operations.
[0013] The feature map after three deconvolution operations is input into the CenterNet network detection head;
[0014] The improved GenterNet network was trained and its parameters tuned using the training set from the entire insulator defect dataset to generate the optimal network model.
[0015] The test set from the insulator defect dataset is input into the trained optimal network model, which outputs the insulator detection results and target location information to realize the detection of insulator defects in transmission lines.
[0016] As a further aspect of the present invention, the establishment of the insulator defect dataset specifically includes:
[0017] To acquire images of insulators on power poles taken by drones;
[0018] Images with resolution below the resolution threshold are deleted, and the remaining photos are used as the inspection dataset.
[0019] The inspection dataset was merged with publicly available insulator datasets online. Data augmentation was performed on the images in the merged dataset to obtain an insulator defect dataset containing several images.
[0020] As a further aspect of the present invention, the different target objects to be detected in all images in the insulator defect dataset have different label categories. After each image is labeled, an XML file containing label information of all target categories in the image is generated.
[0021] As a further aspect of the present invention, the step of dividing the entire insulator defect dataset into a training set and a test set specifically includes: using a random allocation method to randomly divide all the images contained in the insulator defect dataset according to a preset ratio, thereby obtaining training set images and test set images respectively.
[0022] As a further aspect of the present invention, the introduction of the convolutional block attention module (CBAM) specifically includes:
[0023] The original CenterNet backbone network, resnet18, is deepened into a resnet50 network, and a convolutional block attention module (CBAM) is introduced, which is embedded before the connection between the original block and the residual structure.
[0024] As a further aspect of the present invention, the construction of the pyramid pooling module specifically includes:
[0025] The original backbone network of CenterNet, resnet18, is deepened into resnet50, and a convolutional block attention module (CBAM) is introduced. CBAM is embedded before the connection between the original block and the residual structure.
[0026] The feature map output by ResNet50-CBAM is used as the input to the pyramid pooling module. This module uses four different pyramid scales, with each pooling layer having a size of 1×1, 2×2, 3×3, and 6×6. The first layer performs global pooling on the entire feature map. The second to fourth layers divide the feature map into different sub-regions, and each sub-region performs its own pooling operation. Each layer of pooling produces a feature map containing features at different scales. Then, a convolution operation is performed on the obtained feature map with a kernel size of 1×1. The purpose is to reduce the dimensionality of the obtained feature map, reducing the number of channels to a fraction of the original feature map, where represents the number of pyramid layers.
[0027] By using bilinear interpolation, the channel dimension of the obtained feature map is upsampled to be the same as that of the original input feature map. These feature maps are then concatenated with the original feature map to obtain the final global features.
[0028] As a further aspect of the present invention, when performing a convolution operation on the obtained feature map, the size of the convolution kernel is 1×1, so as to reduce the dimensionality of the obtained feature map and reduce the number of channels to a fraction of the original feature map, representing the number of pyramid layers.
[0029] As a further aspect of the present invention, the step of training and parameter tuning the improved CenterNet network using the training set from the entire insulator defect dataset to generate the optimal network model specifically includes:
[0030] The improved CenterNet network was trained using the training set in the dataset, employing a freeze-and-thaw training approach. The batch size was set to 8 during freeze-and-thaw training and 4 during thaw-and-thaw training. Freeze-and-thaw training lasted for 50 epochs, and thaw-and-thaw training lasted for 250 epochs, for a total of 300 epochs. The Adam optimizer was selected for this experiment. The learning rate was set to 0.001 during freeze-and-thaw training and 0.0001 during thaw-and-thaw training. Parameter tuning was performed based on the training results.
[0031] As a further aspect of the present invention, the step of inputting the test set from the dataset into the trained improved CenterNet network model specifically includes:
[0032] The training weights of the model are obtained through training, and the optimal weights are loaded into the improved CenterNet network model. When the insulator image is input into the network model, the detection results will use two different colored detection boxes to detect normal insulators and defective insulators. The identified target categories and the located coordinate information will be displayed on the original image.
[0033] This invention provides an insulator defect detection method based on an improved CenterNet network. By deepening the feature extraction network and constructing a convolutional block attention module, the feature extraction capability of the network is enhanced. In addition, a pyramid pooling module is introduced to extract richer insulator feature information by using local multi-scale feature fusion, thereby avoiding missed detection of insulators and improving detection accuracy. This solves the problem of false detection and missed detection of defective insulators in complex backgrounds. Attached Figure Description
[0034] Figure 1 This is the main flowchart of an insulator defect detection method based on an improved CenterNet network.
[0035] Figure 2 This is a diagram of the CenterNet network structure in an insulator defect detection method based on an improved CenterNet network.
[0036] Figure 3 This is a Resnet50 network structure diagram in an insulator defect detection method based on an improved CenterNet network.
[0037] Figure 4 shows the network structure of the CBAM module in an insulator defect detection method based on an improved CenterNet network.
[0038] Figure 5 This is a pyramid pooling module network structure diagram in an insulator defect detection method based on an improved CenterNet network.
[0039] Figure 6 This is a graph showing the change in the loss function of an improved algorithm in an insulator defect detection method based on an improved CenterNet network.
[0040] Figure 7 It is the mAP value of an improved algorithm in an insulator defect detection method based on an improved CenterNet network.
[0041] Figure 8 This is an example of an insulator defect detection method based on an improved CenterNet network, demonstrating the effectiveness of insulator defect detection. Figure 1 .
[0042] Figure 9 This is an example of an insulator defect detection method based on an improved CenterNet network, demonstrating the effectiveness of insulator defect detection. Figure 2 . Detailed Implementation
[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0044] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.
[0045] With the rise of intelligent power grid inspection and the rapid development of image processing and deep learning technologies, combining deep learning with UAV technology for intelligent insulator detection has become a new direction. Compared with traditional image processing-based insulator detection methods, deep learning methods based on convolutional neural networks can automatically extract image features with higher efficiency, greatly improving the efficiency and accuracy of insulator detection. Therefore, insulator detection based on deep learning methods has attracted widespread attention from relevant scholars.
[0046] like Figures 1-9 As shown, the present invention provides an insulator defect detection method based on an improved GenterNet network, which solves the technical problems in the background art. Specifically, the insulator defect detection method based on an improved CenterNet network includes the following steps:
[0047] S01: Establish an insulator defect dataset;
[0048] Drones were used to collect images of insulators on power poles. Unclear images were deleted, and the remaining images were used as an inspection dataset. The dataset was then merged with publicly available insulator datasets online. Data augmentation was performed on the images, including image rotation, random cropping, adding Gaussian noise, and changing the color temperature. The final dataset contained 6,216 images.
[0049] S02: Establish an image label library for the dataset;
[0050] The image annotation tool Labellmg was used to manually annotate the targets to be detected in each image. Different targets had different label categories. After each image was annotated, an XML file containing the label information of all targets in the image was generated.
[0051] S03: Dataset partitioning;
[0052] The 6216 dataset images were randomly divided in a 9:1 ratio using a random allocation method, with 5594 images in the training set and 622 images in the test set.
[0053] S04: Constructing an improved CenterNet network:
[0054] First, the original GenterNet backbone network, resnet18, is deepened into a resnet50 network. The resnet50 network has four residual modules, each consisting of a Conv Block and a Identity Block. Then, a Convolutional Block Attention (CBAM) module is introduced, which is embedded before the connection between the original block and the residual structure. Next, a pyramid pooling module is constructed, taking the output of the backbone network with embedded CBAM as its output. Then, the feature map output by the pyramid pooling module is subjected to three deconvolution operations. Finally, the feature map after the three deconvolution operations is input into the GenterNet network's detection head.
[0055] S05: Use the training set in the dataset to train and tune the improved CenterNet network, and finally obtain the optimal network model;
[0056] The improved CenterNet network is trained using the training set in the dataset. A freeze-and-thaw training method is adopted. Since the backbone of the model is frozen, the feature extraction network part will not change, so the memory occupied is very small. Therefore, the Batch Size (the number of parameters passed to the program for training at one time) is set to 8 in the frozen training.
[0057] During the unfrozen training phase, the backbone of the model is not frozen, but the feature extraction network changes, thus increasing the memory usage. Therefore, the batch size is set to 4 during unfrozen training.
[0058] The training was frozen for 50 epochs and unfrozen for 250 epochs, for a total of 300 epochs. The Adam optimizer was chosen for this experiment. The learning rate was set to 0.001 during both the frozen and unfrozen training phases, and finally, the parameters were fine-tuned based on the training results.
[0059] S06: Input the test set from the dataset into the trained improved CenterNet network model, and output the insulator detection results and target location information to realize the defect detection of insulators in transmission lines;
[0060] The training weights of the model are obtained through training, and the optimal weights are loaded into the improved CenterNet network model. When the insulator image is input into the network model, the detection results will use two different colored detection boxes to detect normal insulators and defective insulators. The identified target categories and the located coordinate information will be displayed on the original image.
[0061] like Figure 3 As shown, s represents the convolution stride, and the numbers in parentheses represent the length, width, and number of channels after convolution, respectively. Conv Block and Identity Block are two basic blocks in the network. The entire network is divided into 5 stages, namely Stage 1-5.
[0062] The input image first enters Stage 1, which includes a convolutional layer with a kernel size of 7×7, a stride of 2, and 64 channels, a BatchNorm normalization layer, a ReLU activation function, and a max pooling layer with a kernel size of 3×3 and a stride of 2. The final output image is 256×256×64. Stages 2-5 are all composed of Conv Blocks and Identity Blocks.
[0063] The Conv Block structure consists of two parts. The left side is the backbone, where the input undergoes three convolutions. The first two convolutions pass through a BatchNorm layer and a ReLU layer, while the final convolution passes only through a BatchNorm layer. The right side is the residual edge, where the input undergoes one convolution and passes through a BatchNorm layer. The outputs from both parts are summed and then passed through a ReLU layer to become the final output for the next stage. The Conv Block has different input and output dimensions to change the network's dimensionality.
[0064] The Identity Block's structure is also divided into two parts. The left side, like the Conv Block, is the backbone. The input undergoes three convolutions: the first two convolutions pass through a BatchNorm layer and a ReLU layer, while the final convolution only passes through a BatchNorm layer. The right side contains the residual edges. Unlike the Conv Block, the input is not convolved; instead, it is directly added to the output on the left side and passed through a ReLU layer before becoming the final output for the next stage. Because it's a direct addition, the dimensions of the Identity Block's input and output must remain consistent to deepen the network.
[0065] The first layer of each of the last four stages is a Conv Block. After passing through the Conv Block, the height and width of the feature map remain unchanged, but the number of channels is multiplied by 2 to change the network dimension. The subsequent layers are composed of identity blocks. After passing through the identity blocks, the height, width, and number of channels of the feature map remain unchanged, only the depth of the network is increased.
[0066] As shown in Figure 4, the Convolutional Block Attention Module (CBAM) represents the attention mechanism module of the convolutional module, which is a module that combines spatial and channel attention mechanisms. The output of the convolutional layer first passes through a channel attention module to obtain a weighted result, and then passes through a spatial attention module to finally obtain the weighted result.
[0067] The channel attention module is structured as follows: The input feature map is processed through global max pooling and global average pooling based on width and height, respectively, and then through an MLP. The features output from the MLP are then summed element-wise and activated by a sigmoid function to generate the final channel attention feature map. This channel attention feature map and the input feature map are then multiplied element-wise to generate the input features needed by the spatial attention module.
[0068] The spatial attention module is structured as follows: The feature map output from the Channel attention module is used as the input feature map for this module. First, a channel-based global max pooling and global average pooling are performed, and then these two results are concatenated based on the channel. Next, a convolution operation is performed to reduce the dimensionality to one channel. Then, a sigmoid function is used to generate the spatial attention feature. Finally, this feature is multiplied by the input feature of this module to obtain the final generated feature.
[0069] like Figure 5 As shown, this module employs four different pyramid scales, with pooling sizes of 1×1, 2×2, 3×3, and 6×6 for each layer. The first layer performs global pooling on the entire feature map. The following three layers divide the feature map into different sub-regions, each of which undergoes its own pooling operation. Each pooling layer yields feature maps containing features at different scales. A convolution operation with a 1×1 kernel is then performed on these feature maps to reduce their dimensionality, decreasing the number of channels to 1 / N of the original feature map, where N represents the pyramid level. Bilinear interpolation is then used to upsample the channel dimensions of the resulting feature maps to match the original input feature map. Finally, these feature maps are concatenated with the original feature map to obtain the final global features.
[0070] This module uses pooling and fusion to aggregate contextual information from different regions, thereby improving the ability to acquire global information and reducing missed and false detections during insulator testing, thus improving detection accuracy.
[0071] The above embodiments of the present invention provide an insulator defect detection method based on an improved CenterNet network. The network structure of CenterNet is improved by reducing false positives and false negatives and improving detection accuracy. An improved CenterNet network based on attention mechanism and pyramid pooling module is proposed, which can improve the detection accuracy of defective insulators in complex backgrounds.
[0072] In order for the above methods and systems to operate smoothly, the system may include more or fewer components than those described above, or combine certain components, or different components, in addition to the various modules mentioned above. For example, it may include input / output devices, network access devices, buses, processors, and memory.
[0073] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.
[0074] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0075] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.
[0076] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. An insulator defect detection method based on an improved CenterNet network, characterized in that, The method comprises the following steps: Establishing an insulator defect data set; An image label library of the insulator defect data set is established, specifically comprising: labeling all pictures in the insulator defect data set through a LabelImg tool, and arranging the labeled files and the pictures in the insulator defect data set into a data set format of PASCALVOC2007 to generate the entire insulator defect data set; Divide the entire insulator defect data set into a training set and a test set; An improved CenterNet network is constructed, specifically comprising: Deepening the original backbone network resnet18 network of the CenterNet network into a resnet50 network, and introducing a convolution block attention module CBAM; Embedding the CBAM before the original block and residual structure connection, and then constructing a pyramid pooling module; Taking the feature map output by the resnet50-CBAM as the input of the pyramid pooling module, and then performing three deconvolution operations on the feature map output after the pyramid pooling module; Inputting the feature map after the three deconvolution operations into the CenterNet network detection head; Training and parameter adjusting the improved CenterNet network using the training set in the entire insulator defect data set to generate an optimal network model; Inputting the test set in the insulator defect data set into the trained optimal network model to output insulator detection results and target position information, and realizing insulator defect detection of a power transmission line.
2. The insulator defect detection method based on the improved CenterNet network according to claim 1, characterized in that, The establishment of the insulator defect data set specifically comprises: Obtaining pictures collected by a UAV on insulators on a power pole tower; Deleting pictures with a clarity lower than a clarity threshold, and taking the remaining pictures as an inspection data set; Merging the inspection data set with a publicly disclosed insulator data set in a network, performing data enhancement processing on the pictures in the merged data set, and obtaining an insulator defect data set containing a plurality of pictures.
3. The insulator defect detection method based on the improved CenterNet network according to claim 1, characterized in that, The label categories of different target objects to be detected in all pictures in the insulator defect data set are different, and each picture generates an xml file containing all target category label information of the picture after labeling.
4. The insulator defect detection method based on the improved CenterNet network according to claim 3, characterized in that, The division of the entire insulator defect data set into a training set and a test set specifically comprises: using a random allocation method to randomly divide all pictures contained in the insulator defect data set according to a preset ratio to obtain training set pictures and test set pictures respectively.
5. The insulator defect detection method based on the improved CenterNet network according to claim 1, characterized in that, The introduction of the convolution block attention module CBAM specifically comprises: Deepening the original backbone network resnet18 network of the CenterNet network into a resnet50 network, and introducing a convolution block attention module CBAM, and embedding the CBAM before the original block and residual structure connection.
6. The insulator defect detection method based on the improved CenterNet network according to claim 5, characterized in that, The construction of the pyramid pooling module specifically comprises: Deepening the original backbone network resnet18 network of the CenterNet network into a resnet50 network, and introducing a convolution block attention module CBAM, and embedding the CBAM before the original block and residual structure connection. The feature map output by the resnet50-CBAM is taken as the input of the pyramid pooling module, which adopts four different pyramid scales, and the pooling size of each layer is 1x1, 2x2, 3x3 and 6x6, wherein the first layer is global pooling on the entire feature map, and the second to fourth layers divide the feature map into different sub-regions, and each sub-region is internally pooled, and each layer obtains a feature map containing different scale features, and then a convolution operation is performed on the obtained feature map, and the convolution kernel size is 1x1, which is to reduce the dimension of the obtained feature map, and the channel number is reduced to that of the original feature map, and the number of pyramid layers is represented by m. The channel dimension of the obtained feature map is upsampled to the same as the original input feature map by using the bilinear interpolation method, and the feature maps are concatenated with the original feature map in the channel to obtain the final global feature.
7. The insulator defect detection method based on the improved CenterNet network according to claim 6, characterized in that, When the obtained feature map is subjected to a convolution operation, the convolution kernel size is 1x1, so as to reduce the dimension of the obtained feature map, and the channel number is reduced to that of the original feature map, and the number of pyramid layers is represented by m.
8. The insulator defect detection method based on the improved CenterNet network according to claim 1, characterized in that, The training and parameter adjustment of the improved CenterNet network using the training set in the entire insulator defect data set include: The improved CenterNet network is trained using the training set in the data set, and the training method of freezing and unfreezing is adopted, the Batch Size is set to 8 in the freezing training, the Batch Size is set to 4 in the unfreezing training, the freezing training is performed for 50 Epoch, the unfreezing training is performed for 250 Epoch, and the total number of Epoch is 300, wherein the network optimizer in this experiment is selected as Adam optimizer, the learning rate in the freezing training stage is set to 0.001, and the learning rate in the unfreezing training stage is set to 0.0001, and the parameters are optimized according to the training result.
9. The insulator defect detection method based on the improved CenterNet network according to claim 1, characterized in that, The test set in the data set is input into the trained improved CenterNet network model, which includes: The training weight of the model is obtained by training, and the optimal weight is loaded into the improved CenterNet network model, and the insulator picture is input into the network model, and the detection result is detected by using two different colored detection frames to detect normal insulators and defective insulators, and the recognized target category and the located coordinate information are displayed on the original image.