Improved YOLOv9 transformer substation insulator and defect detection method and system thereof

Abstract

The invention discloses an improved YOLOv9 transformer substation insulator and a defect detection method and system thereof, and the method comprises the steps: obtaining and preprocessing an insulator and defect image of a transformer substation, and obtaining a preprocessed image; inputting the preprocessed image into an improved YOLOv9 model for training to obtain an MPDIOU loss function for optimization, and obtaining a transformer substation insulator and defect detection model; and inputting transformer substation insulator and defect images acquired in real time into the transformer substation insulator and defect detection model, and outputting a transformer substation insulator and defect detection result. A diversity branch block DBB is embedded in a RepNCSPELAN, the DBB enhances the representation capability of a single convolution and enriches feature spaces, including convolution sequences, multi-scale convolution and average pooling, by combining diversity branches of different scales and complexity, and the model ensures the running speed and the resource efficiency while improving the performance.

Description

Technical Field

[0001] The present invention relates to the field of electric power computer vision technology, and in particular to an improved YOLOv9 substation insulator and defect detection method and system thereof. Background Art

[0002] With the rapid development and expansion of the power system, the safe and stable operation of substations has become particularly important. As a key component of power transmission and transformation in substations, insulators play an important role in preventing current from returning to the ground and supporting the conductors during the power transmission process. Their quality and status directly affect the safe operation of the entire power grid. Since insulators are exposed to harsh environments such as high-voltage electric fields and strong light for a long time, they are prone to failure, which in turn threatens the reliable operation of electrical equipment. Therefore, in order to ensure the reliability of the power system, it is very important to efficiently and accurately detect the status of insulators and promptly discover hidden dangers of insulator failures for the long-term safe operation of the power grid.

[0003] Traditional insulator detection methods mainly rely on manual inspection and offline detection. However, these methods have problems such as low efficiency, strong subjectivity, and limited coverage. In addition, manual power inspection is a time-consuming, labor-intensive, and dangerous task. In recent years, with the rapid development of computer vision technology, image recognition technology based on deep learning has been widely used in the field of industrial inspection. Traditional target detection algorithms are difficult to find the optimal solution to balance detection speed and detection accuracy. Some single-stage target detection algorithms have a high detection speed, but the detection accuracy decreases significantly, which is not conducive to the accurate identification of insulator defects; although the accuracy of other algorithms meets the requirements, the model complexity is high and the operation speed is slow.

[0004] How to improve the model's reasoning efficiency while maintaining high accuracy is an urgent problem to be solved in the field of insulator and defect detection based on deep learning target detection algorithms.

[0005] In the field of object detection, the YOLO (You Only Look Once) series of algorithms have become the focus of research due to their high speed and high efficiency. In 2015, Joseph Redmon, Santosh Divvala, Ross Girshick and others proposed a one-stage object detection network. The network model processes images in real time at a speed of 45 frames per second. A smaller version of the network can process images at up to 155 frames per second. Due to its high speed and the special method it uses, the authors named it: You Only Look Once, or YOLO. This is the pioneering work of the YOLO series, and the original name of the paper is "You Only Look Once: Unified, Real-Time Object Detection".

[0006] The main feature of the YOLO series of models is that they are fast, which also limits the application scenarios of the YOLO model. In many scenarios with high real-time requirements, the YOLO series of models is obviously a very good choice.

[0007] As of May 2023, many versions of YOLO have been born, including YOLO V1, YOLO9000, YOLO V3, YOLO V4, Scaled YOLO V4, PP-YOLO, YOLO V5, YOLO V6, YOLOX, YOLOR, PP-YOLO V2, DAMOYOLO, PP-YOLOE, YOLO V7, YOLO V6, YOLO V8, and YOLO-NAS.

[0008] More than a year after YOLOv8 was officially released in January 2023, YOLOv9 was released in February 2024. This time, YOLOv9 was jointly developed by Academia Sinica, Taipei University of Technology and other institutions in Taiwan, and the related paper "Learning What You Want to Learn Using Programmable Gradient Information" has been released.

[0009] Today's deep learning methods focus on how to design the most appropriate objective function so that the model's prediction results can be closest to the actual situation. At the same time, an appropriate architecture must be designed to help obtain enough information for prediction. However, existing methods ignore the fact that a lot of information will be lost when the input data undergoes layer-by-layer feature extraction and spatial transformation. Therefore, YOLOv9 deeply studies the important problem of data loss when data is transmitted through deep networks, namely information bottlenecks and reversible functions. Summary of the invention

[0010] In order to solve the above technical problems, the present invention provides a substation insulator and defect detection method and system based on an improved YOLOv9, the method comprising:

[0011] Step S1, obtaining the insulator and defect images of the substation and preprocessing them to obtain a preprocessed image;

[0012] Step S2: input the preprocessed image into the improved YOLOv9 model for training, and then use the MPDIoU loss function for optimization to obtain an optimized substation insulator and defect detection model;

[0013] Step S3: input the substation insulator and defect images collected in real time into the optimized substation insulator and defect detection model, and output the substation insulator and insulator defect detection results.

[0014] Optionally, in step S1, the preprocessing process specifically includes:

[0015] Perform image enhancement on the acquired insulator and defect images of the substation to obtain an enhanced image;

[0016] Use labellmg to annotate the enhanced image to obtain a preprocessed image.

[0017] Optionally, in step S2, the improved YOLOv9 model specifically includes:

[0018] Taking YOLOv9 as the initial framework, the diversity branch block DBB is embedded in the core module RepNCSPELAN of YOLOv9 to extract features and obtain feature maps;

[0019] The downsampling in YOLOv9 is replaced with Haar wavelet downsampling to reduce the spatial resolution of the feature map.

[0020] Optionally, in step S2, the process of using the MPDIoU loss function for optimization specifically includes:

[0021] The MPDIoU loss function is used to make the bounding box predicted by the substation insulator and defect detection model close to the real bounding box:

[0022] Predicted bounding box:

[0023]

[0024] Among them, B prd is the set of predicted bounding boxes, x prd is the x-axis coordinate set of the upper left and lower right corners of the predicted bounding box, and y prd is the y-axis coordinate set of the upper left and lower right corners of the predicted bounding box, w prd is the predicted bounding box width, h prd To predict the bounding box height;

[0025] Predicted bounding box after using MPDIoU loss function:

[0026] B gt =[x gt ,y gt ,w gt ,h gt ] T

[0027] L=minΘ L(B gt ,B prd |Θ)

[0028] Among them, B gt is the set of true bounding boxes, and Θ is the parameter of the regression depth model, x gt is the x-axis coordinate set of the upper left corner and lower right corner of the real bounding box, y gt is the y-axis coordinate set of the upper left corner and lower right corner of the real bounding box, w gt is the actual bounding box width, h gt is the actual bounding box height, and the typical form of L is e n -norm.

[0029] The present invention also discloses an improved YOLOv9 substation insulator and defect detection system thereof, the system comprising: a data acquisition module, a model building module and a target detection module;

[0030] The data acquisition module is used to acquire the insulator and defect images of the substation and perform preprocessing to obtain preprocessed images;

[0031] The model building module is used to input the preprocessed image into the improved YOLOv9 model for training, and then optimize it using the MPDIoU loss function to obtain an optimized substation insulator and defect detection model;

[0032] The target detection module is used to input the substation insulator and defect images collected in real time into the optimized substation insulator and defect detection model, and output the substation insulator and insulator defect detection results.

[0033] Optionally, in the data acquisition module, the preprocessing process specifically includes:

[0034] Perform image enhancement on the acquired insulator and defect images of the substation to obtain an enhanced image;

[0035] Use labellmg to annotate the enhanced image to obtain a preprocessed image.

[0036] Optionally, in the model building module, the improved YOLOv9 model specifically includes:

[0037] Taking YOLOv9 as the initial framework, the diversity branch block DBB is embedded in the core module RepNCSPELAN of YOLOv9 to extract features and obtain feature maps;

[0038] The downsampling in YOLOv9 is replaced with Haar wavelet downsampling to reduce the spatial resolution of the feature map.

[0039] Optionally, in the model building module, the process of optimizing using the MPDIoU loss function specifically includes:

[0040] The MPDIoU loss function is used to make the bounding box predicted by the substation insulator and defect detection model close to the real bounding box:

[0041] Predicted bounding box:

[0042]

[0043] Among them, B prd is the set of predicted bounding boxes, x prd is the x-axis coordinate set of the upper left and lower right corners of the predicted bounding box, and y prd is the y-axis coordinate set of the upper left and lower right corners of the predicted bounding box, w prd is the predicted bounding box width, h prd To predict the bounding box height;

[0044] Predicted bounding box after using MPDIoU loss function:

[0045] B gt =[x gt ,y gt ,w gt ,h gt ] T

[0046] L=min Θ L(B gt ,B prd |Θ)

[0047] Among them, B gt is the set of true bounding boxes, and Θ is the parameter of the regression depth model, x gt is the x-axis coordinate set of the upper left corner and lower right corner of the real bounding box, y gt is the y-axis coordinate set of the upper left corner and lower right corner of the real bounding box, w gt is the actual bounding box width, h gt is the actual bounding box height, and the typical form of L is e n -norm.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] The present invention improves the backbone network core module of the YOLOv9 benchmark model and embeds a diversity branch block DBB in RepNCSPELAN. DBB enhances the representation ability of a single convolution by combining diverse branches of different scales and complexities, enriches the feature space, including convolution sequence, multi-scale convolution and average pooling. The model not only improves performance, but also ensures running speed and resource efficiency. Haar wavelet downsampling HWD is used to replace traditional downsampling in the network. Haar wavelet has the advantages of easy calculation and implementation. The application of Haar wavelet transform can reduce the spatial resolution of the feature map while retaining as much information as possible, effectively reducing information uncertainty. MPDIoU is used as the loss function of the model to train the model. MPDIoU is calculated based on the minimum point distance of the horizontal rectangle. By directly calculating the key point distance between the predicted box and the real box, the average accuracy of the model is improved, and the deviation of the overlapping area, the center point distance, and the width and height can be comprehensively considered. The performance of the trained model is verified on the substation insulator equipment and defect image data set. Compared with the existing insulator and defect detection method, the trained model effectively improves the insulator and defect detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0051] Figure 1 It is a flowchart of a substation insulator and defect detection algorithm based on improved YOLOv9 proposed by the present invention;

[0052] Figure 2 It is a Diverse Branch Block structure and workflow diagram of an embodiment of the present invention;

[0053] Figure 3 This is an example of the operation of the DBB in training and reasoning according to an embodiment of the present invention;

[0054] Figure 4 Schematic diagram of the structure of a Haar wavelet downsampling (HWD) module according to an embodiment of the present invention;

[0055] Figure 5 A diagram of a Haar wavelet transform method according to an embodiment of the present invention;

[0056] Figure 6 Two different bounding box regression result diagrams of the embodiments of the present invention;

[0057] Figure 7 This is a network structure diagram of the improved YOLOv9 insulator and defect detection algorithm model of an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The purpose of the present invention is to provide a substation insulator and defect detection algorithm based on improved YOLOv9, so as to solve the problems of slow speed and large volume of traditional detection network, inability to balance speed and accuracy, and many false detections and missed detections, so as to achieve the purpose of improving detection accuracy under the premise of meeting real-time detection.

[0059] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0060] Embodiment 1

[0061] The present invention provides a substation insulator and defect detection method thereof based on an improved YOLOv9, such as Figure 1 As shown, specifically including:

[0062] Step S1: Acquire the insulator and defect images of the substation and perform preprocessing to obtain a preprocessed image.

[0063] In order to ensure the quality of the constructed substation insulator equipment and defect image data set, a series of image data processing measures were taken. First, the collected image data of insulators and their defects were strictly screened to select high-quality image data with high clarity and rich information. These image data are the basis for subsequent processing and analysis, so ensuring their quality is crucial.

[0064] Next, in order to further improve the quality and diversity of the image data, image enhancement technology was used. Specifically, the selected high-quality images were rotated, cropped, and other operations were performed. These operations not only increase the perspective and morphological changes of the images, but also expand the size of the data set without changing the essential characteristics of the images. Through such processing, more diverse shooting conditions and angles can be simulated, thereby improving the generalization ability of the data set.

[0065] After the image data was processed, the professional labeling tool labelImg was used to perform detailed labeling of the insulator and its defect data in accordance with the standard format requirements of the COCO dataset. This process includes accurately marking the position and size of the insulator, as well as the specific type and location of the defect. Through such labeling, a structured and standardized substation insulator equipment and defect image dataset was constructed, which not only contains rich image information, but also contains detailed labeling information, providing a solid data foundation for subsequent image recognition and defect detection.

[0066] Step S2: After the preprocessed image is input into the improved YOLOv9 model for training, the MPDIoU loss function is used for optimization to obtain an optimized substation insulator and defect detection model.

[0067] During the implementation of this embodiment,

[0068] In view of the fact that most detection network models have many parameters and high demand for hardware computing resources, the detection speed is slow and cannot meet the requirements of real-time detection; at the same time, the large model size also limits its application on operating platforms with relatively limited hardware resources. In addition, traditional neural networks have certain shortcomings in detection accuracy. Therefore, this method introduces the detection network YOLOv9 with excellent performance to improve the accuracy and detection speed of substation insulator equipment and its defective image target detection. Among them, in the step S2, the YOLOv9 network is selected as the basic framework of the detection network, which specifically includes:

[0069] In order to fully verify the superior performance of this research method in detecting substation insulator equipment and its defect images, the performance of several currently popular single-stage and two-stage target detection algorithms was compared and analyzed on the constructed dataset, including but not limited to Faster R-CNN, SSD and multiple versions of the YOLO series. By comprehensively considering multiple indicators such as the model's detection speed, accuracy, recall rate, and model complexity, the YOLOv9 model stood out from many candidate models due to its excellent real-time performance and high detection accuracy, becoming the basis for further research and development, and was used as the benchmark model for research.

[0070] In order to adapt to the particularity of insulator and defect detection tasks, it is necessary to make certain improvements to the baseline model. Therefore, the improved YOLOv9 model specifically includes:

[0071] Taking YOLOv9 as the initial framework, the diversity branch block DBB is embedded in the core module RepNCSPELAN of YOLOv9 to extract features and obtain feature maps;

[0072] The downsampling in YOLOv9 is replaced with Haar wavelet downsampling to reduce the spatial resolution of the feature map.

[0073] Diverse Branch Block DBB is embedded in the core module RepNCSPELAN of YOLOv9. Diverse BranchBlock (DBB) is a general convolutional neural network (ConvNet) building block that can improve performance without increasing any inference time cost. DBB enhances the representation ability of a single convolution by combining diverse branches of different scales and complexities to enrich the feature space, including convolution sequences, multi-scale convolutions, and average pooling. After training, DBB can be equivalently converted to a single convolution layer for deployment. Unlike the advances in novel ConvNet architectures, DBB complicates the microstructure while maintaining the macro architecture during training, so it can be used as a plug-and-play replacement for regular convolution layers in any architecture. In this way, the model can be trained to a higher level of performance and then converted back to the original inference time structure for inference.

[0074] like Figure 2-Figure 3 As shown in the figure, during training, DBB consists of convolutional layers of different sizes and average pooling layers, which are arranged in parallel in a complex way and finally merge the outputs. After training, these complex structures are converted into a single convolutional layer for the inference phase of the model to maintain efficiency during inference. This conversion allows DBB to increase the complexity of the microstructure during training while keeping the macro architecture unchanged.

[0075] The concept of separation of training and inference refers to using a complex DBB structure in the training phase of the model, and converting it to a simplified convolutional structure in the model inference phase. This design allows the model to leverage the diversity of DBB to enhance feature extraction and learning capabilities during training, while maintaining high efficiency by reducing the amount of computation during inference. In this way, the model maintains high performance while also ensuring running speed and resource efficiency.

[0076] Replace the traditional downsampling in the network, including:

[0077] Downsampling operations such as max pooling or strided convolution are commonly used in convolutional neural networks (CNNs) to aggregate local features, expand the receptive field, and minimize computational overhead. However, for semantic segmentation tasks, pooling features over a local neighborhood may lead to the loss of important spatial information, which is beneficial for pixel-level prediction. To address this issue, a Haar wavelet downsampling (HWD) module is introduced. Haar wavelet is one of the simplest wavelet forms and has the advantages of being easy to compute and implement. The HWD module can be easily integrated into CNNs to enhance the performance of semantic segmentation models. The core idea of ​​HWD is to apply Haar wavelet transform to reduce the spatial resolution of feature maps while retaining as much information as possible. Comprehensive experiments show that the HWD module can effectively improve the segmentation performance of different modality image datasets with different CNN architectures, and can effectively reduce information uncertainty compared to traditional downsampling methods.

[0078] like Figure 4 As shown in the figure, the HWD module consists of two modules: (1) the loss less feature encoding block and (2) the feature representation learning block. The lossless feature encoding module is responsible for converting features and reducing spatial resolution. This is achieved through Haar wavelet transform, which can effectively reduce the resolution of feature maps while retaining all information. The representation learning module consists of standard convolutional layers, batch normalization, and ReLU activation layers, which are used to extract discriminative features.

[0079] Figure 5 Depicted is the decomposition process of an image with a resolution of H×W using Haar wavelet transform. Here, H0 and H1 represent low-pass and high-pass decomposition filters, respectively. These filters are used to extract approximate information and high-frequency information from the image, respectively. The symbol ↓2 indicates the downsampling applied to the approximate and detailed components. The Haar wavelet transform generates an approximate component (A) and detail components (high frequencies) in three directions: horizontal (H), vertical (V), and diagonal (D). It should be noted that the resolution of each component is reduced to (H / 2)×(W / 2), while the number of channels of the feature map is increased by four times. In other words, the Haar wavelet transform can encode part of the information of the spatial dimension into the channel dimension without losing any information. Therefore, the subsequent layers of the DCNN can extract representative features from the transformed components, thereby reducing the spatial resolution.

[0080] The MPDIoU loss function is used for optimization, and the optimized substation insulator and defect detection model includes:

[0081] In the process of object detection and instance segmentation, the traditional bounding box regression (BBR) loss function is difficult to optimize the case where the predicted box and the actual box have the same aspect ratio but different specific sizes. To overcome this problem, the present invention uses a better bounding box similarity measurement method - MPDloU (Minimum Point Distance Intersection over Union).

[0082] MPDIoU is used as the loss function of the model to train the model. The optimal model obtained through training is used to detect and identify insulators and defects, and the model effect is evaluated, including:

[0083] MPDIoU is calculated based on the minimum point distance of the horizontal rectangle. It improves the average accuracy of the model by directly calculating the key point distance between the predicted box and the real box, and can comprehensively consider the overlapping area, center point distance, and width and height deviations.

[0084] Figure 6 Two different bounding box regression results are shown, where the green box represents the actual bounding box and the red box represents the predicted bounding box. In these two cases, the loss values ​​calculated by traditional loss functions (such as G1oU, DloU, CoU, and EloU) are the same:

[0085] However, the loss value calculated using the MPDIoU method is different (Left), (Right). This shows that traditional methods may not be able to distinguish different prediction results in certain specific cases, while MPDIoU can more accurately reflect the difference between the predicted box and the true box. This highlights the advantage of MPDIoU in dealing with bounding box regression problems, especially when distinguishing bounding boxes with the same aspect ratio but different sizes or positions. MPDloU provides a more accurate loss measurement method by directly calculating the key point distance between the predicted box and the true box.

[0086] During the training phase, the bounding box predicted by the model is made close to its true bounding box by minimizing the loss function. Specifically, each predicted bounding box:

[0087]

[0088] Among them, B prd is the set of predicted bounding boxes, x prd is the x-axis coordinate set of the upper left and lower right corners of the predicted bounding box, and y prd is the y-axis coordinate set of the upper left and lower right corners of the predicted bounding box, w prd is the predicted bounding box width, h prdTo predict the bounding box height;

[0089] The ground-truth bounding box is approximated by minimizing the following loss function:

[0090] B gt =[x gt ,y gt ,w gt ,h gt ] T

[0091] L=min Θ L(B gt ,B prd |Θ)

[0092] Among them, B gt is the set of true bounding boxes, and Θ is the parameter of the regression depth model, x gt is the x-axis coordinate set of the upper left corner and lower right corner of the real bounding box, y gt is the y-axis coordinate set of the upper left corner and lower right corner of the real bounding box, w gt is the actual bounding box width, h gt is the actual bounding box height, and the typical form of L is e n -norm.

[0093] The trained model uses a variety of evaluation indicators, including precision, recall, and average precision (mAP). mAP is the most commonly used evaluation indicator in target detection, which measures the average detection accuracy under different thresholds. This embodiment uses mAP@0.5 to measure the accuracy of detection. The performance of the trained model was verified on the substation insulator equipment and defect image dataset. The verification results show that this method can effectively improve the comprehensive performance of insulator equipment and defect detection.

[0094] The specific process of the algorithm is as follows Figure 7 , the Slice layer does not perform any operations and is a placeholder layer. In this way, when parse_model, ch[n] can represent the output channel of the nth layer. The upper left corner is the input, which represents the initial input data received by the model. The backbone network includes various convolutional layers and downsampling layers, which are used to extract low-level features. Among them, RepNCSPELAN4 is a combined network of Rep+CSP+ELAN. Rep optimizes calculations, CSP enriches gradients, and ELAN performs efficient feature aggregation. The modules in the neck network are used to upsample feature maps and perform linear transformations and feature fusion operations on feature channels. After multiple ups and downsampling of feature maps, features of different scales are combined into subsequent layers to achieve richer feature expressions. The output stage is on the right. After different output layers pass through multiple convolutional layers, the outputs of each feature scale are spliced ​​together for detection tasks.

[0095] As shown in Table 1, the advantages of the algorithm model of the present invention can be illustrated by comparing SSD, FasterR-CNN, and other models in the YOLO series (YOLOv5n, YOLOv8n). Table 1 is a comparison between the algorithm model of the present invention and the above models. It can be seen that the algorithm model of the present invention performs well in both mAP score and accuracy indicators.

[0096] Table 1

[0097] Model Precision / % Recall / % mAP@0.5 / % mAP@0.5: 0.95 / % FPS SSD 92.3 59.7 76.2 47.9 36.1 Faster-CNN 91.4 88.6 85.2 54.6 13.7 YOLOv5n 91.7 83.5 83.6 52.1 62.1 YOLOv8n 91.0 91.3 92.6 61.2 74.9 YOLOv9 94.3 93.6 94.6 70.1 81.4 Ours 95.4 94.5 96.8 72.6 71.8

[0098] Based on the latest YOLOv9 model, this paper conducts in-depth research and improvement on the insulator defect detection task. By embedding the diversity branch block DBB, using the downsampling HWD of Haar wavelet to replace the traditional downsampling, and using MPDIoU as the loss function of the model, an efficient and robust insulator defect detection model YOLOv9-DHM is designed. The experimental results on the insulator defect detection dataset show that the proposed model has good adaptability and robustness in the real insulator defect detection task.

[0099] Step S3: input the substation insulator and defect images collected in real time into the optimized substation insulator and defect detection model, and output the substation insulator and insulator defect detection results.

[0100] Embodiment 2

[0101] The present invention also provides an improved YOLOv9 substation insulator and defect detection system thereof, the system comprising: a data acquisition module, a model building module and a target detection module;

[0102] The data acquisition module is used to acquire the insulator and defect images of the substation and perform preprocessing to obtain preprocessed images.

[0103] In order to ensure the quality of the constructed substation insulator equipment and defect image data set, a series of image data processing measures were taken. First, the collected image data of insulators and their defects were strictly screened to select high-quality image data with high clarity and rich information. These image data are the basis for subsequent processing and analysis, so ensuring their quality is crucial.

[0104] Next, in order to further improve the quality and diversity of the image data, image enhancement technology was used. Specifically, the selected high-quality images were rotated, cropped, and other operations were performed. These operations not only increase the perspective and morphological changes of the images, but also expand the size of the data set without changing the essential characteristics of the images. Through such processing, more diverse shooting conditions and angles can be simulated, thereby improving the generalization ability of the data set.

[0105] After the image data was processed, the professional labeling tool labelImg was used to perform detailed labeling of the insulator and its defect data in accordance with the standard format requirements of the COCO dataset. This process includes accurately marking the position and size of the insulator, as well as the specific type and location of the defect. Through such labeling, a structured and standardized substation insulator equipment and defect image dataset was constructed, which not only contains rich image information, but also contains detailed labeling information, providing a solid data foundation for subsequent image recognition and defect detection.

[0106] The model building module is used to input the preprocessed image into the improved YOLOv9 model for training, and then use the MPDIoU loss function for optimization to obtain an optimized substation insulator and defect detection model.

[0107] During the implementation of this embodiment, since most detection network models have many parameters and high demand for hardware computing resources, the detection speed is slow and cannot meet the requirements of real-time detection; at the same time, the large model size also limits its application on operating platforms with relatively limited hardware resources. In addition, traditional neural networks have certain shortcomings in detection accuracy. Therefore, this method introduces the detection network YOLOv9 with excellent performance to improve the accuracy and detection speed of substation insulator equipment and its defective image target detection. Among them, in step S2, the YOLOv9 network is selected as the basic framework of the detection network, which specifically includes:

[0108] In order to fully verify the superior performance of this research method in detecting substation insulator equipment and its defect images, the performance of several currently popular single-stage and two-stage target detection algorithms was compared and analyzed on the constructed dataset, including but not limited to Faster R-CNN, SSD and multiple versions of the YOLO series. By comprehensively considering multiple indicators such as the model's detection speed, accuracy, recall rate, and model complexity, the YOLOv9 model stood out from many candidate models due to its excellent real-time performance and high detection accuracy, becoming the basis for further research and development, and was used as the benchmark model for research.

[0109] In order to adapt to the particularity of insulator and defect detection tasks, the baseline model needs to be improved to a certain extent.

[0110] Therefore, the improved YOLOv9 model specifically includes:

[0111] Taking YOLOv9 as the initial framework, the diversity branch block DBB is embedded in the core module RepNCSPELAN of YOLOv9 to extract features and obtain feature maps;

[0112] The downsampling in YOLOv9 is replaced with Haar wavelet downsampling to reduce the spatial resolution of the feature map.

[0113] Diverse Branch Block DBB is embedded in the core module RepNCSPELAN of YOLOv9. Diverse BranchBlock (DBB) is a general convolutional neural network (ConvNet) building block that can improve performance without increasing any inference time cost. DBB enhances the representation ability of a single convolution by combining diverse branches of different scales and complexities to enrich the feature space, including convolution sequences, multi-scale convolutions, and average pooling. After training, DBB can be equivalently converted to a single convolution layer for deployment. Unlike the advances in novel ConvNet architectures, DBB complicates the microstructure while maintaining the macro architecture during training, so it can be used as a plug-and-play replacement for regular convolution layers in any architecture. In this way, the model can be trained to a higher level of performance and then converted back to the original inference time structure for inference.

[0114] During training, DBB consists of convolutional layers of different sizes and average pooling layers, which are arranged in parallel in a complex way and finally merge the outputs. After training, these complex structures are converted into a single convolutional layer for the inference phase of the model to maintain efficiency during inference. This conversion allows DBB to increase the complexity of the microstructure during training while keeping the macro architecture unchanged.

[0115] The concept of separation of training and inference refers to using a complex DBB structure in the training phase of the model, and converting it to a simplified convolutional structure in the model inference phase. This design allows the model to leverage the diversity of DBB to enhance feature extraction and learning capabilities during training, while maintaining high efficiency by reducing the amount of computation during inference. In this way, the model maintains high performance while also ensuring running speed and resource efficiency.

[0116] Replace the traditional downsampling in the network, including:

[0117] Downsampling operations such as max pooling or strided convolution are commonly used in convolutional neural networks (CNNs) to aggregate local features, expand the receptive field, and minimize computational overhead. However, for semantic segmentation tasks, pooling features over a local neighborhood may lead to the loss of important spatial information, which is beneficial for pixel-level prediction. To address this issue, a Haar wavelet downsampling (HWD) module is introduced. Haar wavelet is one of the simplest wavelet forms and has the advantages of being easy to compute and implement. The HWD module can be easily integrated into CNNs to enhance the performance of semantic segmentation models. The core idea of ​​HWD is to apply Haar wavelet transform to reduce the spatial resolution of feature maps while retaining as much information as possible. Comprehensive experiments show that the HWD module can effectively improve the segmentation performance of different modality image datasets with different CNN architectures, and can effectively reduce information uncertainty compared to traditional downsampling methods.

[0118] like Figure 4 As shown in the figure, the HWD module consists of two modules: (1) the loss less feature encoding block and (2) the feature representation learning block. The lossless feature encoding module is responsible for converting features and reducing spatial resolution. This is achieved through Haar wavelet transform, which can effectively reduce the resolution of feature maps while retaining all information. The representation learning module consists of standard convolutional layers, batch normalization, and ReLU activation layers, which are used to extract discriminative features.

[0119] Figure 5 Depicted is the decomposition process of an image with a resolution of H×W using Haar wavelet transform. Here, H0 and H1 represent low-pass and high-pass decomposition filters, respectively. These filters are used to extract approximate information and high-frequency information from the image, respectively. The symbol ↓2 indicates the downsampling applied to the approximate and detailed components. The Haar wavelet transform generates an approximate component (A) and detail components (high frequencies) in three directions: horizontal (H), vertical (V), and diagonal (D). It should be noted that the resolution of each component is reduced to (H / 2)×(W / 2), while the number of channels of the feature map is increased by four times. In other words, the Haar wavelet transform can encode part of the information of the spatial dimension into the channel dimension without losing any information. Therefore, the subsequent layers of the DCNN can extract representative features from the transformed components, thereby reducing the spatial resolution.

[0120] The MPDIoU loss function is used for optimization, and the optimized substation insulator and defect detection model includes:

[0121] In the process of object detection and instance segmentation, the traditional bounding box regression (BBR) loss function is difficult to optimize the case where the predicted box and the actual box have the same aspect ratio but different specific sizes. To overcome this problem, the present invention uses a better bounding box similarity measurement method - MPDloU (Minimum Point Distance Intersection over Union).

[0122] MPDIoU is used as the loss function of the model to train the model. The optimal model obtained through training is used to detect and identify insulators and defects, and the model effect is evaluated, including:

[0123] MPDIoU is calculated based on the minimum point distance of the horizontal rectangle. It improves the average accuracy of the model by directly calculating the key point distance between the predicted box and the real box, and can comprehensively consider the overlapping area, center point distance, and width and height deviations.

[0124] Figure 6 Two different bounding box regression results are shown, where Figure 6 (a) is the regression result of different bounding boxes using traditional methods. Figure 6 (b) is a graph showing the regression results of different bounding boxes using the method used in the present invention. The green box represents the actual bounding box, while the red box represents the predicted bounding box. In both cases, the loss values ​​calculated by traditional loss functions (such as G1oU, DloU, CoU, and EloU) are the same. However, the loss value calculated using the MPDIoU method is different (Left), (Right). This shows that traditional methods may not be able to distinguish different prediction results in certain specific cases, while MPDIoU can more accurately reflect the difference between the predicted box and the true box. This highlights the advantage of MPDIoU in dealing with bounding box regression problems, especially when distinguishing bounding boxes with the same aspect ratio but different sizes or positions. MPDloU provides a more accurate loss measurement method by directly calculating the key point distance between the predicted box and the true box.

[0125] During the training phase, the bounding box predicted by the model is made close to its true bounding box by minimizing the loss function. Specifically, each predicted bounding box:

[0126]

[0127] The ground-truth bounding box is approximated by minimizing the following loss function:

[0128] B gt =[x gt ,y gt,w gt ,h gt ] T

[0129] L=min Θ L(B gt ,B prd |Θ)

[0130] Among them, B gt is the set of true bounding boxes, and Θ is the parameter of the regression deep model.

[0131] The trained model uses a variety of evaluation indicators, including precision, recall, and average precision (mAP). mAP is the most commonly used evaluation indicator in target detection, which measures the average detection accuracy under different thresholds. This embodiment uses mAP@0.5 to measure the accuracy of detection. The performance of the trained model was verified on the substation insulator equipment and defect image dataset. The verification results show that this method can effectively improve the comprehensive performance of insulator equipment and defect detection.

[0132] As shown in Table 2, the advantages of the algorithm model of the present invention can be illustrated by comparing SSD, FasterR-CNN, and other models in the YOLO series (YOLOv5n, YOLOv8n). Table 2 is a comparison between the algorithm model of the present invention and the above models. It can be seen that the algorithm model of the present invention performs well in both mAP score and accuracy indicators.

[0133] Table 2

[0134]

[0135] Based on the latest YOLOv9 model, this paper conducts in-depth research and improvement on the insulator defect detection task. By embedding the diversity branch block DBB, using the downsampling HWD of Haar wavelet to replace the traditional downsampling, and using MPDIoU as the loss function of the model, an efficient and robust insulator defect detection model YOLOv9-DHM is designed. The experimental results on the insulator defect detection dataset show that the proposed model has good adaptability and robustness in the real insulator defect detection task.

[0136] The target detection module is used to input the substation insulator and defect images collected in real time into the optimized substation insulator and defect detection model, and output the substation insulator and insulator defect detection results.

[0137] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. An improved YOLOv9 substation insulator and defect detection method thereof, characterized in that: Methods include: Step S1, obtaining the insulator and defect images of the substation and preprocessing them to obtain a preprocessed image; Step S2: input the preprocessed image into the improved YOLOv9 model for training, and then use the MPDIoU loss function for optimization to obtain an optimized substation insulator and defect detection model; Step S3: input the substation insulator and defect images collected in real time into the optimized substation insulator and defect detection model, and output the substation insulator and insulator defect detection results.

2. According to the improved YOLOv9 of claim 1, the substation insulator and defect detection method thereof is characterized in that: In step S1, the preprocessing process specifically includes: Perform image enhancement on the acquired insulator and defect images of the substation to obtain an enhanced image; Use labellmg to annotate the enhanced image to obtain a preprocessed image.

3. According to the improved YOLOv9 of claim 1, the substation insulator and defect detection method thereof is characterized in that: In step S2, the improved YOLOv9 model specifically includes: Taking YOLOv9 as the initial framework, the diversity branch block DBB is embedded in the core module RepNCSPELAN of YOLOv9 to extract features and obtain feature maps; The downsampling in YOLOv9 is replaced with Haar wavelet downsampling to reduce the spatial resolution of the feature map.

4. The substation insulator and defect detection method thereof according to the improved YOLOv9 of claim 2 is characterized in that: In step S2, the process of optimizing using the MPDIoU loss function specifically includes: The MPDIoU loss function is used to make the bounding box predicted by the substation insulator and defect detection model close to the real bounding box: Predicted bounding box: Among them, B prd is the set of predicted bounding boxes, x prd is the x-axis coordinate set of the upper left and lower right corners of the predicted bounding box, and y prd is the y-axis coordinate set of the upper left and lower right corners of the predicted bounding box, w prd is the predicted bounding box width, h prd To predict the bounding box height; Predicted bounding box after using MPDIoU loss function: B gt =[x gt ,y gt ,w gt ,h gt ] T L=min Θ L(B gt ,B prd |I) Among them, B gt is the set of true bounding boxes, and Θ is the parameter of the regression depth model, x gt is the x-axis coordinate set of the upper left corner and lower right corner of the real bounding box, y gt is the y-axis coordinate set of the upper left corner and lower right corner of the real bounding box, w gt is the actual bounding box width, h gt is the actual bounding box height, and L is typically in the form of e n -norm.

5. An improved YOLOv9 substation insulator and defect detection system thereof, the defect detection system is used to implement the substation insulator and defect detection method thereof according to any one of claims 1 to 4, characterized in that: The system includes: a data acquisition module, a model building module and a target detection module; The data acquisition module is used to acquire the insulator and defect images of the substation and perform preprocessing to obtain preprocessed images; The model building module is used to input the preprocessed image into the improved YOLOv9 model for training, and then optimize it using the MPDIoU loss function to obtain an optimized substation insulator and defect detection model; The target detection module is used to input the substation insulator and defect images collected in real time into the optimized substation insulator and defect detection model, and output the substation insulator and insulator defect detection results.

6. The substation insulator and defect detection system thereof according to the improved YOLOv9 of claim 5 is characterized in that: In the data acquisition module, the preprocessing process specifically includes: Perform image enhancement on the acquired insulator and defect images of the substation to obtain an enhanced image; Use labellmg to annotate the enhanced image to obtain a preprocessed image.

7. The substation insulator and defect detection system thereof according to the improved YOLOv9 of claim 5 is characterized in that: In the model building module, the improved YOLOv9 model specifically includes: Taking YOLOv9 as the initial framework, the diversity branch block DBB is embedded in the core module RepNCSPELAN of YOLOv9 to extract features and obtain feature maps; The downsampling in YOLOv9 is replaced with Haar wavelet downsampling to reduce the spatial resolution of the feature map.

8. The substation insulator and defect detection system thereof according to the improved YOLOv9 of claim 5 is characterized in that: In the model building module, the process of optimizing using the MPDIoU loss function specifically includes: The MPDIoU loss function is used to make the bounding box predicted by the substation insulator and defect detection model close to the real bounding box: Predicted bounding box: Among them, B prd is the set of predicted bounding boxes, x prd is the x-axis coordinate set of the upper left and lower right corners of the predicted bounding box, and y prd is the y-axis coordinate set of the upper left and lower right corners of the predicted bounding box, w prd is the predicted bounding box width, h prd To predict the bounding box height; Predicted bounding box after using MPDIoU loss function: B gt =[x gt ,y gt ,w gt ,h gt ] T L=min Θ L(B gt ,B prd |I) Among them, B gt is the set of true bounding boxes, and Θ is the parameter of the regression depth model, x gt is the x-axis coordinate set of the upper left corner and lower right corner of the real bounding box, y gt is the y-axis coordinate set of the upper left corner and lower right corner of the real bounding box, w gt is the actual bounding box width, h gt is the actual bounding box height, and the typical form of L is e n -norm.

Images