Composite insulator fault detection method based on YOLOV8

Abstract

The invention discloses a composite insulator fault detection method based on YOLOV8, and relates to the technical field of graphic data reading, and the method comprises the steps: synchronously collecting an infrared image and a visible light image of a composite insulator, taking the infrared image and the visible light image as dual-mode images, carrying out the cooperative preprocessing, obtaining a registered dual-mode image, and generating a significance hot area graph; constructing an improved target detection network, and inputting the registered bimodal image into the improved target detection network to obtain a detection result of the composite insulator and a pixel-level contour of a fault area; obtaining a final overheating grade based on a physical mechanism diagnosis model, and calculating an intensity loss rate estimated value of a current fault based on a pixel-level contour of a fault area; and generating and outputting a structured diagnosis report based on the fault type, the absolute temperature data matrix, the final overheating level, the strength loss rate estimated value and the fault area pixel level contour. According to the invention, the technical problem of poor detection effect caused by incomplete composite insulator fault detection in the prior art is solved.

Description

Technical Field

[0001] This invention relates to the field of graphic data reading technology, and more specifically to a method for detecting faults in composite insulators based on YOLOV8. Background Technology

[0002] Composite insulators are critical components in power transmission systems, and their normal operation is essential for power grid safety. However, due to long-term exposure to harsh environments, composite insulators are prone to various faults, such as cracks, damage, contamination, and overheating. These faults can lead to a decline in insulation performance and even power outages. Currently, fault detection of composite insulators mainly relies on manual inspection or image-based detection methods. Manual inspection is inefficient, highly subjective, and risky in high-altitude or harsh environments, and has been gradually replaced by automated detection technologies. Image-based automated detection methods often employ a single mode, resulting in limited detection accuracy, especially in complex backgrounds or multi-scale fault conditions, where they may miss small targets or falsely detect noise. Summary of the Invention

[0003] This application provides a composite insulator fault detection method based on YOLOV8, which is used to address the technical problem that the incomplete fault detection of composite insulators in the prior art leads to poor detection results.

[0004] In view of the above problems, this application provides a fault detection method for composite insulators based on YOLOv8, the method comprising: Infrared and visible light images of composite insulators are acquired simultaneously as dual-modal images and co-processed to obtain registered dual-modal images. Then, an absolute temperature data matrix is ​​extracted from the infrared images, and a significant thermal map is generated based on the temperature data matrix. Based on the aforementioned saliency heat map, an improved target detection network is constructed, and the registered dual-modal image is input into the improved target detection network to obtain the detection results of the composite insulator and the pixel-level contour of the fault area. The detection results include the fault type. Based on the physical mechanism diagnostic model, combined with the absolute temperature data matrix and the pixel-level contour of the fault area, the severity of the current fault is quantified to obtain the final overheating level, and the intensity loss rate estimate of the current fault is calculated based on the pixel-level contour of the fault area. Based on the fault type, absolute temperature data matrix, final overheating level, strength loss rate estimate, and pixel-level contour of the fault area, a structured diagnostic report is generated and output.

[0005] One or more technical solutions provided in this application have at least the following technical effects or advantages: This application proposes a YOLOv8-based method for fault detection in composite insulators. It constructs a registration data foundation by simultaneously acquiring and coordinating the preprocessing of infrared and visible light dual-modal images. Then, an improved target detection network guided by saliency heatmaps is used to achieve accurate fault identification and pixel-level contour extraction. Furthermore, a physical mechanism-based diagnostic model is integrated to quantitatively assess fault severity, ultimately generating a comprehensive structured diagnostic report. This significantly improves the accuracy, robustness, and information completeness of composite insulator fault detection. Compared to traditional methods, the technical solution provided in this application significantly overcomes the information limitations of single-modal detection and the shortcomings of traditional network structures in multi-scale feature extraction and cross-modal fusion. It achieves integrated, automated, and intelligent diagnosis of composite insulator faults, from precise location and refined identification to in-depth quantitative assessment, effectively improving the intelligence level and operational efficiency of power equipment condition inspection. Attached Figure Description

[0006] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0007] Figure 1 This is a flowchart illustrating the composite insulator fault detection method based on YOLOV8 provided in the embodiments of this application.

[0008] Figure 2 This is a schematic diagram illustrating the process of constructing an improved target detection network in the YOLOV8-based composite insulator fault detection method provided in the embodiments of this application. Detailed Implementation

[0009] This application provides a composite insulator fault detection method based on YOLOV8, which aims to address the technical problem of incomplete fault detection in composite insulators in the prior art, resulting in poor detection performance.

[0010] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0011] It should be noted that the terms "comprising" and "having" are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to these processes, methods, products, or devices.

[0012] Examples, such as Figure 1 As shown, this application provides a fault detection method for composite insulators based on YOLOv8, wherein the method includes: S10: Simultaneously acquire infrared and visible light images of composite insulators as dual-modal images, perform collaborative preprocessing to obtain registered dual-modal images, extract absolute temperature data matrix from infrared images, and generate a significant thermal map based on the temperature data matrix.

[0013] In fault detection of composite insulators, single-type image data has inherent limitations. While visible light images can reveal surface texture and physical morphology, they cannot directly reflect temperature distribution information closely related to internal faults or electrical performance of the insulator. Infrared images, while capable of capturing temperature anomalies, lack sufficient spatial detail to accurately locate faults and identify their physical properties. Furthermore, simply processing the two types of images separately, due to differences in imaging principles and viewing angles, can lead to inaccurate spatial correspondence of information, introducing registration errors into subsequent fusion analysis.

[0014] Step S10 in the method provided in this application embodiment includes: Infrared and visible light images of the same insulator were acquired using a calibrated infrared thermal imager and a visible light camera in a synchronous triggering manner. Visible light images are processed using an adaptive histogram equalization algorithm with limited contrast to enhance texture details and suppress overall overexposure. A feature point matching and perspective transformation algorithm based on scale-invariant feature transformation is used to perform pixel-level fine registration between the filtered infrared image and the enhanced visible light image. For infrared images acquired by infrared thermal imagers, temperature calculations are performed based on the reference blackbody temperature and camera response model built into the infrared thermal imager to generate the absolute temperature data matrix. The absolute temperature data matrix is ​​then subjected to bad pixel replacement and Gaussian filtering to eliminate noise. The absolute temperature data matrix is ​​normalized and linearly mapped to the interval between 0 and 1. An algorithm based on Gaussian difference and threshold segmentation is used to extract abnormal regions from the normalized absolute temperature data matrix. The extracted abnormal regions are morphologically closed to fill the holes and smooth the boundaries, ultimately generating a salient thermal map of the same size as the original infrared image that highlights the temperature anomalies.

[0015] In this embodiment, infrared and visible light images of the composite insulator are acquired simultaneously as dual-modal images and undergo collaborative preprocessing to obtain registered dual-modal images. Then, an absolute temperature data matrix is ​​extracted from the infrared images, and a significant thermal map is generated based on the temperature data matrix.

[0016] Specifically, firstly, calibrated infrared thermal imagers, such as FLIR imagers, and visible light cameras are used to simultaneously acquire images of the same composite insulator via a synchronization triggering device, such as a hardware trigger. For example, the two devices are fixed on the same bracket to ensure overlapping fields of view, and a hardware trigger signal causes them to expose and capture images at the same time, obtaining spatiotemporally consistent infrared and visible light images, reducing registration errors caused by shooting time differences. Ultimately, an infrared image reflecting temperature distribution and a visible light image displaying texture details are obtained.

[0017] Visible light images are processed using an adaptive histogram equalization algorithm with limited contrast to enhance texture details and suppress overall overexposure. For example, the image is divided into multiple small regions, such as 8×8 blocks, and histogram equalization is performed within each region. A contrast limit is set to prevent local over-enhancement, for example, setting the contrast limit to 2.0. Then, bilinear interpolation is used to smooth the boundaries between blocks, enhancing the visibility of textures such as cracks and dirt on the insulator surface while avoiding an overall image that is too bright or too dark.

[0018] A feature point matching and perspective transformation algorithm based on scale-invariant feature transform (SIT) is employed to perform pixel-level fine registration of the filtered infrared image and the enhanced visible light image. Specifically, SIT and perspective transformation, such as using OpenCV, are used to register the preprocessed infrared and visible light images. First, SIFT feature points (keypoints) and descriptors are extracted from the two images. Then, a nearest neighbor matching algorithm, such as the FLANN algorithm, is used to find corresponding point pairs. Mismatches are removed through random consistency sampling. Finally, the perspective transformation matrix is ​​calculated based on the matching points to transform the infrared image into the coordinate system of the visible light image, thereby eliminating the spatial offset between the two modal images and ensuring pixel-level alignment of temperature data and texture details.

[0019] For infrared images acquired by an infrared thermal imager, temperature calculation is performed based on the imager's built-in reference blackbody temperature and camera response model to generate an absolute temperature data matrix. This matrix is ​​then subjected to defect replacement and Gaussian filtering to eliminate noise. Specifically, the absolute temperature value of each pixel is calculated based on the original pixel values ​​of the infrared image and the reference blackbody temperature, forming a matrix of these values. Defect replacement is then performed on the matrix; for example, obvious outliers are replaced with the average value of their surrounding 3×3 neighborhood. Gaussian filtering, such as using a Gaussian kernel with a standard deviation of 1.0 for convolution, is then applied to smooth the noise. Finally, the image data is converted into reliable temperature field information, and sensor noise is eliminated.

[0020] The absolute temperature data matrix is ​​normalized and linearly mapped to the interval between 0 and 1. Specifically, the minimum and maximum temperature values ​​in the absolute temperature matrix are obtained, and then each temperature value is normalized. The normalized temperature value = (temperature value to be normalized - minimum temperature value) ÷ (maximum temperature value - minimum temperature value).

[0021] An algorithm based on Gaussian difference and threshold segmentation is used to extract anomalous regions from the normalized absolute temperature data matrix. Specifically, firstly, Gaussian difference is applied to filter the matrix using two Gaussian filters with different standard deviations, such as a Gaussian filter with σ1=1.0 and σ2=2.0, and the difference matrix is ​​calculated to enhance the identification of edges and anomalous regions. Further, threshold segmentation is performed, setting a high-temperature threshold of 0.7 and a low-temperature threshold of 0.3 to mark regions with values ​​higher than the high-temperature threshold or lower than the low-temperature threshold as anomalous, thereby identifying significant temperature anomalies, such as overheated or undercooled areas.

[0022] The extracted anomalous regions undergo morphological closing operations to fill holes and smooth boundaries, ultimately generating a salient thermal map of the same size as the original infrared image, highlighting the temperature anomalies. For example, a circular structuring element with a radius of 5 pixels is used. The mask is first dilated to connect adjacent regions, then eroded to restore its approximate shape, thus filling small holes and smoothing boundaries to generate a continuous and complete anomalous region map, reducing noise interference. The resulting salient thermal map, the same size as the original infrared image, prominently displays the temperature anomalies and serves as a guide for feature fusion in subsequent target detection.

[0023] Through synchronous acquisition and collaborative preprocessing, a high-quality, complementary dual-modal data foundation was established for subsequent analysis. Synchronous acquisition ensured the spatiotemporal consistency of infrared and visible light images, providing a prerequisite for deep information fusion. Pixel-level registration eliminated spatial misalignment between modalities, enabling precise overlay of temperature information with texture details. The absolute temperature data matrix extracted from the infrared images provided objective, quantifiable temperature field information. The resulting salient thermal map effectively highlighted fault-related abnormal temperature regions and suppressed background interference.

[0024] S20: Combining the aforementioned saliency heat map, an improved target detection network is constructed, and the registered dual-modal image is input into the improved target detection network to obtain the detection results of the composite insulator and the pixel-level contour of the fault area. The detection results include the fault type.

[0025] Traditional target detection networks face challenges when processing multimodal images of composite insulators. First, standard network structures may not effectively process and fuse heterogeneous information from different modalities; simple feature splicing or early fusion can easily lead to information confusion and fail to fully utilize their complementarity. Second, for faults with varying shapes and scales, convolutional operations with fixed receptive fields struggle to adaptively capture key features, potentially resulting in missed detections of small targets or inaccurate boundary localization.

[0026] Step S20 in the method provided in the embodiments of this application, such as Figure 2 As shown, it includes: The YOLOV8 architecture with the last downsampling layer removed is used as the base network. The backbone network of the base network is split into two branches with non-shared parameters after the input, which receive the registered infrared image and the visible light image respectively. In the backbone network and feature pyramid network of the two branches, dynamic convolution modules are used to replace all 3×3 standard convolutional layers. The dynamic convolution modules generate convolutional kernel weights based on the input feature map through lightweight sub-networks to achieve adaptive adjustment of the receptive field. The feature pyramid contains three key fusion levels, each of which embeds a cross-modal attention fusion module. The execution logic of the cross-modal attention fusion module is constructed based on the saliency heatmap. The execution logic of the cross-modal attention fusion module includes: At any given level, the feature map of the infrared image branch is used as the query, and the feature map of the visible light image branch is used as both the key and value. Calculate the dot product of Query and Key to obtain the initial attention map, and then multiply the initial attention map element-wise with the saliency heatmap to obtain the enhanced attention map weighted by temperature information; The enhanced attention map is Softmax normalized to obtain the final fusion weight, and the Value is weighted and summed using this weight. The summation result is then channel-stitched with the infrared branch feature map of the current level. The feature map after channel concatenation is reduced in dimensionality and integrated with information through a 1×1 convolutional layer, and the final fused feature at the specified level is output. The registered historical infrared images and historical visible light images are labeled to form a training sample set. The labeling information of each sample includes at least the bounding box coordinates of the fault region, the fault type label, and the pixel-level segmentation mask of the fault region. The target detection network is trained using the training sample set until convergence. The registered infrared and visible light images are then input into the target detection network, which outputs the detection results of the composite insulator and the pixel-level contour of the fault area.

[0027] In this embodiment, an improved target detection network is constructed by combining a saliency heat map, and the registered dual-modal image is input into the improved target detection network to obtain the detection results of the composite insulator and the pixel-level contour of the fault area. The detection results include the fault type.

[0028] Specifically, firstly, the YOLOv8 architecture is used as the foundation, and the last downsampling layer in its backbone network is removed. Downsampling reduces the spatial resolution of feature maps, and some faults in composite insulators, such as minute cracks, account for a very small proportion in the image. Removing the last downsampling layer prevents the features of these minute faults from being over-compressed and lost in the deep network, thus ensuring that there are sufficiently high-resolution feature maps for accurate detection. Furthermore, the backbone network is split into two branches after the input. One branch is dedicated to receiving the registered infrared image, and the other branch is dedicated to receiving the registered visible light image. Infrared and visible light images contain different modalities of information; infrared images mainly contain temperature distribution, while visible light images mainly contain texture details. Using a dual-branch network with non-shared parameters allows each branch to independently and focusedly learn the unique features of its corresponding modality, avoiding the feature confusion and mutual interference that may occur when a single shared-parameter network processes dual-modal data, providing a more representative feature foundation for subsequent deep fusion.

[0029] Furthermore, in the backbone network and feature pyramid network of both branches, dynamic convolutional modules replace all 3×3 standard convolutional layers. These dynamic convolutional modules generate convolutional kernel weights based on the input feature map through a lightweight sub-network, enabling adaptive adjustment of the receptive field. Specifically, the core of the dynamic convolutional module is a lightweight sub-network. For example, it can consist of a global average pooling layer, a fully connected layer, and a ReLU activation function. This sub-network dynamically generates a set of convolutional kernel weight parameters based on the current input feature map, instead of using fixed convolutional kernels. This gives the convolutional operation the ability to adaptively adjust the receptive field. For composite insulator fault detection, the scale of the fault target varies greatly, ranging from minute cracks to large hot spots. Feature pyramids are a general multi-scale feature fusion technique. YOLOv8 integrates feature pyramids into its network architecture as one of the key modules for improving detection performance. Combined with the dynamic convolutional module, it can adaptively adjust the convolutional kernel parameters according to the scale and shape of the fault features in the input feature map, thereby capturing the key features of these multi-scale targets more accurately and flexibly.

[0030] The feature pyramid comprises three key fusion layers, each embedding a cross-modal attention fusion module. The execution logic of this module is based on a saliency heatmap. Specifically, the feature pyramid network contains three key feature fusion layers, each designed to handle targets of different scales, such as small, medium, and large targets. Each key fusion layer embeds a cross-modal attention fusion module, whose execution logic relies on the construction of a saliency heatmap.

[0031] Specifically, the execution logic of the cross-modal attention fusion module includes: At any given level, the feature map of the infrared image branch is used as the query, and the feature map of the visible light image branch is used as both the key and the value.

[0032] The dot product of the query and the key is calculated to obtain an initial attention map. This initial attention map is then multiplied element-wise with the saliency heatmap. Using the prior temperature anomaly information provided by the saliency heatmap, the attention map is weighted to obtain an enhanced attention map that is more focused on potential fault regions and weighted by temperature information.

[0033] The enhanced attention map is Softmax normalized to obtain the final fusion weights, and the Value is weighted and summed using these weights. The summation result is then concatenated with the infrared branch feature map of the current level.

[0034] Furthermore, the feature maps after channel concatenation are used for dimensionality reduction and information integration through 1×1 convolutional layers to output the final fused features at the specified level. This step enables the network to selectively focus on regions that exhibit abnormalities in temperature and have corresponding visible light textures when fusing bimodal information, greatly improving the targeting and effectiveness of the fusion.

[0035] The registered historical infrared and visible light images are labeled to form a training sample set. The labeling information of each sample includes at least the bounding box coordinates of the fault region, the fault type label, and the pixel-level segmentation mask of the fault region. Specifically, the bounding box coordinates of the fault region include the bounding rectangle vertex bounding box of the fault region, the fault type label may include "crack", "stain", etc., and the pixel-level segmentation mask of the fault region is used to extract the contour of the fault region.

[0036] The target detection network is trained using a training sample set until convergence. Registered infrared and visible light images are then input into the network, outputting the detection results of composite insulators and pixel-level contours of fault areas. For example, using a training sample set, a standard deep learning training process is employed, using the Adam optimizer with an initial learning rate of 0.001, along with a cosine annealing strategy and a multi-task loss function including classification loss, bounding box regression loss, and segmentation loss. The network parameters are iteratively updated using backpropagation until the model's performance on the validation set stabilizes. For instance, if the total loss value on the validation set decreases by less than a preset threshold (e.g., 0.001) for 10 consecutive cycles, the model is considered to have converged in terms of loss metrics. The registered infrared and visible light images are then input into the target detection network, outputting the detection results of composite insulators and pixel-level contours of fault areas.

[0037] By constructing an improved target detection network incorporating salient heatmaps, accurate identification and refined segmentation of composite insulator faults were achieved. The improved network architecture, through a dual-branch design, allows the network to learn temperature features in the infrared mode and texture features in the visible light mode separately, avoiding intermodal interference. The introduction of dynamic convolutional modules endows the network with the ability to adaptively adjust its receptive field, enabling it to more effectively capture multi-scale fault features ranging from minute to macroscopic. Most importantly, the use of salient heatmaps to guide the cross-modal attention fusion module allows the network to prioritize regions related to temperature anomalies when fusing features, significantly improving the targeting and effectiveness of feature fusion.

[0038] S30: Based on the physical mechanism diagnostic model, combined with the absolute temperature data matrix and the pixel-level contour of the fault area, the severity of the current fault is quantified to obtain the final overheating level, and the intensity loss rate estimate of the current fault is calculated based on the pixel-level contour of the fault area.

[0039] Many existing fault detection methods for composite insulators often stop at fault identification and location, lacking in-depth quantitative assessment of fault severity. However, for operation and maintenance decisions, simply knowing that a fault exists is insufficient; it is also necessary to understand its severity to determine the urgency and priority of maintenance. For example, the severity of overheating faults is not determined by a single temperature point, but is related to multiple factors such as relative temperature rise and temperature distribution uniformity; the degree to which physical damage such as cracks and breaks weaken the mechanical strength of insulators is also closely related to their geometry, size, and orientation. Without a quantitative model based on physical mechanisms, it is difficult to achieve accurate and objective severity assessment based solely on image appearance or a single temperature threshold, resulting in a lack of data support for operation and maintenance strategies.

[0040] Step S30 in the method provided in this application embodiment includes: A multi-level temperature difference threshold is preset to correspond to the fault type. The multi-level temperature difference threshold divides the fault into at least three overheating levels: concern level, severe level, and critical level. The pixel-level contour of the fault area output by the target detection network is accurately mapped onto the absolute temperature data matrix, and the temperature values ​​of all pixels within the pixel-level contour mask area are extracted to form a set of temperatures to be analyzed. The final overheating level of the fault is obtained based on the set of temperatures to be analyzed; The final overheating level of the fault is obtained based on the set of temperatures to be analyzed, including: Calculate the statistical characteristics of the temperature set to be analyzed, including the maximum value, mean value, and standard deviation; Select a known normal area without defects on the composite insulator body and calculate its average temperature as the reference temperature. Calculate the differences between the maximum temperature and the average temperature of the temperature set to be analyzed and the reference temperature to obtain the maximum temperature difference and the average temperature difference; The maximum temperature difference and the average temperature difference are compared with the multi-level temperature difference thresholds to determine their respective overheating levels; finally, the one with the highest overheating level is output as the final overheating level of the fault. Calculate the minimum bounding rectangle of the pixel-level contour of the fault region, take the long side of the minimum bounding rectangle as the main direction of the defect, and calculate the maximum cross-sectional width of the defect along the direction perpendicular to the long side. The contour compactness is obtained by calculating the ratio of the total pixel area of ​​the pixel-level contour of the fault region to the minimum convex hull area. The maximum cross-sectional width and contour compactness are used as key geometric parameters and input into a pre-trained random forest regression model to output the current fault intensity loss rate estimate. The random forest regression model is obtained by analyzing the nonlinear relationship between contour compactness, maximum cross-sectional width and laboratory-measured strength loss rate in historical samples.

[0041] In this embodiment, multiple temperature difference thresholds are pre-set for different types of faults. For example, based on industry standards and historical experience data, multiple temperature difference thresholds are set, with a temperature difference of 5 degrees Celsius classified as "attention level", 10 degrees Celsius as "serious level", and 15 degrees Celsius as "critical level", to initially classify the severity of the fault.

[0042] Furthermore, the pixel-level contours of the fault region output by the target detection network are precisely mapped onto the absolute temperature data matrix, and the temperature values ​​of all pixels within the pixel-level contour mask area are extracted to form a set of temperatures to be analyzed. For example, using the contour mask as an index, the temperature values ​​of all pixels covered by the mask are extracted from the temperature matrix and integrated to obtain the set of temperatures to be analyzed. For instance, if the mask of a certain fault region covers 150 pixels, then a set containing 150 temperature values ​​is obtained.

[0043] Based on the set of temperatures to be analyzed, the final overheating level of the fault is obtained.

[0044] Specifically, the statistical characteristics of the temperature set to be analyzed are calculated, and the maximum value, arithmetic mean and standard deviation of the temperature set to be analyzed are obtained.

[0045] Furthermore, a known, defect-free, normal region on the composite insulator body is selected, and its average temperature is calculated and used as a reference temperature. For example, a known defect-free normal region is selected, and the average temperature of all pixels within that region is calculated and used as the reference temperature.

[0046] Calculate the differences between the maximum temperature, average temperature, and reference temperature of the temperature set to be analyzed, to obtain the maximum temperature difference and average temperature difference. Maximum temperature difference = maximum temperature - reference temperature, average temperature difference = average temperature - reference temperature.

[0047] The maximum and average temperature differences are compared with multi-level temperature difference thresholds to determine their respective overheating levels. The highest overheating level among the two is then output as the final overheating level for the fault. For example, a maximum temperature difference of 12 degrees Celsius corresponds to "Severe," while an average temperature difference of 8 degrees Celsius corresponds to "Concern." Therefore, the highest level, "Severe," is selected as the final overheating level for the fault.

[0048] Calculate the minimum bounding rectangle of the pixel-level contour of the fault region. Take the long side of the minimum bounding rectangle as the main direction of the defect. Calculate the maximum projected width of the defect contour in the direction perpendicular to the main direction, and take it as the maximum cross-sectional width of the defect.

[0049] The contour compactness is obtained by calculating the ratio of the total pixel area to the minimum convex hull area of ​​the pixel-level contour of the fault region. Contour compactness = total pixel area ÷ minimum convex hull area. The closer the contour compactness is to 1, the more compact and full the contour is; the smaller the contour compactness is, the narrower and more irregular the contour is.

[0050] Obtain a historical strength dataset, in which each sample contains: the geometric parameters of the fault profile, namely the maximum cross-sectional width and profile compactness, and the strength loss rate of the faulty insulator as measured by mechanical tests.

[0051] A random forest regression model is trained based on a historical intensity dataset. The random forest regression model is derived by analyzing the nonlinear relationship between contour compactness, maximum cross-sectional width, and laboratory-measured intensity loss rate in historical samples. For example, the random forest regression model consists of 100 decision trees. A training subset is constructed using a bagging method for bootstrapping. Each tree randomly selects two features from all features to find the optimal split point when splitting at a node. The mean of the predictions from all trees is the final output of the model. The model's performance is quantified by calculating the mean squared error between the model's predictions and the actual laboratory-measured values ​​on out-of-bag samples. Training of the random forest regression model is considered complete when the mean squared error is less than a set threshold, such as 0.001. Maximum cross-sectional width and contour compactness are used as key geometric parameters and input into the random forest regression model to output an estimated intensity loss rate for the current fault.

[0052] By introducing a physical mechanism diagnostic model, crucial quantitative evidence is provided for condition-based maintenance. By mapping precise pixel-level contours onto an absolute temperature data matrix, the actual temperature data of the fault area can be accurately extracted. Based on preset multi-level temperature difference thresholds and statistical characteristics, a comprehensive judgment is made, resulting in a more scientific and reliable final overheating level, avoiding subjective misjudgments. Simultaneously, utilizing the geometric characteristics of the fault contour and a random forest regression model trained on historical data, the estimated strength loss rate caused by the fault can be calculated. The overheating level and strength loss rate together constitute the core parameters for assessing the impact of faults on the electrical and mechanical properties of insulators, greatly enhancing the practical value and guiding significance of the diagnostic results.

[0053] S40: Based on the fault type, absolute temperature data matrix, final overheating level, strength loss rate estimate, and pixel-level contour of the fault area, generate and output a structured diagnostic report.

[0054] After fault detection and severity quantification are completed, the resulting information is multi-dimensional and fragmented, including fault type, location, precise profile, temperature data, overheating level, and intensity loss rate. If this information is presented in an unstructured manner, it will be difficult for operations and maintenance personnel to quickly and comprehensively understand the diagnostic conclusions, and it will also be difficult to support automatic reading and integration.

[0055] Step S40 in the method provided in this application embodiment includes: Structured diagnostic reports are output simultaneously in both JSON and PDF formats; The JSON format report contains structured data and includes a list of faults. Each fault entry in the list records: bounding box coordinates, fault type, overheating level, strength loss rate estimate, and the maximum and average temperatures extracted from the absolute temperature data matrix. The PDF report is a visual document containing a diagnostic results map overlaid on the original visible light image, showing pixel-level outlines of all faults, category labels, and severity levels.

[0056] In this embodiment, a structured diagnostic report is generated and output based on the fault type, absolute temperature data matrix, final overheating level, strength loss rate estimate, and pixel-level contour of the fault area.

[0057] Specifically, the structured diagnostic report is output simultaneously in both JSON and PDF formats.

[0058] The JSON-formatted report contains structured data, including a list of faults. Each fault entry in the list records: bounding box coordinates, fault type, overheating level, estimated intensity loss rate, and the maximum and average temperatures extracted from the absolute temperature data matrix. For example, each element in the fault list is a dictionary representing a detected fault entry. Each fault entry dictionary contains the following key-value pairs: the key "bounding box coordinates" corresponds to a list of four numbers from the output of the object detection network; the key "fault type" corresponds to a string such as "crack" or "overheating"; the key "overheating level" corresponds to a string such as "severity"; the key "estimated intensity loss rate" corresponds to a floating-point number from the prediction output of the random forest model; and the keys "maximum temperature" and "average temperature" are obtained by extracting all temperature values ​​from the absolute temperature data matrix mapped to the fault contour region and then calculating their maximum and arithmetic mean. Finally, the json.dump() function is used to serialize this dictionary structure and write it to a text file with the .json extension, resulting in a machine-readable JSON format diagnostic report that fully records the quantification and location information of each fault.

[0059] The PDF report is a visual document containing a diagnostic result image overlaid on the original visible light image, showing pixel-level outlines, category labels, and severity levels of all faults. Specifically, the original visible light image is used as the base image. For each detected fault, a contour boundary line is drawn at the corresponding location based on its pixel-level contour coordinates. Simultaneously, a text label is added near the contour, consisting of the fault type and the final overheating level, such as "Crack - Level of Concern." Finally, a PDF document object is created using the reportlab library, and the visual image with all fault outlines and labels is inserted into the PDF page. The document is then saved, resulting in a PDF diagnostic report that is easy for humans to review intuitively, clearly showing the location, shape, and severity level of all faults on the original image.

[0060] By generating and outputting structured diagnostic reports, effective integration and efficient delivery of diagnostic results are achieved. The structured reports systematically organize and encapsulate key information. JSON-formatted reports provide machine-readable structured data, facilitating integration into digital operation and maintenance platforms for automatic alarms, historical trend analysis, or asset status tracking, meeting the needs of automated and intelligent operation and maintenance. PDF-formatted reports, on the other hand, present information in a visual format, such as overlaying fault outlines and labels onto the original image, enabling maintenance personnel to intuitively and quickly grasp the overall distribution, specific location, type, and severity of faults, facilitating on-site verification and decision-making. This dual-format output method simultaneously meets the dual needs of automated machine processing and intuitive human interpretation, ensuring that diagnostic information accurately and efficiently serves the maintenance and management of power equipment, forming a seamless closed loop from detection and analysis to result application.

[0061] In summary, the embodiments of this application have at least the following technical effects: This application proposes a fault detection method for composite insulators based on YOLOv8. By simultaneously acquiring and coordinating the preprocessing of infrared and visible light dual-modal images to construct a registration data foundation, an improved target detection network guided by saliency heatmaps is used to achieve accurate fault identification and pixel-level contour extraction. Furthermore, a diagnostic model based on physical mechanisms is integrated to quantitatively assess the severity of the fault, and finally, a comprehensive structured diagnostic report is generated, which significantly improves the accuracy, robustness, and information integrity of fault detection for composite insulators. Specifically, this method effectively preserves and adaptively learns unique features and multi-scale fault morphologies of different modalities through a dual-branch non-shared parameter network structure and dynamic convolutional modules. It utilizes a cross-modal attention fusion mechanism driven by temperature saliency to achieve deep complementarity and precise alignment of infrared thermal features and visible light texture features, enabling stable fault location and accurate delineation even under complex backgrounds and variable operating conditions. Simultaneously, the solution combines data-driven detection results with a physical mechanism model, not only identifying fault types but also providing in-depth diagnostic information for operation and maintenance decisions—directly applicable to assessing fault risk and prioritizing maintenance—by quantifying overheating levels and intensity loss rates, thus enhancing the practical value of the detection results. Compared to traditional methods, the technical solution provided in this application significantly overcomes the information limitations of single-modal detection and the shortcomings of traditional network structures in multi-scale feature extraction and cross-modal fusion. It achieves integrated, automated, and intelligent diagnosis of composite insulator faults, from precise location and refined identification to in-depth quantitative assessment, effectively improving the intelligence level and operational efficiency of power equipment condition inspection.

[0062] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0063] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0064] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and modifications fall within the scope of this application and its equivalents, this application intends to include such modifications and modifications.

Claims

1. A method for fault detection of composite insulators based on YOLOv8, characterized in that, The method includes: Infrared and visible light images of composite insulators are acquired simultaneously as dual-modal images and co-processed to obtain registered dual-modal images. Then, an absolute temperature data matrix is ​​extracted from the infrared images, and a significant thermal map is generated based on the temperature data matrix. Based on the aforementioned saliency heat map, an improved target detection network is constructed, and the registered dual-modal image is input into the improved target detection network to obtain the detection results of the composite insulator and the pixel-level contour of the fault area. The detection results include the fault type. Based on the physical mechanism diagnostic model, combined with the absolute temperature data matrix and the pixel-level contour of the fault area, the severity of the current fault is quantified to obtain the final overheating level, and the intensity loss rate estimate of the current fault is calculated based on the pixel-level contour of the fault area. Based on the fault type, absolute temperature data matrix, final overheating level, strength loss rate estimate, and pixel-level contour of the fault area, a structured diagnostic report is generated and output.

2. The method according to claim 1, characterized in that, The simultaneous acquisition of infrared and visible light images of the composite insulator, followed by collaborative preprocessing, yields a registered dual-modal image. An absolute temperature data matrix is ​​extracted from the infrared image, including: Infrared and visible light images of the same insulator were acquired using a calibrated infrared thermal imager and a visible light camera in a synchronous triggering manner. Visible light images are processed using an adaptive histogram equalization algorithm with limited contrast to enhance texture details and suppress overall overexposure. A feature point matching and perspective transformation algorithm based on scale-invariant feature transformation is used to perform pixel-level fine registration between the filtered infrared image and the enhanced visible light image. The infrared images acquired by the infrared thermal imager are used to calculate the temperature based on the reference blackbody temperature and camera response model built into the infrared thermal imager, generating the absolute temperature data matrix. The absolute temperature data matrix is ​​then subjected to bad pixel replacement and Gaussian filtering to eliminate noise.

3. The method according to claim 1, characterized in that, Generate a significant heat map based on the temperature data matrix, including: The absolute temperature data matrix is ​​normalized and linearly mapped to the interval between 0 and 1. An algorithm based on Gaussian difference and threshold segmentation is used to extract abnormal regions from the normalized absolute temperature data matrix. The extracted abnormal regions are morphologically closed to fill the holes and smooth the boundaries, ultimately generating a salient thermal map of the same size as the original infrared image that highlights the temperature anomalies.

4. The method according to claim 1, characterized in that, Based on the aforementioned saliency heatmap, an improved target detection network is constructed, including: The YOLOV8 architecture with the last downsampling layer removed is used as the base network. The backbone network of the base network is split into two branches with non-shared parameters after the input, which receive the registered infrared image and the visible light image respectively. In the backbone network and feature pyramid network of the two branches, dynamic convolution modules are used to replace all 3×3 standard convolutional layers. The dynamic convolution modules generate convolutional kernel weights based on the input feature map through lightweight sub-networks to achieve adaptive adjustment of the receptive field. The feature pyramid contains three key fusion levels, each of which embeds a cross-modal attention fusion module. The execution logic of the cross-modal attention fusion module is constructed based on the saliency heatmap.

5. The method according to claim 4, characterized in that, The execution logic of the cross-modal attention fusion module includes: At any given level, the feature map of the infrared image branch is used as the query, and the feature map of the visible light image branch is used as both the key and value. Calculate the dot product of Query and Key to obtain the initial attention map, and then multiply the initial attention map element-wise with the saliency heatmap to obtain the enhanced attention map weighted by temperature information; The enhanced attention map is Softmax normalized to obtain the final fusion weight, and the Value is weighted and summed using this weight. The summation result is then channel-stitched with the infrared branch feature map of the current level. The feature map after channel splicing is reduced in dimensionality and integrated with information through a 1×1 convolutional layer, and the final fused feature at the specified level is output.

6. The method according to claim 5, characterized in that, The registered dual-modal image is input into the improved target detection network to obtain the detection results of the composite insulator and the pixel-level contour of the fault region, including: The registered historical infrared images and historical visible light images are labeled to form a training sample set. The labeling information of each sample includes at least the bounding box coordinates of the fault region, the fault type label, and the pixel-level segmentation mask of the fault region. The target detection network is trained using the training sample set until convergence. The registered infrared and visible light images are then input into the target detection network, which outputs the detection results of the composite insulator and the pixel-level contour of the fault area.

7. The method according to claim 1, characterized in that, Based on the physical mechanism diagnostic model, and combining the absolute temperature data matrix with the pixel-level contour of the fault area, the severity of the current fault is quantified, including: A multi-level temperature difference threshold is preset to correspond to the fault type. The multi-level temperature difference threshold divides the fault into at least three overheating levels: concern level, severe level, and critical level. The pixel-level contour of the fault area output by the target detection network is accurately mapped onto the absolute temperature data matrix, and the temperature values ​​of all pixels within the pixel-level contour mask area are extracted to form a set of temperatures to be analyzed. The final overheating level of the fault is obtained based on the set of temperatures to be analyzed.

8. The method according to claim 7, characterized in that, The final overheating level of the fault is obtained based on the set of temperatures to be analyzed, including: Calculate the statistical characteristics of the temperature set to be analyzed, including the maximum value, mean value, and standard deviation; Select a known normal area without defects on the composite insulator body and calculate its average temperature as the reference temperature. Calculate the differences between the maximum temperature and the average temperature of the temperature set to be analyzed and the reference temperature to obtain the maximum temperature difference and the average temperature difference; The maximum temperature difference and the average temperature difference are compared with the multi-level temperature difference thresholds to determine their respective overheating levels; finally, the one with the highest overheating level is output as the final overheating level of the fault.

9. The method according to claim 1, characterized in that, Calculate the current fault intensity loss rate estimate, including: Calculate the minimum bounding rectangle of the pixel-level contour of the fault region, take the long side of the minimum bounding rectangle as the main direction of the defect, and calculate the maximum cross-sectional width of the defect along the direction perpendicular to the long side. The contour compactness is obtained by calculating the ratio of the total pixel area of ​​the pixel-level contour of the fault region to the minimum convex hull area. The maximum cross-sectional width and contour compactness are used as key geometric parameters and input into a pre-trained random forest regression model to output the current fault intensity loss rate estimate. The random forest regression model is obtained by analyzing the nonlinear relationship between contour compactness, maximum cross-sectional width and laboratory-measured strength loss rate in historical samples.

10. The method according to claim 1, characterized in that, Based on the fault type, absolute temperature data matrix, final overheating level, strength loss rate estimate, and pixel-level contour of the fault area, a structured diagnostic report is generated and output, including: Structured diagnostic reports are output simultaneously in both JSON and PDF formats; The JSON format report contains structured data and includes a list of faults. Each fault entry in the list records: bounding box coordinates, fault type, overheating level, strength loss rate estimate, and the maximum and average temperatures extracted from the absolute temperature data matrix. The PDF report is a visual document containing a diagnostic results map overlaid on the original visible light image, showing pixel-level outlines of all faults, category labels, and severity levels.

Images