A power transmission line insulator fault detection method and system based on semi-supervised ensemble learning and edge computing

Abstract

This invention proposes a method and system for fault detection of power transmission line insulators based on semi-supervised ensemble learning and edge computing, comprising: Step S1: Constructing a hierarchical imbalanced insulator image dataset; constructing a high imbalance dataset in conjunction with actual power inspection scenarios; Step S2: Training a lightweight two-stage deep learning model on the host side; including training a lightweight two-stage model that meets the requirements for edge deployment on the development board; Step S3: Designing a semi-supervised ensemble learning fault diagnosis framework based on Fourier feature enhancement, achieving dual utilization of labels and data distribution through distance threshold collaborative decision-making; Step S4: Model conversion and lightweight deployment on edge devices, including implementing model inference on the development board; Step S5: Integrated fault detection and real-time response at the edge, including realizing an integrated process from image acquisition to target detection to fault diagnosis to alarm output.

Description

Technical Field

[0001] This invention proposes a method and system for detecting faults in power transmission line insulators based on semi-supervised ensemble learning and edge computing, which relates to the field of intelligent inspection and fault diagnosis of power equipment. Background Technology

[0002] Insulators on power transmission lines are critical components ensuring the safe and stable operation of the power grid. Faults in these insulators (such as damage, flashover, and contamination) that are not detected in time can easily lead to major accidents such as line tripping, large-scale power outages, and even fires. Traditional manual inspection methods are inefficient, costly, and pose safety risks. In recent years, image-based intelligent recognition technology using drones or inspection robots has become a research hotspot. However, this technology faces two major challenges in practical applications: First, insulator fault samples account for a very small percentage of the massive number of inspection images, resulting in a severe imbalance in training data, making it difficult for deep learning models to guarantee the accuracy and recall of identifying minority fault samples. Second, existing high-precision models are typically computationally complex and have a large number of parameters, making them difficult to deploy directly on edge computing devices (such as OrangePi AIpro) with limited computing power, memory, and power consumption, thus failing to meet the real-time and on-site independent decision-making requirements of power inspection.

[0003] An existing method utilizes convolutional neural networks to automatically extract features and classify them from inspection images, effectively improving recognition accuracy. However, this method relies on a large amount of labeled data for supervised learning. When faced with real-world scenarios where insulator fault samples are scarce and data is imbalanced, the model performance drops sharply. Furthermore, it does not consider the lightweight deployment of the model on edge devices, does not introduce feature enhancement and semi-supervised learning strategies, and cannot utilize massive amounts of unlabeled inspection data.

[0004] An existing method, combining a sliding window with an improved YOLOv5 network, enhances the detection rate of small targets in high-resolution images, showing potential application value in scenarios such as drone aerial photography and industrial quality inspection. However, this method only focuses on target localization and does not address fine-grained classification or state discrimination of the detected targets (such as identifying specific fault types like insulator damage, contamination, and flashover). Furthermore, it relies heavily on sliding window inference, resulting in significant computational overhead and making it difficult to meet the real-time requirements of practical deployments. Additionally, despite the introduction of attention mechanisms and shallow feature maps, it fails to effectively address fundamental data challenges such as sparse small target samples and class imbalance, leaving its detection accuracy and robustness for small defect regions in real-world complex scenarios insufficient. Summary of the Invention

[0005] In view of this, and to fill the gaps and deficiencies in existing technologies, this invention proposes a method and system for fault detection of transmission line insulators based on semi-supervised ensemble learning and edge computing. This invention provides a method for real-time target detection and fault diagnosis of transmission line insulators on resource-constrained edge devices. It integrates deep learning, unsupervised clustering, and feature enhancement techniques, resolving the contradiction between imbalanced insulator fault data and the real-time performance and accuracy requirements of edge device deployment.

[0006] This invention proposes a method and system for detecting faults in power transmission line insulators based on semi-supervised ensemble learning and edge computing, comprising the following:

[0007] According to a first aspect of the present invention, a method for detecting faults in transmission line insulators based on semi-supervised ensemble learning and edge computing is characterized by comprising the following:

[0008] Step S1: Construct a hierarchical insulator image imbalance dataset; including using a hierarchical annotation strategy and combining it with the actual scenario of power inspection to construct a high imbalance dataset;

[0009] Step S2: Train a lightweight two-stage deep learning model on the host side; this includes training a lightweight two-stage model that meets the requirements for edge deployment on the development board, wherein all models adopt a lightweight network design;

[0010] Step S3: Design a semi-supervised ensemble learning fault diagnosis framework based on Fourier feature enhancement. This includes proposing a semi-supervised ensemble learning framework that integrates a supervised CNN classifier and an unsupervised Mini-Batch K-means clustering algorithm to address the data imbalance problem caused by the scarcity of fault samples. Fourier transform feature enhancement is introduced to improve the separability of fault features, and distance threshold collaborative decision-making is used to achieve dual utilization of labels and data distribution.

[0011] Step S4: Model conversion and lightweight deployment on edge devices, including implementing model inference on the development board, completing model format conversion, inference engine adaptation and system deployment;

[0012] Step S5: Integrated fault detection and real-time response at the edge, including enabling the system to run independently in a closed loop on the development board after deployment, and realizing an integrated process from image acquisition to target detection to fault diagnosis to alarm output.

[0013] Further, step S1 includes the following:

[0014] Step S11: The first stage of coarse annotation includes using rectangular boxes to annotate all complete insulator targets in the inspection images, without distinguishing their health status. The annotation category is uniformly set to insulator, which is used to train a robust insulator target detector to achieve accurate localization and cropping of insulators in complex backgrounds.

[0015] Step S12: The second stage of fine annotation includes fine pixel-level or rectangular box annotation only on the images of insulators containing faults, targeting specific fault areas for training the fault classifier; this includes labeling fault areas as damaged, flashover ablation areas as flashover, and contamination accumulation areas as contamination.

[0016] Step S13: Dataset construction, including collecting real images of transmission line insulators, and using data augmentation techniques to construct an imbalanced dataset with a significant disparity in the number of healthy insulators and various types of faulty insulators, simulating the data distribution characteristics in real inspection scenarios. Data augmentation techniques include image mirroring, rotation, and brightness perturbation.

[0017] Further, step S2 includes the following:

[0018] Step S21: Training the insulator target detection model, including using the lightweight YOLOv8s as the basic framework based on the first stage coarsely labeled dataset, scaling the input images to the same pixel size, setting the initial learning rate, batch size, and number of training epochs, enabling Mosaic data augmentation and non-maximum suppression optimization, and training to obtain a target detection model that can quickly locate insulators in complex backgrounds, ensuring that there are no missed detections and few false detections of insulator targets.

[0019] Step S22: Training the fault area classification model, including using a lightweight convolutional neural network as the fault classifier based on the second-stage fine-annotated dataset. The input is the single insulator image cropped in step S1, and the output includes the probability distribution of categories such as healthy, damaged, flashover, or contaminated. During training, pre-trained weights are loaded for transfer learning, and the cross-entropy loss function is set to ensure the generalization ability of the classifier on small sample fault data.

[0020] Step S23: Lightweight constraints, including converting the insulator target detection model and fault area classification model into a format that meets the requirements of the development board and ensuring that the model size is ≤100MB, so that the two models meet the memory and computing power limitations of the development board.

[0021] Further, step S3 includes the following:

[0022] Step S31: Feature extraction and Fourier transform enhancement, including:

[0023] Step S311: For the input fault region image, extract a high-dimensional feature vector X using the lightweight convolutional neural network trained in step S2 as a fault classifier; where...

[0024] ;

[0025] Where D is the feature dimension;

[0026] Step S312: Perform a discrete Fourier transform on the eigenvector X to decompose it into its real part. and the virtual part The calculation formula is:

[0027] ;

[0028] ;

[0029] ;

[0030] Step S313: Concatenate the real and imaginary parts with the original feature vector X to form the enhanced fused feature vector. This enriches data representation, improves the separability between fault features and normal features, and solves the problem of unclear fault features in small samples.

[0031] Furthermore, step S3 also includes the following:

[0032] Step S32: Training the Mini-Batch K-means clustering model, including the following:

[0033] Step S321: Using fused feature vectors As input, train a Mini-Batch K-means clustering algorithm, where the number of clusters k is the same as the number of fault categories;

[0034] Step S322: Using Euclidean distance as the similarity metric, calculate the distance from the sample to the cluster center, and update the cluster center by minimizing the sum of squared errors to obtain the cluster centers C1, C2, C3, and C4 for each category of insulator; corresponding to healthy, damaged, flashover, and contaminated, respectively.

[0035] Furthermore, step S3 also includes the following:

[0036] Step S33: Employ a distance threshold collaborative decision-making mechanism, which utilizes the distribution information of unlabeled data to compensate for the insufficient training of supervised lightweight convolutional neural networks on minority class fault samples, thereby enhancing the model's robustness to minority class faults. The distance threshold collaborative decision-making mechanism includes the following:

[0037] Step S331: For a fault region of an insulator to be diagnosed, fuse the feature vector. The framework yields two results simultaneously: the probabilistic predicted label y from the lightweight convolutional neural network classifier. CNN The majority vote label y in Mini-Batch K-means clustering Kmeans The specific decision-making rules are as follows:

[0038] If y CNN =y Kmeans If the result is positive, then the result will be adopted directly as the final fault diagnosis label.

[0039] If y CNN ≠y Kmeans Then calculate the distance d from the sample to the K-means cluster center. Kmeans The distance d to the cluster center of the predicted category by the lightweight convolutional neural network CNN ;

[0040] in ;

[0041] in ;

[0042] Step S332: Set an empirical threshold M, if This indicates that the sample is close to the center of the CNN's predicted class in the feature space, indicating confidence in the CNN's supervised discriminative ability and acceptance of its prediction result y. CNN Otherwise, it indicates that the sample is significantly close to the K-means cluster center in the feature space, and the K-means clustering result y is adopted. Kmeans .

[0043] Further, step S4 includes the following:

[0044] Step S41: Model format conversion, including converting the YOLOv8s object detection model trained in step S2 and the lightweight convolutional neural network classifier in step S3 into ONNX.

[0045] Step S42: Edge device environment configuration, including setting up the inference environment on the development board, installing the ONNX Runtime lightweight inference engine, OpenCV image processing library, GPIO control library, configuring the Python 3.9 runtime environment, and ensuring compatibility between model inference, image acquisition and peripheral control;

[0046] Step S43: System deployment, which includes deploying the converted ONNX format model file, semi-supervised ensemble learning decision logic code, Mini-Batch K-means cluster center parameter file, and USB camera driver to the development board. All code is optimized for lightweighting to avoid redundant calculations and ensure inference speed at the edge.

[0047] Further, step S5 includes the following:

[0048] Step S51: Image Acquisition: The high-definition industrial camera connected to the development board's USB interface captures images of the power transmission line inspection scene in real time. The image resolution is 1920×1080 and the frame rate is ≥20fps, which meets the requirements for real-time inspection.

[0049] Step S52: First-stage insulator target detection: The YOLOv8s-ONNX model is called to perform inference on the acquired scene images, quickly and accurately locate all insulator targets from the complex background, output the bounding box coordinates of the insulator targets, crop out individual insulator images based on the coordinates, and scale the cropped image resolution to a uniform resolution to prepare for subsequent fault diagnosis; the complex background includes the sky, trees, and iron towers.

[0050] Step S53: Two-stage insulator fault diagnosis: For each cropped single insulator image, feature extraction, Fourier transform enhancement, CNN classification, and K-means clustering are performed sequentially. Through the distance threshold collaborative decision-making mechanism described in step S3, its working status is finally determined. The diagnosis reasoning time for a single image is ≤1.5s. The working status includes healthy, damaged, flashover, and contaminated.

[0051] Step S54: Real-time Alarms and Data Upload

[0052] Local alarm: Based on the diagnostic results, the external LED indicator and buzzer are controlled through the GPIO pins of the development board to provide real-time audible and visual alarms. The buzzer will sound continuously when a fault is detected until manual confirmation is received.

[0053] Remote upload: Fault alarm information and fault images are compressed and uploaded to the cloud power grid monitoring platform through the Wi-Fi / Ethernet interface of the development board, realizing remote synchronization of fault information and facilitating timely handling by operation and maintenance personnel.

[0054] According to a second aspect of the present invention, a fault detection system for transmission line insulators based on semi-supervised ensemble learning and edge computing includes an electronic device, wherein the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements a fault detection method for transmission line insulators based on semi-supervised ensemble learning and edge computing as described in any one of the present invention.

[0055] According to a third aspect of the present invention, a fault detection system for transmission line insulators based on semi-supervised ensemble learning and edge computing includes a computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements a fault detection method for transmission line insulators based on semi-supervised ensemble learning and edge computing as described in any one of the present invention.

[0056] The present invention has the following advantages:

[0057] This invention addresses the challenges of data imbalance and edge deployment in power transmission line insulator inspection by integrating semi-supervised ensemble learning, Fourier feature enhancement, and edge computing techniques. Compared to existing technologies, it offers the following significant advantages:

[0058] This invention can efficiently address the problem of data imbalance and improve the accuracy of detecting a minority of fault types.

[0059] A hierarchical labeling strategy is adopted to separate and detect insulator targets from fault areas, solving the problem of unclear fault features for small targets. Fourier transform feature enhancement is introduced, which concatenates the original features with the real and imaginary parts of Fourier transforms, enriching the fault feature representation and improving the separability of fault features from normal features. A CNN-Kmeans semi-supervised ensemble decision framework is designed to effectively utilize the distribution information of massive unlabeled inspection data, making up for the insufficient training of supervised models on minority class fault samples. This enables the model to alleviate the problem of missed and false fault detections under imbalanced data in the industry, even in scenarios where the proportion of fault samples is extremely low.

[0060] This invention achieves a balance between detection accuracy and real-time performance, and is suitable for edge device deployment:

[0061] A two-stage lightweight architecture of YOLOv8s and lightweight CNN is adopted and converted to ONNX format to accelerate edge inference. On OrangePi AIpro (8T), the total inference time of "detection + diagnosis" for a single image is ≤3s and the frame rate is ≥20fps, which meets the real-time requirements of power inspection. At the same time, the two-stage model ensures high recognition accuracy for minor fault areas, achieving a dual balance between accuracy and real-time performance.

[0062] This invention enables independent intelligent decision-making at the edge, improving the reliability and practicality of the inspection system.

[0063] The entire process of "image acquisition - target detection - fault diagnosis - alarm output" is completed in a closed loop on the OrangePi AIpro (8T) edge device, without relying on cloud servers, which greatly reduces communication latency and bandwidth costs and avoids inspection failures when the network is interrupted. The system supports local audible and visual alarms and remote fault information uploads, making it suitable for power line inspection scenarios in the field and remote areas, and improving the automation and intelligence level of power grid inspection. At the same time, the edge device is small in size and low in power consumption, and can be directly mounted on drones and inspection robots to realize mobile real-time inspection and expand application scenarios.

[0064] This invention enables the model and system to have good generalization ability and engineering applicability:

[0065] The semi-supervised ensemble learning framework of this invention does not rely on a specific neural network structure and can be adapted to various lightweight models such as YOLO series and CNN series. The dataset is constructed based on real power inspection scenarios, and the model training introduces image enhancement and transfer learning to improve the model's generalization ability to different environments (such as different lighting, weather, and background). The system deployment process is standardized and can be quickly ported to other models of edge computing devices, which has good engineering promotion value. Attached Figure Description

[0066] Figure 1 This is a schematic diagram of the steps of the present invention.

[0067] Figure 2 This is a schematic diagram of the overall process of a power transmission line insulator fault detection system according to an embodiment of the present invention.

[0068] Figure 3 This is a schematic diagram of the decision logic of the semi-supervised ensemble learning framework of the present invention.

[0069] Figure 4 This is a schematic diagram of the Fourier transform feature enhancement process of the present invention. Detailed Implementation

[0070] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0071] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0072] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the present invention; as used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise; furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0073] like Figures 1 to 4 As shown, this invention proposes a method and system for detecting faults in transmission line insulators based on semi-supervised ensemble learning and edge computing, including the following:

[0074] In one embodiment of the present invention, step 1 includes the following:

[0075] Step 1: Construct a hierarchical insulator image imbalance dataset:

[0076] Images of ordinary insulator inspections often feature complex backgrounds, and fault areas (such as damaged points or contaminated areas) are typically small, making them typical small-target detection scenarios. Therefore, a hierarchical annotation strategy is adopted, and a highly imbalanced dataset (healthy insulators:faulty insulators = 5:1~25:1) is constructed based on actual power line inspection scenarios. The specific steps are as follows:

[0077] The first stage is coarse annotation: all complete insulator targets in the inspection images are annotated with rectangular boxes, without distinguishing their health status. The annotation category is uniformly "insulator", which is used to train a robust insulator target detector and achieve accurate localization and cropping of insulators in complex backgrounds.

[0078] The second stage of fine annotation involves performing detailed pixel-level / rectangular box annotations only on images of insulators containing faults, targeting specific fault areas (such as damaged cracks, flashover ablation areas, and contamination accumulation areas). The annotation categories are "damaged", "flashover", and "contamination", which are used to train the fault classifier.

[0079] Dataset Construction: Collect real images of insulators on transmission lines, and combine data augmentation techniques such as image mirroring, rotation, and brightness perturbation to construct an imbalanced dataset with a significant disparity in the number of healthy insulators and various types of faulty insulators. The training set:test set = 8:2, simulating the data distribution characteristics in real inspection scenarios.

[0080] In one embodiment of the present invention, step 2 includes the following:

[0081] Step 2: Train a lightweight two-stage deep learning model on the host machine

[0082] On a high-performance host (equipped with a GPU acceleration module), a lightweight two-stage model that meets the edge deployment requirements of OrangePi AIpro (8T) is trained. All models adopt a lightweight network design, controlling the number of parameters and computational load to avoid inference lag on edge devices, including:

[0083] Insulator target detection model training: Based on the first-stage coarse-labeled dataset, the lightweight YOLOv8s was selected as the basic framework. The input images were uniformly scaled to 640×640 pixels. The initial learning rate was set to 0.01, the batch size to 16, and the number of training epochs to 150. Mosaic data augmentation and non-maximum suppression (NMS) optimization were enabled. The training yielded a target detection model that can quickly locate insulators in complex backgrounds, ensuring that there are no missed detections and few false detections of insulator targets.

[0084] Fault area classification model training: Based on the second-stage fine-annotated dataset, a lightweight convolutional neural network (CNN) (composed of Conv+BatchNorm+SiLU basic modules, containing 3 convolutional layers + 2 pooling layers + 2 fully connected layers) was selected as the fault classifier. The input was the single insulator image cropped in step 1, and the output was the probability distribution of the categories "healthy / damaged / flashover / dirty". During training, pre-trained weights were loaded for transfer learning, and the cross-entropy loss function was set to ensure the generalization ability of the classifier on small sample fault data.

[0085] Lightweight constraint: After converting the two models to ONNX format, the model size is ≤100MB, which meets the memory and computing power limits of OrangePiAIpro (8T).

[0086] In one embodiment of the present invention, step 3 includes the following:

[0087] Step 3: Design a semi-supervised ensemble learning fault diagnosis framework based on Fourier feature enhancement:

[0088] To address the data imbalance problem caused by the scarcity of fault samples, a semi-supervised ensemble learning framework is proposed, which integrates a supervised CNN classifier and an unsupervised Mini-Batch K-means clustering algorithm. Fourier transform feature enhancement is introduced to improve the separability of fault features, and distance threshold collaborative decision-making achieves dual utilization of labels and data distribution. The specific process is as follows:

[0089] Feature extraction and Fourier transform enhancement:

[0090] For the input fault area image, high-dimensional feature vectors are extracted through the backbone network of the lightweight CNN classifier trained in step 2. (D is the feature dimension);

[0091] Perform a discrete Fourier transform on the eigenvector X to decompose it into its real part. and the virtual part The calculation formula is:

[0092] ;

[0093] ;

[0094] ;

[0095] The real and imaginary parts are concatenated with the original feature vector X to form the enhanced fused feature vector. This enriches data representation, improves the separability between fault features and normal features, and solves the problem of unclear fault features in small samples.

[0096] Mini-Batch K-means clustering model training:

[0097] By fusing feature vectors As input, train the Mini-Batch K-means clustering algorithm (which reduces computational complexity compared to the traditional K-means and is suitable for edge devices). The number of clusters k is the same as the number of fault categories (including healthy categories, for a total of 4 categories).

[0098] Using Euclidean distance as a similarity metric, the distance from the sample to the cluster center is calculated. The cluster centers are updated by minimizing the sum of squared errors (SSE) to obtain the cluster centers C1, C2, C3, and C4 for each category of insulator (healthy / damaged / flashover / polluted).

[0099] Distance threshold collaborative decision-making mechanism:

[0100] For a fault region of an insulator to be diagnosed, the feature vector is fused. The framework simultaneously yields two results: the probability prediction of the label y from the CNN classifier. CNN (The category corresponding to the highest probability), the majority vote label y of Mini-Batch K-means clustering. Kmeans (The category corresponding to the cluster to which the sample belongs), the specific decision rule is:

[0101] If y CNN =y Kmeans If the result is positive, then the result will be adopted directly as the final fault diagnosis label.

[0102] If y CNN ≠y Kmeans Then calculate the distance d from the sample to the K-means cluster center. Kmeans The distance d to the cluster center of the predicted category by the lightweight convolutional neural network CNN ;

[0103] in ;

[0104] in ;

[0105] An empirical threshold M is set (determined through experimental debugging; in this invention, M=0.15).

[0106] like This indicates that the sample is close to the center of the CNN's predicted class in the feature space, indicating confidence in the CNN's supervised discriminative ability and acceptance of its prediction result y. CNN Otherwise, it indicates that the sample is significantly close to the K-means cluster center in the feature space, and the K-means clustering result y is adopted. Kmeans .

[0107] This mechanism effectively utilizes the distribution information of unlabeled data, compensates for the lack of training of supervised CNNs on minority class fault samples, and enhances the robustness of the model to minority class faults.

[0108] In one embodiment of the present invention, step 4 includes the following:

[0109] Step 4: Model Conversion and Lightweight Deployment of Edge Devices

[0110] To achieve efficient inference of the model on OrangePi AIpro (8T), the following steps are taken to complete model format conversion, inference engine adaptation, and system deployment:

[0111] Model format conversion: The YOLOv8s object detection model trained in step 2 and the lightweight CNN classifier in step 3 are uniformly converted to the ONNX (Open Neural Network Exchange) format. The ONNX format has the advantages of cross-platform compatibility and high compatibility. It defines a unified computational structure and standardized operation operators, which can be efficiently loaded by the OrangePi AIpro (8T) inference engine. During the conversion process, ONNX simplification is enabled, redundant operators are removed, and the amount of inference computation is reduced.

[0112] Edge device environment configuration: Build an inference environment on OrangePi AIpro (8T), install the ONNX Runtime lightweight inference engine, OpenCV image processing library, GPIO control library, and configure the Python 3.9 runtime environment to ensure compatibility between model inference, image acquisition and peripheral control;

[0113] System Deployment: The converted ONNX format model file, semi-supervised ensemble learning decision logic code, Mini-Batch K-means cluster center parameter file, and USB camera driver were deployed together on the OrangePi AIpro (8T) development board. All code was optimized for lightweighting to avoid redundant calculations and ensure the speed of edge inference.

[0114] In one embodiment of the present invention, step 3 includes the following:

[0115] Step 5: Integrated fault detection and real-time response at the edge:

[0116] After deployment, the system runs independently in a closed loop on OrangePi AIpro (8T), without relying on a cloud server, realizing an integrated workflow of "image acquisition - target detection - fault diagnosis - alarm output". The specific steps are as follows:

[0117] Image acquisition: A high-definition industrial camera connected via the development board's USB interface captures real-time images of power transmission line inspection scenes. The image resolution is 1920×1080, and the frame rate is ≥20fps, meeting the requirements for real-time inspection.

[0118] First-stage insulator target detection: The YOLOv8s-ONNX model is called to perform inference on the acquired scene images, quickly and accurately locate all insulator targets from complex backgrounds (such as sky, trees, and iron towers), output the bounding box coordinates of the insulator targets, and crop out individual insulator images based on the coordinates. The cropped image resolution is scaled to 640×640 to prepare for subsequent fault diagnosis.

[0119] Two-stage insulator fault diagnosis: For each cropped single insulator image, feature extraction, Fourier transform enhancement, CNN classification, and K-means clustering are performed sequentially. Through the distance threshold collaborative decision-making mechanism described in step 3, its health status (healthy / damaged / flashover / pollution) is finally determined. The diagnosis inference time for a single image is ≤1.5s.

[0120] Real-time alarms and data uploads:

[0121] Local alarm: Based on the diagnostic results, the external LED indicator (green = healthy, red = damaged, yellow = flashover, blue = dirty) and buzzer are controlled through the GPIO pins of the development board to provide real-time audible and visual alarms. The buzzer will sound continuously when a fault is detected until manual confirmation is received.

[0122] Remote upload: The fault alarm information (fault type, detection time, insulator position) and fault images are compressed and uploaded to the cloud power grid monitoring platform through the Wi-Fi / Ethernet interface of the development board, realizing remote synchronization of fault information and facilitating timely handling by operation and maintenance personnel.

[0123] In addition to the above, the present invention also has related embodiments, including the following:

[0124] like Figures 2 to 4 As shown, this embodiment uses OrangePi AIpro (8T) as the core edge computing hardware and combines it with a high-definition industrial camera to specifically illustrate the technical solution of the present invention.

[0125] Step 1: Hierarchical Dataset Construction and Preprocessing:

[0126] First, images of power transmission lines under different weather and lighting conditions were collected.

[0127] Coarse annotation and detector preparation: Use rectangular boxes to annotate all insulators in the original image, regardless of the fault type, and classify them all as "insulator".

[0128] Fine-grained annotation and classifier preparation: For images of faulty insulators, pixel-level or fine rectangular box annotations are performed on areas of “damage”, “flashover”, and “contamination”.

[0129] Data augmentation: Perform operations such as mirroring, rotation, and brightness perturbation on faulty samples to make the ratio of training set to test set reach 8:2, simulating the high imbalance distribution (5:1~25:1) of healthy and faulty samples in real-world scenarios.

[0130] Step 2: Lightweight model training on the host side:

[0131] On high-performance host machines, GPUs are used for parallel training of the two-stage model:

[0132] The first-stage model uses the YOLOv8s framework with an input size of 640*640. Mosaic data augmentation and NMS optimization are used to ensure the recall rate of insulator localization.

[0133] Two-stage model: Construct a lightweight CNN classifier with 3 convolutional layers, 2 pooling layers and 2 fully connected layers, load pre-trained weights through transfer learning, and output the probability distribution of each category.

[0134] Lightweight constraint: Ensure that the total size of the two models after conversion to ONNX format is ≤100MB to adapt to the memory limitations of the edge.

[0135] Step 3: Semi-supervised ensemble learning decision logic:

[0136] This stage involves the core feature extraction and diagnostic logic of this invention:

[0137] Fourier feature enhancement (e.g.) Figure 4 ):

[0138] Extracting feature vectors generated by the CNN backbone network .

[0139] According to the formula Perform a discrete Fourier transform.

[0140] Extracting the real part With the imaginary part .

[0141] The original feature X, its real part, and its imaginary part are concatenated to output a fused feature vector. .

[0142] Step 2: Collaborative decision-making (e.g.) Figure 2 ):

[0143] Simultaneously obtain the CNN predicted label y CNN and Mini-Batch K-means clustering labels y Kmeans .

[0144] Decision path A: If y CNN= y Kmean If so, the final diagnostic label will be output directly.

[0145] Decision path B: If the results are inconsistent, calculate the Euclidean distance d from the sample to the predicted class centers of the two models. CNN and d Kmeans .

[0146] Threshold determination: Set an empirical threshold M = 0.15. If Trust the results of CNN; otherwise, trust the results of K-means clustering.

[0147] Edge deployment and integrated detection:

[0148] Step 4: Model Conversion and Environment Configuration: Convert the model to ONNX format and enable operator simplification. Install the ONNX Runtime, OpenCV, and GPIO control library on OrangePiAIpro.

[0149] Step 5-1: Image Acquisition ( Figure 2 -Step 5-1): Access the industrial camera via USB interface to capture 1920*1080 resolution images, maintaining a frame rate of over 20fps.

[0150] Step 5-2: Target Detection and Cropping ( Figure 2-Step 5-2): Call the YOLOv8s-ONNX model to locate the insulator, and crop out a 640*640 single-unit image according to the coordinates.

[0151] Step 5-3: Diagnostic Reasoning ( Figure 2 -Step 5-3): Perform the integrated decision logic of step 3 on the cropped image, with a single image inference time ≤1.5S.

[0152] Step 5-4: Response Output ( Figure 2 -Step 5-4): Control the external LED to switch colors (green, red, yellow, and blue correspond to health and three types of faults, respectively) via GPIO pins and drive the buzzer alarm. Use Wi-Fi / Ethernet to synchronize the fault type, time, and image compression package to the cloud platform in real time.

[0153] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for detecting faults in transmission line insulators based on semi-supervised ensemble learning and edge computing, characterized in that, Includes the following: Step S1: Construct a hierarchical insulator image imbalance dataset; including using a hierarchical annotation strategy and combining it with the actual scenario of power inspection to construct a high imbalance dataset; Step S2: Train a lightweight two-stage deep learning model on the host side; this includes training a lightweight two-stage model that meets the requirements for edge deployment on the development board, wherein all models adopt a lightweight network design; Step S3: Design a semi-supervised ensemble learning fault diagnosis framework based on Fourier feature enhancement. This includes proposing a semi-supervised ensemble learning framework that integrates a supervised CNN classifier and an unsupervised Mini-Batch K-means clustering algorithm to address the data imbalance problem caused by the scarcity of fault samples. Fourier transform feature enhancement is introduced to improve the separability of fault features, and distance threshold collaborative decision-making is used to achieve dual utilization of labels and data distribution. Step S4: Model conversion and lightweight deployment on edge devices, including implementing model inference on the development board, completing model format conversion, inference engine adaptation and system deployment; Step S5: Integrated fault detection and real-time response at the edge, including enabling the system to run independently in a closed loop on the development board after deployment, and realizing an integrated process from image acquisition to target detection to fault diagnosis to alarm output.

2. The method for detecting faults in transmission line insulators based on semi-supervised ensemble learning and edge computing according to claim 1, characterized in that, Step S1 includes the following: Step S11: The first stage of coarse annotation includes using rectangular boxes to annotate all complete insulator targets in the inspection images, without distinguishing their health status. The annotation category is uniformly set to insulator, which is used to train a robust insulator target detector to achieve accurate localization and cropping of insulators in complex backgrounds. Step S12: The second stage of fine annotation includes fine pixel-level or rectangular box annotation only on the images of insulators containing faults, targeting specific fault areas for training the fault classifier; this includes labeling fault areas as damaged, flashover ablation areas as flashover, and contamination accumulation areas as contamination. Step S13: Dataset construction, including collecting real images of transmission line insulators, and using data augmentation techniques to construct an imbalanced dataset with a significant disparity in the number of healthy insulators and various types of faulty insulators, simulating the data distribution characteristics in real inspection scenarios. Data augmentation techniques include image mirroring, rotation, and brightness perturbation.

3. The method for detecting faults in transmission line insulators based on semi-supervised ensemble learning and edge computing according to claim 2, characterized in that, Step S2 includes the following: Step S21: Training the insulator target detection model, including using the lightweight YOLOv8s as the basic framework based on the first stage coarsely labeled dataset, scaling the input images to the same pixel size, setting the initial learning rate, batch size, and number of training epochs, enabling Mosaic data augmentation and non-maximum suppression optimization, and training to obtain a target detection model that can quickly locate insulators in complex backgrounds, ensuring that there are no missed detections and few false detections of insulator targets. Step S22: Training the fault area classification model, including using a lightweight convolutional neural network as the fault classifier based on the second-stage fine-annotated dataset. The input is the single insulator image cropped in step S1, and the output includes the probability distribution of categories such as healthy, damaged, flashover, or contaminated. During training, pre-trained weights are loaded for transfer learning, and the cross-entropy loss function is set to ensure the generalization ability of the classifier on small sample fault data. Step S23: Lightweight constraints, including converting the insulator target detection model and fault area classification model into a format that meets the requirements of the development board and ensuring that the model size is ≤100MB, so that the two models meet the memory and computing power limitations of the development board.

4. The method for detecting faults in transmission line insulators based on semi-supervised ensemble learning and edge computing according to claim 3, characterized in that, Step S3 includes the following: Step S31: Feature extraction and Fourier transform enhancement, including: Step S311: For the input fault region image, extract a high-dimensional feature vector X using the lightweight convolutional neural network trained in step S2 as a fault classifier; where... ； Where D is the feature dimension; Step S312: Perform a discrete Fourier transform on the eigenvector X to decompose it into its real part. and the virtual part The calculation formula is: ； ； ； Step S313: Concatenate the real and imaginary parts with the original feature vector X to form the enhanced fused feature vector. This enriches data representation, improves the separability between fault features and normal features, and solves the problem of unclear fault features in small samples.

5. The method for detecting faults in transmission line insulators based on semi-supervised ensemble learning and edge computing according to claim 4, characterized in that, Step S3 also includes the following: Step S32: Training the Mini-Batch K-means clustering model, including the following: Step S321: Using fused feature vectors As input, train a Mini-Batch K-means clustering algorithm, where the number of clusters k is the same as the number of fault categories; Step S322: Using Euclidean distance as the similarity metric, calculate the distance from the sample to the cluster center, and update the cluster center by minimizing the sum of squared errors to obtain the cluster centers C1, C2, C3, and C4 for each category of insulator; corresponding to healthy, damaged, flashover, and contaminated, respectively.

6. The method for detecting faults in transmission line insulators based on semi-supervised ensemble learning and edge computing according to claim 5, characterized in that, Step S3 also includes the following: Step S33: Employ a distance threshold collaborative decision-making mechanism, which utilizes the distribution information of unlabeled data to compensate for the insufficient training of supervised lightweight convolutional neural networks on minority class fault samples, thereby enhancing the model's robustness to minority class faults. The distance threshold collaborative decision-making mechanism includes the following: Step S331: For a fault region of an insulator to be diagnosed, fuse the feature vector. The framework yields two results simultaneously: the probabilistic predicted label y from the lightweight convolutional neural network classifier. CNN The majority vote label y in Mini-Batch K-means clustering Kmeans The specific decision-making rules are as follows: If y CNN =y Kmeans If the result is positive, then the result will be adopted directly as the final fault diagnosis label. If y CNN ≠y Kmeans Then calculate the distance d from the sample to the K-means cluster center. Kmeans The distance d to the cluster center of the predicted category by the lightweight convolutional neural network CNN ; in ; in ; Step S332: Set an empirical threshold M, if This indicates that the sample is close to the center of the CNN's predicted class in the feature space, indicating confidence in the CNN's supervised discriminative ability and acceptance of its prediction result y. CNN Otherwise, it indicates that the sample is significantly close to the K-means cluster center in the feature space, and the K-means clustering result y is adopted. Kmeans .

7. The method for detecting faults in transmission line insulators based on semi-supervised ensemble learning and edge computing according to claim 6, characterized in that, Step S4 includes the following: Step S41: Model format conversion, including converting the YOLOv8s object detection model trained in step S2 and the lightweight convolutional neural network classifier in step S3 into ONNX. Step S42: Edge device environment configuration, including setting up the inference environment on the development board, installing the ONNX Runtime lightweight inference engine, OpenCV image processing library, GPIO control library, configuring the Python 3.9 runtime environment, and ensuring compatibility between model inference, image acquisition and peripheral control; Step S43: System deployment, which includes deploying the converted ONNX format model file, semi-supervised ensemble learning decision logic code, Mini-Batch K-means cluster center parameter file, and USB camera driver to the development board. All code is optimized for lightweighting to avoid redundant calculations and ensure inference speed at the edge.

8. The method for detecting faults in transmission line insulators based on semi-supervised ensemble learning and edge computing according to claim 7, characterized in that, Step S5 includes the following: Step S51: Image Acquisition: The high-definition industrial camera connected to the development board's USB interface captures images of the power transmission line inspection scene in real time. The image resolution is 1920×1080 and the frame rate is ≥20fps, which meets the requirements for real-time inspection. Step S52: First-stage insulator target detection: The YOLOv8s-ONNX model is called to perform inference on the acquired scene images, quickly and accurately locate all insulator targets from the complex background, output the bounding box coordinates of the insulator targets, crop out individual insulator images based on the coordinates, and scale the cropped image resolution to a uniform resolution to prepare for subsequent fault diagnosis; the complex background includes the sky, trees, and iron towers. Step S53: Two-stage insulator fault diagnosis: For each cropped single insulator image, feature extraction, Fourier transform enhancement, CNN classification, and K-means clustering are performed sequentially. Through the distance threshold collaborative decision-making mechanism described in step S3, its working status is finally determined. The diagnosis reasoning time for a single image is ≤1.5s. The working status includes healthy, damaged, flashover, and contaminated. Step S54: Real-time Alarms and Data Upload Local alarm: Based on the diagnostic results, the external LED indicator and buzzer are controlled through the GPIO pins of the development board to provide real-time audible and visual alarms. The buzzer will sound continuously when a fault is detected until manual confirmation is received. Remote upload: Fault alarm information and fault images are compressed and uploaded to the cloud power grid monitoring platform through the Wi-Fi / Ethernet interface of the development board, realizing remote synchronization of fault information and facilitating timely handling by operation and maintenance personnel.

9. A fault detection system for transmission line insulators based on semi-supervised ensemble learning and edge computing, comprising an electronic device, wherein the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements a method for detecting faults in transmission line insulators based on semi-supervised ensemble learning and edge computing as described in any one of claims 1 to 8.

10. A fault detection system for power transmission line insulators based on semi-supervised ensemble learning and edge computing, comprising a computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a method for detecting faults in power transmission line insulators based on semi-supervised ensemble learning and edge computing as described in any one of claims 1 to 8.

Images