Power grid inspection system, method and terminal based on edge intelligence

The power grid inspection system based on edge intelligence solves the problems of unclear candidate target area generation and insufficient small sample enhancement in power grid inspection. It achieves efficient detection under changing lighting conditions and rain and fog, reduces false alarm rate and bandwidth consumption, enhances data privacy protection, supports closed-loop labeling and model distribution mechanism between edge and cloud, and realizes continuous iteration and engineering implementation.

CN121033637AActive Publication Date: 2025-11-28TAIZHOU KAITAI POWER DESIGN CO LTD +2

Patent Information

Application Number
CN202511136656.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-08-14
Publication Date
2025-11-28
Estimated Expiration
2045-08-14

AI Technical Summary

Technical Problem

Existing technologies in power grid inspection suffer from inconsistent candidate target area generation strategies and unclear ROI sources, leading to ambiguity in the detection process logic. In small sample scenarios, detection models are difficult to generalize effectively, existing solutions lack a dynamic online enhancement system, and traditional cloud-based backhaul processing is limited by communication bandwidth and latency, making it difficult to meet real-time alarm requirements. At the same time, existing technologies cannot effectively solve data privacy and security issues.

Method used

An edge-intelligence-based power grid inspection system is adopted, including a data acquisition module, a preprocessing module, a single-sample online enhancement module, a sensing module, a time-series and anomaly fusion module, and a decision module. Through the data acquisition module, preprocessing module, online enhancement module, single-sample detection module, time-series and anomaly detection module, and decision module, enhanced candidate region samples are generated. These samples are then detected by a lightweight feature extraction network and subjected to time-series and anomaly fusion processing to generate alarm information.

Benefits of technology

Significantly improves detection robustness and recall in scenarios with varying lighting, rain, fog, and small sample sizes, reduces false alarm rate and bandwidth consumption, enhances data privacy protection, supports closed-loop annotation and model delivery mechanisms from edge to cloud, and enables continuous iteration and engineering implementation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121033637A_ABST
    Figure CN121033637A_ABST
Patent Text Reader

Abstract

The invention provides a power grid inspection system, method and terminal based on edge intelligence, and the system comprises an acquisition module which is used for collecting a visual image sequence and position information at an inspection site; the preprocessing module is used for performing image enhancement and defogging processing on the visual image sequence; the single sample online enhancement module is used for applying a dynamic enhancement operator set to the candidate region-of-interest set to generate an enhanced candidate region sample; the sensing module is used for receiving the enhanced candidate region sample and outputting a defect bounding box, a defect category and a classification confidence coefficient; the time sequence and anomaly fusion module is used for performing fusion processing on the defect targets at the same spatial position and performing smoothing processing on a confidence coefficient sequence of the same target in a continuous preset number of frames so as to output a smoothed confidence coefficient; and the decision module is used for calculating a final comprehensive score based on the smoothed confidence coefficient and the normalized abnormal score, and generating alarm information when the final comprehensive score is higher than a preset threshold value.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of power grid inspection technology, and in particular to a power grid inspection system, method and terminal based on edge intelligence. Background Technology

[0002] With the continuous expansion of the power grid, the aging of equipment, and the accelerated integration of renewable energy, the frequency and complexity of inspections have increased significantly. The changing on-site environment, including sudden changes in lighting, rain and fog, strong backlighting, and varying shooting angles, results in inconsistent quality of the acquired visual images. Furthermore, many defects are small in size or subtle in appearance, leading to a scarcity of positive samples and uneven distribution of categories, making it difficult for supervised learning-based detection models to achieve stable generalization capabilities. On the other hand, the traditional approach of relying on transmitting large amounts of images back to the cloud for centralized processing is limited by communication bandwidth and latency, making it difficult to meet real-time alarm requirements. Large-scale transmission also increases data privacy and security risks, hindering engineering deployment.

[0003] Currently, existing technologies have shortcomings in several key aspects: for example, the strategies for generating candidate target areas (ROIs) are not uniform, and some solutions do not clearly define the source and temporal relationship of ROIs, resulting in logical ambiguity in the input and output of the detection process; and the enhancement strategies for small sample / single sample scenarios are mostly offline expansion or simple affine transformations, lacking a dynamic online enhancement system that can run in edge environments. Summary of the Invention

[0004] In view of this, it is necessary to provide a power grid inspection system, method and terminal based on edge intelligence, which can overcome at least one of the above defects.

[0005] In a first aspect, embodiments of this application provide a power grid inspection system based on edge intelligence, the system comprising:

[0006] The data acquisition module is used to acquire visual image sequences and location information at the inspection site.

[0007] A preprocessing module is used to perform image enhancement and dehazing processing on the visual image sequence, and generate a set of candidate target regions based on the processed image, wherein each candidate target region corresponds to an image region that may contain power equipment;

[0008] The single-sample online augmentation module is used to apply a dynamic set of augmentation operators to a set of candidate regions of interest to generate augmented candidate region samples.

[0009] The perception module is used to receive enhanced candidate region samples, extract features through a lightweight feature extraction network, and determine whether the candidate region includes valid power equipment. If the candidate region includes valid power equipment, the module outputs the defect bounding box, defect category, and classification confidence.

[0010] The temporal and anomaly fusion module is used to perform fusion processing on defect targets at the same spatial location, smooth the confidence sequence of the same target in a consecutive preset number of frames to output the smoothed confidence; it is also used to extract features from the image region corresponding to the defect box in the current frame and calculate the normalized anomaly score.

[0011] The decision module is used to calculate a final comprehensive score based on the smoothed confidence score and the normalized outlier score, and to generate an alarm message when the final comprehensive score is higher than a preset threshold. The formula for calculating the final comprehensive score is as follows:

[0012] S=λ·P′+(1-λ)·A

[0013] Where S is the final comprehensive score, P′ is the smoothed confidence score, A is the normalized outlier score, and λ∈[0,1] is the weighting coefficient.

[0014] In one embodiment of this application, the set of dynamic enhancement operators includes:

[0015] Gaussian blur operator, median blur operator, random erase operator, superpixel perturbation operator, pixel additive noise operator, and affine or perspective distortion operator;

[0016] The single-sample online enhancement module generates enhanced candidate region samples by applying different operators according to preset probabilities.

[0017] In one embodiment of this application, the timing and anomaly fusion module is further used for:

[0018] The confidence sequence from the perception module is smoothed using a dilated temporal convolutional network, and the output formula of the dilated temporal convolution is:

[0019]

[0020] Among them, y t This represents the smoothed output value at time point t, i.e., the defect detection confidence after temporal fusion processing, where d is the dilation factor, K is the convolution kernel width, and w n denoted as the weight coefficients of the convolution kernel, x represents the original confidence sequence from the perception module, t represents the current time point, and td·n represents the position d·n time steps backward from the current time point t, i.e., the selected historical data point.

[0021] In one embodiment of this application, the preprocessing module is further used for:

[0022] When the acquisition module acquires thermal imaging data simultaneously, it performs coarse registration between the thermal imaging data and the visual image, and generates the candidate target region based on the registered thermal imaging data.

[0023] In one embodiment of this application, the preprocessing module is further used for:

[0024] The imaging quality of the thermal imaging data is detected based on the signal-to-noise ratio, contrast, and temperature distribution uniformity. When the imaging quality of the thermal imaging data is detected to be lower than a preset threshold, image enhancement and dehazing processing are performed on the visual image sequence, and a set of candidate target regions is generated based on the processed image.

[0025] In one embodiment of this application, the perception module is further used to fuse the detection confidence score with the category probability vector output by the classification module to obtain a fused confidence score. The formula for calculating the fused confidence score is as follows:

[0026] P fused =β·P + (1-β)·max(q)

[0027] Among them, P fused P is the fusion confidence, q is the class probability vector, max(q) represents the maximum component in q, and β is the fusion coefficient.

[0028] In one embodiment of this application, the weight coefficients in the decision module are dynamically generated by a lightweight neural network based on on-site environmental variables, including light intensity, estimated wind speed, historical false alarm rate, and temperature.

[0029] In one embodiment of this application, the alarm information includes: alarm timestamp, location information, corresponding candidate target region bounding box, defect category label, fusion confidence sequence, normalized anomaly score, and severity level.

[0030] Secondly, embodiments of this application provide a power grid inspection method based on edge intelligence, applied to the power grid inspection system based on edge intelligence as described in the first aspect, the method comprising:

[0031] Collect visual image sequences and location information at the inspection site;

[0032] Image enhancement and dehazing are performed on the visual image sequence, and a set of candidate target regions is generated based on the processed images, wherein each candidate target region corresponds to an image region that may contain power equipment;

[0033] A dynamic set of enhancement operators is applied to the candidate region of interest set to generate enhanced candidate region samples.

[0034] The system receives enhanced candidate region samples and extracts features through a lightweight feature extraction network to determine whether the candidate region includes valid power equipment. If the candidate region includes valid power equipment, the system outputs the defect bounding box, defect category, and classification confidence score.

[0035] Perform fusion processing on defect targets at the same spatial location, and smooth the confidence sequence of the same target in a consecutive preset number of frames to output the smoothed confidence.

[0036] Extract features from the image region corresponding to the defect box in the current frame and calculate the normalized anomaly score;

[0037] The final comprehensive score is calculated based on the smoothed confidence score and the normalized outlier score, and an alarm message is generated when the final comprehensive score is higher than a preset threshold.

[0038] Thirdly, embodiments of this application provide a terminal, including:

[0039] Processor; and

[0040] The memory stores computer-readable instructions for controlling the processor to execute the edge intelligence-based power grid inspection method as described in the second aspect.

[0041] The power grid inspection system, method, and terminal based on edge intelligence provided in this application integrate candidate target region generation, edge-side single-sample online enhancement, lightweight perception, temporal smoothing and unsupervised anomaly scoring, and adaptive decision-making. This solves the problems of unclear ROI sources, insufficient small-sample enhancement, and inconsistent confidence / anomaly scoring in existing technologies. Specifically, the preprocessing module explicitly generates candidate ROIs; the single-sample online enhancement module expands sample diversity under training and optional test-time enhancement (TTA); the perception module achieves low-latency and efficient inference at the edge with a lightweight backbone and ultra-lightweight detection head; the temporal and anomaly fusion module smooths confidence through expanded temporal convolution and calculates normalized anomaly scores in parallel; and the decision module provides robust alarms through a co-directional fusion strategy. This solution significantly improves detection robustness and recall capabilities under varying lighting conditions, rain, fog, and small-sample scenarios while ensuring low power consumption and real-time performance. It reduces false alarm rates and bandwidth consumption, enhances data privacy protection, and supports continuous iteration and engineering implementation through an edge-cloud closed-loop annotation and model delivery mechanism. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of a power grid inspection system based on edge intelligence provided in an embodiment of this application.

[0043] Figure 2a A Gaussian blurred visual image provided in one embodiment of this application.

[0044] Figure 2b A median-blurred visual image provided in one embodiment of this application.

[0045] Figure 2c A visual image after random erasure provided in one embodiment of this application.

[0046] Figure 2d This is a visual image after superpixel processing provided in one embodiment of this application.

[0047] Figure 2e A pixel-added visual image provided in one embodiment of this application.

[0048] Figure 2f This is a distorted visual image provided in one embodiment of this application.

[0049] Figure 3 This is a schematic diagram of a method flow provided in an embodiment of this application.

[0050] Figure 4 This is a schematic diagram of a terminal provided in an embodiment of this application.

[0051] Explanation of main component symbols

[0052] Edge-based power grid inspection system 10

[0053] Acquisition Module 11

[0054] Preprocessing module 12

[0055] Single-sample online augmentation module 13

[0056] Perception Module 14

[0057] Timing and Anomaly Fusion Module 15

[0058] Decision Module 16

[0059] Terminal 20

[0060] Processor 21

[0061] Memory 22

[0062] Method steps S100-700 Detailed Implementation

[0063] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0064] It should be noted that, in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0065] It should be noted that in the embodiments of this application, the terms "first," "second," etc., are used only for descriptive purposes and should not be construed as indicating or implying relative importance, nor as indicating or implying order. Features specified as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0066] Based on the embodiments described in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0067] With the continuous expansion of the power grid, the aging of equipment, and the accelerated integration of renewable energy, the frequency and complexity of inspections have increased significantly. The changing on-site environment, including sudden changes in lighting, rain and fog, strong backlighting, and varying shooting angles, results in inconsistent quality of the acquired visual images. Furthermore, many defects are small in size or subtle in appearance, leading to a scarcity of positive samples and uneven distribution of categories, making it difficult for supervised learning-based detection models to achieve stable generalization capabilities. On the other hand, the traditional approach of relying on transmitting large amounts of images back to the cloud for centralized processing is limited by communication bandwidth and latency, making it difficult to meet real-time alarm requirements. Large-scale transmission also increases data privacy and security risks, hindering engineering deployment.

[0068] Currently, existing technologies have shortcomings in several key aspects: First, the strategies for generating candidate target areas (ROIs) are not uniform, and some solutions do not clearly define the source and temporal relationship of ROIs, resulting in logical ambiguity in the input and output of the detection process; Second, the enhancement strategies for small sample / single sample scenarios are mostly offline expansion or simple affine transformations, lacking a dynamic online enhancement system that can run in edge environments.

[0069] In view of this, this application provides a power grid inspection system, method, and terminal based on edge intelligence. By integrating candidate target region generation, edge-side single-sample online enhancement, lightweight perception, temporal smoothing and unsupervised anomaly scoring, and adaptive decision-making, it solves the problems of unclear ROI sources, insufficient small-sample enhancement, and inconsistent confidence / anomaly scoring in existing technologies. Specifically, the preprocessing module explicitly generates candidate ROIs, the single-sample online enhancement module expands sample diversity under training and optional test-time enhancement (TTA), the perception module achieves low-latency and efficient inference at the edge with a lightweight backbone and ultra-lightweight detection head, the temporal and anomaly fusion module smooths confidence through temporal convolution and calculates normalized anomaly scores in parallel, and the decision module provides robust alarms through a co-directional fusion strategy. This solution significantly improves detection robustness and recall capabilities under varying lighting conditions, rain, fog, and small-sample scenarios while ensuring low power consumption and real-time performance, reduces false alarm rate and bandwidth consumption, and enhances data privacy protection. Furthermore, it supports continuous iteration and engineering implementation through a closed-loop annotation and model delivery mechanism between the edge and cloud.

[0070] Figure 1 This is a schematic diagram of a power grid inspection system based on edge intelligence provided in an embodiment of this application, as shown below. Figure 1 The power grid inspection system 10 based on edge intelligence shown includes at least the following components: a data acquisition module 11, a preprocessing module 12, a single-sample online enhancement module 13, a sensing module 14, a time series and anomaly fusion module 15, and a decision module 16.

[0071] In this embodiment of the application, the acquisition module 11 is used to acquire visual image sequences and location information at the inspection site.

[0072] Specifically, the acquisition module 11 includes a high-resolution visible light camera, an optional thermal imaging camera, a GPS receiver unit, and an inertial measurement unit (IMU). The camera supports automatic / manual exposure, gain, and white balance control, and can perform continuous shooting or trigger shooting at a preset frame rate (e.g., 10–30 fps, adjustable according to the task). The acquisition module 11 is responsible for timestamping and initially annotating metadata (such as heading, pitch, latitude and longitude, altitude, and camera intrinsic and extrinsic parameters) of the images and location information, and managing the acquired data locally, enabling breakpoint resume and local encrypted storage for subsequent preprocessing and edge analysis. The acquisition module can also trigger low-latency shooting strategies based on the scene (e.g., increasing the sampling frequency when motion or abnormal vibration is detected), and provide necessary exposure / focus information to the preprocessing module to facilitate dynamic adjustment of the image enhancement algorithm.

[0073] Understandably, the acquisition module 11 not only undertakes the basic responsibilities of "image capture and positioning," but also needs to ensure data quality and the availability of downstream processing. First, precise synchronization between GPS / IMU and image timestamps ensures accurate matching of the same physical target during target matching, trajectory tracking, and temporal fusion across different frames. Second, the acquisition module should have basic self-diagnostic and image quality assessment capabilities (such as detecting overexposure, underexposure, blur, or motion artifacts), and issue commands to retake or adjust exposure when image quality fails to meet minimum standards, reducing subsequent false detections and redundant calculations. Third, in engineering deployments, the acquisition module needs to balance energy consumption and stability—for example, in drone-mounted scenarios, sampling strategies need to be optimized to extend battery life, while in portable terminal scenarios, image resolution and real-time inference latency need to be balanced. Finally, for data privacy and security considerations, the acquisition module should locally support temporary encryption and hierarchical storage strategies for raw images and metadata, ensuring compliance with on-site security policies and requirements before uploading to the edge gateway or cloud platform, thereby providing reliable, secure, and traceable raw data input for the preprocessing module and the entire subsequent system.

[0074] In this embodiment of the application, the preprocessing module 12 is used to perform image enhancement and dehazing processing on the visual image sequence, and generate a set of candidate target regions based on the processed image, wherein each candidate target region corresponds to an image region that may contain power equipment.

[0075] Specifically, the preprocessing module 12 first performs denoising, automatic exposure / white balance correction, and dehazing / contrast enhancement on the input frame (using a zero-reference or attention-based enhancement network). Then, it executes semantic segmentation or an edge / superpixel-based candidate region generation algorithm (or a combination of both) to output a set of candidate target regions. When the system also has thermal image input, the preprocessing module 12 can also first perform quality assessment on the thermal image and (after coarse registration) use the thermal image as an auxiliary channel for generating candidate regions to improve recall of thermal defects or occluded targets. The preprocessing module outputs metadata for each candidate region (such as source frame ID, spatial coordinates, initial confidence estimate, thermal image temperature statistics, etc.) and submits the candidate regions to the single-sample online enhancement module or directly to the perception module (determined by system configuration).

[0076] Understandably, the preprocessing module 12 plays a crucial role in the entire pipeline by "transforming noisy data into detectable data." Its design must consider both algorithm performance and engineering overhead. On the one hand, edge terminals have limited computing power and memory, so dehazing and enhancement algorithms need to be as lightweight as possible or adopt tiered strategies (e.g., perform fast shallow enhancement first, and only call more complex deep dehazing when the quality is still insufficient). On the other hand, the generation of candidate target regions needs to balance recall and accuracy—too many candidates will increase the burden on the backend perception module, while too few will miss real defects. To this end, the preprocessing module often uses a combination of multi-scale segmentation and confidence filtering: first, lightweight semantic segmentation is used to quickly identify possible device regions, then candidate boxes are simplified using heuristic rules based on color / temperature / shape or small region proposers, and the thermal image information is quality-checked (e.g., signal-to-noise ratio, whether the temperature distribution is uniform, whether the thermal noise exceeds the standard) to determine whether to use thermal image assistance. In addition, the preprocessing module needs to output rich metadata for future traceability and model iteration, such as recording enhancement parameters, dehazing coefficients, and registration matrices, to support data cleaning and label matching during offline training in the cloud. In summary, the preprocessing module is both a data quality gatekeeper and a candidate search space designer, and its robustness directly affects the efficiency and accuracy of the entire detection process.

[0077] In this embodiment, the single-sample online enhancement module 13 is used to apply a set of dynamic enhancement operators to a set of candidate regions of interest to generate enhanced candidate region samples.

[0078] Specifically, the single-sample online augmentation module 13 maintains a set of configurable augmentation operators (including but not limited to Gaussian blur, median blur, random erasure, superpixel perturbation, pixel additive noise, affine / perspective distortion, etc.). During the training phase, it randomly combines and augments candidate regions according to preset or adaptive strategies to generate rich training samples. During the inference phase, the module can selectively perform test-time augmentation (TTA), which generates several augmented views for each candidate region and fuses the perception results of each view (e.g., averaging, weighting, or voting) to improve robustness to small samples or occlusion conditions. The augmentation module also records the operators used and their parameters as metadata, supporting the synchronization and iterative optimization of augmentation strategies between the edge and the cloud. To control the computational overhead on the edge, the single-sample online augmentation module can enable a threshold triggering strategy during inference, enabling TTA only for low-confidence or uncertain candidate regions. In the training pipeline, augmentation can be performed offline in the cloud or at the edge gateway to reduce the burden on the terminal.

[0079] Understandably, the introduction of the single-sample online augmentation module 13 aims to address the generalization problems caused by small on-site samples, occlusion, and sudden changes in lighting. While traditional offline augmentation can increase data diversity, it often fails to cover the unique characteristics of each on-site shot. Online augmentation, on the other hand, can dynamically adjust the augmentation type and intensity based on the real-time features of candidate regions (such as texture details, contrast, and occlusion levels), thereby generating training or test samples that more closely resemble the on-site distribution. In implementation, this module serves both as a means of training data augmentation and as a robust augmentation tool during inference: various transformations are widely applied during the training phase to improve the model's tolerance to deformation and noise; during the inference phase, low-cost transformations are applied to candidate regions with unstable confidence, and the results are fused to reduce isolated false positives or false negatives. In engineering, it is necessary to balance the augmentation benefits with computational costs, so a layered strategy is typically adopted: large-scale augmentation and model training are first performed in the cloud, and then the distilled lightweight model is distributed to the terminal; the terminal only enables the lightweight TTA when necessary, and in conjunction with a time window strategy (e.g., only when multiple consecutive frames have low confidence) to ensure that real-time performance and energy consumption constraints are not broken. In addition, the traceability of the original sample and enhancement parameters must be maintained during the enhancement process to facilitate subsequent manual verification and sample reuse.

[0080] In this embodiment of the application, the perception module 14 is used to receive enhanced candidate region samples, extract features through a lightweight feature extraction network, and determine whether the candidate region includes valid power equipment. If the candidate region includes valid power equipment, the defect bounding box, defect category and classification confidence are output.

[0081] Specifically, the perception module 14 consists of a lightweight backbone network (e.g., FBNetV3 / MobileNetV3 subnets searched through differentiable architectures or hand-designed) and an ultra-lightweight detection / classification head (e.g., a YOLOX-Tiny+Mobile-Former classifier). First, it performs multi-scale feature extraction on each candidate region and executes target regression and class prediction. For multi-view input from TTA, the perception module can return multiple candidate boxes and confidence scores, and perform NMS or time-based fusion on them. The output annotates each final candidate box with a class probability vector, detection confidence score, box coordinates, and a feature description vector (such as intermediate layer features or patch embeddings) for subsequent time-based / anomaly fusion. The perception module 14 also implements lightweight post-processing logic (e.g., fast NMS, soft NMS, confidence calibration) to adapt to real-time operation on the edge.

[0082] Understandably, the perception module 14 is the core decision-making unit that transforms "candidates" into "determinable objects": its design goal is to maintain high recall while reducing false positives under limited computing power. To this end, the perception module adopts several engineering optimizations: First, it uses a combination of a lightweight backbone and an efficient detection head, along with structured pruning, channel pruning, and quantized perceptual training (QAT), to achieve low-latency forward inference on the NPU / embedded CPU; second, it performs size normalization and multi-scale pyramid processing on the input candidate regions to balance the detection capabilities of large-sized devices and small-scale defects; third, the output not only includes detection boxes and confidence scores, but also carries deep feature vectors that can be used for anomaly scoring (such as local patch embedding or bottleneck layer activation), so that the temporal and anomaly fusion modules can perform more refined unsupervised anomaly discrimination; fourth, moderate TTA fusion and confidence calibration are allowed in the inference process, but the default strategy is to prioritize ensuring that the end-to-end latency of a single frame meets real-time indicators, and only call more expensive fusion strategies in cases of low confidence or uncertainty. Finally, to support on-site model updates, the perception module needs to be able to output valuable metadata of false positive samples and low-confidence samples as candidate data to be uploaded to the cloud for incremental training and distillation. This will continuously improve model performance and shorten the engineering implementation cycle.

[0083] In this embodiment, the temporal and anomaly fusion module 15 is used to perform fusion processing on defect targets at the same spatial location, smooth the confidence sequence of the same target in a consecutive preset number of frames to output the smoothed confidence; it is also used to extract features from the image region corresponding to the defect box in the current frame and calculate the normalized anomaly score.

[0084] Specifically, the temporal and anomaly fusion module 15 first performs cross-frame association of candidate boxes from the perception module based on spatial coordinates and timestamps (using IoU, position projection, or feature-based matching). It then applies an extended temporal convolutional network (ED-TCN) or a lightweight LSTM / TCN to a series of confidence scores associated with the same physical target to obtain smoothed confidence scores. Simultaneously, the temporal and anomaly fusion module 15 performs unsupervised anomaly scoring on the local image or perception features corresponding to the current candidate box (e.g., density / distance measurement of patch features based on a reversible flow model or PatchCore) to obtain a normalized anomaly score. The temporal and anomaly fusion module 15 uses the smoothed confidence score and the normalized anomaly score as input to the decision module 16 and can record the confidence time series to generate a confidence trend map or for manual review.

[0085] Understandably, the temporal and anomaly fusion module 15 plays a dual role in improving detection stability and discovering novel defects: First, by smoothing the temporal sequence, it eliminates instantaneous false alarms or missed alarms caused by single-frame noise, so that high-priority alarms are triggered only when high-confidence signals appear in multiple consecutive frames, which is crucial for controlling the false alarm rate of the inspection platform; Second, the anomaly scoring supplements the blind spots of the supervised classifier through unsupervised methods—for example, when a certain novel corrosion or thermal deviation does not appear in the training set, the model may not be able to directly give a high classification confidence, but the anomaly scoring can indicate that "the patch deviates significantly from the normal distribution in the feature space," thus serving as an additional basis for alarms. In implementation, cross-frame association needs to handle target occlusion, viewpoint changes, and inter-frame drift. Therefore, it is usually combined with spatial projection and extrinsic parameters (GPS / IMU) for multimodal association. To ensure real-time performance on the edge, anomaly detection adopts patch-level fast feature comparison or lightweight flow model, leaving more complex anomaly analysis to cloud offline processing. The timing and anomaly fusion module 15 also undertakes the task of recording historical confidence curves and anomaly scores. These sequence data are important basis for realizing adaptive weight generation (such as dynamic adjustment of λ) and subsequent policy optimization.

[0086] In this embodiment, the decision module 16 is used to calculate the final comprehensive score based on the smoothed confidence score and the normalized outlier score, and to generate an alarm message when the final comprehensive score is higher than a preset threshold. The formula for calculating the final comprehensive score is as follows:

[0087] S=λ·P′+(1-λ)·A

[0088] Where S is the final comprehensive score, P′ is the smoothed confidence score, A is the normalized outlier score, and λ∈[0,1] is the weighting coefficient.

[0089] Specifically, the decision module 16 first receives the smoothed confidence score and normalized anomaly score from the time series and anomaly fusion module 15, and calculates the comprehensive score S according to the preset or adaptively generated weight coefficient λ. When S ≥ τ (threshold τ, which can be issued by the backend or adaptively adjusted), the decision module 16 triggers a local alarm, generates a structured alarm package (including timestamp, GPS, candidate box coordinates, defect category, time series of smoothed confidence score and normalized anomaly score, schematic diagram and confidence curve), and uploads the alarm to the edge gateway through the communication channel. At the same time, according to the strategy, samples with low confidence but high anomaly scores or samples in the uncertainty range are reported in batches to the cloud platform for manual review and incremental annotation. The decision module 16 also provides an interface to adjust λ, τ and the reporting strategy, and records decision logs for subsequent evaluation and model optimization.

[0090] Understandably, decision module 16 is the final checkpoint for synthesizing multi-source evidence into executable maintenance actions: the same-direction fusion form of the above formula ensures that classification confidence (P') and anomaly indication (A) reinforce defect indication in the same direction (the larger the value, the more likely it is a defect), avoiding the logical paradox caused by opposite-direction coupling. The value of λ can be a fixed strategy parameter or dynamically generated by the adaptive submodule (based on environmental variables such as on-site lighting, wind speed, and historical false alarm rate), thereby balancing the contributions of the classifier and anomaly detection under different operating conditions. In addition, decision module 16 needs to consider both real-time performance and robustness: when S is slightly high but the target is in an uncertain range, the module can first generate a low-priority alarm and trigger a delayed review (e.g., requesting additional frame confirmation or enabling TTA), while when S is significantly higher than the threshold, a high-priority alarm is generated and reported immediately. In terms of engineering, the decision module 16 is also responsible for the local buffering, encryption and transmission retry mechanism of alarms to ensure that important alarm records are not lost when the edge network is unstable. At the same time, the decision module 16 provides decision confidence logs and sample metadata to the cloud to support subsequent manual review, model retraining and policy adjustment, thereby forming a closed-loop adaptive capability.

[0091] In this embodiment, the dynamic enhancement operator set includes: Gaussian blur operator, median blur operator, random erase operator, superpixel perturbation operator, pixel additive noise operator, and affine or perspective distortion operator. The single-sample online enhancement module applies different operators with preset probabilities to generate enhanced candidate region samples.

[0092] Specifically, the single-sample online enhancement module maintains a set of configurable enhancement operators (e.g., Gaussian blur operator, median blur operator, random erase operator, superpixel perturbation operator, pixel additive noise operator, affine / perspective distortion operator). For each candidate ROI passed from the preprocessing module, the module performs enhancement operations according to the following pipeline: First, it samples one to several operators (repetition is allowed) for the ROI based on the current strategy and a random number generator, with each operator being triggered with its own probability; second, it applies the selected operators to the ROI in a predetermined or random order to generate an enhanced sample view (enhancement can be performed on a copy of the original ROI to preserve the original data); for superpixel perturbations, the module first segments the ROI into superpixels (e.g., SLIC or other lightweight segmentation methods), and then applies brightness / hue perturbations or texture replacement to the selected superpixel blocks; for random erasure, the module randomly samples rectangular occlusion areas within the ROI and fills them with local statistical noise or background patches; for affine / perspective distortion, the module uses geometric transformation mapping to maintain pixel consistency and performs boundary padding after the transformation. The module also supports Test-Time Augmentation (TTA): When TTA is triggered during the inference phase (typically by low-confidence or uncertain samples), several augmented views are generated in parallel for the same ROI and fed into the perception module. The perception results are then fused (e.g., arithmetic average, weighted average, or voting). The type, order, parameters, and random seed of all augmentation operations are recorded as metadata and saved with the samples for backtracking, synchronizing edge-cloud strategies, and use in subsequent offline training.

[0093] Please refer to the following: Figures 2a to 2f . Figure 2a The visual image after Gaussian blurring. Figure 2b The visual image after median blurring. Figure 2c The visual image after random erasure. Figure 2d The visual image after superpixel processing. Figure 2e The visual image after pixel-level enhancement. Figure 2f This is the visual image after the image has been distorted.

[0094] Understandably, the core purpose of a single-sample online augmentation module is to dynamically expand the performance of a single candidate region during training and inference with minimal additional computational overhead and controllable physical plausibility on the edge, thereby alleviating generalization problems caused by small on-site samples, occlusion, and sudden changes in illumination. Implementation requires a trade-off between "augmentation benefits" and "computational / energy costs": online augmentation during the training phase is mainly used to expand the sample distribution and improve model robustness, while during the inference phase, it typically uses triggered TTA, only activating for low-confidence or uncertain ROIs to avoid significantly increasing latency and power consumption. To ensure the realism of the augmented samples, extreme transformations (such as excessively large scales or excessive noise) should be limited, and physically plausible transformations should be prioritized (e.g., local superpixel perturbations more closely approximate surface damage, affine distortion simulates changes in shooting angle). Furthermore, when multimodal data such as thermal images are present, the augmentation strategy must consider intermodal consistency—for example, if the visible light region is occluded or distorted, the temperature characteristics of the corresponding thermal image region should not be arbitrarily destroyed; if necessary, the thermal image branch of the TTA should be parallel and maintain its original state or undergo temperature-consistent transformation. The module should retain complete metadata (including operators, parameters, and random seeds) of the original ROI and augmented samples, enabling the cloud to perform offline quality measurement and augmentation strategy iteration based on this information. Furthermore, leveraging the capabilities of edge gateways, more complex or computationally intensive augmentations can be outsourced to the gateway or cloud for batch processing, thereby gradually optimizing augmentation distribution while ensuring real-time performance on the terminal. Finally, to avoid augmentations introducing false defects or masking real defects, augmentation strategies should undergo offline verification (A / B testing) and expert review before design and deployment to ensure that augmentations improve recall without significantly increasing the false positive rate. During operation, the frequency of augmentation triggering, TTA cost, and model performance changes should be continuously monitored as feedback signals for adaptive strategy adjustment.

[0095] In one embodiment of this application, the temporal and anomaly fusion module 15 is further configured to: apply an extended temporal convolutional network to smooth the confidence sequence from the perception module, wherein the output formula of the extended temporal convolution is:

[0096]

[0097] Among them, y t This represents the smoothed output value at time point t, i.e., the defect detection confidence after temporal fusion processing, where d is the dilation factor, K is the convolution kernel width, and w n denoted as the weight coefficients of the convolution kernel, x represents the original confidence sequence from the perception module, t represents the current time point, and td·n represents the position d·n time steps backward from the current time point t, i.e., the selected historical data point.

[0098] Specifically, after receiving the candidate bounding boxes and their confidence scores output by the perception module 14, the temporal and anomaly fusion module 15 first performs cross-frame association based on the spatial location of the candidate bounding boxes in the image plane or geographic coordinate system (e.g., matching strategies based on IoU, spatial projection, or feature similarity), constructing a confidence score sequence for each associated physical target. Subsequently, a pre-trained / online fine-tuned dilated temporal convolutional network (ED-TCN) is applied to this sequence: the ED-TCN running on the edge uses a one-dimensional convolutional kernel and extracts contextual information at different time scales according to a set dilation factor d and kernel width K, with the network output serving as the smoothed confidence score. To adapt to edge resource constraints, this module supports pruning of network depth and width (based on OFA / pruned light quantum network), and employs padding or history replay strategies to maintain stable output when the input sequence is insufficient; in TTA or multi-view parallel processing, the module also supports fusing the time series results of multiple views (e.g., weighted averaging) to obtain a more robust result. The convolutional kernel weights of ED-TCN can be obtained through offline training or fine-tuned based on on-site false alarm feedback; the module provides a configuration interface for latency / window size to make engineering trade-offs between real-time performance and smoothness.

[0099] Understandably, the core objective of using dilated temporal convolution for confidence smoothing is to suppress isolated noise and amplify persistent signals by leveraging temporal context, thereby significantly reducing the impact of sudden false alarms or momentary missed detections on alarm decisions. Dilated convolution, by sampling historical frames in a skip-based manner (controlled by the dilation factor d), can cover a longer time range while maintaining the complexity of shallow networks, capturing both short-term jitter and responding to long-term trends; this is particularly important for confidence fluctuations caused by conductor swaying, camera jitter, or drone attitude changes in inspection scenarios. In engineering implementation, it is necessary to balance the robustness of sequence association (matching strategies under occlusion, viewpoint changes, and frame rate fluctuations) with edge latency budget. Therefore, it is often combined with spatial projection and feature similarity constraints for cross-frame association, and a backoff strategy (such as using the most recent valid confidence or weighted empty frames) is adopted when association fails or short-term frame loss occurs. In addition, temporal smoothing is not a panacea: for fast, sudden, and real fault events (such as instantaneous arcs), excessive smoothing should be avoided to prevent delayed alarms. Therefore, the system can set differentiated response thresholds for different types of events or confidence rise rates. Finally, to support online operation and maintenance and model improvement, the module should record the comparison data between the original confidence sequence and the smoothed curve, as a basis for adjusting the judgment threshold, retraining ED-TCN, or optimizing the alarm priority strategy.

[0100] In this embodiment of the application, the preprocessing module 12 is also used to perform coarse registration between the thermal imaging data and the visual image when the acquisition module 11 acquires thermal imaging data at the same time, and generate candidate target regions based on the registered thermal imaging data.

[0101] Specifically, the preprocessing module 12 first timestamps and aligns the visible light image and thermal imaging frame from the acquisition module 11 with metadata. Then, it employs a feasible cross-domain registration process: first, a lightweight feature detector extracts matching points in both modalities; then, RANSAC is used to estimate the homography / affine transformation matrix or a projection mapping based on depth information; and finally, a geometric transformation is performed on the thermal image to align it pixel-level with the visible light frame. After registration, the preprocessing module generates thermal priority candidate regions based on the temperature distribution of the thermal image (e.g., a set of hotspots above the background or local temperature differences), and performs cross-validation with the visible light segmentation / edge detection results to output the final set of candidate target regions. The candidate regions also carry source labels (visible light only, thermal image only, or fusion) and preliminary confidence estimates, so that the downstream sensing module can adopt different strategies (e.g., prioritizing the use of regions indicated by the thermal image for TTA or improving recall calibration). The registration process also includes steps such as multi-resolution interpolation, temperature-pixel mapping, and interpolation filtering to reduce errors caused by differences in modal resolution and sampling frequency.

[0102] It is understandable that using thermal imaging as an auxiliary channel for visible light processing can significantly improve the detection rate of faults with significant temperature differences (such as localized heating, poor contact, or hot spot corrosion) and targets in occluded scenes; however, cross-modal registration presents several engineering challenges, thus requiring careful consideration in preprocessing design: First, thermal images and visible light typically differ in resolution, field of view, and imaging delay, so the registration step must be able to tolerate scale and rotation differences and rigorously screen matching points (robust estimation algorithms such as RANSAC are necessary); second, thermal images have weaker texture information, directly... Traditional keypoint detectors may not perform well; therefore, deep learning-based cross-domain matchers or pseudo-color mapping of the thermal image can be used to enhance structural information. Furthermore, the reliability of thermal images fluctuates under different weather, day / night, or reflection conditions. The preprocessing module needs to evaluate the thermal image quality (e.g., signal-to-noise ratio, temperature distribution uniformity) and fall back to the visible light path only if the quality is insufficient. Additionally, to avoid false regions introduced by registration errors, the final candidate generation typically combines thermal image cues with visible light segmentation cross-validation, increasing candidate confidence only when both or multiple pieces of evidence are consistent. In engineering implementation, a balance needs to be struck between the complexity of edge registration and gateway / cloud capabilities. Complex registration or registration failure analysis can be outsourced to edge gateways or cloud batch processing to reduce the terminal load and provide registration quality feedback and parameter updates at the backend.

[0103] In this embodiment of the application, the preprocessing module 12 is further configured to: detect the imaging quality of thermal imaging data based on signal-to-noise ratio, contrast and temperature distribution uniformity; when the imaging quality of thermal imaging data is detected to be lower than a preset threshold, perform image enhancement and dehazing processing on the visual image sequence, and generate a set of candidate target regions based on the processed image.

[0104] Specifically, the preprocessing module 12 calculates several quality indicators for each frame of thermal imaging data: (1) Signal-to-noise ratio (SNR), which can be the ratio of local signal mean to noise variance or the noise energy estimated based on smoothing filter residuals; (2) Thermal contrast or temperature difference significance, which measures the temperature difference between hotspots and background.

[0105] (3) Temperature distribution uniformity or saturation index: Detects whether there are a large number of saturation values ​​or sensor failure areas in the thermal image. If any index is below a preset threshold, or the joint score of multiple indexes indicates insufficient thermal image quality, a thermal image degradation strategy is triggered: The preprocessing module will abandon or reduce the weight of the thermal image in candidate generation, and instead perform stronger image enhancement (e.g., adaptive dehazing, local contrast enhancement and sharpening) and multi-scale segmentation on the visible light image sequence to ensure that at least a high-recall candidate target region is generated from the visual channel; the module will also annotate the thermal image quality information as metadata to the candidate region for reference in subsequent decision-making and cloud review. This quality evaluation and degradation strategy can be adjusted according to the on-site strategy and supports the edge gateway to issue update rules.

[0106] Understandably, while thermal imaging can supplement the blind spots of visible light in many situations, its image quality is significantly affected by the sensor, environment, and maintenance status—for example, sensor noise, improper cooling, extremely low ambient temperature differences, or strong reflections can all render thermal image information unusable or misleading detection. To ensure system robustness, the preprocessing module must automatically and promptly assess the usability of thermal images and gracefully degrade to a visible light-only detection path when unreliable; this strategy avoids false alarms caused by incorrectly presenting low-quality thermal information as high-confidence evidence. Quality assessment metrics should balance sensitivity to real thermal anomalies with robustness to noise: SNR and temperature difference directly reflect the effective signal in the thermal image, while the uniformity of temperature distribution helps identify adverse situations such as sensor saturation or excessively small temperature differences across the scene. The degradation strategy should not simply ignore thermal images but should also trigger a stronger enhancement process for the visible light channel and record the thermal image quality so that the cloud can consider these differential samples during subsequent batch training or expert review. In engineering, quality assessment thresholds and degradation logic should be determined through scenario-based A / B testing and expert verification, and can be continuously optimized over time through a closed-loop reporting mechanism, thereby balancing accuracy, recall and system availability in actual operation and maintenance.

[0107] In one embodiment of this application, the perception module 14 is further used to fuse the detection confidence with the category probability vector output by the classification module to obtain a fused confidence. The formula for calculating the fused confidence is as follows:

[0108] P fused =β·P + (1-β)·max(q)

[0109] Among them, P fusedP is the fusion confidence, q is the class probability vector, max(q) represents the maximum component in q, and β is the fusion coefficient.

[0110] Specifically, after receiving the detection output of the candidate region, the sensing module 14 sequentially performs the following steps to obtain P. fused (1) Confidence calibration—Temperature calibration or Platt calibration is performed on the detection confidence P and the classifier output q respectively to correct systematic bias and ensure the comparability of the two in terms of numerical dimensions; (2) Class probability processing—Softmax is applied to the classifier output vector q if it is not yet normalized, and its maximum element max(q) is taken as the classification support; (3) Fusion strategy selection—

[0111] —P is obtained by linearly weighting P and max(q) with a fixed or adaptive fusion coefficient β (which can be provided by the running strategy or adaptive weight unit). fused (4) Multi-frame processing - When the same candidate region or overlapping detection generates multiple P fused When the final candidate entries are output using NMS (non-maximum suppression) or a confidence-weighted merging strategy, the following steps are taken: (5) Metadata generation – The original P, q, β and calibrator version metadata records used for fusion are reported along with the detection results. For TTA or multi-view scenarios, the sensing module first calculates P independently for each view. fused These values ​​are then averaged or weighted to obtain the final fusion confidence score for use by the timing module. All steps implemented on the edge should employ low-overhead operations (such as vectorized softmax, pre-computed tables, or lightweight calibration models) to meet real-time requirements.

[0112] Understandably, the initial intention of fusing detection confidence and classification probability is to leverage the complementary information from the detection head and classification head to improve the robustness of the final confidence level: the detection head (P) is typically sensitive to the existence and location of the target, quickly providing target candidates and coarse confidence; the classifier (q) is more sensitive to category discrimination and detailed features, and its maximum probability reflects the strength of evidence that the candidate "belongs to a certain defect category." When both are consistent, fusion can amplify the true signal; when they conflict, fusion can suppress the misleading influence of a single source. In implementation, attention must be paid to numerical comparability: the original detection confidence and classification probability often have different distributions and biases. Direct linear fusion without prior calibration will introduce bias. Therefore, lightweight calibration (e.g., binning calibration or a small calibration network) should be implemented at the end, and calibration parameters should be included in model version management for iteration. Furthermore, merging strategies under multiple bounding boxes and views need to be considered—especially for targets of equal length like wires, where different boxes overlap significantly and local details differ greatly. Simply taking the maximum value may lead to information loss; therefore, NMS and confidence-weighted merging are commonly used methods. Finally, the fusion coefficient β can be statically set to an empirical value to achieve stable performance, or it can be adjusted through an online strategy or an adaptive weight generator to cope with changes in day / night, fog, or thermal imaging availability. Regardless of the strategy, the fusion decision and original evidence should be recorded for future auditing and model retraining, thereby ensuring the interpretability and maintainability of the system.

[0113] In one embodiment of this application, the weight coefficients in the decision module 16 are dynamically generated by a lightweight neural network based on on-site environmental variables, including light intensity, estimated wind speed, historical false alarm rate, and temperature.

[0114] Specifically, the decision module 16 internally implements a lightweight adaptive weight generation unit. This unit receives a standardized environment vector (e.g., the average brightness / contrast index of the current frame, estimated camera / wind-induced jitter intensity, historical short-term false alarm rate statistics of the terminal, device temperature and sensor status codes, etc.), passes it through one or two layers of fully connected networks (with a small number of nodes per layer, enabling BatchNorm and lightweight regularization such as Dropout or L2 regularization) to output a scalar value, which is then normalized by the Sigmoid function to obtain λ∈(0,1). During the training phase, the adaptive weight generation unit can be trained or fine-tuned offline using a sample set with scene labels distributed from the cloud. During the deployment phase, if the edge side can update using statistical data or manual strategies, periodic online fine-tuning or updates distributed by the edge gateway can also be implemented. To ensure stability, the module provides a fault rollback strategy: when the output of the adaptive weight generation unit is unreliable or the environment vector is missing, the system rolls back to a preset static λ value. In addition, to avoid frequent jitter, changes to λ can be exponentially smoothed or a minimum adjustment interval can be set. When implementing the entire computation process on the edge, matrix operation optimization and quantization models are prioritized to control latency and energy consumption.

[0115] Understandably, the significance of adaptive weight generation units lies in making decision-making more flexible and robust in changing environments: in clear, textured scenes, the classification and detection confidence of the perception module is usually more reliable, and the system can relatively increase the weight of P′; conversely, in foggy, jittery, or when thermal imaging information is available but visually blurred, λ should be reduced and more weights allocated to the unsupervised anomaly score A to utilize thermal or anomaly patterns to indicate potential defects. Engineering principles must be followed when constructing adaptive weight generation units: input features must be reliable and can be estimated in real-time on the edge; the network should be as small as possible to reduce computational burden; and the training objective must be aligned with the final operational metrics (such as recall / false alarm balance, average alarm latency). While adaptive weights can improve overall performance, they also bring potential risks: if the environment estimation is inaccurate or the training data distribution is biased, incorrect weight allocation may occur, affecting alarm judgment. Therefore, it is necessary to combine backoff mechanisms, smoothing constraints, and manually configurable strategies, and maintain online monitoring and regular calibration in the cloud. In addition, for auditing and compliance purposes, the historical changes and triggering conditions of λ should be recorded so that operations and maintenance personnel can trace back why a certain alarm was judged as high / low priority, ensuring the interpretability and reliability of the system.

[0116] In one embodiment of this application, the alarm information includes: alarm timestamp, location information, corresponding candidate target region bounding box, defect category label, fusion confidence sequence, normalized anomaly score, and severity level.

[0117] Specifically, when the decision module 16 triggers an alarm, it constructs a structured alarm package, the main fields of which include: (1) alarm ID and timestamp; (2) geographic location information (latitude and longitude, altitude, positioning accuracy, coordinate reference system); (3) original frame ID and pixel bounding box and normalized coordinates of the candidate target area; (4) defect category label and corresponding category probability vector; (5) fusion confidence sequence; (6) normalized anomaly score A and its calculation version information; (7) if available, attach thermal image thumbnail or temperature statistics; (8) metadata (perception model version, enhancement operator information, whether TTA is triggered, fusion coefficient β, adaptive weight λ and threshold τ); (9) evidence package (small-size schematic diagram, heat map / activation map, confidence time series map); (10) security and compliance fields (local signature, encryption identifier and retention policy indication). Alarm packets are serialized in JSON format and saved locally and transmitted encrypted along with compressed image blocks. Based on priority, the system selects three processing strategies: immediate reporting, delayed batch reporting, or local caching only. Multi-channel reporting (e.g., 4G / 5G+edge gateway) is enabled for high-priority alarms.

[0118] Understandably, alarm information serves as both a direct output of on-site automated operation and maintenance decisions and a crucial data source for subsequent manual review, work assignment, and model iteration. Therefore, its design must balance availability, verifiability, and security: On one hand, the evidence (time series, heatmaps, diagrams) in the alarm package must be sufficient to support on-site engineers or cloud auditors in quickly determining priorities and handling solutions, reducing the cost of repeated deployments due to misjudgments; on the other hand, sufficient metadata must be retained for traceability and auditing, such as indicating the model version used to generate the alarm, enhancement strategies, and parameters, to quickly locate the cause and roll back or adjust the model in the event of systemic false alarms. For privacy and compliance reasons, original images or sensitive information in alarms should be hierarchically encrypted or obfuscated before transmission, and the reporting strategy must support local retention periods and automatic cleanup mechanisms to comply with data retention regulations. The transmission strategy must also consider network availability: when the network is unstable, prioritize uploading summaries and low-resolution evidence, and schedule high-resolution data for batch reporting in the background. Finally, the alarm structure should be aligned with the cloud platform's data entry / tracking system (field consistency) to automatically channel alarms into the work order system, manual review queue, and training data pool, completing closed-loop iteration and gradually improving system reliability.

[0119] Figure 3 This is a schematic diagram of a method flow provided in an embodiment of this application, as shown below. Figure 3 The edge intelligence-based power grid inspection method shown includes at least the following steps: S100: Collect visual image sequences and location information at the inspection site; S200: Perform image enhancement and dehazing processing on the visual image sequences, and generate a set of candidate target regions based on the processed images; S300: Apply a set of dynamic enhancement operators to the set of candidate regions of interest to generate enhanced candidate region samples; S400: Receive the enhanced candidate region samples, extract features through a lightweight feature extraction network to determine whether the candidate region includes valid power equipment. If the candidate region includes valid power equipment, output the defect bounding box, defect category, and classification confidence score; S500: Perform fusion processing on defect targets at the same spatial location, and smooth the confidence score sequence of the same target in a consecutive preset number of frames to output a smoothed confidence score; S600: Extract features from the image region corresponding to the defect box in the current frame and calculate a normalized anomaly score; S700: Calculate the final comprehensive score based on the smoothed confidence score and the normalized anomaly score, and generate an alarm message when the final comprehensive score is higher than a preset threshold.

[0120] S100: Collects visual image sequences and location information at the inspection site.

[0121] In this embodiment of the application, the power grid inspection method based on edge intelligence includes collecting visual image sequences and location information at the inspection site in step S100. Please refer to the specific collection method for details. Figure 1The details of 1, 2 and their corresponding descriptions are not repeated here.

[0122] S200: Perform image enhancement and dehazing on the visual image sequence, and generate a set of candidate target regions based on the processed image.

[0123] In this embodiment of the application, the power grid inspection method based on edge intelligence includes performing image enhancement and dehazing processing on the visual image sequence in step S200, and generating a set of candidate target regions based on the processed image. Please refer to the following for details on the processing method. Figure 1 The details of 1, 2 and their corresponding descriptions are not repeated here.

[0124] S300: Apply a set of dynamic enhancement operators to the set of candidate regions of interest to generate enhanced candidate region samples.

[0125] In this embodiment of the application, the power grid inspection method based on edge intelligence includes applying a set of dynamic enhancement operators to the candidate region of interest set in step S300. For details on the processing of the enhanced candidate region samples, please refer to the following documentation. Figure 1 The details of 1, 2 and their corresponding descriptions are not repeated here.

[0126] S400: Receives enhanced candidate region samples, extracts features through a lightweight feature extraction network to determine whether the candidate region includes valid power equipment. If the candidate region includes valid power equipment, it outputs the defect bounding box, defect category, and classification confidence.

[0127] In this embodiment of the application, the power grid inspection method based on edge intelligence includes receiving enhanced candidate region samples in step S400, extracting features through a lightweight feature extraction network to determine whether the candidate region includes valid power equipment. If the candidate region includes valid power equipment, the method outputs the defect bounding box, defect category, and classification confidence score. For details on the processing method, please refer to the relevant documentation. Figure 1 The details of 1, 2 and their corresponding descriptions are not repeated here.

[0128] S500: Performs fusion processing on defect targets at the same spatial location, and smooths the confidence sequence of the same target in a consecutive preset number of frames to output the smoothed confidence.

[0129] In this embodiment of the application, the power grid inspection method based on edge intelligence includes performing fusion processing on defect targets at the same spatial location and smoothing the confidence sequence of the same target in a consecutive preset number of frames to output a smoothed confidence score. For details of the processing method, please refer to the relevant documentation. Figure 1 The details of 1, 2 and their corresponding descriptions are not repeated here.

[0130] S600: Extract features from the image region corresponding to the defect box in the current frame and calculate a normalized anomaly score. In this embodiment, the power grid inspection method based on edge intelligence includes extracting features from the image region corresponding to the defect box in the current frame and calculating a normalized anomaly score in step S600. Please refer to the documentation for the specific calculation method. Figure 1 The details of 1, 2 and their corresponding descriptions are not repeated here.

[0131] S700: Calculates the final comprehensive score based on smoothed confidence and normalized outlier score, and generates an alarm message when the final comprehensive score is higher than a preset threshold.

[0132] In this embodiment of the application, the power grid inspection method based on edge intelligence includes calculating a final comprehensive score based on smoothed confidence and normalized anomaly score in step S700, and generating an alarm message when the final comprehensive score is higher than a preset threshold. Please refer to the specific calculation method for details. Figure 1 The details of 1, 2 and their corresponding descriptions are not repeated here.

[0133] Figure 4 This is terminal 20 provided in one embodiment of this application. For example... Figure 4 As shown, terminal 20 includes at least the following components: processor 21 and memory 22.

[0134] In this embodiment, the memory 22 is used to store executable instructions of the processor 21, which, when configured to execute instructions, implement... Figure 3 The power grid inspection method based on edge intelligence is shown.

[0135] In one embodiment of this application, the program operating in the terminal 20 may be a program that controls a central processing unit (CPU) or similar device to achieve the functions described in the above-described embodiments of the present invention (a program that enables the computer to function). Information processed by these devices is then temporarily stored in random access memory (RAM) during processing, and subsequently stored in various ROMs such as read-only memory (Flash ROM) and hard disk drives (HDDs), and read, corrected, and written by the CPU as needed.

[0136] It should be noted that a portion of the terminal 20 described above can also be implemented using a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and the program recorded on the recording medium can be read into the computer and executed.

[0137] It should be noted that the "computer" mentioned here refers to the computer built into terminal 20, which uses hardware including an operating system and peripheral devices. Furthermore, "computer-readable recording media" refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage devices such as hard drives built into the computer.

[0138] Furthermore, a "computer-readable recording medium" can include: a medium that dynamically stores a program for a short period of time, such as a communication line used when transmitting a program via a network such as the Internet or a communication line such as a telephone line; or a medium that stores a program for a fixed period of time, such as volatile memory inside a computer that serves as a server or client in this case. In addition, the aforementioned program can be a program used to implement the above-mentioned functions, or it can be a program that can implement the above-mentioned functions by combining with programs already recorded in the computer.

[0139] Furthermore, the terminal 20 in the above embodiments can also be implemented as an assembly (device group) composed of multiple devices. Each device constituting the device group can possess some or all of the functions or functional blocks of the terminal 20 in the above embodiments. As a device group, it is sufficient to have all the functions or functional blocks of the terminal 20.

[0140] It is understood that the power grid inspection system, method, and terminal based on edge intelligence provided in this application integrate candidate target region generation, edge-side single-sample online enhancement, lightweight perception, temporal smoothing and unsupervised anomaly scoring, and adaptive decision-making, solving problems such as unclear ROI sources, insufficient small-sample enhancement, and inconsistent confidence / anomaly scoring in existing technologies. Specifically, the preprocessing module explicitly generates candidate ROIs, the single-sample online enhancement module expands sample diversity under training and optional test-time enhancement (TTA), the perception module achieves low-latency and efficient inference on the edge with a lightweight backbone and ultra-lightweight detection head, the temporal and anomaly fusion module smooths confidence through expanded temporal convolution and calculates normalized anomaly scores in parallel, and the decision module provides robust alarms through a co-directional fusion strategy. This solution significantly improves detection robustness and recall capabilities under varying lighting conditions, rain, fog, and small-sample scenarios while ensuring low power consumption and real-time performance, reduces false alarm rate and bandwidth consumption, enhances data privacy protection, and supports continuous iteration and engineering implementation through an edge-cloud closed-loop annotation and model delivery mechanism.

[0141] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.

Claims

1. A power grid inspection system based on edge intelligence, characterized in that, The system includes: The data acquisition module is used to acquire visual image sequences and location information at the inspection site. A preprocessing module is used to perform image enhancement and dehazing processing on the visual image sequence, and generate a set of candidate target regions based on the processed image, wherein each candidate target region corresponds to an image region that may contain power equipment; The single-sample online augmentation module is used to apply a dynamic set of augmentation operators to a set of candidate regions of interest to generate augmented candidate region samples. The perception module is used to receive enhanced candidate region samples, extract features through a lightweight feature extraction network, and determine whether the candidate region includes valid power equipment. If the candidate region includes valid power equipment, the module outputs the defect bounding box, defect category, and classification confidence. The temporal and anomaly fusion module is used to perform fusion processing on defect targets at the same spatial location, smooth the confidence sequence of the same target in a consecutive preset number of frames to output the smoothed confidence; it is also used to extract features from the image region corresponding to the defect box in the current frame and calculate the normalized anomaly score. The decision module is used to calculate a final comprehensive score based on the smoothed confidence score and the normalized outlier score, and to generate an alarm message when the final comprehensive score is higher than a preset threshold. The formula for calculating the final comprehensive score is as follows: S=λ·P′+(1-λ)·A Where S is the final comprehensive score, P′ is the smoothed confidence score, A is the normalized outlier score, and λ∈[0,1] is the weighting coefficient.

2. The power grid inspection system based on edge intelligence according to claim 1, characterized in that, The set of dynamic enhancement operators includes: Gaussian blur operator, median blur operator, random erase operator, superpixel perturbation operator, pixel additive noise operator, and affine or perspective distortion operator; The single-sample online enhancement module generates enhanced candidate region samples by applying different operators according to preset probabilities.

3. The power grid inspection system based on edge intelligence according to claim 1, characterized in that, The timing and anomaly fusion module is also used for: The confidence sequence from the perception module is smoothed using a dilated temporal convolutional network, and the output formula of the dilated temporal convolution is: Among them, y t This represents the smoothed output value at time point t, i.e., the defect detection confidence after temporal fusion processing, where d is the dilation factor, K is the convolution kernel width, and w n denoted as the weight coefficients of the convolution kernel, x represents the original confidence sequence from the perception module, t represents the current time point, and td·n represents the position d·n time steps backward from the current time point t, i.e., the selected historical data point.

4. The power grid inspection system based on edge intelligence according to claim 1, characterized in that, The preprocessing module is also used for: When the acquisition module acquires thermal imaging data simultaneously, it performs coarse registration between the thermal imaging data and the visual image, and generates the candidate target region based on the registered thermal imaging data.

5. The power grid inspection system based on edge intelligence according to claim 4, characterized in that, The preprocessing module is also used for: The imaging quality of the thermal imaging data is detected based on the signal-to-noise ratio, contrast, and temperature distribution uniformity. When the imaging quality of the thermal imaging data is detected to be lower than a preset threshold, image enhancement and dehazing processing are performed on the visual image sequence, and a set of candidate target regions is generated based on the processed image.

6. The power grid inspection system based on edge intelligence according to claim 1, characterized in that, The perception module is also used to fuse the detection confidence score with the category probability vector output by the classification module to obtain a fused confidence score. The formula for calculating the fused confidence score is as follows: P fused =β·P+(1-β)·max(q) Among them, P fused P is the fusion confidence, q is the class probability vector, max(q) represents the maximum component in q, and β is the fusion coefficient.

7. The power grid inspection system based on edge intelligence according to claim 1, characterized in that, The weighting coefficients in the decision-making module are dynamically generated by a lightweight neural network based on on-site environmental variables, including light intensity, estimated wind speed, historical false alarm rate, and temperature.

8. The power grid inspection system based on edge intelligence according to claim 1, characterized in that, The alarm information includes: alarm timestamp, location information, corresponding candidate target area bounding box, defect category label, fusion confidence sequence, normalized anomaly score, and severity level.

9. A power grid inspection method based on edge intelligence, applied to the power grid inspection system based on edge intelligence as described in any one of claims 1 to 8, characterized in that, The method includes: Collect visual image sequences and location information at the inspection site; Image enhancement and dehazing are performed on the visual image sequence, and a set of candidate target regions is generated based on the processed images, wherein each candidate target region corresponds to an image region that may contain power equipment; A dynamic set of enhancement operators is applied to the candidate region of interest set to generate enhanced candidate region samples. The system receives enhanced candidate region samples and extracts features through a lightweight feature extraction network to determine whether the candidate region includes valid power equipment. If the candidate region includes valid power equipment, the system outputs the defect bounding box, defect category, and classification confidence score. The system performs fusion processing on defect targets at the same spatial location, smooths the confidence sequence of the same target in a consecutive preset number of frames to output the smoothed confidence; it is also used to extract features from the image region corresponding to the defect box in the current frame and calculate the normalized anomaly score. The final comprehensive score is calculated based on the smoothed confidence score and the normalized outlier score, and an alarm message is generated when the final comprehensive score is higher than a preset threshold.

10. A terminal, characterized in that, include: processor; as well as The memory stores computer-readable instructions for controlling the processor to execute the power grid inspection method based on edge intelligence as described in claim 9.

Citation Information

Patent Citations

  • Intelligent defect analysis system for joint inspection of transformer substation

    CN116797576A

  • Circuit board defect automatic detection method and system based on machine vision

    CN120253851A

  • Corn abnormal ear detection system based on image recognition

    CN120411652A

  • Utilizing Temporal Indicators to Weight Semantic Values

    US20150293901A1

  • Devices, systems, and methods for topological normalization for anomaly detection

    US20200311944A1

Cited By

  • Wire fault positioning method and system based on multi-source power distribution network connecting device

    CN121414846A

  • A method and system for locating conductor faults based on multi-source distribution network connection devices

    CN121414846B

  • Catenary component thermal defect detection method based on space-time fusion and vehicle speed normalization

    CN121456768A