An edge-oriented power transmission line defect multi-fidelity progressive refinement method

Abstract

The application discloses an edge-oriented power transmission line defect multi-fidelity progressive inspection method, and belongs to the technical field of power transmission line defect detection. The method is oriented to the edge computing environment of the unmanned aerial vehicle side or the nest side, and performs adaptive inspection distribution on the candidate defect area under the constraints of computing power, time delay or energy consumption. The steps include: acquiring an inspection image and performing quality evaluation to obtain an image quality feature vector containing definition, exposure, motion blur and compression noise; outputting a candidate ROI set through a low-cost candidate generator; constructing a multi-fidelity action library to provide actions with different inspection intensities and calculation forms; estimating the cost and gain of each candidate under each action by a cost-gain predictor; and generating an inspection plan by a constraint distribution solver according to a cost-performance ratio sorting or grading strategy under frame-level budget constraints.

Description

Technical Field

[0001] This invention relates to the field of technology, and more particularly to a multi-fidelity progressive inspection method for defects in transmission lines at the edge. Background Technology

[0002] Defect detection in power transmission lines uses inspection images as input to detect and classify defects in components such as insulators, fittings, clamps, and conductors. The core challenges and difficulties in aerial inspections lie in the defects on small targets, fine-grained image differences, and interference from complex backgrounds. Furthermore, edge devices are limited by computing power and real-time performance, resulting in a significant trade-off between accuracy and efficiency. Existing research and engineering methods mainly fall into two categories: manual inspection and automated detection methods based on computer vision technology and deep learning.

[0003] Traditional manual inspection is currently one of the main methods for detecting defects in power transmission lines. While it can provide a relatively intuitive assessment of defects, its drawbacks are also obvious: First, manual inspection is time-consuming and easily affected by various factors such as weather, lighting, and terrain, leading to missed or misjudged defects; second, manual inspection is highly subjective, making it difficult to detect small defects such as small cracks or loose screws, resulting in a high rate of missed detection; most importantly, manual inspection relies on a large labor force, making it very costly and difficult to implement in remote areas.

[0004] In the field of automated defect detection, solutions based on computer vision and deep learning have gradually become mainstream in recent years. These methods primarily utilize deep neural networks to automatically detect defects in power transmission lines using high-resolution images. Their advantage lies in providing detailed detection results, especially with a high recognition rate for subtle defects. However, their disadvantages include high computational demands on high-resolution image processing, slow processing speed, and inability to meet real-time requirements. For edge devices such as drones and drone nests, the computational burden is enormous, thus limiting their practical application on these devices.

[0005] To address the problem of excessive computational burden, some studies have employed low-resolution image processing to reduce computational load. However, while offering faster computation speeds, this comes at the cost of precision and detail, making it easy to miss small defects and resulting in unstable detection results in complex environments. To resolve the trade-off between computational load and accuracy, some studies have proposed coarse-to-fine detection algorithms, which first rapidly detect candidate defect regions from low-resolution images and then perform fine-grained identification on these regions. However, these methods rely excessively on manually set thresholds, lacking flexibility and adaptability, and suffer from severe false positives or false negatives in dynamic environments (such as changes in lighting, occlusion, and motion blur).

[0006] Another shortcoming of existing technologies is the imperfect uncertainty assessment. Many deep learning methods rely on fixed thresholds to determine defects, failing to consider the impact of environment and image quality on the detection results. This makes it difficult to guarantee stable and accurate detection results in complex real-world inspection scenarios. In summary, existing technologies have shortcomings in terms of limited computing resources, high real-time requirements, and accuracy in real-world scene recognition. Especially in edge computing environments, how to achieve efficient and high-accuracy defect detection under limited computing power and bandwidth conditions remains an urgent problem to be solved. Summary of the Invention

[0007] This invention addresses the limitations of existing transmission line defect detection technologies in edge computing environments, including constraints in computing power, latency, and bandwidth. It also addresses issues such as low-resolution detection leading to missed small defects and the reliance on fixed thresholds in the coarse-to-fine process, which lacks dynamic adaptability. The invention provides a multi-fidelity progressive fine-tuning method for transmission line defects at the edge. This method aims to resolve the contradiction between accuracy and efficiency when computing resources are limited, as well as the fixed and unadaptable nature of the fine-tuning process. Through budget-constrained resource allocation optimization and uncertainty-driven fine-tuning upgrades, it improves the accuracy and reliability of defect detection while ensuring computational efficiency.

[0008] This invention proposes a multi-fidelity progressive fine-tuning method for transmission line defects at the edge. It is designed for edge computing environments such as UAVs or UAV nests. Under operational constraints such as computing power, latency, or energy consumption, the fine-tuning calculations for candidate defect regions are adaptively allocated, and the fine-tuning is driven by reliability metrics to upgrade or stop.

[0009] For ease of description, this invention uses symbolic conventions for key objects. The edge end at time... Acquire inspection images And calculate the image quality feature vector. This includes, but is not limited to, metrics such as sharpness, exposure, motion blur, and compression noise. The candidate generator CG... Perform low-cost inference and output a set of candidate ROIs. Each candidate At least includes candidate spatial regions and candidate feature representation And may further include coarse categories. With coarse confidence This is used for subsequent detailed investigation decisions and risk weighting.

[0010] This invention introduces a Multi-Fidelity Action Set (MAS) during the candidate refinement stage to express different refinement intensities and calculation methods that can be applied to the same candidate ROI. The action set is denoted as... Each of the actions Composed of executable parameters, including at least the resolution scaling. (For example , , Cutting expansion factor (Appropriately expand the candidate region to cover the context) and iteration intensity (For example, refining the number of inference rounds or the number of boundary refinement rounds). Different actions correspond to different refinement costs and different potential recognition gains, enabling the refinement process to have a progressive capability from low to high.

[0011] This invention explicitly models motion costs. (Candidate) In action The cost is denoted as This cost can be characterized by both the number of pixels and the iteration intensity, and can be mapped to actual latency or energy consumption through offline calibration at the edge. For example, the cost can be defined as a function positively correlated with the candidate area, cropping expansion, and resampling rate:

[0012]

[0013] in , , These are calibration parameters used to represent the edge cost per unit pixel processing and per unit iteration. The above expression is only an example; in practice, it can be implemented using a lookup table or a linear / piecewise linear approximation. The core point is that the cost can be calculated quickly at runtime and reflect the difference in action intensity.

[0014] Regarding gain, this invention introduces a cost-gain predictor (CGP) to estimate the reliability improvement that performing a specific investigation action on a candidate may result in. Candidate In action The expected gain is denoted as Its value can be determined by candidate features, coarse judgment output, image quality, and motion parameters.

[0015]

[0016] To reflect the principle of prioritizing engineering risks, this invention assigns risk weights to candidates. It can be determined by component type and coarse defect category or obtained by data-driven learning, so that high-risk defects are given priority in detailed inspection calculation in resource competition.

[0017] Based on the quantifiable cost and gain mentioned above, this invention formalizes the precise selection of edge nodes as a constraint allocation problem. Let the edge node be in frame... The operating budget is (This can be one or a combination of the maximum allowable latency, maximum pixel processing volume, maximum number of detailed checks, or energy consumption limit), introducing decision variables. Indicates whether to support the candidate Select Action When a detailed investigation is performed, the detailed investigation allocation satisfies the following objectives and constraints:

[0018]

[0019]

[0020] The significance of this modeling lies in the fact that, under budget constraints, the system no longer relies on fixed thresholds or fixed Top-K rules, but instead allocates detailed investigation resources among candidates based on "risk-weighted unit cost gain." Considering the need for low-overhead solutions at the edge, the constraint allocation solver CAS of this invention can be implemented using approximate solution methods, such as based on cost-effectiveness. By sorting and greedily selecting, or by employing a tiered strategy to first cover more candidates with low-fidelity actions, and then upgrading a small number of high-risk and highly uncertain candidates to higher-fidelity actions, the above approximate solutions can satisfy budget constraints and achieve stable delay control effects within limited computational overhead.

[0021] The fine-check executor FI performs fine-check inference on the selected candidates based on the CAS allocation results, outputting the defect category, location, and optional boundary representations (such as masks or key points). To further improve the effective identification capability of small defects, this invention allows for a "local refinement" approach under high-fidelity operation. That is, an uncertain region indication is first generated from the low-fidelity result, and then higher-resolution inference or stronger iterative refinement is performed only on the local sub-blocks corresponding to the uncertain region, thereby reducing the high-cost computation of the entire ROI and improving the quality of boundary and texture details.

[0022] In determining the results of a detailed investigation, this invention introduces a Reliability Assessment and Upgrade Controller (REC) to elevate a single confidence assessment to a stable, reliable output. For candidate... Based on the detailed analysis output, this invention constructs an uncertainty and consistency metric and generates a reliability score. For example, uncertainty can be measured by the output distribution entropy, and consistency can be measured by the overlap across fidelity or across pruning results, thus yielding:

[0023]

[0024] in To refine the output confidence level, As an indicator of uncertainty, As a consistency indicator, For quality mapping function, For calibrable parameters, For the sigmoid function. This reliability score is used to drive incremental refinement: when Reaching the confirmation threshold Stop the detailed inspection and output "Confirm"; when Below the upgrade threshold Furthermore, when the budget allows, a higher-fidelity action is triggered to upgrade and refine the candidate selection; when the budget is insufficient or the confirmation conditions still cannot be met after the upgrade, a "suspected" output is given, and the candidate ROI, action sequence, and reliability evidence are recorded for verification; for obviously invalid candidates, a "negative" output is given. This forms a more stable three-state output mechanism under complex environments and quality fluctuations, reducing false alarm jitter and improving the verifiability of results.

[0025] Regarding training and calibration, cost model parameters It can be obtained through offline calibration using measured data at the edge; the gain predictor CGP can be trained by constructing supervision signals (such as improved consistency with annotations under different actions, improved IoU, and reduced error rate) based on the offline dataset. It has interpretable statistical significance; reliability model parameters The reliability threshold can be calibrated using a reliability calibration method to ensure consistency with the actual accuracy rate. The above training and calibration methods guarantee that the detailed allocation and upgrade decisions of this invention are not based on a stacking of empirical thresholds, but rather on an edge-constrained reasoning mechanism built upon quantifiable costs, learnable gains, and calibrable reliability.

[0026] Through the above technical solution, the present invention transforms defect detection from fixed configuration reasoning to a constrained progressive fine inspection allocation process at the edge, so that the fine inspection cost adapts to the candidate density and quality conditions and meets the budget constraints, while prioritizing the reliability of high-risk defect identification, and providing stable and verifiable output results in complex inspection scenarios.

[0027] The beneficial effects of this invention are:

[0028] 1. This invention explicitly incorporates the candidate ROI refinement process into budget constraints by introducing a budget into the constraint assignment solver CAS. And the cost given by the cost-gain predictor CGP With gain This enables adaptive allocation of computational resources for detailed screening. Consequently, it maintains stable edge-side overhead when the number of candidates fluctuates, avoiding latency outages or excessive degradation caused by fixed thresholds or fixed Top-K strategies.

[0029] 2. This invention parameterizes the fine-check intensity into a set of executable actions through the Multi-Fidelity Action Library (MAS), supporting progressive fine-checking from low to high, and prioritizing computing resources for high-risk or high-uncertainty candidates, thereby improving the recognition quality of minor defects and complex backgrounds without significantly increasing the overall overhead.

[0030] 3. This invention constructs a reliability-driven upgrade / stop closed loop using a reliability assessment and upgrade controller (REC), introducing reliability... With threshold , When reliability is insufficient and budget allows, an upgraded detailed investigation is triggered. When budget is insufficient or it is still difficult to confirm, a "suspected" result is output and evidence is recorded, thereby reducing false alarm jitter in complex environments and enhancing the verifiability of results.

[0031] 4. This invention can introduce local uncertain region refinement in high-fidelity operations, perform high-cost refinement only on uncertain sub-regions within the ROI and merge the results, and obtain clearer boundary and detail performance under the same budget, further alleviating the contradiction between "limited computing power and detailed requirements" at the edge.

[0032] 5. This invention achieves a closed-loop system of cost calibration, gain learning, and reliability calibration, enabling key parameters to be calibrated and adjusted according to platform and data distribution, reducing reliance on empirical thresholds, and improving the portability and reproducibility of cross-device and cross-scenario deployments. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in this invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0034] Figure 1 This is a schematic diagram of the overall process of a multi-fidelity progressive fine inspection method for transmission line defects at the edge end according to the present invention.

[0035] Where 100 is the input image 110 is the quality assessment module, used to obtain indicators such as sharpness, exposure, motion blur, and compression noise, and generate image quality feature vectors. ;120 is the candidate generator CG, used to perform low-cost inference on the input image and output a set of candidate ROIs; 130 is the multi-fidelity action library MAS, used to provide a set of optional actions with different levels of detail. ;140 is the cost-gain predictor (CGP), used to estimate the cost of each candidate under different actions. (191) and gain (192); 150 is the constraint assignment solver CAS, used for frame-level budgeting. (190) Generate a detailed investigation plan under constraints; 160 is the detailed investigation executor FI, used to perform inference based on the detailed investigation plan; 170 is the reliability assessment and upgrade controller REC, used to calculate reliability. (194) and threshold (195) Compare and output the final judgment result; 180 is the output and evidence recording module, used to output structured results and save suspected evidence.

[0036] Figure 2 This is a block diagram of the functional modules and data flow structure implemented at the edge of the present invention.

[0037] Specifically, the candidate generator CG (120) outputs the candidate set and candidate features to the cost-gain predictor CGP (140) and the constraint assignment solver CAS (150); the multi-fidelity action library MAS (130) provides the action set and action parameters to CGP and CAS; CGP outputs the cost of the candidate-action pairs. (191) and gain (192); CAS receives budget (190) Calculate the cost-effectiveness after input. (193) and form action assignment, output to the fine inspection actuator FI (160); FI outputs the fine inspection results to the reliability assessment and upgrade controller REC (170); REC calculates and outputs the reliability. (194) and escalation / stop decision; all results are ultimately incorporated into the output and evidence record (180).

[0038] Figure 3 This is a schematic diagram of the reliability-driven upgrade and stop decision process of the progressive refinement investigation of this invention.

[0039] like Figure 3 As shown, the reliability assessment and upgrade controller REC is based on reliability. (194) is used as the decision criterion to confirm the threshold. and upgrade threshold (195) serves as a boundary. When Output "Confirm" and stop detailed investigation; when At that time, further assessment of the remaining budget (190) Whether to allow upgrade; if allowed, trigger a higher fidelity action; if budget is insufficient, output "suspected" and record evidence; when In When the timeframe is within a certain range, a conservative strategy should be implemented based on risk weights.

[0040] Figure 4 This is a schematic diagram illustrating the motion levels and parameterization of the Multifidelity Action Library (MAS) of this invention.

[0041] like Figure 4 As shown, the multi-fidelity motion library includes three levels of motion: low, medium, and high. Motion A1 is a low-fidelity motion, with the resolution multiplier set. Cutting expansion coefficient Iteration intensity Action A2 is a medium-fidelity action, set... , , Action A3 is a high-fidelity action, set... , , It also enables fine-tuning of local uncertain areas. The action parameters reflect a monotonic relationship of "increasing cost and increasing reliability".

[0042] Figure 5 This is a schematic diagram illustrating the allocation process for generating a detailed investigation plan under budget constraints according to the present invention.

[0043] like Figure 5 As shown, the system combines the candidate ROI list with the action set. Candidate-action pairs are constructed, and the cost of each pair is output by the cost-gain predictor (CGP). (191) and gain (192) The cost-effectiveness is calculated by the constraint assignment solver CAS. (193) and sorted in descending order in the budget (190) Select each item one by one under the constraint until the condition is met. The final output is a detailed investigation plan that includes candidates and their corresponding actions.

[0044] Figure 6 This is a schematic diagram of the process for refining the local uncertain areas in this invention.

[0045] like Figure 6 As shown, after performing low-fidelity or medium-fidelity fine-tuning on the candidate ROI (200), the system extracts uncertain regions (210) and generates an uncertainty heatmap (220). Based on this, local sub-blocks (230) are selected, and local high-fidelity fine-tuning is performed on the local sub-blocks (240). Finally, the local fine-tuning results are fused with the original ROI fine-tuning results (250) to output fine-tuning results with clearer boundaries and more reliable details.

[0046] Figure 7 This is a schematic diagram of the closed-loop process of cost calibration, gain learning, and reliability calibration of this invention.

[0047] like Figure 7 As shown, the system uses the calibration / training data (300) as a basis to perform cost calibration (310) to obtain cost model parameters (320); constructs a gain supervision signal (330) and completes gain predictor training (340); performs reliability calibration (350) and sets a threshold (360); finally, it deploys the cost mapping, gain model, and threshold parameters to the edge running link through deployment parameter distribution (370). Detailed Implementation

[0048] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should also be noted that, to make the embodiments more comprehensive, the following embodiments are the best and preferred embodiments, and those skilled in the art can use other alternative methods to implement some well-known technologies; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0049] Example 1: Overall Process and Input / Output

[0050] Implementation conditions: Edge computing units are deployed at the drone end or on the drone nest side, and the input is a stream of inspection images. A single frame input is an input image. (100), Example resolution is Candidate number The value fluctuates between 60 and 180 depending on the complexity of the scene.

[0051] Implementation steps: such as Figure 1 As shown, the system first performs a quality assessment. (110). In this embodiment, the specific implementation of quality assessment is as follows: the sharpness index is calculated using the Tenengrad function based on gradient magnitude. The horizontal and vertical gradients of the image are extracted using the Sobel operator, and the mean of the sum of squared gradient magnitudes is calculated as the sharpness score; the exposure index uses the average gray value of the image, and the deviation from the standard exposure range (such as gray value 80-180) is normalized and used as the exposure score; the motion blur index is obtained by analyzing the attenuation degree of high-frequency components in the image spectrum; the compression noise index is obtained by statistically analyzing the consistency of variance between image blocks. The above indices together form a quality feature vector. This information is then used as a quality constraint for subsequent decisions.

[0052] Subsequently, the candidate generator CG(120) completes low-cost candidate generation. "Low-cost inference" refers to using a lightweight neural network backbone (such as MobileNet or EfficientNet-Lite) to perform a single forward propagation on the input image, outputting a set of candidate ROIs, each containing a spatial region. Intermediate layer feature representation Coarse category and coarse confidence .

[0053] The system calls the Multi-Fidelity Actions Library (MAS) (130) to provide an optional set of detailed actions, and the Cost-Gain Predictor (CGP) (140) estimates the cost of each candidate action under different conditions. (191) and gain (192). In frame-level budget Under constraints (190), the constraint assignment solver CAS (150) generates a detailed investigation plan and sends it to the detailed investigation executor FI (160) to perform detailed investigation reasoning. Subsequently, the reliability assessment and upgrade controller REC (170) calculates the reliability of the candidates. (194) and threshold , (195) Compare and output the results of “confirmed / suspected / negative”. Finally, the output and evidence record (180) output the structured results and save the evidence fragments of the suspected items.

[0054] Implementation results: Under the condition of fluctuations in the number of candidates and image quality, this process can keep the latency at the edge controllable through budget constraints and reliability-driven mechanisms, and prioritize the allocation of fine-tuning resources to high-risk candidates, thereby improving the reliability of key defect identification.

[0055] Example 2: Module Structure and Data Interface

[0056] Implementation conditions: The edge is implemented in the form of software modules. Candidate, action and scoring information are passed between modules through memory queues or function interfaces to avoid data copying overhead.

[0057] Implementation steps: such as Figure 2 As shown, the candidate generator CG (120) outputs the candidate set and candidate features to the cost-gain predictor CGP (140) and the constraint assignment solver CAS (150); the multi-fidelity action library MAS (130) provides the action set and action parameters to CGP and CAS. CGP outputs two types of decision quantities: the cost of the candidate-action pair. (191) and gain (192). CAS receives budget (190) After input, calculate the cost-effectiveness for each candidate action pair. (193) and form an action assignment, outputting it to the fine-check executor FI (160). FI outputs the fine-check results to the reliability assessment and upgrade controller REC (170). REC receives the output distribution, candidate features, and image quality features after the current action is executed, and calculates the reliability. (194) and upgrade / stop decision, and finally integrate the confirmed / suspected / negative results and evidence into the output and evidence record (180).

[0058] Implementation Results: Through clearly defined data interfaces and functional boundaries, this invention can achieve closed-loop decision-making on the edge without relying on cloud computing power, which facilitates system integration and modular replacement.

[0059] Example 3: Reliability-Driven Upgrades and Shutdowns

[0060] Implementation conditions: For any candidate ROI, a corresponding reliability score can be obtained after performing a certain fidelity-preserving action. And can check the remaining budget. .

[0061] Implementation steps: such as Figure 3 As shown, REC is based on reliability (194) is used as the decision criterion. In this embodiment, Specifically, this is achieved by meticulously examining the entropy of the category probability distribution output. Calculations show that a higher entropy value indicates greater uncertainty. The reliability model parameters are obtained by comparing the intersection-over-union (IoU) of the bounding boxes output under the current high-fidelity action and the bounding boxes output under the previous low-fidelity action. A higher IoU indicates better consistency. Obtained in advance through offline calibration.

[0062] REC with threshold , (195) serves as the dividing line, in which To confirm the threshold, To upgrade the threshold, and The logic behind the judgment is as follows:

[0063] like If the reliability is sufficient, "Confirm" is output and the detailed investigation is stopped;

[0064] like If the reliability is insufficient, then the remaining budget needs to be further evaluated. (190) Is it sufficient to support a higher fidelity action (at least the action one level above the current action)? If allowed, trigger an upgraded detailed investigation and call the higher fidelity action back to FI for execution. After completion, recalculate the reliability to form a closed loop; if the budget is insufficient, output "suspected" and record "insufficient budget" as well as the current action sequence and evidence fragments;

[0065] like In For intervals, the system combines risk weights. Adopt a conservative strategy: prioritize stopping low-risk candidates and marking them as "negative" or "suspected," and prioritize escalating high-risk candidates within budget limits.

[0066] Implementation Results: This mechanism transforms the output from a single binary confidence level judgment to a stable three-state output of "confirmed / suspected / negative," and limits the refinement upgrade to within the budget, thereby reducing false alarm jitter and ensuring real-time performance at the edge. Evidence records (including candidate ROI cropped images, action sequences, and intermediate reliability scores) provide traceable evidence for subsequent manual review.

[0067] Example 4: Parametric Example of a Multi-Fidelity Action Library

[0068] Implementation conditions: The action library is configured and embedded at the edge; action parameters can be set during deployment or adaptively adjusted according to the scenario. The edge inference engine supports dynamic input size, i.e., it can adapt to different resolution magnifications. The input tensor is efficiently resampled and forward propagated.

[0069] Implementation steps: such as Figure 4 As shown, the motion library contains at least three motion levels: low, medium, and high. The specific meanings of the motion parameters are: resolution scaling. This indicates the factor by which the candidate region cropping blocks are resampled to a multiple of the original resolution; the cropping dilation factor. Indicates the candidate spatial region The proportion of isotropic expansion used to introduce contextual information; iteration intensity. This indicates the number of rounds of iterative refinement of the bounding box in the detailed investigation output or the number of random pruning inferences.

[0070] Example location:

[0071] Action A1 (Low Fidelity), Settings , , This action directly performs lightweight, detailed network inference on the candidate regions at the original resolution, without introducing additional upsampling.

[0072] Action A2 (Medium Fidelity), Settings , , This action upsamples the candidate region to twice the resolution for inference and appropriately expands the context.

[0073] Motion A3 (High Fidelity), Settings , , This action upsamples the candidate region to 4 times the resolution for fine inference and performs two rounds of boundary refinement, while enabling the "refinement of local uncertain regions" function.

[0074] The action library also reflects the monotonic relationship of "cost ↑, reliability ↑", which enables progressive detailed investigation to have clear action-level definitions.

[0075] Implementation results: By using action-level multi-fidelity parameterization, this invention transforms "intensity of detailed investigation" from an abstract concept into a set of configurable and executable actions, providing an operable means of detailed investigation for budget constraint allocation and reliability upgrade.

[0076] Example 5: Budget Constraint Allocation and Detailed Investigation Plan Generation

[0077] Implementation conditions: The edge device sets a budget for each frame. In this embodiment, the budget Specifically, it is expressed as a latency limit, which is dynamically given by the edge hardware real-time scheduler based on the current expected frame rate. For example... ,in For the target frame rate, This represents the fixed overhead for image acquisition and preprocessing. Additionally, the budget can also be represented as a maximum pixel processing limit or a maximum number of refining operations.

[0078] Implementation steps: such as Figure 5 As shown, the system organizes the candidate set into a "candidate ROI list" and associates it with the action set. Candidate-action pairs are constructed. CGP outputs the cost of each pair. (191) and gain (192). CAS calculation of cost-effectiveness (193) and execute according to Sort in descending order, and select item by item under budget constraints. The selection rule is: add the candidate-action pair with the highest cost-effectiveness to the detailed investigation plan in turn, until adding the next item would result in... Stop when the time is right, and output "Detailed Investigation Plan (Candidate + Action)".

[0079] Example location:

[0080] When the number of candidates is large (e.g.) ) or quality assessment When the image quality is indicated as “Medium / Poor” (e.g., low sharpness, strong motion blur), CAS prioritizes assigning high-fidelity action A3 to a small number of high-risk candidates (e.g., candidates corresponding to insulator damage or broken conductor strands), and assigns low-fidelity action A1 or no detailed investigation to the remaining candidates.

[0081] When the number of candidates is small (e.g.) When the quality assessment is "excellent", CAS can assign more candidates to the medium-fidelity action A2 to improve overall reliability.

[0082] Implementation results: This allocation mechanism keeps the cost of detailed inspection under the condition of candidate density fluctuation, and prioritizes the improvement of the reliability of high-risk candidates within a limited budget, avoiding the loss of control over time delay or the omission of small defects caused by fixed thresholds or fixed Top-K at the edge.

[0083] Example 6: Implementation conditions for refining local uncertain regions: After performing low-fidelity or medium-fidelity refining on candidate ROIs, the system can obtain the probability map output by the refining network (such as the probability map of segmentation mask or the category score map of target detection).

[0084] Implementation steps: such as Figure 6As shown, after performing low-fidelity or medium-fidelity detailed inspection on the candidate ROI (200), the system extracts uncertain regions within the ROI based on the detailed inspection output (210). Specifically, based on the probability map of the detailed inspection output, the information entropy of each pixel is calculated to form an uncertainty heatmap. (220). Set a threshold. (In this embodiment) Take 1.2 times the average of the heatmap values, and... Binarization yields an uncertain region mask. Through connected component analysis, the top K largest sub-blocks (K is 3 to 5 in this embodiment) are selected as local sub-blocks (230).

[0085] Then, local high-fidelity refinement (240) is performed on the selected local sub-blocks, for example, by using a higher resolution magnification (such as upsampling the sub-blocks to...). or stronger iteration intensity ( The refinement reasoning only increases computational overhead in a local area. After refinement, the local refinement results are merged with the original ROI refinement results (250). The fusion method can be weighted average (the weight is determined by the uncertainty normalization of the local sub-block) or logical maximum value (take the one with higher confidence between the refinement result and the global result) to obtain a more clearly defined and more reliable output.

[0086] Implementation results: This method focuses high-fidelity calculations on areas with insufficient information, which can improve the reliability of identifying minor defects (such as cracks and broken strands) and reduce unnecessary high-cost calculations for the overall ROI under budget constraints.

[0087] Example 7: Cost Calibration, Gain Learning, and Reliability Calibration Closed Loop

[0088] Implementation conditions: Before system deployment, parameter calibration must be performed using offline data and edge-end measurements. The edge-end testing environment must be consistent with the operating environment (same CPU / GPU / NPU model, same inference framework version).

[0089] Implementation steps: such as Figure 7 As shown, the system constructs a deployment closed loop based on the calibration / training data (300).

[0090] First, perform cost calibration (310): for different combinations of motion parameters ( , , ) and different ROI area ranges (e.g. , (etc.), the average latency / energy consumption of 100 inference iterations at the edge is collected as the measured cost. A least-squares fitting formula is used. The cost model parameters (320) are obtained (or a lookup table mapping is directly established) for fast cost estimation of candidate-action pairs at runtime.

[0091] Next, the gain supervision signal is constructed (330): based on the real defects labeled in the offline dataset, the improvement in label consistency (such as F1 score improvement), IoU improvement, or false positive rate reduction under different actions relative to the coarse detection stage is calculated, which serves as the supervision label for the gain predictor CGP. The gain predictor training (340) is completed to make the gain estimation statistically significant.

[0092] Finally, reliability calibration (350) is performed. Isotonic regression is used to calibrate the confidence score output by the precision network against the true accuracy. Based on this calibration, a threshold (360) is set according to the calibrated reliability score to ensure consistency between the reliability score and the actual accuracy, thereby determining... , Available values ​​at different quality levels. After completing the above process, the cost mapping, gain model and threshold parameters are deployed to the edge running link through deployment parameter distribution (370).

[0093] Implementation Results: Through the aforementioned closed-loop calibration and adjustment, budget constraint allocation and reliability-driven upgrades are reproducible and transferable across different platforms and scenarios, avoiding cross-device performance degradation caused by relying on empirical thresholds. This invention covers any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments, while those skilled in the art can fully understand the invention without these details. Furthermore, to avoid unnecessary confusion regarding the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0094] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A multi-fidelity progressive fine inspection method for defects in transmission lines oriented towards edge ends, characterized in that, Includes the following steps: Step S1, obtain the edge endpoint at time... Inspection images For the inspection image Perform quality assessment to obtain image quality feature vectors. The image quality feature vector includes sharpness index, exposure index, motion blur index and compression noise index; Step S2, use the candidate generator CG to process the inspection image. Perform low-cost inference and output a set of candidate ROIs. Each candidate At least includes candidate spatial regions and candidate feature representation The candidate feature characterization Output from the intermediate layer of the candidate generator CG; Step S3: Construct a multi-fidelity action set Each action Composed of executable parameters, including at least the resolution scaling. Cutting expansion factor and iteration intensity ; Step S4, for each candidate and each action Calculation cost The gain is estimated using the cost-gain predictor (CGP). ,in and These represent the coarse category and coarse confidence level output by the candidate generator CG, respectively. Step S5, Set frame-level budget Introducing decision variables To maximize the risk-weighted total gain, the detailed investigation allocation plan is solved under budget constraints: in The risk weights for the candidates are determined by the candidate... The component type and the category of coarse defects are used to determine this; Step S6: The detailed inspection executor FI performs detailed inspection reasoning on the selected candidates according to the detailed inspection allocation plan, and outputs the defect category, location, and optional boundary representation. Step S7: Reliability assessment and upgrade controller REC calculates the reliability score of the detailed inspection output. Based on the relationship between the reliability score and the preset threshold, the system outputs a "confirmed", "suspected", or "negative" result, and triggers an upgraded detailed investigation when the reliability is insufficient and the budget allows.

2. The multi-fidelity progressive fine inspection method for transmission line defects oriented towards the edge end according to claim 1, characterized in that, The sharpness index in step S1 is calculated based on the image gradient magnitude; the exposure index is calculated based on the deviation between the image's average grayscale value and the standard exposure range; the motion blur index is calculated based on the attenuation of high-frequency components in the image spectrum; and the compression noise index is calculated based on the inter-block variance statistics. The quality assessment is related to the image quality feature vectors in steps S4 and S7. Correspondingly, the quality feature vector is used for gain prediction and reliability score calculation.

3. The multi-fidelity progressive fine inspection method for transmission line defects oriented towards the edge end according to claim 1, characterized in that, The resolution magnification of the motion parameters mentioned in step S3 Pick , or Cutting expansion coefficient Pick , or Iteration intensity Pick or .

4. The multi-fidelity progressive fine inspection method for transmission line defects oriented towards edge ends according to claim 1, characterized in that, The cost mentioned in step S4 Calculate according to the following formula: in , , The cost model parameters are obtained through offline calibration using measured data from the edge. The area of ​​the candidate spatial region.

5. The multi-fidelity progressive fine inspection method for transmission line defects oriented towards edge ends according to claim 1, characterized in that, The constraint assignment solver CAS described in step S5 adopts a greedy approximation solution method, based on cost-effectiveness. Sort in descending order and select items one by one under budget constraints until the remaining budget cannot meet the next minimum cost action; or adopt a tiered strategy, first cover more candidates with low-fidelity actions, and then upgrade high-risk and high-uncertainty candidates within the remaining budget to higher-fidelity actions.

6. The multi-fidelity progressive fine inspection method for transmission line defects oriented towards edge ends according to claim 1, characterized in that, The reliability score described in step S7 Construct it according to the following formula: in To refine the output confidence level, As an uncertainty indicator, the probability distribution entropy output by the detailed investigation. measure; The consistency metric is the intersection-over-union ratio (IoU) between the output of the current fidelity action and the output of the previous fidelity action or different clipping offsets. This is a quality mapping function that maps quality feature vectors to quality scores. For calibrable parameters, This is the sigmoid function.

7. The multi-fidelity progressive fine inspection method for transmission line defects at the edge end as described in claim 1, characterized in that, The decision logic of the reliability assessment and upgrade controller REC in step S7 is as follows: when Reaching the confirmation threshold Stop the detailed inspection and output "Confirm"; when Below the upgrade threshold If the remaining budget allows, trigger an upgraded, more detailed investigation of the candidate selection using higher-fidelity actions; if the budget is insufficient or the confirmation criteria cannot be met after the upgrade, output "suspected" and record the candidate ROI, action sequence, and reliability evidence; when Below the negation threshold Output "negation" when the time is right. .

8. The multi-fidelity progressive fine inspection method for transmission line defects oriented towards edge ends according to claim 1, characterized in that, In step S6, the fine-tuning actuator FI employs local uncertainty region refinement under high-fidelity operation: it calculates the information entropy of each pixel based on the probability map of the fine-tuning output, forming an uncertainty heatmap. Through threshold The uncertainty heatmap is binarized to obtain an uncertainty region mask. The top K sub-blocks with the largest areas are selected as local sub-blocks through connected component analysis. Refinement inference with higher resolution or stronger iteration is performed only on the local sub-blocks. The refinement results are then fused with the global results through weighted average or logical maximum value.

9. The multi-fidelity progressive fine inspection method for transmission line defects oriented towards the edge end according to claim 1, characterized in that, It also includes pre-deployment cost calibration, gain predictor training, and reliability calibration steps, where cost model parameters... The model parameters were calibrated using least-squares fitting based on measured data from the edge; the gain predictor (CGP) was trained using the improvement in annotation consistency, IoU improvement, or error rate reduction under multi-fidelity actions as supervisory signals; and the reliability model parameters were... The reliability threshold is calibrated using isotonic regression or orthogonal regression to ensure consistency with the actual accuracy.

Images