A power transmission line video panoramic fusion online monitoring system

Abstract

The application discloses a kind of transmission line video panorama fusion online monitoring systems, the system includes: the video acquisition unit, for obtaining transmission line along line continuous monitoring video, and the monitoring video is executed time synchronization, imaging correction, key frame extraction, view overlap identification and local video segment division;The structure prior construction unit is used to analyze the tower main body, cross arm, insulator string, wire, hardware connection area and passageway boundary in local video segment, and the application relates to the technical field of power system intelligent inspection.This kind of transmission line video panorama fusion online monitoring system, the dynamic change in video is classified, which is divided into environmental disturbance change, imaging disturbance change and line real state change, and is respectively given inhibition or retention mark, so as to effectively inhibit the influence of non-target disturbance on fusion result, improve the stability of system in complex environment.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection of power systems, and more specifically, to an online monitoring system for transmission lines that integrates video panoramic fusion. Background Technology

[0002] With the continuous expansion of the power grid, transmission lines are characterized by cross-regional, long-distance, and complex environmental distribution. Traditional methods relying on manual inspections or single-point monitoring equipment are no longer sufficient to meet the needs of efficient, continuous, and refined operation and maintenance. In recent years, online inspection technology for transmission lines based on video monitoring has been gradually applied. By deploying camera equipment on towers, corridors, or mobile carriers, real-time collection and analysis of the line's operating status can be achieved.

[0003] However, existing technologies still have the following problems in using video data for panoramic construction and condition monitoring of transmission lines:

[0004] Video stitching suffers from several problems. First, the stability of the stitching is insufficient. Due to the elongated, repetitive, and scale-varying characteristics of power transmission line scenes, traditional texture- or feature-point-based stitching methods are prone to mismatches, leading to issues such as conductor breakage and structural misalignment. Second, environmental and imaging disturbances have a significant impact. Power transmission lines are typically located in complex natural environments, affected by wind swaying, lighting changes, fog, and camera shake. This results in numerous unstructured dynamic changes in the video, easily interfering with the accuracy of the fusion results. Third, there is a lack of structural constraints and topological consistency guarantees. Existing methods rely heavily on low-level visual features and lack modeling of the inherent structural relationships of power transmission lines, making it difficult to guarantee the physical plausibility of the fusion results. Fourth, errors propagate and lack self-correction capabilities. Once a local fusion error occurs, traditional methods often propagate it directly to subsequent stitching results, lacking effective backoff and adaptive correction mechanisms, thus affecting the overall panoramic quality. Fifth, the reliability of the results is insufficient. Existing systems typically only output fused images or videos, lacking assessment and labeling of the reliability of the fusion results, making it difficult to provide a reliable basis for subsequent operation and maintenance decisions.

[0005] Therefore, there is an urgent need for an online monitoring system that can combine the prior knowledge of the transmission line structure, suppress the influence of complex disturbances, and has the ability to perform topology verification and adaptive correction. Summary of the Invention

[0006] The purpose of this invention is to provide an online monitoring system for video panoramic fusion of transmission lines, which solves the problems of insufficient video stitching stability, significant impact of environmental and imaging disturbances, lack of structural constraints and topology consistency guarantee mechanisms, error propagation and lack of self-correction capabilities, and insufficient reliability of results in the process of using video data to construct panoramic views and monitor the status of transmission lines, thus failing to meet the application requirements.

[0007] This invention achieves the above objectives through the following technical solution: an online monitoring system for video panoramic fusion of transmission lines, the system comprising:

[0008] The system comprises a video acquisition unit, a structure prior construction unit, a disturbance decoupling unit, a constraint fusion unit, a topology verification unit, a backoff correction unit, and a reliable output unit.

[0009] The video acquisition unit is used to acquire continuous monitoring videos along the transmission line, and to perform time synchronization, imaging correction, key frame extraction, field of view overlap recognition, and local video segmentation on the monitoring videos.

[0010] The structural prior construction unit is used to analyze the main body of the tower, crossarm, insulator string, conductor, hardware connection area and channel boundary in the local video segment, and construct the line structure prior constraint diagram based on the connection relationship, adjacency relationship, direction relationship and positioning relationship between the aforementioned targets;

[0011] The disturbance decoupling unit is used to identify dynamic change information in local video segments and divide the dynamic change information into environmental disturbance change, imaging disturbance change and line real state change, so as to generate corresponding suppression markers or retention markers.

[0012] The constraint fusion unit is used to implement structural consistency constraints on candidate matching regions based on the prior constraint map of the line structure, and adjust the fusion participation weights of each region in the local video segment based on the suppression flag or retention flag, so as to output the local fusion result.

[0013] The topology verification unit is used to perform a physical topology consistency verification of the transmission line on the local fusion result;

[0014] The rollback correction unit is used to trigger a local rollback recalculation when the topology consistency check does not meet the preset conditions, and to adaptively correct the subsequent fusion control parameters.

[0015] The trusted output unit is used to cascade multiple local fusion results that have passed the topology consistency verification according to the tower section order and conductor topology extension relationship to form a panoramic monitoring result of the transmission line, and attach a trusted tag corresponding to each local fusion result.

[0016] Furthermore, the video acquisition unit includes:

[0017] Acquisition control subunit and segmented preprocessing subunit;

[0018] The acquisition and control subunit is used to control the video acquisition equipment set on the tower, the channel side or the pan-tilt carrier to acquire monitoring videos containing continuous line sections;

[0019] The segmented preprocessing subunit is used to perform local video segmentation on the original video stream based on the degree of overlap of the field of view between adjacent frames, the direction of wire extension and the positional relationship of line segments, so that each local video segment corresponds to a single continuous line segment or adjacent continuous line segments.

[0020] Furthermore, the structural prior construction unit includes:

[0021] Target analysis sub-unit and prior graph generation sub-unit;

[0022] The target analysis subunit is used to extract the main outline of the tower, the position of the crossarm, the position of the insulator string, the trajectory of the conductor extension, the hardware connection area and the background boundary information.

[0023] The prior diagram generation subunit is used to construct a prior constraint diagram of the line structure that characterizes the inherent structural relationship of the transmission line, based on the constraints of continuous extension of conductors, connection of insulator strings and crossarms, adjacent connection of fittings and conductors, and positioning constraints of the tower body and line section.

[0024] Furthermore, the segmentation preprocessing subunit is also used to perform key frame screening on the video frame sequence before performing local video segmentation. The key frame screening is based on the intensity of content change and structural stability between adjacent frames, and duplicate frames that do not meet the valid change conditions are removed.

[0025] Furthermore, when performing local video segment division, the segmented preprocessing subunit uses the continuity of the overlapping field of view between keyframes as the connection condition and the main extension direction of the conductor as the segmentation axis, so that the boundary of the local video segment corresponds to the boundary of the physical section of the line.

[0026] Furthermore, the constraint fusion unit includes a candidate screening subunit;

[0027] The candidate screening subunit is used to calculate the structural consistency result based on the prior constraint diagram of the line structure for candidate matching areas in a local video segment. The structural consistency result includes at least the continuity of conductor extension, the compliance of splicing relationship, the compliance of connection adjacency, and the compliance of segment positioning.

[0028] Only candidate matching regions that meet the preset structural consistency conditions are retained as valid fusion regions, while candidate matching regions that do not meet the preset structural consistency conditions are excluded from the local fusion process.

[0029] Furthermore, the disturbance decoupling unit includes:

[0030] Feature extraction subunit, perturbation classification subunit, and label generation subunit;

[0031] The feature extraction subunit is used to extract inter-frame displacement features, local illumination change features, edge stability features, regional texture fluctuation features, and target shape continuity features;

[0032] The disturbance classification subunit is used to identify changes in environmental disturbances, changes in imaging disturbances, and changes in the actual state of the line based on the extracted features.

[0033] The tag generation subunit is used to generate suppression tags for changes in environmental disturbances and imaging disturbances, and to generate retention tags for changes in the actual state of the line.

[0034] Furthermore, the constraint fusion unit includes a weight control subunit;

[0035] The weight control subunit is used to map the suppression mark or retention mark together with the prior constraint map of the line structure into fusion control weights. Specifically, regions with high structural consistency and which are determined to be changes in the actual state of the line are given high retention weights, regions with high structural consistency and which are determined to be changes in environmental disturbances or imaging disturbances are given low participation weights, and regions with insufficient structural consistency are restricted from participating in local fusion.

[0036] The constraint fusion unit outputs a local fusion result that preserves the true state information of the line and suppresses the propagation of non-target disturbances.

[0037] Furthermore, the topology verification unit is used to jointly verify the continuity of conductor extension direction, the correctness of insulator string connection position, the rationality of tower relative position and conductor cross-frame connection integrity after the local fusion result is generated, and output the topology consistency evaluation result.

[0038] When the topology consistency evaluation result meets the preset conditions, the current local fusion result is determined to be a valid result;

[0039] When the topology consistency evaluation result does not meet the preset conditions, a distortion indication is sent to the rollback correction unit.

[0040] Furthermore, the backoff correction unit is used to re-execute fusion control on the corresponding local video segment after receiving the distortion indication, and adaptively correct at least one of the disturbance identification threshold, fusion participation threshold and candidate matching retention threshold according to the current topology consistency evaluation result;

[0041] The rollback correction unit is also used to resubmit the corrected fusion result to the topology verification unit for review, and retain the result with better topology consistency as the effective local fusion result among multiple candidate fusion results.

[0042] Furthermore, the trusted output unit is used to generate a trusted evaluation result for each local fusion segment based on the success of one fusion, the backoff correction, the effectiveness of disturbance suppression, and the pass of topology verification during the local fusion process, and to output the trusted evaluation result synchronously with the corresponding panoramic monitoring image or panoramic monitoring video.

[0043] The trusted output unit is also used to cascade and splice multiple effective local fusion results according to the tower section sequence and conductor topology extension relationship, so that the output results simultaneously include the panoramic monitoring content of the transmission line and the trusted information of the section.

[0044] The beneficial effects of this invention are as follows:

[0045] 1. By constructing a priori constraint diagram of the transmission line structure, the connection relationships, adjacency relationships, and spatial positioning relationships among key targets such as conductors, insulator strings, and towers are modeled. Structural consistency constraints are implemented during the fusion process, which effectively reduces the probability of mismatch and improves the structural correctness of panoramic stitching.

[0046] 2. By classifying the dynamic changes in the video into environmental disturbance changes, imaging disturbance changes, and changes in the actual state of the line, and assigning suppression or retention labels to each, the influence of non-target disturbances on the fusion results can be effectively suppressed, thereby improving the stability of the system in complex environments.

[0047] 3. By jointly mapping structural consistency and disturbance markings into fusion weights, higher retention weights are assigned to areas of actual state change, while participation is reduced in areas of interference, thereby ensuring structural correctness while preserving the true state information of the line to the greatest extent.

[0048] 4. By jointly verifying the continuity of conductors, the connection relationship of insulators, and the spatial position of towers, we ensure that the fusion result conforms to the actual physical topology of the transmission line, effectively avoiding structural breakage or incorrect connection problems.

[0049] 5. When a topology inconsistency is detected, the system can automatically trigger a local backoff recalculation and adaptively adjust the disturbance identification threshold and fusion parameters to correct errors and optimize the subsequent fusion process, thereby avoiding error accumulation.

[0050] 6. By comprehensively considering the success rate of fusion, the effect of disturbance suppression, and the topology verification results, a credibility evaluation is generated for each local fusion segment, and the evaluation is output synchronously with the panoramic results, providing quantifiable decision-making basis for operation and maintenance personnel.

[0051] 7. By cascading and splicing multiple local fusion results according to the tower section sequence and conductor topology, a continuous and complete panoramic monitoring result of the transmission line is formed, which meets the overall observation needs of long-distance lines. Attached Figure Description

[0052] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0053] Figure 1 This is a flowchart illustrating the overall system flow of the present invention.

[0054] Figure 2 This is a flowchart of the video acquisition unit of the present invention;

[0055] Figure 3 This is a flowchart of the constraint fusion unit of the present invention. Detailed Implementation

[0056] The present application will now be described in further detail with reference to the accompanying drawings. It should be noted that the following specific embodiments are only used to further illustrate the present application and should not be construed as limiting the scope of protection of the present application. Those skilled in the art can make some non-essential improvements and adjustments to the present application based on the above application content.

[0057] Example 1:

[0058] Please see Figure 1-3 This invention provides a technical solution: an online monitoring system for panoramic video fusion of power transmission lines, the system comprising:

[0059] The system comprises a video acquisition unit, a structure prior construction unit, a disturbance decoupling unit, a constraint fusion unit, a topology verification unit, a backoff correction unit, and a reliable output unit.

[0060] The video acquisition unit is used to continuously acquire the video stream of the transmission line monitoring through video acquisition equipment deployed on the tower, channel side or pan-tilt device. It performs time synchronization, distortion correction, key frame screening and view overlap interval identification on the acquired video frame sequence. According to the tower section location, conductor extension direction and the overlap of the view field of adjacent frames, the original video stream is divided into multiple independently processable local video segments.

[0061] Among them, video acquisition equipment, such as cameras, is deployed on poles, towers, corridor sides, or pan-tilt units to continuously acquire video streams for power transmission line monitoring, and is the source of raw video data; video frame sequences, since video is composed of a series of consecutive image frames, the video data acquired by the video acquisition equipment is a sequence of video frames arranged in chronological order; time synchronization, which processes the acquired video frame sequences to ensure that video frames acquired from different locations or devices are consistent in time, ensuring the accuracy of subsequent analysis; and distortion correction, which addresses the distortion that may exist in the acquired images due to lens and other factors, such as barrel distortion and pincushion distortion. Distortion correction eliminates distortions, restoring the image to its normal geometry; keyframe filtering selects representative frames from the video frame sequence that reflect the main information, reducing the amount of data for subsequent processing and improving processing efficiency; viewpoint overlap region identification identifies the overlapping parts between adjacent video frames, which is crucial for subsequent operations such as panoramic fusion, determining which areas can be matched and fused; local video segments divide the original video stream into multiple independently processable parts based on the tower section location, conductor extension direction, and the overlap of the viewpoints of adjacent frames, with each local video segment containing transmission line information within a certain range.

[0062] The structural prior construction unit is used to analyze the image content of each local video segment to identify the transmission line target, the tower body, crossarm, insulator string, conductor, hardware connection point and channel background boundary, and construct the line structure prior constraint map based on the spatial adjacency, connection, direction and relative position relationship between each target. This map includes the continuous extension constraint of the conductor, the connection constraint between the insulator string and the crossarm, the adjacency constraint of the hardware and conductor connection area, and the positioning constraint of the tower body and the line section.

[0063] The analysis includes: transmission line target resolution, which analyzes the image content in each local video segment to identify transmission line-related targets such as tower bodies, crossarms, insulator strings, conductors, hardware connection points, and corridor background boundaries; spatial adjacency, which refers to the spatial proximity or adjacent relationship between transmission line targets, such as the tower body and crossarm being adjacent; connection relationship, which indicates the connection method between transmission line targets, such as the connection between insulator strings and crossarms, and the connection between hardware and conductors; and directional relationship, which describes the extension direction of transmission line targets. Examples of constraints include: the direction of the conductor; relative positional relationships, the position of each target relative to other targets in space, such as the position of a hardware connection point relative to the conductor; and a priori constraint diagram of the line structure, which is a diagram constructed based on the spatial adjacency, connection, direction, and relative positional relationships between each target. This diagram includes constraints on the continuous extension of the conductor, the connection between the insulator string and the crossarm, the adjacency constraints between the hardware and the conductor connection area, and the positioning constraints between the tower body and the line section. These constraints are used to guide subsequent panoramic fusion operations and ensure that the fusion result conforms to the actual structure of the transmission line.

[0064] The perturbation decoupling unit is used to extract inter-frame displacement features, local illumination change features, edge stability features, regional texture fluctuation features, and target morphology continuity features for dynamic change information in local video segments. It establishes a multi-perturbation hierarchical decoupling model, which divides the changes in the video into environmental perturbation changes, imaging perturbation changes, and line real state changes. It generates suppression labels for environmental perturbation changes and imaging perturbation changes, and generates retention labels for line real state changes.

[0065] Among them, inter-frame displacement features reflect the changes in target position between adjacent video frames, indicating the target's motion; local illumination variation features show changes in light intensity and color in local areas of the video, which may be caused by factors such as weather changes or changes in lighting angle; edge stability features indicate the stability of target edges across different frames, with edge instability potentially caused by imaging problems or environmental interference; regional texture fluctuation features show changes in texture in specific areas of the image across different frames, with texture fluctuations potentially reflecting environmental changes or imaging anomalies; and target morphology continuity features indicate the degree to which the target's morphology remains continuous across different frames. The system employs several features: a degree feature, indicating potential real-world changes or interference if the target shape is discontinuous; a multi-disturbance hierarchical decoupling model, designed for dynamic changes in local video segments, categorizes changes into environmental disturbances, imaging disturbances, and changes in the actual line state, separating different types of disturbances by extracting these features; suppression labels, generated for environmental and imaging disturbances, are used to suppress the impact of these interference factors on the results during subsequent processing; and retention labels, generated for changes in the actual line state, are used to retain this real-world change information during subsequent processing for accurate monitoring of the transmission line's condition.

[0066] The constraint fusion unit is used to perform structural consistency screening of candidate matching regions based on the prior constraint map of the line structure when performing panoramic fusion on each local video segment. At the same time, it adjusts the fusion weights based on the disturbance decoupling results to complete the reliable panoramic fusion of the local video segment and output the local fusion result.

[0067] Among these, candidate matching refers to the image regions that may be used for matching and fusion during panoramic fusion, which need to be determined according to certain rules and features; structural consistency screening filters candidate matching regions based on the prior constraint map of the line structure to ensure that the fused region structurally conforms to the actual structure of the transmission line, avoiding unreasonable fusion results; fusion weight is used to adjust the degree of contribution of different regions or different frames to the fusion result during panoramic fusion, and adjusting the fusion weight according to the perturbation decoupling result can make the fusion result more reliable; local fusion result is the result obtained after performing panoramic fusion on each local video segment, which contains the fused image information of the transmission line within that local area;

[0068] The topology verification unit is used to perform physical topology consistency verification of transmission lines on the local fusion results.

[0069] Among them, the physical topology consistency verification of transmission lines checks the local fusion results to verify whether they conform to the actual physical topology of the transmission lines, such as whether the connection relationship of conductors and the relative positions of towers and lines are correct.

[0070] The rollback correction unit is used to trigger a local rollback recalculation when the topology consistency check does not meet the preset conditions, and to adaptively correct the subsequent fusion control parameters.

[0071] Among them, the local rollback recalculation is performed when the topology consistency check does not meet the preset conditions, and the local fusion process is rolled back and recalculated and processed to correct possible problems; the fusion control parameters are various parameters used to control the fusion process during the panoramic fusion process, such as fusion weights and matching thresholds. The rollback correction unit will adaptively correct these parameters according to the situation to improve the accuracy and reliability of subsequent fusion.

[0072] The trusted output unit is used to perform panoramic cascading of multiple local fusion results that have passed the consistency verification according to the tower section order and conductor topology extension relationship to generate a panoramic image or panoramic video of the transmission line online monitoring. At the same time, it adds a trusted monitoring mark to the corresponding panoramic area and outputs it.

[0073] Among them, panoramic cascading connects and combines multiple local fusion results that have passed consistency verification according to the order of tower sections and the topological extension relationship of conductors to form a complete panoramic image or panoramic video of online monitoring of transmission lines; monitoring credibility markers are markers attached to the corresponding panoramic areas to indicate that the monitoring results of the panoramic areas have high credibility, which are derived from the previous processing and analysis process.

[0074] It should be noted that during use, the video acquisition unit deploys equipment at multiple locations and divides the video into local segments, enabling comprehensive and targeted acquisition of line information, reducing data processing volume. The structure prior construction unit analyzes the target and constructs a constraint graph, providing a structural basis for fusion and ensuring that the results conform to the actual line layout. The disturbance decoupling unit distinguishes and marks different changes, effectively suppressing interference and preserving true state information. The constraint fusion unit filters and adjusts weights based on the constraint graph, improving fusion reliability. The topology verification unit ensures the correct physical topology of the results. The backoff correction unit can correct in time when verification fails, enhancing system stability. Finally, the reliable output unit is cascaded panoramically and adds reliable markers, providing intuitive and reliable panoramic images or videos for monitoring, helping to detect line problems in a timely manner and ensuring the safe and stable operation of transmission lines.

[0075] In one embodiment, the fusion weight is adjusted based on the perturbation decoupling result using a quantitative calculation method. Specifically, the final fusion weight of each pixel region in the image is determined by the fusion weight calculation formula, which is:

[0076] W_i = W_(s,i) × W_(d,i);

[0077] Wherein, W_i is the final fusion weight of the i-th image region, with a value range of [0ⓜ,1]. The higher the weight value, the higher the participation and fidelity requirement of the region in the panoramic fusion process; W_(s,i) is the structural credibility weight of the i-th image region, which is assigned according to the structural matching degree between the region and the prior constraint map of the line structure. When the matching degree is 100%, W_(s,i)=1. For every 10% decrease in the matching degree, W_(s,i) decreases linearly by 0.1. The lower the structural matching degree, the smaller the structural credibility weight of the region; W_(d,i) is the disturbance decoupling weight of the i-th image region, which is assigned differently according to the disturbance type of the region and whether it is a change in the real state of the line. Among them, W_(d,i)=1 for the region with a change in the real state of the line, W_(d,i)=0.2 for the region with a change in the environmental disturbance, W_(d,i)=0.1 for the region with a change in the imaging disturbance, and W_(d,i)=0.5 for the background region with no disturbance and no change in the real state.

[0078] This design determines the final fusion weight of each pixel region in the image through a fusion weight calculation formula. The structural reliability weight is multiplied by the disturbance decoupling weight, which comprehensively considers the matching degree between the image region and the line structure, as well as the disturbance type. The structural reliability weight is assigned based on the structural matching degree, which can ensure that the fusion result conforms to the actual structure of the line. The disturbance decoupling weight is assigned differently according to the disturbance type, which can effectively suppress environmental and imaging disturbances and retain the true changes in the line's state. The combination of the two allows the fusion weight to accurately reflect the participation and fidelity requirements of each region in the panoramic fusion, improve the accuracy and reliability of the fusion result, and provide a more realistic and effective panoramic image for transmission line monitoring.

[0079] In one embodiment, the structural consistency screening of candidate matching regions is a structured screening method based on quantitative calculation. Specifically, the structural matching degree M_s between the candidate matching region and the prior constraint diagram of the line structure is calculated using the following formula:

[0080] M_s=(∑_(k=1)^n▒S_k ×ω_k) / (∑_(k=1)^n▒ω_k )×100%;

[0081] Where n represents the number of line structure constraint types, and in this system n=4, corresponding to the four core structural constraints: conductor continuous extension constraint, insulator string and crossarm connection constraint, hardware and conductor connection and adjacency constraint, and tower body and line section positioning constraint; S_k is the matching value of the k-th type of structural constraint. When the matching conforms to the preset line physical structure constraint, S_k=1, and when it does not conform to the preset constraint, S_k=0; ω_k is the weight of the k-th type of structural constraint, which is assigned according to the importance of each constraint to the integrity of the transmission line structure. Among them, ω_1=0.35, ω_2=0.25, ω_3=0.25, ω_4=0.15, and satisfying ∑_(k=1)^4▒ω_k =1;

[0082] The selection criteria are as follows: only candidate matching regions with M_s≥80% are judged as having consistent structure and are allowed to participate in subsequent panoramic fusion calculations; regions with a structural matching degree of less than 80% are directly eliminated and do not participate in fusion.

[0083] This design employs a structured screening method based on quantitative calculations for candidate matching regions. By calculating the structural matching degree, the matching status and weight of each structural constraint are clarified. Only regions with a matching degree ≥80% are retained for fusion. This quantitative screening method strictly avoids regions that do not conform to the physical structural constraints of the transmission line from participating in the fusion, ensuring the structural integrity of the fusion result. The four core structural constraints and their corresponding weight settings comprehensively consider the key structural characteristics of the transmission line, making the screening more targeted and scientific, effectively improving the quality of panoramic fusion, and laying the foundation for accurate monitoring of the transmission line status in the future.

[0084] In one embodiment, the topology consistency verification is a topology structure verification method based on quantitative scoring. Specifically, it calculates the topology consistency score S_t of the local fusion result using a formula to determine whether the fusion result meets the physical topology rules of the transmission line. The calculation formula is as follows:

[0085] S_t=100-∑_(m=1)^4▒D_m ×λ_m;

[0086] Where D_m is the deviation value of the m-th type of topology, m=1 is the deviation of the conductor extension direction, m=2 is the deviation of the insulator string connection position, m=3 is the deviation of the relative position of the tower, and m=4 is the deviation of the conductor continuity. The deviation value is normalized according to the ratio of the actual deviation to the allowable deviation.

[0087] When the actual deviation is less than or equal to the preset allowable deviation, D_m = 0, indicating that there is no deviation for this type of topology.

[0088] When the actual deviation is greater than the preset allowable deviation, D_m = actual deviation value / allowable deviation value × 25. The larger the deviation value, the higher the degree of distortion of this type of topology.

[0089] λ_m is the weighting coefficient of the m-th type of topology. In this method, λ_1=λ_2=λ_3=λ_4=1 is taken to indicate that the influence of the deviation of the four types of topology is consistent.

[0090] The criterion for determining topological distortion is that when S_t < 90%, the current local fusion result is determined to have structural distortion, and an abnormal rollback mechanism is immediately triggered to correct the fusion result.

[0091] This design employs a topology verification method based on quantitative scoring. By calculating the topology consistency score, it determines whether the fusion result meets the physical topology rules of the transmission line. It transforms complex topology deviations into quantifiable scores, intuitively reflecting the topology accuracy of the fusion result. It clarifies the calculation methods and weighting coefficients for various topology deviation values, comprehensively considers the impact of deviations in key structures such as conductors, insulator strings, and towers, and triggers an abnormal rollback mechanism when the score is below 90%. This allows for timely detection and correction of structurally distorted fusion results, ensuring that the output panoramic image conforms to the actual physical topology of the transmission line and improving the reliability of monitoring.

[0092] In one embodiment, in the abnormal rollback mechanism, the disturbance suppression threshold τ of the local segment is adjusted using an adaptive adjustment method based on the topology consistency score, and the specific adjustment formula is as follows:

[0093] τ_new=τ_old×(1-(90-S_t) / 100);

[0094] Where τ_new is the adjusted perturbation suppression threshold, and τ_old is the initial perturbation suppression threshold before adjustment. The initial value is 0.7. The perturbation suppression threshold is used to determine whether an image region is a perturbation region. The higher the threshold, the stricter the perturbation recognition.

[0095] To avoid excessively large threshold adjustments affecting disturbance recognition, the value range of τ is limited to [0.5ⓜ, 0.9]. If the calculated τ_new < 0.5, then τ_new = 0.5 is taken as the adjusted threshold. If the calculated τ_new > 0.9, then τ_new = 0.9 is taken as the adjusted threshold.

[0096] This design allows the perturbation suppression threshold in the anomaly backoff mechanism to be adaptively adjusted based on the topology consistency score. The perturbation suppression threshold is dynamically adjusted according to the topology consistency score, ensuring that the strictness of perturbation identification is adapted to the quality of the fusion result. When the topology consistency score is low, the perturbation suppression threshold is appropriately lowered to relax the perturbation identification standard and avoid over-suppression leading to misjudgment of changes in the true state. At the same time, the threshold value range is limited to prevent excessive adjustment from affecting the perturbation identification effect. This adaptive adjustment method improves the flexibility and effectiveness of the anomaly backoff mechanism, helps to more accurately correct the fusion result, and enhances the quality and stability of panoramic fusion.

[0097] In one embodiment, the monitoring credibility marker includes the credibility level of each panoramic segment. The credibility level G is determined by a quantitatively calculated credibility score S_c, and the credibility score is calculated using the following formula:

[0098] S_c = W_f × 80 + W_r × 20;

[0099] Wherein, W_f is the fusion success coefficient of the local fusion segment, accounting for 80% of the credibility score. When there is no abnormal backtracking, it means that the fusion is successful in one go, and W_f=1. Each time an abnormal backtracking is triggered, it means that there is a deviation in the fusion process, and W_f decreases by 0.2. In order to ensure the validity of the score, W_f is at least 0.4. W_r is the perturbation suppression effectiveness coefficient, accounting for 20% of the credibility score. W_r = area of ​​effectively suppressed perturbation region / total area of ​​perturbation region, and the value range is [0ⓜ,1]. The closer this coefficient is to 1, the better the perturbation suppression effect.

[0100] The credibility level is divided into four levels based on the credibility score. The specific grading criteria are as follows:

[0101] When S_c≥90, it is Grade A, highly reliable;

[0102] When 75 ≤ S_c < 90, it is classified as Grade B, which is of medium reliability.

[0103] When 60 ≤ S_c < 75, it is classified as Grade C, low reliability;

[0104] When S_c < 60, it is classified as Grade D, meaning it is unreliable.

[0105] The grading results are output synchronously with the panoramic fusion results.

[0106] This design allows for the quantitative calculation of credibility scores for tiered assessment of monitoring credibility markers. It comprehensively considers the fusion success coefficient and disturbance suppression effectiveness coefficient of the local fusion segment to fully evaluate the credibility of the panoramic fusion results. The fusion success coefficient reflects the smoothness of the fusion process, while the disturbance suppression effectiveness coefficient reflects the effectiveness of disturbance handling. Both are weighted differently to calculate the credibility score, making the assessment more scientific and reasonable. Four credibility levels are assigned based on the scores and output synchronously with the panoramic fusion results, providing monitoring personnel with an intuitive credibility reference. This helps to quickly determine the reliability of monitoring results, enabling timely implementation of appropriate measures and improving the efficiency and accuracy of transmission line monitoring.

[0107] In one embodiment, the multi-perturbation hierarchical decoupling model is built based on a convolutional neural network (CNN), specifically including a feature extraction sub-model, a perturbation classification sub-model, and a confidence evaluation sub-model. The training steps of the multi-perturbation hierarchical decoupling model include:

[0108] The model preprocessing step involves constructing a video disturbance sample dataset of transmission lines, collecting at least 1000 monitoring videos of transmission lines under different operating conditions, including environmental scenes such as sunny days, rainy days, foggy days, and strong winds, as well as imaging disturbance scenes such as camera shake, focal length fluctuations, and sudden changes in illumination. At the same time, samples of actual changes in the line status such as foreign object hanging, broken conductor strands, and loose hardware are collected. The samples are uniformly scaled to a resolution of 256×256, normalized to the pixel value range [0ⓜ,1], and divided into training set, validation set, and test set in a ratio of 8:1:1.

[0109] The training steps for the feature extraction sub-model are as follows: ResNet50 is used as the backbone network for feature extraction. Preprocessed sample data is input, with an initial learning rate of 0.001. The Adam optimizer is used, and the loss function is cross-entropy loss. A learning rate decay strategy is introduced during training, with a decay rate of 0.95 per training epoch. The number of epochs is set to 50, and the batch size is set to 32. Training stops when the feature classification accuracy on the validation set no longer improves for 10 consecutive epochs, resulting in a converged feature extraction sub-model.

[0110] The training steps for the perturbation classification sub-model are as follows: Using the output of the feature extraction sub-model as input, a classification head containing three fully connected layers is constructed. The number of neurons in the fully connected layers is 512, 256, and 3 respectively, with ReLU as the activation function. The output layer uses the Softmax function to predict the probability distributions of three types of perturbations: environmental perturbation, imaging perturbation, and changes in the actual state of the line. The initial learning rate is set to 0.0005, using the AdamW optimizer with Focal Loss as the loss function. The training epochs are 30, and the batch size is 64. Dropout regularization is added during training with a dropout probability of 0.3 to prevent overfitting, resulting in a converged perturbation classification sub-model.

[0111] The confidence evaluation sub-model training steps are as follows: input the feature vector extracted by the feature extraction sub-model, construct an evaluation head containing two fully connected layers with 256 and 1 neurons respectively, and output the confidence score of the sample belonging to the corresponding perturbation category; the initial learning rate is set to 0.0003, the mean squared error loss function is used, the number of training epochs is 20, the batch size is 64, and training stops when the mean absolute error (MAE) on the validation set is less than 0.02, thus completing the overall training of the multi-perturbation hierarchical decoupling model;

[0112] The model inference step involves inputting frames of the local video segment to be processed into the trained multi-perturbation hierarchical decoupled model. The feature extraction sub-model outputs a 128-dimensional feature vector, the perturbation classification sub-model outputs the probability values ​​of each frame belonging to three types of perturbations, and the category corresponding to the maximum probability is taken as the perturbation type of the frame. The confidence evaluation sub-model outputs the confidence score of the category. When the confidence score is ≥0.85, the perturbation category is determined to be valid; otherwise, the perturbation category of the neighboring frame is used for interpolation to complete the perturbation.

[0113] This design, with its multi-perturbation hierarchical decoupling model, is built upon a convolutional neural network and features a detailed training plan. Leveraging the powerful feature extraction capabilities of CNNs, it accurately identifies different types of perturbations. By constructing sub-models for feature extraction, perturbation classification, and confidence assessment, and processing video data hierarchically, it improves the model's accuracy in perturbation classification and the reliability of confidence assessment. The detailed training steps, including sample preprocessing and parameter settings for each sub-model, ensure that the model can effectively learn perturbation features under different operating conditions. The judgment and interpolation of confidence scores during the model inference process further enhance the accuracy of perturbation classification, providing a reliable basis for subsequent fusion weight adjustments and improving the quality of panoramic fusion.

[0114] In one embodiment, environmental disturbance changes are video image changes without practical monitoring significance caused by natural environmental factors such as wind, rain, fog, cloud shadows, and vegetation swaying; imaging disturbance changes are video image changes without practical monitoring significance caused by equipment imaging factors such as camera shake, focal length fluctuations, and exposure changes; and changes in the actual state of the line are video image changes with practical hazard monitoring significance caused by transmission line-specific factors such as foreign object hanging, icing development, discharge traces, conductor damage, or hardware abnormalities. Differentiated labels are generated for different types of changes, suppression labels are generated for environmental disturbance changes and imaging disturbance changes, and retention labels are generated for changes in the actual state of the line. These results form the fusion evidence screening results for subsequent fusion weight adjustment.

[0115] This design generates differentiated markers for environmental disturbances, imaging disturbances, and changes in the actual state of the transmission line, clearly distinguishing different types of changes and providing a clear basis for subsequent fusion weight adjustments. Changes in environmental disturbances and imaging disturbances generate suppression markers, which can prevent these changes without practical monitoring significance from interfering with panoramic fusion, reduce data processing volume, and improve fusion efficiency. Changes in the actual state of the transmission line generate retention markers, ensuring that changes with practical hidden danger monitoring significance can be accurately reflected in the panoramic image, facilitating the timely detection of abnormal conditions in transmission lines. Differentiated markers form the fusion evidence screening results, making panoramic fusion more in line with monitoring needs and improving the accuracy and effectiveness of monitoring.

[0116] In one embodiment, the keyframe selection of the acquired video frame sequence adopts a quantitative selection method of inter-frame difference, which determines whether to retain the next frame as a keyframe by calculating the similarity Sim between adjacent frames.

[0117] When Sim≥95%, it means that there is no significant change between two adjacent frames, and the next frame is removed to reduce the amount of data processing.

[0118] When Sim < 95%, it means that there is a valid change between two adjacent frames, and the next frame is retained as the keyframe.

[0119] The similarity calculation formula for the inter-frame difference method is:

[0120] Sim=(1-(∑_(x=1)^W▒∑_(y=1)^H▒| I_t (x,y)-I_(t-1) (x,y)|) / (W×H×255))×100%;

[0121] Where I_t (xⓜ,y) is the pixel value at coordinates (xⓜ,y) in the t-th frame image, I_(t-1) (xⓜ,y) is the pixel value at coordinates (xⓜ,y) in the (t-1)-th frame image, W is the image width, H is the image height, and 255 is the maximum value of the pixel value. This formula is used to calculate the normalization of the similarity between adjacent frames, and the value range is [0,100%].

[0122] This design employs inter-frame difference (IPD) to filter keyframes in the acquired video frame sequence. By calculating the similarity between adjacent frames, it quantitatively determines whether to retain a keyframe, effectively reducing data processing volume. When the similarity between adjacent frames is ≥95%, the next frame is discarded to avoid repetitive image processing and improve processing efficiency. When the similarity is <95%, the next frame is retained to ensure that no effectively changing images are lost and to guarantee the integrity of key information. The similarity calculation formula of IPD achieves normalized calculation of the similarity between adjacent frames, with a clear value range, making the selection criteria uniform and objective. This provides high-quality keyframe sequences for subsequent panoramic fusion, improving the accuracy and reliability of the fusion results.

[0123] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0124] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A video panoramic fusion online monitoring system for power transmission lines, characterized in that, The system includes: The system comprises a video acquisition unit, a structure prior construction unit, a disturbance decoupling unit, a constraint fusion unit, a topology verification unit, a backoff correction unit, and a reliable output unit. The video acquisition unit is used to acquire continuous monitoring videos along the transmission line, and to perform time synchronization, imaging correction, key frame extraction, field of view overlap recognition, and local video segmentation on the monitoring videos. The structural prior construction unit is used to analyze the main body of the tower, crossarm, insulator string, conductor, hardware connection area and channel boundary in the local video segment, and construct the line structure prior constraint diagram based on the connection relationship, adjacency relationship, direction relationship and positioning relationship between the aforementioned targets; The disturbance decoupling unit is used to identify dynamic change information in local video segments and divide the dynamic change information into environmental disturbance change, imaging disturbance change and line real state change, so as to generate corresponding suppression markers or retention markers. The constraint fusion unit is used to implement structural consistency constraints on candidate matching regions based on the prior constraint map of the line structure, and adjust the fusion participation weights of each region in the local video segment based on the suppression flag or retention flag, so as to output the local fusion result. The topology verification unit is used to perform a physical topology consistency verification of the transmission line on the local fusion result; The rollback correction unit is used to trigger a local rollback recalculation when the topology consistency check does not meet the preset conditions, and to adaptively correct the subsequent fusion control parameters. The trusted output unit is used to cascade multiple local fusion results that have passed the topology consistency verification according to the tower section order and conductor topology extension relationship to form a panoramic monitoring result of the transmission line, and attach a trusted tag corresponding to each local fusion result.

2. The online monitoring system for panoramic video fusion of transmission lines according to claim 1, characterized in that, The video acquisition unit includes: Acquisition control subunit and segmented preprocessing subunit; The acquisition and control subunit is used to control the video acquisition equipment set on the tower, the channel side or the pan-tilt carrier to acquire monitoring videos containing continuous line sections; The segmented preprocessing subunit is used to perform local video segmentation on the original video stream based on the degree of overlap of the field of view between adjacent frames, the direction of wire extension and the positional relationship of line segments, so that each local video segment corresponds to a single continuous line segment or adjacent continuous line segments.

3. The online monitoring system for panoramic video fusion of transmission lines according to claim 1, characterized in that, The structural prior construction unit includes: Target analysis sub-unit and prior graph generation sub-unit; The target analysis subunit is used to extract the main outline of the tower, the position of the crossarm, the position of the insulator string, the trajectory of the conductor extension, the hardware connection area and the background boundary information. The prior diagram generation subunit is used to construct a prior constraint diagram of the line structure that characterizes the inherent structural relationship of the transmission line, based on the constraints of continuous extension of conductors, connection of insulator strings and crossarms, adjacent connection of fittings and conductors, and positioning constraints of the tower body and line section.

4. The online monitoring system for panoramic video fusion of transmission lines according to claim 2, characterized in that: The segmentation preprocessing subunit is also used to perform key frame screening on the video frame sequence before performing local video segmentation. The key frame screening is based on the intensity of content change and structural stability between adjacent frames, and duplicate frames that do not meet the effective change conditions are removed. Furthermore, when performing local video segment division, the segmented preprocessing subunit uses the continuity of the overlapping field of view between keyframes as the connection condition and the main extension direction of the conductor as the segmentation axis, so that the boundary of the local video segment corresponds to the boundary of the physical section of the line.

5. The online monitoring system for panoramic video fusion of transmission lines according to claim 3, characterized in that: The constraint fusion unit includes a candidate screening subunit; The candidate screening subunit is used to calculate the structural consistency result based on the prior constraint diagram of the line structure for candidate matching areas in a local video segment. The structural consistency result includes at least the continuity of conductor extension, the compliance of splicing relationship, the compliance of connection adjacency, and the compliance of segment positioning. Only candidate matching regions that meet the preset structural consistency conditions are retained as valid fusion regions, while candidate matching regions that do not meet the preset structural consistency conditions are excluded from the local fusion process.

6. The online monitoring system for panoramic video fusion of transmission lines according to claim 1, characterized in that, The disturbance decoupling unit includes: Feature extraction subunit, perturbation classification subunit, and label generation subunit; The feature extraction subunit is used to extract inter-frame displacement features, local illumination change features, edge stability features, regional texture fluctuation features, and target shape continuity features; The disturbance classification subunit is used to identify changes in environmental disturbances, changes in imaging disturbances, and changes in the actual state of the line based on the extracted features. The tag generation subunit is used to generate suppression tags for changes in environmental disturbances and imaging disturbances, and to generate retention tags for changes in the actual state of the line.

7. The online monitoring system for panoramic video fusion of transmission lines according to claim 6, characterized in that: The constraint fusion unit includes a weight control subunit; The weight control subunit is used to map the suppression mark or retention mark together with the prior constraint map of the line structure into fusion control weights. Specifically, regions with high structural consistency and which are determined to be changes in the actual state of the line are given high retention weights, regions with high structural consistency and which are determined to be changes in environmental disturbances or imaging disturbances are given low participation weights, and regions with insufficient structural consistency are restricted from participating in local fusion. The constraint fusion unit outputs a local fusion result that preserves the true state information of the line and suppresses the propagation of non-target disturbances.

8. The online monitoring system for video panoramic fusion of transmission lines according to claim 1, characterized in that: The topology verification unit is used to jointly verify the continuity of conductor extension direction, the correctness of insulator string connection position, the rationality of tower relative position and conductor cross-frame connection integrity after the local fusion result is generated, and outputs the topology consistency evaluation result. When the topology consistency evaluation result meets the preset conditions, the current local fusion result is determined to be a valid result; When the topology consistency evaluation result does not meet the preset conditions, a distortion indication is sent to the rollback correction unit.

9. The online monitoring system for video panoramic fusion of transmission lines according to claim 8, characterized in that: The backoff correction unit is used to re-execute fusion control on the corresponding local video segment after receiving the distortion indication, and adaptively correct at least one of the disturbance identification threshold, fusion participation threshold and candidate matching retention threshold according to the current topology consistency evaluation result. The rollback correction unit is also used to resubmit the corrected fusion result to the topology verification unit for review, and retain the result with better topology consistency as the effective local fusion result among multiple candidate fusion results.

10. The online monitoring system for video panoramic fusion of transmission lines according to claim 7, characterized in that: The trusted output unit is used to generate a trusted evaluation result for each local fusion segment based on the success of one fusion, the backoff correction, the effectiveness of disturbance suppression, and the pass of topology verification during the local fusion process, and to output the trusted evaluation result synchronously with the corresponding panoramic monitoring image or panoramic monitoring video. The trusted output unit is also used to cascade and splice multiple effective local fusion results according to the tower section sequence and conductor topology extension relationship, so that the output results simultaneously include the panoramic monitoring content of the transmission line and the trusted information of the section.

Images