Real-time monitoring system and method for power transmission line digital twin based on multi-modal data fusion

Abstract

The present application relates to the technical field of power transmission line monitoring, in particular to a multi-modal data fusion power transmission line digital twin real-time monitoring system and method, comprising: a data acquisition module responsible for collecting optical images, environmental feature data, current waveform data and point cloud data, and transmitting these data to a digital twin module, the digital twin module constructs a three-dimensional digital twin of the power transmission line, and fuses these multi-modal data through a graph neural network and an attention mechanism to generate multi-modal fusion features, based on these features, a dynamic correlation model of conductor temperature-sag-load flow is established and transmitted to an online monitoring module, the online monitoring module fuses real-time collected data and historical data, analyzes the line state, generates multi-level early warning information, and outputs line state analysis results and fault prediction information, the innovative data fusion method of the system improves the sensitivity and accuracy of fault detection, and helps to improve the operation safety and efficiency of the power transmission line.

Description

Technical Field

[0001] This invention relates to the field of transmission line monitoring technology, specifically to a real-time monitoring system and method for digital twins of transmission lines based on multimodal data fusion. Background Technology

[0002] As a crucial component of the power system, the safe and stable operation of transmission lines is of paramount importance to ensuring power supply. Traditional transmission line monitoring relies primarily on manual inspections and single-sensor monitoring, which suffers from limitations such as simplistic monitoring methods, limited accuracy, poor real-time performance, and weak early warning capabilities. Especially under complex meteorological conditions, such as icing and strong winds, traditional monitoring methods struggle to detect potential hazards in a timely manner, increasing the risk of line failures.

[0003] With the development of sensing, communication, and artificial intelligence technologies, the collaborative utilization of multiple data sources has become an important way to improve monitoring efficiency. However, existing technologies face many challenges in multimodal data fusion: First, different types of data (such as optical images, point cloud data, and current waveforms) have heterogeneous formats and inconsistent spatiotemporal distributions, making effective integration difficult; second, traditional data fusion methods often employ simple splicing or weighted averaging, lacking in-depth exploration of the complex relationships between modes; third, in terms of data analysis models, static models struggle to adapt to the dynamic changes in transmission lines under different environmental conditions, affecting prediction accuracy; and finally, once deployed, the parameters are fixed and cannot adapt to long-term changes in system operating conditions and the environment.

[0004] Existing transmission line monitoring systems have significant shortcomings in processing multimodal data, constructing dynamic correlation models, and achieving adaptive prediction, making it difficult to meet the requirements of high-reliability power grid operation. Therefore, there is an urgent need to develop a transmission line monitoring system that can deeply integrate multimodal data, accurately model dynamic relationships, and possess self-evolution capabilities. Summary of the Invention

[0005] The purpose of this invention is to provide a real-time monitoring system and method for digital twins of transmission lines based on multimodal data fusion, aiming to solve the technical problems in the prior art such as insufficient depth of multimodal data fusion, limited dynamic correlation modeling capability, and weak adaptability, and to improve the accuracy, real-time performance, and predictive ability of transmission line condition monitoring.

[0006] This invention proposes a real-time monitoring system for digital twins of transmission lines based on multimodal data fusion, comprising:

[0007] The data acquisition module is used for:

[0008] Acquire optical images, environmental feature data, current waveform data, and point cloud data;

[0009] The optical image, the environmental feature data, the current waveform data, and the point cloud data are transmitted to the digital twin module;

[0010] The digital twin module is electrically connected to the data acquisition module and is used for:

[0011] Receive the optical image, the environmental feature data, the current waveform data, and the point cloud data;

[0012] Constructing a three-dimensional digital twin of a power transmission line;

[0013] Multimodal fusion features are generated by fusing the optical image, the environmental feature data, the current waveform data, and the point cloud data through a graph neural network and an attention mechanism.

[0014] A dynamic correlation model of conductor temperature-sag-current carrying capacity is established based on the aforementioned multimodal fusion features;

[0015] The dynamic correlation model of conductor temperature-sag-current carrying capacity is transmitted to the online monitoring module;

[0016] The online monitoring module is electrically connected to the digital twin module and is used for:

[0017] Receive the dynamic correlation model of conductor temperature-sag-current carrying capacity;

[0018] By integrating real-time and historical data, the status of the line can be analyzed.

[0019] Generate multi-level early warning information;

[0020] Output line status analysis results and fault prediction information.

[0021] Preferably, the digital twin module includes:

[0022] The raw image analysis module is used to establish the correlation between image features and partial discharge waveform features;

[0023] The sag prediction module is used to calculate the conductor-tower distance using fused features;

[0024] The model fusion module is used for point cloud registration and model updating of point cloud data;

[0025] The partial discharge analysis module is used to establish the correlation between the original image and the partial discharge waveform;

[0026] The sag calculation module is used to calculate the current conductor sag information based on the actual line model and the digital twin model.

[0027] Preferably, the online monitoring module includes:

[0028] The data fusion and mining module is used to calculate the correlation between line environmental data and line current waveforms, as well as the changing trends of historical data.

[0029] The results analysis and evaluation module is used to analyze and evaluate the results data and send the results to staff in the form of charts.

[0030] Preferably, the original image analysis module, the sag prediction module, the model fusion module, the partial discharge analysis module, and the sag calculation module employ a graph neural network combined with an attention mechanism to fuse multimodal data.

[0031] The graph neural network includes three graph convolutional layers for extracting node features and capturing high-order topological relationships between nodes;

[0032] The attention mechanism includes intramodal attention and intermodal cross-attention, which are used to adaptively allocate weights for data from different modalities;

[0033] The graph neural network and the attention mechanism are combined to achieve deep fusion of multimodal data.

[0034] Preferably, the optical image is obtained through a line-scan camera mounted on an unmanned inspection vehicle, the environmental feature data is obtained through wireless sensor nodes, and the current waveform data and point cloud data are obtained through a power transmission line inspection robot; wherein...

[0035] The resolution of the line scan camera is no less than 2048×1536;

[0036] The wireless sensor node collects environmental feature data every 5 minutes.

[0037] The power transmission line inspection robot is equipped with a lidar with a point cloud density of no less than 100 points per square meter.

[0038] Preferably, the digital twin module is based on the principle of digital twins, uses lidar technology and infrared thermal imaging technology to realize three-dimensional digital twin imaging, and combines real-time current waveform data collected by current sensors to digitally twinnize the collected images, current data and point cloud data to generate a digital twin circuit. The temperature information of the collection points, the current data collected by the current sensors and the images are associated to update the digital twin model and calculate the sag and sag prediction value.

[0039] Preferably, the input of the online monitoring module is the data detected by the sensor, and the output is the fault prediction result and the fault identification; wherein,

[0040] The fault prediction results include short-term prediction results, medium-term prediction results and long-term prediction results;

[0041] The short-term forecast results predict state changes within the next 10 minutes and are updated every 10 seconds.

[0042] The intermediate-term forecast results predict state changes from 10 minutes to 2 hours and are updated every 10 minutes.

[0043] The long-term prediction results predict state changes over a period of more than 2 hours and are updated hourly.

[0044] The fault identification includes partial discharge faults, abnormal sag, and icing risk.

[0045] Preferably, the dynamic correlation model of conductor temperature-sag-current carrying capacity is implemented through a spatiotemporal fusion mechanism, which includes:

[0046] Time information encoding is used to capture the periodic changes and long-term trends of data;

[0047] Spatial relationship modeling is used to evaluate the physical distance and functional similarity between nodes;

[0048] Spatiotemporal fusion processing is used to integrate information from both temporal and spatial dimensions;

[0049] Multiscale spatiotemporal analysis is used to analyze state changes at different spatiotemporal scales.

[0050] Preferably, the online monitoring module includes a self-evolutionary prediction and feedback correction mechanism, which includes:

[0051] The error analysis and correction unit is used to calculate the deviation between the prediction results and the actual observations, and to adjust the prediction parameters based on the error distribution.

[0052] Model self-evolutionary units are used to update the model structure and parameters based on performance evaluations and environmental changes.

[0053] Knowledge accumulation units are used to retain key historical experiences and patterns, preventing new knowledge from overwriting useful old knowledge.

[0054] The feedback correction unit triggers an update when the prediction accuracy falls below a threshold or when environmental conditions change significantly.

[0055] The specific steps of the real-time monitoring method for digital twins of transmission lines based on the aforementioned system are as follows:

[0056] Step 1: Construct a three-dimensional digital twin of the transmission line using laser point cloud, infrared thermal imaging, and current waveform data fusion technology;

[0057] Step 2: Design a cross-modal attention mechanism and establish a dynamic correlation model of conductor temperature-sag-current carrying capacity through a graph neural network to achieve real-time evaluation of conductor thermal stability;

[0058] Step 3: Integrate edge computing nodes to complete partial discharge feature extraction within 10ms, and combine meteorological data to predict icing risk, with a classification accuracy of over 98.5%.

[0059] Step 4: Through self-evolutionary prediction and feedback correction mechanisms, combined with a multi-level prediction framework, short-term, medium-term, and long-term fault prediction and early warning are achieved.

[0060] Step 5: Based on the abnormal status and prediction results, generate multi-level early warning information and send it to staff in the form of charts to realize intelligent operation and maintenance decision support.

[0061] The beneficial effects of this invention include:

[0062] 1. By innovatively combining graph neural networks with attention mechanisms, deep fusion of optical images, environmental feature data, current waveform data, and point cloud data is achieved, breaking through the limitations of traditional data fusion. It can mine hidden fault features from multi-source heterogeneous data, improving the sensitivity and accuracy of fault detection.

[0063] 2. A dynamic correlation model of conductor temperature-sag-current carrying capacity was established, which can accurately capture the variation patterns and mutual influences of these three key parameters in time and space, solve the problem that static models cannot adapt to complex environmental changes, and make the prediction results more consistent with physical reality.

[0064] 3. A self-evolutionary prediction and feedback correction mechanism was designed. The system can continuously optimize model parameters based on real-time monitoring data, achieve self-learning and adaptation, and achieve an accuracy rate of over 98.5% in predicting icing risk, which is significantly higher than traditional methods.

[0065] 4. By adopting edge computing optimization and lightweight design, while maintaining the model's expressive power, the system achieves the real-time requirement of extracting partial discharge features within 10ms, thus improving the system's response speed to sudden events.

[0066] 5. By using time-based fault prediction (short-term, medium-term, and long-term) and a multi-level early warning mechanism, comprehensive decision support is provided for operation and maintenance personnel, effectively reducing maintenance costs, extending equipment lifespan, and improving the reliability of power grid operation. Attached Figure Description

[0067] Figure 1 This is a schematic diagram of the overall architecture of the real-time monitoring system for digital twins of transmission lines based on multimodal data fusion, as described in an embodiment of the present invention.

[0068] Figure 2 This is a schematic diagram of the structure of the digital twin module in an embodiment of the present invention;

[0069] Figure 3 This is a schematic diagram of the structure of the online monitoring module in an embodiment of the present invention;

[0070] Figure 4 This is a schematic diagram of the framework for fusing graph neural networks and attention mechanisms in an embodiment of the present invention;

[0071] Figure 5 This is a schematic diagram illustrating the construction process of the dynamic correlation model between conductor temperature, sag, and current carrying capacity in an embodiment of the present invention.

[0072] Figure 6 This is a schematic diagram illustrating the workflow of the self-evolutionary prediction and feedback correction mechanism in an embodiment of the present invention.

[0073] Figure 7 This is a flowchart illustrating the real-time monitoring method for digital twins of transmission lines in an embodiment of the present invention. Detailed Implementation

[0074] Please refer to the attached document. Figure 1-7 The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0075] like Figure 1 As shown, the multimodal data fusion-based digital twin real-time monitoring system for transmission lines provided by this invention includes a data acquisition module 1, a digital twin module 2, and an online monitoring module 3. These three main modules are electrically connected to form a complete data flow path, realizing full-process monitoring from data acquisition, processing and analysis to result output.

[0076] Preferably, the data acquisition module 1 is used to collect various types of transmission line data, including optical images, environmental feature data, current waveform data, and point cloud data. In one embodiment of the invention, optical images are acquired by a line scan camera mounted on an unmanned inspection vehicle, environmental feature data is collected in real time by wireless sensor nodes distributed around the line, and current waveform data and point cloud data are obtained by a dedicated transmission line inspection robot. The data acquisition module 1 transmits this data to the digital twin module 2 via wired or wireless communication for further processing.

[0077] Specifically, the line scan camera has a resolution of no less than 2048×1536 to ensure sufficiently clear images of the power transmission lines. Environmental characteristic data includes parameters such as temperature, humidity, wind speed, and wind direction. The wireless sensor nodes collect data every 5 minutes, which meets real-time requirements without creating excessive data transmission burden. The lidar equipped on the power transmission line inspection robot has a point cloud density of no less than 100 points / square meter. This density parameter is determined based on the accuracy requirements of the 3D reconstruction of the power transmission lines. A density lower than this may lead to missed detection of minor defects, while a density higher than this will increase the computational burden of data processing.

[0078] Digital twin module 2 is electrically connected to data acquisition module 1. This module is the core processing unit of this system, responsible for receiving multimodal data, constructing a three-dimensional digital twin of the transmission line, and generating comprehensive features through advanced data fusion technology. For example... Figure 2 As shown, the digital twin module 2 includes an original image analysis module 21, a sag prediction module 22, a model fusion module 23, a partial discharge analysis module 24, and a sag calculation module 25.

[0079] The original image analysis module 21 is used to establish the correlation between image features and partial discharge waveform features. This module first preprocesses the acquired optical image, including denoising, enhancement, and normalization, and then extracts key features from the image, such as the surface condition of insulators and the surface features of conductors. Simultaneously, it analyzes the waveform data collected by the current transformer to extract partial discharge features. By establishing a mapping relationship between the two, it achieves a correlation analysis from visual features to electrical characteristics.

[0080] The sag prediction module 22 calculates the conductor-to-tower distance using fused features. This module receives the fused feature data and, based on a combination of physical models and data-driven methods, predicts conductor sag changes under different environmental conditions. In this invention, the conductor sag prediction employs a comprehensive model that considers the influence of multiple factors such as temperature, tension, and current carrying capacity. Compared to traditional methods that only consider temperature, the prediction accuracy is improved by approximately 20%.

[0081] The model fusion module 23 is responsible for point cloud registration and model updating of point cloud data. This module receives point cloud data collected by lidar, unifies point cloud data collected at different times and angles into the same coordinate system using a point cloud registration algorithm, and then compares it with the existing digital model to identify changes and update the digital twin model in a timely manner. Preferably, the point cloud registration adopts the Iterative Closest Point (ICP) algorithm combined with feature matching, achieving a registration accuracy better than 5mm, which meets the accuracy requirements for transmission line monitoring.

[0082] The partial discharge analysis module 24 establishes a correlation between the original image and the partial discharge waveform. Based on deep learning methods, this module analyzes the partial discharge signs that may exist in the optical image (such as flashover marks on the insulator surface) and the partial discharge waveform characteristics in the electrical signal, establishes a mapping relationship between the two, and realizes the early detection of potential fault points.

[0083] The sag calculation module 25 calculates the current conductor sag information based on the actual line model and the digital twin model. This module comprehensively utilizes point cloud data, environmental parameters, and mechanical models to accurately calculate the conductor sag value under the current condition and compare it with the safety threshold to promptly detect abnormalities.

[0084] In one embodiment of the present invention, the core innovation of the digital twin module 2 lies in the use of graph neural networks combined with an attention mechanism to fuse multimodal data. For example... Figure 4 As shown, the fusion framework consists of a graph structure representation layer, a graph convolutional layer, and an attention fusion layer.

[0085] The graph representation layer first transforms data from different modalities into a unified graph structure. Specifically, each component of a transmission line (such as towers, conductor segments, and insulators) is defined as a node in the graph, and each node is assigned a multi-dimensional feature vector containing all modal data features related to that node. The connections (edges) between nodes are determined based on physical connections, electrical relationships, and spatial proximity. In this way, multimodal heterogeneous data is unified into a graph representation, laying the foundation for subsequent processing.

[0086] The graph convolutional layer consists of a three-layer graph convolutional network used to extract node features and capture high-order topological relationships between nodes. The first graph convolutional layer primarily captures local topological structure and first-order neighbor relationships, with an output dimension of 64; the second graph convolutional layer integrates second-order neighbor information to form medium-scale features, with an output dimension of 32; and the third graph convolutional layer captures high-order graph structure and global dependencies, with an output dimension of 16. This hierarchical design effectively extracts multi-scale features while controlling computational complexity.

[0087] The graph convolution operation can be represented as:

[0088]

[0089] Among them, H (l+1) This is the node feature matrix of the (l+1)th layer, with dimensions N×F. l+1 N is the number of nodes, F l+1 Let l+1 be the feature dimension of the layer. To add a self-loop to the adjacency matrix, Vicon is N×N, A is the original adjacency matrix representing the connection relationship between nodes, and I is the identity matrix. Adding a self-loop ensures that the node's own information is also considered. for The degree matrix is ​​a diagonal matrix, with diagonal elements H represents the degree of node i, i.e., the number of edges connected to it; (l) Let F be the node feature matrix of the l-th layer, with dimension N×F. l ,F l W represents the feature dimension of the l-th layer. (l) Let F be the learnable weight matrix of the l-th layer. l ×F l+1 σ represents a nonlinear activation function; in this embodiment, the ReLU function is used, defined as ReLU(x) = max(0,x), to increase the nonlinear expressive power of the model. In transmission line monitoring scenarios, each node typically represents a monitoring point or line component, and its features include multimodal information such as temperature, current, and image features. Through graph convolution operations, the system can simultaneously consider the node's own features and neighboring node information, effectively capturing the mutual influence between components and the overall line status.

[0090] The attention fusion layer consists of two parts: intra-modal attention and inter-modal cross-attention. Intra-modal attention calculates the correlation between elements within the same modality, identifies and enhances important features; inter-modal cross-attention calculates the correlation between different modalities, identifies complementary information and performs synergistic enhancement.

[0091] The computation of the attention mechanism can be expressed as:

[0092]

[0093] Where Q is the query matrix, with dimensions N×d. k It is obtained from the original features through linear transformation; K is the key matrix with dimension N×d. k This corresponds to the query matrix; V is the value matrix with dimensions N×d. v , including the eigenvalues ​​that actually need to be weighted; d k The dimension of the key vector, divided by This is to scale the dot product result and prevent the gradient vanishing problem; the softmax function normalizes the attention score into a probability distribution, defined as follows: In practical applications of transmission line monitoring, Q, K, and V can be understood as feature representations of different dimensions. For example, when dealing with the relationship between temperature and sag, the temperature feature can be represented as Q, and the sag feature as K and V. Attention calculations can be used to identify the part of the sag that has the most significant impact from temperature changes.

[0094] In this embodiment, to adapt to the specific needs of transmission line monitoring, the standard attention mechanism was improved by incorporating weight adjustments based on prior knowledge. For example, when environmental conditions are severe (such as strong winds or snow), the weight of environmental feature data is increased; when abnormal current waveforms are detected, the weight of current data is increased. This adaptive adjustment mechanism significantly improves the system's ability to perceive key information.

[0095] Specifically, the improved formula for calculating the attention mechanism is as follows:

[0096]

[0097] Where M is the prior knowledge matrix, and its dimension is the same as QK. T Same, element M ij This indicates adjustments to the attention score based on prior knowledge. In practical applications, when strong winds are detected (wind speed exceeding 10 m / s), the attention weight related to environmental features is increased by approximately 50%; when the temperature change rate exceeds 5°C / hour, the weight of temperature features is increased by approximately 30%; and when the harmonic content in the current waveform exceeds 5%, the weight of current features is increased by approximately 40%. These adjustment thresholds and weighting coefficients are determined based on the analysis of a large amount of transmission line operation data and can effectively capture key information under different operating conditions.

[0098] By combining graph neural networks with attention mechanisms, this system can effectively fuse multimodal data to generate fused features with rich semantic information, providing a solid foundation for subsequent state analysis and prediction. Taking the monitoring of a 500kV transmission line as an example, the system integrates the surface state features of insulators from optical images, temperature and humidity data collected by environmental sensors, waveform features recorded by current transformers, and point cloud data from lidar scanning. Under severe weather conditions, it successfully issued an early warning of insulator flashover risk, providing an advance warning of 6 hours, giving maintenance personnel sufficient time to handle the situation and avoiding a potential large-scale power outage.

[0099] Based on the fusion features, digital twin module 2 established a dynamic correlation model of conductor temperature, sag, and current carrying capacity. For example... Figure 5 As shown, the model is implemented through a spatiotemporal fusion mechanism, which includes four parts: temporal information encoding, spatial relationship modeling, spatiotemporal fusion processing, and multi-scale spatiotemporal analysis.

[0100] The time information encoding section uses positional encoding to represent time information, capturing the periodic changes and long-term trends of the data. The positional encoding function is defined as:

[0101]

[0102] Where PE(pos,2i) is the sine code value of time position pos in the 2i-th dimension; PE(pos,2i+1) is the cosine code value of time position pos in the (2i+1)-th dimension; pos is the time position, representing the index of the data point in the time series, such as pos=1 for the first hour data, pos=2 for the second hour data, etc.; i is the dimension index, with a value range of 0≤i <d model / 2;d model The model dimension, typically 128 or 256, represents the vector dimension of the time encoding. This encoding method effectively represents time location information and has good extrapolation performance. In transmission line monitoring, time encoding is crucial for capturing daily temperature variations, periodic load changes, and seasonal variations. For example, the system can identify that 14:00-16:00 in summer is a high-incidence period for conductor temperature peaks, increasing monitoring frequency and early warning sensitivity during this period.

[0103] The spatial relationship modeling section evaluates the physical distance and functional similarity between nodes. Physical distance is calculated based on actual geographic coordinates, while functional similarity considers the node's functional role within the power grid. The spatial similarity function is defined as:

[0104]

[0105] Where, ρ ij The spatial similarity between node i and node j is expressed as d, with values ​​ranging from (0,1], where a larger value indicates a higher similarity. ij σ represents the distance between node i and node j, which can be a physical distance (meters) or a measure of functional similarity; σ is a scaling parameter adjusted according to different types of spatial relationships. In this embodiment, for physical distance, σ is set to 50 meters, meaning that when the distance between two monitoring points is 50 meters, their similarity drops to approximately 0.61; for functional similarity, σ is set to 0.5, which is determined based on the similarity of functional roles. In practical applications, for example, if the distance between adjacent towers on a 330kV transmission line is approximately 300 meters, the system will automatically calculate the spatial similarity between the towers. When an anomaly is detected in the insulator of one tower, the monitoring priority of neighboring towers will be increased accordingly, achieving "correlated monitoring".

[0106] The spatiotemporal fusion processing section adopts a "space-first, time-later" strategy, that is, spatial information is aggregated first, and then temporal dependencies are processed. Specifically, this is implemented using a two-stage attention mechanism:

[0107] Z space =SpatialAttention(X),

[0108] Z time =TemporalAttention(Z)space ),

[0109] Z = g·Z space +(1-g)·Z time ,

[0110] Where X is the input feature matrix with dimensions N×T×F, where N is the number of nodes, T is the time series length, and F is the feature dimension; Zspace is the output of spatial attention, with the same dimension as X, representing the feature representation after considering spatial relationships; Ztime is the output of temporal attention, with the same dimension as Zspace, representing the feature representation after further considering time dependencies; g is the gating parameter, ranging from [0,1], used to dynamically adjust the fusion ratio of spatial and temporal information, which is adaptively determined by the model based on the current state. SpatialAttention and TemporalAttention are the spatial attention and temporal attention calculation functions, respectively, implemented based on the aforementioned attention mechanism. In practical applications, such as monitoring changes in conductor temperature, the system first considers spatial distribution (e.g., temperature distribution at different locations), then considers temporal changes (e.g., temperature rise and fall trends), and finally integrates these two parts of information through a gating mechanism to obtain a comprehensive temperature state representation.

[0111] The multi-scale spatiotemporal analysis component performs analyses at the local spatiotemporal scale (changes at a single monitoring point over a short period), the meso-scale spatiotemporal scale (evolution over a medium time period within a region), and the macro-scale spatiotemporal scale (long-term evolution of the entire system), forming a comprehensive spatiotemporal understanding. For example, the system can simultaneously analyze minute-level changes in the surface temperature of a certain insulator (local scale), hourly trends in the load changes of that section of the line (meso-scale), and the evolution of the entire line's operational status over a month (macro-scale), thus providing a multi-angle, multi-level understanding of the line's condition.

[0112] Through this spatiotemporal fusion mechanism, this system can accurately model the dynamic relationship between three key parameters: conductor temperature, sag, and current carrying capacity, capturing their variation patterns under different environmental conditions, thus providing a theoretical basis for accurate prediction and early warning. In an application case of a 220kV transmission line, the system successfully captured the abnormal increase in conductor sag under high temperature and high load conditions in summer, predicting three hours in advance that the conductor might be approaching the lower limit of the safe distance, giving the dispatching department sufficient time to adjust load distribution and avoid potential safety hazards.

[0113] The online monitoring module 3 is electrically connected to the digital twin module 2, and is responsible for receiving the dynamic correlation model transmitted from the digital twin module 2, combining it with real-time data for status analysis and early warning. For example... Figure 3 As shown, the online monitoring module 3 includes a data fusion and mining module 31 and a result analysis and evaluation module 32.

[0114] The data fusion and mining module 31 calculates the correlation between line environmental data and line current waveforms, as well as the changing trends of historical data. This module receives real-time data from sensors, compares and analyzes it with historical data, and identifies abnormal patterns and potential risks. Preferably, this module uses a sliding window method to process time-series data. The window size is dynamically adjusted according to the changing characteristics of different parameters, typically ranging from 5 minutes to 1 hour. For example, a 15-minute window is used for temperature data; a 30-minute window is used for load data; and a 1-hour window is used for meteorological data. This dynamic window design can capture rapidly changing parameter characteristics while reducing the computational burden.

[0115] The results analysis and evaluation module 32 analyzes and evaluates the results data and sends the results to staff in chart form. Based on the results of the fusion analysis, this module determines the current line status, generates different levels of early warning information, and displays it to maintenance personnel in an intuitive chart format to support decision-making. Early warning levels are typically divided into four levels: information prompt (green), attention reminder (yellow), warning alert (orange), and emergency handling (red), each with clear triggering conditions and handling suggestions.

[0116] In one embodiment of the present invention, one of the innovations of the online monitoring module 3 is the introduction of a self-evolutionary prediction and feedback correction mechanism. For example... Figure 6 As shown, the mechanism includes an error analysis and correction unit, a model self-evolution unit, and a knowledge accumulation unit.

[0117] The error analysis and correction unit calculates the deviation between the predicted results and the actual observed values, analyzes the error distribution characteristics, and adjusts the prediction parameters based on the error distribution. Two indicators are used for error calculation: mean square error (MSE) and mean absolute error (MAE).

[0118]

[0119] Where MSE is the mean squared error, used to measure the accuracy of the prediction, and is more sensitive to larger errors; MAE is the mean absolute error, used to measure the degree of deviation in the prediction, and treats all errors equally; y i These are actual observed values, such as measured temperature or sag values; is the corresponding predicted value; n is the number of samples, usually taken from the observation data points in the most recent 24 hours. This represents summing over all sample points, with i iterating from 1 to n. In transmission line monitoring, MSE and MAE provide different perspectives on prediction quality assessment. For example, when monitoring conductor temperature, an MAE value controlled within 1℃ and an MSE value controlled within 1.5℃² are considered to have good prediction performance.

[0120] When the error exceeds a preset threshold, the system triggers a parameter adjustment process. For temperature prediction, the threshold is set at 1.5℃; for sag prediction, it's set at 5cm; and for current carrying capacity prediction, it's set at 3% of the line's rated current carrying capacity. These threshold values ​​are determined based on power system operating experience and safety margins, ensuring timely detection of anomalies while avoiding frequent and unnecessary adjustments. For example, the typical rated current carrying capacity of a 330kV transmission line is approximately 2000A, therefore the threshold for current carrying capacity prediction is 60A. After a thunderstorm, the system detected that the conductor temperature prediction error consistently exceeded 1.5℃, triggering parameter adjustment. Analysis revealed that this was due to rainwater erosion altering the conductor surface condition. The system automatically adjusted the parameters of the thermal balance model, restoring the prediction to high accuracy.

[0121] The model self-evolutionary unit updates the model structure and parameters based on performance evaluations and environmental changes. This unit employs an incremental learning strategy, incorporating information from new data while retaining existing knowledge. Parameter updates utilize gradient descent.

[0122]

[0123] Where, θ t+1 The updated model parameter vector; θ t This is the current model parameter vector, including the weights and biases of the neural network; η is the learning rate, which controls the step size for parameter updates. The loss function L with respect to the parameter θ t The gradient vector represents the direction and rate of change of the loss function at the current parameter point. An adaptive learning rate strategy is employed, initially set to 0.01 and gradually decreased during training, but never falling below 0.001, to maintain the model's adaptability to new data. In practical applications, for example, when the system identifies a new weather pattern (such as the first encounter with frost), the learning rate is increased to quickly adapt to the new situation; while during stable operation, a smaller learning rate is used for fine-tuning.

[0124] The knowledge accumulation unit retains key historical experiences and patterns to prevent new knowledge from overwriting useful old knowledge. This unit employs an experience replay mechanism, regularly reviewing key historical cases, especially anomalous cases under extreme weather conditions, to ensure the model does not "forget" important experiences. In a practical application on a 500kV transmission line, the system retained data and patterns from the previous winter's extreme icing conditions. Although no similar extreme conditions occurred throughout the winter of that year, the system still maintained its ability to identify and warn of such situations, demonstrating the value of knowledge accumulation.

[0125] In this embodiment, the feedback correction mechanism triggers an update when the prediction accuracy falls below 90% or when environmental conditions change significantly. "Significant changes" in environmental conditions are defined as: temperature changes exceeding 10°C / hour, wind speed changes exceeding 5 m / s, or the occurrence of special weather events such as rain or snow. These triggering conditions ensure that the system can respond promptly to environmental changes while avoiding excessively frequent updates that would waste computational resources. For example, during a cold air mass passage, if the temperature in a certain area drops by 15°C within 3 hours, the system automatically triggers a model update process, optimizing the temperature-sag relationship model to make the prediction results more consistent with the actual situation under rapid cooling conditions.

[0126] Through a self-evolving prediction and feedback correction mechanism, the online monitoring module 3 can continuously optimize the prediction model, improve prediction accuracy, and adapt to constantly changing environments and operating conditions. Taking the monitoring of an important power transmission channel as an example, after the system was put into use for 6 months, the average error of temperature prediction decreased from the initial 1.2℃ to 0.7℃, and the average error of sag prediction decreased from 4cm to 2.5cm, demonstrating the system's self-optimization capability.

[0127] Another innovation of the online monitoring module 3 is the adoption of a multi-level forecasting framework. This framework divides forecasts into three levels—short-term, medium-term, and long-term—based on time scales.

[0128] Short-term forecasts predict state changes within the next 10 minutes, updating every 10 seconds, and are primarily used for real-time monitoring and immediate early warning. Short-term forecasts employ a direct prediction method based on the current state and short-term trends, offering rapid response and suitability for handling emergencies. For example, when a tree branch is detected approaching a conductor on a section of the line, the system will generate an early warning within seconds, providing time for emergency response.

[0129] The intermediate-term forecast predicts state changes from 10 minutes to 2 hours, updated every 10 minutes, and is primarily used for fault prevention and operation scheduling. The intermediate-term forecast combines recursive prediction and intermediate-term trend analysis, balancing accuracy and real-time performance. For example, the system can predict load change trends for the next hour, providing a reference for power allocation decisions by the scheduling department.

[0130] Long-term forecasts predict status changes over a period of more than two hours and are updated hourly. They are primarily used for maintenance planning and resource allocation. Long-term forecasts incorporate weather forecast data to comprehensively consider the long-term impact of meteorological factors on line status. For example, based on 24-hour weather forecasts, the system can predict potential icing risk periods, assisting maintenance departments in scheduling inspections and protective measures in advance.

[0131] This hierarchical prediction strategy enables the system to provide accurate prediction results at different time scales, meeting the needs of various application scenarios. In the monitoring of a transmission line spanning a complex mountainous area, the system's multi-level prediction capability improved operation and maintenance efficiency by approximately 35% and reduced the line failure rate by approximately 25%, demonstrating the practical value of the technical solution.

[0132] This invention also provides a method for real-time monitoring of digital twins of transmission lines based on the above system. For example... Figure 7 As shown, the method includes the following steps:

[0133] Step 1: Construct a three-dimensional digital twin of the transmission line using laser point cloud, infrared thermal imaging and current waveform data fusion technology.

[0134] Specifically, the system first acquires point cloud data from a lidar sensor, then uses a point cloud registration algorithm to unify multiple scans into the same coordinate system. Next, it acquires temperature distribution images from an infrared thermal imaging camera, registers them with the point cloud data, and adds temperature information to the 3D model. Then, it acquires current waveform data from a current transformer, extracts features, and correlates them with spatial location. Finally, based on this multimodal data, a complete 3D digital twin with geometric shape, temperature distribution, and electrical characteristics is constructed. In an application example on a 500kV transmission line, the system used a 1024×768 resolution infrared thermal imager with a temperature resolution of 0.05℃ and a lidar point cloud density of 200 points / square meter, successfully constructing a high-precision digital twin model with geometric accuracy better than 2cm and temperature mapping accuracy better than 0.5℃.

[0135] Step 2: Design a cross-modal attention mechanism and establish a dynamic correlation model of conductor temperature-sag-current carrying capacity through a graph neural network to achieve real-time evaluation of conductor thermal stability.

[0136] In this step, each component of the transmission line is first represented as a node in a graph structure, with node features containing multimodal data information. Then, a three-layer graph convolutional network is used to extract node features and high-order topological relationships. Next, a cross-modal attention mechanism is applied to adaptively allocate weights for different modal data. Then, a dynamic correlation model between conductor temperature, sag, and current carrying capacity is established based on fused features. Finally, a spatiotemporal fusion mechanism is used to capture the variation of parameters with time and environment, enabling real-time assessment of conductor thermal stability. For example, the system can establish a dynamic relationship model in which a 1°C increase in conductor temperature leads to an increase in sag of approximately 1.5-2 cm, while also considering the influence of environmental factors such as wind speed and solar radiation, providing more accurate assessment results than static models.

[0137] Step 3: Integrate edge computing nodes to complete partial discharge feature extraction within 10ms, and combine meteorological data to predict icing risk, with a classification accuracy of over 98.5%.

[0138] In this step, edge computing nodes are first deployed at key monitoring points, with hardware configurations including industrial-grade ARM processors, 4GB RAM, and GPU acceleration support. Then, a lightweight algorithm model is used to optimize the calculation process, achieving partial discharge feature extraction within 10ms. Next, meteorological data, including parameters such as temperature, humidity, wind speed, and precipitation, is acquired. Then, an icing risk value is calculated using a FROST-like icing model. Finally, by combining preliminary analysis from edge nodes with in-depth analysis from the cloud, high-precision icing risk prediction is achieved, with a classification accuracy rate exceeding 98.5%. In a practical application on a transmission line in a northern province, the system successfully issued 18 icing risk warnings during a complete winter monitoring period, with an average warning lead time of 4.5 hours, allowing the maintenance department ample time to take preventative measures. All warnings were proven effective, with no missed or false alarms.

[0139] Step 4: Through self-evolutionary prediction and feedback correction mechanisms, combined with a multi-level prediction framework, short-term, medium-term and long-term fault prediction and early warning are achieved.

[0140] This process first constructs a three-tiered prediction framework—short-term, medium-term, and long-term—each responsible for predictions at different time scales. Then, the deviation between the predicted results and actual observations is calculated in real time, and the error distribution characteristics are analyzed. Next, model parameters are adjusted based on feedback to optimize prediction performance. Then, key historical experiences are retained through knowledge accumulation units to prevent "catastrophic forgetting." Finally, a closed-loop feedback mechanism is formed, enabling the system to continuously learn and evolve, thereby continuously improving prediction accuracy. In an application case of a major power transmission channel, after one year of system operation, the fault prediction accuracy increased from the initial 85% to over 95%, and the average early warning time increased by 30%, demonstrating the long-term value of the self-evolutionary mechanism.

[0141] Step 5: Based on the abnormal status and prediction results, generate multi-level early warning information and send it to staff in the form of charts to realize intelligent operation and maintenance decision support.

[0142] This step first sets multi-level warning thresholds based on different types of anomalies, such as excessive sag, high temperature, and frequent partial discharge. Then, it generates different levels of warning information by comparing the predicted results with the thresholds. Next, the warning information is displayed in intuitive charts, including trend graphs, heat maps, and 3D visualizations. Finally, it provides maintenance personnel with handling suggestions to support intelligent maintenance decision-making. For example, when the system detects a gradual increase in the surface pollution of insulators on a certain section of the line and predicts a potential flashover risk, it generates an orange warning and suggests prioritizing this section during the next planned maintenance. It also provides a heat map of the pollution distribution and historical comparison data to help maintenance personnel fully understand the situation and make informed decisions.

[0143] Through the above steps, the method of the present invention can comprehensively monitor the status of transmission lines, promptly identify potential hidden dangers, predict possible fault risks, provide a scientific basis for operation and maintenance decisions, and significantly improve the safety and reliability of power grid operation.

[0144] The above embodiments are merely preferred embodiments of the present invention, and not all embodiments. Those skilled in the art, based on the descriptions of the above embodiments, can make various modifications and improvements to the embodiments without departing from the basic spirit of the present invention. Therefore, these modifications or improvements all fall within the protection scope of the present invention.

[0145] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A real-time monitoring system for digital twins of transmission lines based on multimodal data fusion, characterized in that... It includes a data acquisition module, a digital twin module, and an online monitoring module; The data acquisition module is used for: Acquire optical images, environmental feature data, current waveform data, and point cloud data; The optical image, the environmental feature data, the current waveform data, and the point cloud data are transmitted to the digital twin module; The digital twin module is electrically connected to the data acquisition module and is used for: Receive the optical image, the environmental feature data, the current waveform data, and the point cloud data; Constructing a three-dimensional digital twin of a power transmission line; Each component of the transmission line is represented as a node in a graph structure. Each node contains the multimodal data features of the corresponding component, and the edges between nodes represent the physical connection relationship, electrical relationship and spatial proximity relationship between the components. A three-layer graph convolutional network is used to extract node features and capture high-order topological relationships between nodes. The three-layer graph convolutional network includes a first graph convolutional layer to extract local topological structure and first-order neighbor relationships, a second graph convolutional layer to integrate second-order neighbor information to form medium-scale features, and a third graph convolutional layer to capture high-order graph structure and global dependencies. The weights of different modal data are adaptively allocated by using intramodal attention and intermodal cross-attention, and the optical image, the environmental feature data, the current waveform data and the point cloud data are fused to generate multimodal fusion features; Based on the aforementioned multimodal fusion characteristics, a dynamic correlation model of conductor temperature-sag-current carrying capacity is established through a spatiotemporal fusion mechanism. The spatiotemporal fusion mechanism includes four components: time information encoding, spatial relationship modeling, spatiotemporal fusion processing, and multi-scale spatiotemporal analysis. The dynamic correlation model of conductor temperature-sag-current carrying capacity is transmitted to the online monitoring module; The online monitoring module is electrically connected to the digital twin module and is used for: Receive the dynamic correlation model of conductor temperature-sag-current carrying capacity; By integrating real-time and historical data, the status of the line can be analyzed. Generate multi-level early warning information; Output line status analysis results and fault prediction information.

2. The real-time monitoring system for digital twin of transmission lines based on multimodal data fusion according to claim 1, characterized in that... The digital twin module includes: The original image analysis module is used to establish the correlation between image features and partial discharge waveform features; The sag prediction module is used to calculate the conductor-tower distance using fused features; The model fusion module is used to perform point cloud registration and model updates on point cloud data. The partial discharge analysis module is used to establish the correlation between the original image and the partial discharge waveform; The sag calculation module is used to calculate the current conductor sag information based on the actual line model and the digital twin model.

3. The real-time monitoring system for digital twin of transmission lines based on multimodal data fusion according to claim 1, characterized in that... The online monitoring module includes: The data fusion and mining module is used to calculate the correlation between line environmental data and line current waveforms, as well as the changing trends of historical data. The results analysis and evaluation module is used to analyze and evaluate the results data and send the results to staff in the form of charts.

4. The real-time monitoring system for digital twin of transmission lines based on multimodal data fusion according to claim 2, characterized in that... The original image analysis module, the sag prediction module, the model fusion module, the partial discharge analysis module, and the sag calculation module employ a graph neural network combined with an attention mechanism to fuse multimodal data. The graph neural network includes three graph convolutional layers, which are used to extract node features and capture high-order topological relationships between nodes. The attention mechanism includes intramodal attention and intermodal cross-attention, which are used to adaptively allocate weights for data from different modalities. The graph neural network and the attention mechanism are combined to achieve deep fusion of multimodal data.

5. The real-time monitoring system for digital twin of transmission lines based on multimodal data fusion according to claim 1, characterized in that... The optical images are obtained through a line-scan camera mounted on an unmanned inspection vehicle; the environmental feature data are obtained through wireless sensor nodes; and the current waveform data and point cloud data are obtained through a power transmission line inspection robot. The resolution of the line scan camera is no less than 2048×1536; The wireless sensor node collects environmental feature data every 5 minutes. The power transmission line inspection robot is equipped with a lidar with a point cloud density of no less than 100 points per square meter.

6. The real-time monitoring system for digital twin of transmission lines based on multimodal data fusion according to claim 1, characterized in that... The digital twin module is based on the principle of digital twins. It uses lidar technology and infrared thermal imaging technology to realize three-dimensional digital twin imaging. Combined with real-time current waveform data collected by current sensors, the collected images, current data and point cloud data are digitally twinned to generate a digital twin circuit. The temperature information of the collection points, the current data collected by the current sensors and the images are associated to update the digital twin model and calculate the sag and sag prediction value.

7. The real-time monitoring system for digital twin of transmission lines based on multimodal data fusion according to claim 1, characterized in that... The online monitoring module takes as input data detected by sensors and outputs as fault prediction results and fault identification; wherein, The fault prediction results include short-term prediction results, medium-term prediction results and long-term prediction results; The short-term forecast results predict state changes within the next 10 minutes and are updated every 10 seconds. The intermediate-term forecast results predict state changes from 10 minutes to 2 hours and are updated every 10 minutes. The long-term prediction results predict state changes over a period of more than 2 hours and are updated hourly. The fault identification includes partial discharge faults, abnormal sag, and icing risk.

8. The real-time monitoring system for digital twin of transmission lines based on multimodal data fusion according to claim 1, characterized in that... The dynamic correlation model of conductor temperature-sag-current carrying capacity is realized through a spatiotemporal fusion mechanism, which includes: Time information encoding is used to capture the periodic changes and long-term trends of data; Spatial relationship modeling is used to evaluate the physical distance and functional similarity between nodes; Spatiotemporal fusion processing is used to integrate information from both temporal and spatial dimensions; Multiscale spatiotemporal analysis is used to analyze state changes at different spatiotemporal scales.

9. The real-time monitoring system for digital twin of transmission lines based on multimodal data fusion according to claim 1, characterized in that... The online monitoring module includes a self-evolutionary prediction and feedback correction mechanism, which includes: The error analysis and correction unit is used to calculate the deviation between the prediction results and the actual observations, and to adjust the prediction parameters based on the error distribution. Model self-evolutionary units are used to update the model structure and parameters based on performance evaluations and environmental changes. Knowledge accumulation units are used to retain key historical experiences and patterns, preventing new knowledge from overwriting useful old knowledge. The feedback correction mechanism triggers an update when the prediction accuracy falls below a threshold or when environmental conditions change significantly.

10. A method for real-time monitoring of digital twins of transmission lines based on the system described in claim 1, characterized in that... The specific steps are as follows: Step 1: Construct a three-dimensional digital twin of the transmission line using laser point cloud, infrared thermal imaging, and current waveform data fusion technology; Step 2: Design a cross-modal attention mechanism and establish a dynamic correlation model of conductor temperature-sag-current carrying capacity through a graph neural network to achieve real-time evaluation of conductor thermal stability. Step 3: Integrate edge computing nodes to complete partial discharge feature extraction within 10ms, and combine meteorological data to predict icing risk, with a classification accuracy of over 98.5%. Step 4: Through self-evolutionary prediction and feedback correction mechanisms, combined with a multi-level prediction framework, short-term, medium-term, and long-term fault prediction and early warning are achieved. Step 5: Based on the abnormal status and prediction results, generate multi-level early warning information and send it to staff in the form of charts to realize intelligent operation and maintenance decision support.

Images