Multi-modal data fusion power transmission line digital twinning real-time monitoring system and method

Abstract

The invention relates to the technical field of power transmission line monitoring, in particular to a multi-modal data fusion power transmission line digital twinning real-time monitoring system and method, and the system comprises a data collection module which is responsible for collecting optical images, environment characteristic data, current waveform data and point cloud data, and transmitting the data to a digital twinning module; the digital twinborn module constructs a three-dimensional digital twinborn body of the power transmission line, fuses the multi-modal data through a graph neural network and an attention mechanism, generates multi-modal fusion features, establishes a dynamic correlation model of conductor temperature-sag-current-carrying capacity based on the features, and transmits the dynamic correlation model to the online monitoring module; the on-line monitoring module fuses real-time collected data and historical data, analyzes the line state, generates multi-level early warning information and outputs a line state analysis result and fault prediction information, the system adopts an innovative data fusion method, the sensitivity and accuracy of fault detection are improved, and the operation safety and efficiency of the power transmission line are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of transmission line monitoring, and in particular to a digital twin real-time monitoring system and method for transmission lines using multimodal data fusion. Background Art

[0002] As a vital component of the power system, the safe and stable operation of transmission lines is crucial for ensuring power supply. Traditional transmission line monitoring relies primarily on manual inspections and single-sensor monitoring. These methods suffer from limited accuracy, poor real-time performance, and weak early warning capabilities. Especially under complex weather conditions, such as icing and strong winds, traditional monitoring methods struggle to detect potential hazards in a timely manner, increasing the risk of line failures.

[0003] With the development of sensing, communication, and artificial intelligence technologies, the coordinated utilization of multiple data sources has become an important approach to improving monitoring efficiency. However, existing technologies face numerous challenges in multimodal data fusion: First, different types of data (such as optical images, point cloud data, and current waveforms) have heterogeneous formats and inconsistent temporal and spatial distributions, making effective integration difficult. Second, traditional data fusion methods often rely on simple concatenation or weighted averaging, lacking in-depth exploration of the complex relationships between modalities. Third, static data analysis models struggle to adapt to the dynamic characteristics of transmission lines under varying environmental conditions, impacting prediction accuracy. Finally, once deployed, the model's parameters are fixed, making it unable to adapt to long-term changes in the system's operating state and environment.

[0004] Existing transmission line monitoring systems have significant shortcomings in processing multimodal data, building dynamic correlation models, and implementing adaptive prediction, making them difficult to meet the requirements of high-reliability grid operation. Therefore, there is an urgent need to develop a transmission line monitoring system that can deeply integrate multimodal data, accurately model dynamic relationships, and possess self-evolutionary capabilities. Summary of the Invention

[0005] The purpose of the present invention is to provide a real-time monitoring system and method for digital twins of transmission lines with multimodal data fusion, aiming to solve technical problems in the existing technology such as insufficient depth of multimodal data fusion, limited dynamic correlation modeling capabilities, and weak adaptability, and to improve the accuracy, real-time performance and predictive capabilities of transmission line status monitoring.

[0006] The present invention proposes a digital twin real-time monitoring system for transmission lines with multimodal data fusion, comprising:

[0007] The data acquisition module is used to:

[0008] Collect optical images, environmental feature data, current waveform data and point cloud data;

[0009] transmitting the optical image, the environmental characteristic data, the current waveform data, and the point cloud data to the digital twin module;

[0010] The digital twin module is electrically connected to the data acquisition module and is used to:

[0011] receiving the optical image, the environmental characteristic data, the current waveform data, and the point cloud data;

[0012] Build a three-dimensional digital twin of the transmission line;

[0013] fusing the optical image, the environmental feature data, the current waveform data, and the point cloud data through a graph neural network and an attention mechanism to generate a multimodal fusion feature;

[0014] Establishing a dynamic correlation model of conductor temperature, sag and current carrying capacity based on the multimodal fusion features;

[0015] Transmitting the conductor temperature-sag-current carrying capacity dynamic correlation model to the online monitoring module;

[0016] The online monitoring module is electrically connected to the digital twin module and is used to:

[0017] receiving a dynamic correlation model of conductor temperature, sag and current carrying capacity;

[0018] Integrate real-time collected data with historical data to analyze line status;

[0019] Generate multi-level warning information;

[0020] Output line status analysis results and fault prediction information.

[0021] Preferably, the digital twin module includes:

[0022] The original image analysis module is used to establish the correlation between image features and partial discharge waveform features;

[0023] The sag prediction module is used to calculate the conductor-tower distance using fused features;

[0024] Model fusion module, used for point cloud registration and model update of point cloud data;

[0025] Partial discharge analysis module, used to establish the correlation between the original image and the partial discharge waveform;

[0026] The sag calculation module is used to calculate the current conductor sag information based on the real line model and the digital twin model.

[0027] Preferably, the online monitoring module includes:

[0028] Data fusion mining module, used to calculate the correlation between line environment data and line current waveform and the change trend of historical data;

[0029] The result analysis and evaluation module is used to analyze and evaluate the result data and send the results to the staff in the form of charts.

[0030] Preferably, the original image analysis module, the sag prediction module, the model fusion module, the partial discharge analysis module and the sag calculation module use a graph neural network combined with an attention mechanism to fuse multimodal data, wherein:

[0031] The graph neural network includes three layers of graph convolutional layers, which are used to extract node features and capture high-order topological relationships between nodes;

[0032] The attention mechanism includes intra-modality attention and inter-modality cross-attention, which are used to adaptively assign weights to data of different modalities;

[0033] The graph neural network and the attention mechanism are combined to achieve deep fusion of multimodal data.

[0034] Preferably, the optical image is obtained by a linear array camera mounted on an unmanned patrol vehicle, the environmental feature data is obtained by a wireless sensor node, and the current waveform data and the point cloud data are obtained by a power transmission line inspection robot; wherein,

[0035] The resolution of the linear array camera is not less than 2048×1536;

[0036] The wireless sensor node collects environmental feature data every 5 minutes;

[0037] The transmission line inspection robot is equipped with a laser radar, and the point cloud density is not less than 100 points per square meter.

[0038] Preferably, the digital twin module is based on the digital twin principle, uses lidar technology and infrared thermal imaging technology to achieve three-dimensional digital twin imaging, combines the real-time current waveform data collected by the current sensor, digitally twins the collected images, current data and point cloud data, generates a digital twin circuit, associates the temperature information of the collection point, the current data collected by the current sensor and the image, realizes the update of the digital twin model, and calculates the sag and sag prediction value.

[0039] Preferably, the input of the online monitoring module is the data detected by the sensor, and the output is the fault prediction result and fault identification; wherein,

[0040] The fault prediction results include short-term prediction results, medium-term prediction results and long-term prediction results;

[0041] The short-term prediction results predict the state changes within the next 10 minutes and are updated every 10 seconds;

[0042] The medium-term forecast results predict the state changes of 10 minutes to 2 hours and are updated every 10 minutes;

[0043] The long-term prediction results predict state changes of more than 2 hours and are updated every hour;

[0044] The fault identification includes partial discharge fault, sag abnormality and icing risk.

[0045] Preferably, the conductor temperature-sag-current carrying capacity dynamic correlation model is implemented by a time-space fusion mechanism, and the time-space fusion mechanism includes:

[0046] Temporal information encoding is used to capture periodic changes and long-term trends in data;

[0047] Spatial relationship modeling, used to assess the physical distance and functional similarity between nodes;

[0048] Spatiotemporal fusion processing, used to integrate information in time and space dimensions;

[0049] Multi-scale spatiotemporal analysis is used to analyze state changes at different spatiotemporal scales.

[0050] Preferably, the online monitoring module includes a self-evolution prediction and feedback correction mechanism, and the self-evolution prediction and feedback correction mechanism includes:

[0051] Error analysis and correction unit, used to calculate the deviation between the predicted results and the actual observed values, and adjust the prediction parameters based on the error distribution;

[0052] Model self-evolution unit, used to update model structure and parameters based on performance evaluation and environmental changes;

[0053] Knowledge accumulation unit, used to preserve key historical experiences and patterns and prevent new knowledge from overwriting useful old knowledge;

[0054] The feedback correction unit triggers an update when the prediction accuracy is lower than a threshold or the environmental conditions change significantly.

[0055] The specific steps of the real-time monitoring method of digital twins of transmission lines based on the system are as follows:

[0056] Step 1: Use laser point cloud, infrared thermal imaging, and current waveform data fusion technology to build a 3D digital twin of the transmission line;

[0057] Step 2: Design a cross-modal attention mechanism and establish a dynamic correlation model between conductor temperature, sag, and current carrying capacity through a graph neural network to achieve real-time assessment of conductor thermal stability.

[0058] Step 3: Integrate edge computing nodes to complete partial discharge feature extraction within 10ms and combine meteorological data to predict icing risk, with a classification accuracy of over 98.5%;

[0059] Step 4: Through the self-evolution prediction and feedback correction mechanism, combined with a multi-level prediction framework, short-term, medium-term and long-term fault prediction and warning are achieved;

[0060] Step 5: Based on the abnormal status and prediction results, generate multi-level warning information and send it to the staff in the form of charts to achieve intelligent operation and maintenance decision support.

[0061] The beneficial effects of the present invention include:

[0062] 1. Through the innovative combination of graph neural networks and attention mechanisms, deep fusion of optical images, environmental feature data, current waveform data, and point cloud data is achieved, breaking through the limitations of traditional data fusion. It can mine implicit fault features from multi-source heterogeneous data, thereby improving the sensitivity and accuracy of fault detection.

[0063] 2. A dynamic correlation model of conductor temperature, sag and current carrying capacity was established, which can accurately capture the changing patterns and mutual influence of these three key parameters in time and space dimensions, solve the problem that static models cannot adapt to complex environmental changes, and make the prediction results more consistent with physical reality.

[0064] 3. A self-evolution prediction and feedback correction mechanism has been designed. The system can continuously optimize model parameters based on real-time monitoring data, achieve self-learning and adaptation, and the accuracy of icing risk prediction reaches more than 98.5%, which is significantly higher than traditional methods.

[0065] 4. By adopting edge computing optimization and lightweight design, while maintaining the model's expressiveness, the real-time requirement of partial discharge feature extraction is achieved within 10ms, improving the system's response speed to emergencies.

[0066] 5. Through the time stratification (short-term, medium-term, and long-term) of fault prediction and the multi-level early warning mechanism, comprehensive decision-making support is provided for operation and maintenance personnel, effectively reducing maintenance costs, extending equipment service life, and improving grid operation reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 Schematic diagram of the overall architecture of a digital twin real-time monitoring system for transmission lines with multimodal data fusion in an embodiment of the present invention;

[0068] Figure 2 Schematic diagram of the structure of the digital twin module in an embodiment of the present invention;

[0069] Figure 3 This is a schematic structural diagram of an online monitoring module in an embodiment of the present invention;

[0070] Figure 4 Schematic diagram of the fusion framework of the graph neural network and the attention mechanism in an embodiment of the present invention;

[0071] Figure 5 Schematic diagram of the process of constructing a conductor temperature-sag-current carrying capacity dynamic correlation model in an embodiment of the present invention;

[0072] Figure 6 Schematic diagram of the workflow of the self-evolution prediction and feedback correction mechanism in an embodiment of the present invention;

[0073] Figure 7 Schematic diagram of the process of real-time monitoring method of digital twin of transmission line in an embodiment of the present invention. DETAILED DESCRIPTION

[0074] Please refer to the attached Figure 1-7 The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0075] like Figure 1 As shown in the figure, the multimodal data fusion digital twin real-time monitoring system for power transmission lines provided by the present invention includes a data acquisition module 1, a digital twin module 2, and an online monitoring module 3. These three main modules are electrically connected to form a complete data flow path, realizing full-process monitoring from data acquisition, processing and analysis to result output.

[0076] Preferably, the data acquisition module 1 is used to collect various types of transmission line data, including optical images, environmental characteristic data, current waveform data, and point cloud data. In one embodiment of the present invention, optical images are acquired by a linear array camera mounted on an unmanned patrol vehicle, environmental characteristic data is collected in real time by wireless sensor nodes distributed around the transmission line, and current waveform data and point cloud data are acquired by a dedicated transmission line inspection robot. The data acquisition module 1 transmits this data via wired or wireless communication to the digital twin module 2 for subsequent processing.

[0077] Specifically, the resolution of the line array camera must be no less than 2048×1536 to ensure sufficiently clear line images. Environmental characteristic data, including temperature, humidity, wind speed, and wind direction, is collected every five minutes by wireless sensor nodes, meeting real-time requirements without incurring excessive data transmission burdens. The laser radar onboard the power transmission line inspection robot maintains a point cloud density of no less than 100 points per square meter. This density is determined based on the accuracy requirements for 3D reconstruction of power transmission lines. Densities below this threshold may result in missed detection of minor defects, while those above this threshold increase the computational burden of data processing.

[0078] The digital twin module 2 is electrically connected to the data acquisition module 1. This module is the core processing unit of the system and is responsible for receiving multimodal data, building a three-dimensional digital twin of the transmission line, and generating comprehensive features through advanced data fusion technology. Figure 2 As shown, the digital twin module 2 includes an original image analysis module 21 , a sag prediction module 22 , a model fusion module 23 , a partial discharge analysis module 24 and a sag calculation module 25 .

[0079] The raw image analysis module 21 is used to establish a correlation between image features and partial discharge waveform characteristics. This module first preprocesses the acquired optical image, including denoising, enhancement, and normalization. It then extracts key features from the image, such as the surface condition of the insulator and the surface characteristics of the conductor. Simultaneously, it analyzes the waveform data collected by the current transformer to extract partial discharge characteristics. By establishing a mapping relationship between the two, correlation analysis from visual features to electrical characteristics is achieved.

[0080] The sag prediction module 22 uses fused features to calculate the conductor-tower distance. This module receives the fused feature data and, based on a combination of physical models and data-driven methods, predicts conductor sag changes under different environmental conditions. This invention utilizes a comprehensive model that considers multiple factors, including temperature, tension, and current carrying capacity. Compared to traditional methods that only consider temperature, the prediction accuracy is improved by approximately 20%.

[0081] The model fusion module 23 is responsible for performing point cloud registration and model updates on the point cloud data. This module receives point cloud data collected by the LiDAR and, using a point cloud registration algorithm, unifies the point cloud data collected at different times and angles into a common coordinate system. This data is then compared with the existing digital model, identifying changes and promptly updating the digital twin model. Preferably, point cloud registration utilizes an iterative closest point (ICP) algorithm combined with feature matching, achieving a registration accuracy of better than 5mm, meeting the precision requirements for transmission line monitoring.

[0082] The partial discharge analysis module 24 establishes a correlation between the original image and the partial discharge waveform. Based on deep learning methods, this module analyzes possible partial discharge signs in the optical image (such as flashover traces on the insulator surface) with the partial discharge waveform characteristics in the electrical signal, establishing a mapping between the two and enabling early detection of potential fault points.

[0083] The sag calculation module 25 calculates the current conductor sag information based on the actual line model and the digital twin model. This module uses point cloud data, environmental parameters, and mechanical models to accurately calculate the conductor's current sag value and compares it with safety thresholds to detect anomalies in a timely manner.

[0084] In one embodiment of the present invention, the core innovation of the digital twin module 2 is to fuse multimodal data using a graph neural network combined with an attention mechanism. Figure 4 As shown in Figure 3, the fusion framework consists of a graph structure representation layer, a graph convolution layer, and an attention fusion layer.

[0085] The graph representation layer first converts data from different modalities into a unified graph structure. Specifically, each component of a transmission line (such as towers, conductor segments, and insulators) is defined as a node in the graph. Each node is assigned a multidimensional feature vector containing all the modal data features associated with that node. The connections (edges) between nodes are determined based on physical connections, electrical relationships, and spatial proximity. In this way, multimodal, heterogeneous data is unified into a graph representation, laying the foundation for subsequent processing.

[0086] The graph convolution layer consists of three layers of graph convolutional networks, which are used to extract node features and capture high-order topological relationships between nodes. The first layer of graph convolution primarily captures local topological structure and first-order neighbor relationships, with an output dimension of 64. The second layer of graph convolution integrates second-order neighbor information to form medium-scale features, with an output dimension of 32. The third layer of graph convolution captures high-order graph structure and global dependencies, with an output dimension of 16. This hierarchical design effectively extracts multi-scale features while controlling computational complexity.

[0087] The graph convolution operation can be expressed as:

[0088]

[0089] Among them, H (l+1) is the node feature matrix of the l+1th layer, with a dimension of N×F l+1 ,N is the number of nodes, F l+1 is the feature dimension of the l+1th layer; To add the adjacency matrix of self-loops, the Weikang is N×N, A is the original adjacency matrix, which represents the connection relationship between nodes, and I is the identity matrix. Adding self-loops ensures that the node's own information is also considered; for The degree matrix is ​​a diagonal matrix with diagonal elements represents the degree of node i, that is, the number of edges connected to it; H (l) is the node feature matrix of the lth layer, with a dimension of N×F l ,F l is the feature dimension of the lth layer; W (l) is the learnable weight matrix of layer l, with dimension F l ×F l+1 σ is a nonlinear activation function. In this embodiment, the ReLU function is used, defined as ReLU(x) = max(0, x), to enhance the model's nonlinear expressiveness. In power transmission line monitoring scenarios, each node typically represents a monitoring point or line component, and its characteristics include multimodal information such as temperature, current, and image features. Through graph convolution, the system can simultaneously consider the node's own characteristics and neighboring node information, effectively capturing the mutual influence between components and the overall line status.

[0090] The attention fusion layer consists of two parts: intra-modal attention and inter-modal cross-attention. Intra-modal attention calculates the correlation between elements within the same modality, identifying and enhancing important features; inter-modal cross-attention calculates the correlation between different modalities, identifying complementary information and synergistically enhancing it.

[0091] The calculation of the attention mechanism can be expressed as:

[0092]

[0093] Among them, Q is the query matrix (Query), the dimension is N×d k , obtained by linear transformation of the original features; K is the key matrix (Key), with a dimension of N×d k , corresponding to the query matrix; V is the value matrix (Value), with a dimension of N×d v , contains the eigenvalues ​​that actually need to be weighted; d k is the dimension of the key vector, divided by It is to scale the dot product result to prevent the gradient disappearance problem; the softmax function normalizes the attention score into a probability distribution, which is defined as In practical applications of transmission line monitoring, Q, K, and V can be understood as feature representations of different dimensions. For example, when dealing with the relationship between temperature and sag, the temperature feature can be represented as Q, and the sag features can be represented as K and V. Attention calculations can be used to identify the areas where temperature changes have the most significant impact on sag.

[0094] In this embodiment, to meet the specialized needs of transmission line monitoring, the standard attention mechanism has been improved by incorporating weight adjustments based on prior knowledge. For example, when environmental conditions are severe (such as strong winds, ice, and snow), the weight of environmental feature data is increased; when abnormal current waveforms are detected, the weight of current data is increased. This adaptive adjustment mechanism significantly improves the system's ability to perceive critical information.

[0095] Specifically, the calculation formula of the improved attention mechanism is:

[0096]

[0097] Among them, M is the prior knowledge matrix, and its dimension is the same as QK T Same, element M ij Represents an adjustment to the attention score based on prior knowledge. In practice, when strong winds (wind speeds exceeding 10m / s) are detected, the attention weight associated with environmental features is increased by approximately 50%. When the temperature change rate exceeds 5°C / hour, the weight of the temperature feature is increased by approximately 30%. When the harmonic content in the current waveform exceeds 5%, the weight of the current feature is increased by approximately 40%. These adjustment thresholds and weighting coefficients are determined based on analysis of extensive transmission line operation data and effectively capture key information under different operating conditions.

[0098] By combining graph neural networks with attention mechanisms, this system can effectively fuse multimodal data, generating fused features rich in semantic information, providing a solid foundation for subsequent state analysis and prediction. Taking the monitoring of a 500kV transmission line as an example, the system integrated insulator surface characteristics from optical images, temperature and humidity data collected by environmental sensors, waveform characteristics recorded by current transformers, and point cloud data from lidar scans. It successfully warned of the risk of insulator flashover during severe weather conditions, issuing a warning six hours in advance, giving operations and maintenance personnel ample time to respond and avoiding a potential large-scale power outage.

[0099] Based on the fusion characteristics, the digital twin module 2 establishes a dynamic correlation model of conductor temperature, sag and current carrying capacity. Figure 5 As shown in the figure, the model is implemented through a spatiotemporal fusion mechanism, which includes four parts: temporal information encoding, spatial relationship modeling, spatiotemporal fusion processing, and multi-scale spatiotemporal analysis.

[0100] The temporal information encoding part uses the position encoding method to represent the temporal information and capture the periodic changes and long-term trends of the data. The position encoding function is defined as:

[0101]

[0102] Where PE(pos,2i) is the sine code value of the time position pos in the 2i dimension; PE(pos,2i+1) is the cosine code value of the time position pos in the 2i+1 dimension; pos is the time position, indicating the position index of the data point in the time series, such as pos=1 for the first hour data and pos=2 for the second hour data; i is the dimension index, with a value range of 0≤i <d model / 2;d model The model dimension, typically 128 or 256, represents the dimension of the time-encoded vector. This encoding method effectively represents temporal location information and has good extrapolation performance. In transmission line monitoring, time encoding is crucial for capturing diurnal temperature variations, cyclical load changes, and seasonal variations. For example, the system can identify that conductor temperature peaks are most frequent between 2:00 PM and 4:00 PM in the summer, increasing monitoring frequency and warning sensitivity during this period.

[0103] The spatial relationship modeling part evaluates the physical distance and functional similarity between nodes. The physical distance is calculated based on the actual geographic coordinates, while the functional similarity considers the functional role of the node in the power grid. The spatial similarity function is defined as:

[0104]

[0105] Among them, ρ ij is the spatial similarity between node i and node j, with a value range of (0,1], and a larger value indicates a higher similarity; d ij is the distance between node i and node j, which can be a physical distance (meters) or a measure of functional similarity; σ is a scaling parameter, which is adjusted according to different types of spatial relationships. In this embodiment, for physical distance, σ is taken as 50 meters, which means that when the distance between two monitoring points is 50 meters, their similarity drops to about 0.61; for functional similarity, σ is taken as 0.5, which is determined based on the similarity of functional roles. In actual applications, for example, the distance between adjacent towers on a 330kV transmission line is about 300 meters. The system will automatically calculate the spatial similarity between the towers. When an abnormality is detected in the insulator of a tower, the monitoring priority of the adjacent tower will be increased accordingly to achieve "associated monitoring."

[0106] The spatiotemporal fusion processing part adopts the strategy of "space first, time second", that is, spatial information aggregation is performed first, and then the temporal dependency is processed. The specific implementation is a two-stage attention mechanism:

[0107] Z space =SpatialAttention(X),

[0108] Z time =TemporalAttention(Zspace ),

[0109] Z=g·Z space +(1-g)·Z time ,

[0110] Where X is the input feature matrix, with dimensions N × T × F, where N is the number of nodes, T is the length of the time series, and F is the feature dimension. Zspace is the output of spatial attention, with the same dimensions as X, representing the feature representation after considering spatial relationships. Ztime is the output of temporal attention, with the same dimensions as Zspace, representing the feature representation after further considering temporal dependencies. g is a gating parameter with a value range of [0, 1], which is used to dynamically adjust the fusion ratio of spatial and temporal information and is adaptively determined by the model based on the current state. SpatialAttention and TemporalAttention are the spatial attention and temporal attention calculation functions, respectively, implemented based on the aforementioned attention mechanism. In practical applications, for example, when monitoring wire temperature changes, the system first considers spatial distribution (e.g., temperature distribution at different locations), then temporal changes (e.g., temperature fluctuation trends). Finally, these two pieces of information are integrated through a gating mechanism to obtain a comprehensive representation of the temperature state.

[0111] The multi-scale spatiotemporal analysis component analyzes data at the local spatiotemporal scale (short-term changes at a single monitoring point), the mesoscale (evolution over medium-term periods within a region), and the macroscale (long-term evolution of the entire system), providing a comprehensive understanding of the spatiotemporal state. For example, the system can simultaneously analyze minute-by-minute variations in the surface temperature of a specific insulator (local scale), hourly trends in line load variations in that section (mesoscale), and the evolution of the entire line's operating status over a month (macroscale), providing a multi-faceted and multi-layered understanding of the line's status.

[0112] Through this spatiotemporal fusion mechanism, the system accurately models the dynamic relationship between three key parameters: conductor temperature, sag, and current carrying capacity. It captures their variations under varying environmental conditions, providing a theoretical basis for precise prediction and early warning. In a case study of a 220kV transmission line, the system successfully captured the abnormal increase in conductor sag under high summer temperatures and loads, predicting three hours in advance that the conductor might approach the lower limit of the safe distance, giving the dispatching department ample time to adjust load distribution and avoid potential safety hazards.

[0113] The online monitoring module 3 is electrically connected to the digital twin module 2 and is responsible for receiving the dynamic correlation model from the digital twin module 2 and performing status analysis and early warning in combination with real-time data. Figure 3 As shown, the online monitoring module 3 includes a data fusion mining module 31 and a result analysis and evaluation module 32 .

[0114] The data fusion mining module 31 calculates the correlation between line environmental data and line current waveforms, as well as the changing trends of historical data. This module receives real-time data from sensors, compares and analyzes it with historical data, and identifies abnormal patterns and potential risks. Preferably, this module uses a sliding window method to process time series data. The window size is dynamically adjusted based on the changing characteristics of different parameters, typically ranging from 5 minutes to 1 hour. For example, for temperature data, a 15-minute window is used; for load data, a 30-minute window is used; and for meteorological data, a 1-hour window is used. This dynamic window design can capture rapidly changing parameter characteristics while reducing the computational burden.

[0115] The result analysis and evaluation module 32 analyzes and evaluates the resulting data and sends the results to staff in the form of charts. Based on the results of the integrated analysis, this module determines the current line status and generates warning information of different levels. It then presents these warnings to operations and maintenance personnel in the form of intuitive charts to support decision-making. Warning levels are generally divided into four levels: information prompt (green), attention reminder (yellow), early warning (orange), and emergency treatment (red). Each level has clear triggering conditions and handling suggestions.

[0116] In one embodiment of the present invention, one of the innovations of the online monitoring module 3 is the introduction of a self-evolution prediction and feedback correction mechanism. Figure 6 As shown in Figure 3, the mechanism includes an error analysis and correction unit, a model self-evolution unit, and a knowledge accumulation unit.

[0117] The error analysis and correction unit calculates the deviation between the predicted results and the actual observed values, analyzes the error distribution characteristics, and adjusts the prediction parameters based on the error distribution. Error calculation uses two indicators: mean square error (MSE) and mean absolute error (MAE):

[0118]

[0119] Among them, MSE is the mean square error, which is used to measure the accuracy of the prediction and is more sensitive to larger errors; MAE is the mean absolute error, which is used to measure the degree of deviation of the prediction and treats all errors equally; i is the actual observed value, such as the measured temperature or sag value; is the corresponding predicted value; n is the number of samples, usually the observed data points within the last 24 hours. represents the sum of all sample points, with i traversing from 1 to n. In transmission line monitoring, MSE and MAE provide different perspectives on prediction quality. For example, when monitoring conductor temperature, MAE values ​​within 1°C and MSE values ​​within 1.5°C² are considered good prediction performance.

[0120] When the error exceeds a preset threshold, the system triggers a parameter adjustment process. For temperature prediction, the threshold is set at 1.5°C; for sag prediction, the threshold is set at 5cm; and for current-carrying capacity prediction, the threshold is set at 3% of the line's rated current. These thresholds are determined based on power system operating experience and safety margins, ensuring timely detection of anomalies while avoiding frequent, unnecessary adjustments. For example, the typical rated current of a 330kV transmission line is approximately 2000A, so the current-carrying capacity prediction threshold is 60A. After a thunderstorm, the system detected that the conductor temperature prediction error persisted above 1.5°C, triggering parameter adjustments. Analysis revealed that this was due to rainwater erosion, which altered the conductor surface condition. The system automatically adjusted the parameters of the thermal balance model, restoring high-precision predictions.

[0121] The model self-evolution unit updates the model structure and parameters based on performance evaluation and environmental changes. This unit adopts an incremental learning strategy, incorporating information provided by new data while retaining the original knowledge. Parameter updates are performed using the gradient descent method:

[0122]

[0123] Among them, θ t+1 is the updated model parameter vector; θ t is the current model parameter vector, including the weights and bias values ​​of the neural network; η is the learning rate, which controls the step size of parameter updates; is the loss function L with respect to the parameter θ t The gradient vector represents the direction and rate of change of the loss function at the current parameter point. The learning rate uses an adaptive strategy, initially set to 0.01 and gradually decreasing as training progresses, but never falling below 0.001, to maintain the model's adaptability to new data. In practical applications, for example, when the system recognizes a new weather pattern (such as rime weather encountered for the first time), the learning rate is increased to quickly adapt to the new situation; during stable operation, a smaller learning rate is used for fine-tuning.

[0124] The knowledge accumulation unit retains key historical experiences and patterns, preventing new knowledge from overwriting useful older knowledge. This unit utilizes an experience playback mechanism to regularly review key historical cases, particularly those under extreme weather conditions, to ensure the model doesn't "forget" important lessons. In an actual application on a 500kV transmission line, the system retained data and patterns from extreme icing conditions the previous winter. Although no similar extreme conditions occurred throughout that winter, the system still maintained its ability to identify and warn of such situations, demonstrating the value of knowledge accumulation.

[0125] In this embodiment, the feedback correction mechanism triggers an update when the prediction accuracy falls below 90% or when environmental conditions change significantly. "Significant changes" in environmental conditions are defined as temperature changes exceeding 10°C / hour, wind speed changes exceeding 5m / s, or the presence of unusual weather conditions such as rain or snow. These triggering conditions ensure that the system can respond promptly to environmental changes while avoiding the waste of computing resources caused by overly frequent updates. For example, during a cold front pass, the temperature in a certain area dropped 15°C over three hours. The system automatically triggered a model update, optimizing the temperature-sag relationship model to make the predictions more consistent with the actual conditions under rapid temperature drops.

[0126] Through its self-evolving prediction and feedback correction mechanism, Online Monitoring Module 3 continuously optimizes its prediction model, improving prediction accuracy and adapting to changing environmental and operating conditions. For example, in monitoring a key transmission channel, six months after the system was put into operation, the average error in temperature prediction decreased from an initial 1.2°C to 0.7°C, and the average error in sag prediction decreased from 4 cm to 2.5 cm, demonstrating the system's self-optimization capabilities.

[0127] Another innovation of the online monitoring module 3 is the use of a multi-level forecasting framework. This framework divides forecasts into three levels: short-term, medium-term, and long-term according to the time scale:

[0128] Short-term forecasts predict state changes within the next 10 minutes, updated every 10 seconds. They are primarily used for real-time monitoring and immediate warnings. Using a direct prediction method based on current state and short-term trends, short-term forecasts offer fast response times and are suitable for handling emergencies. For example, if a branch is detected approaching a conductor on a certain section of a line, the system will generate an alert within seconds, providing time for emergency response.

[0129] Medium-term forecasts predict state changes for 10 minutes to 2 hours, updated every 10 minutes, and are primarily used for fault prevention and operational scheduling. These forecasts combine recursive prediction with medium-term trend analysis, ensuring both accuracy and real-time performance. For example, the system can predict load trends within the next hour, providing a reference for the dispatch department's power allocation decisions.

[0130] Long-term forecasts predict status changes over a period of more than two hours, updated hourly, and are primarily used for maintenance planning and resource allocation. These forecasts incorporate weather forecast data, comprehensively considering the long-term impact of meteorological factors on line conditions. For example, based on the 24-hour forecast, the system can predict periods of potential icing risk, assisting operations and maintenance departments in pre-arranging inspections and protective measures.

[0131] This hierarchical prediction strategy enables the system to provide accurate forecasts at different timescales, meeting the needs of diverse application scenarios. In monitoring a power transmission line across a complex mountainous area, the system's multi-level prediction capabilities improved operational efficiency by approximately 35% and reduced line failure rates by approximately 25%, demonstrating the practical value of this technical solution.

[0132] The present invention also provides a real-time monitoring method for digital twins of power transmission lines based on the above system. Figure 7 As shown, the method includes the following steps:

[0133] Step 1: Use laser point cloud, infrared thermal imaging and current waveform data fusion technology to construct a three-dimensional digital twin of the transmission line.

[0134] Specifically, the system first acquires point cloud data from a lidar (LiDAR), unifying the multiple scans into a common coordinate system using a point cloud registration algorithm. Temperature distribution images captured by an infrared thermal imaging camera are then registered with the point cloud data to add temperature information to the 3D model. Current waveform data from a current transformer is then acquired, features extracted, and associated with spatial locations. Finally, based on this multimodal data, a complete 3D digital twin is constructed, complete with geometric shape, temperature distribution, and electrical characteristics. In an application example involving a 500kV transmission line, the system used an infrared thermal imager with a resolution of 1024×768 and a temperature resolution of 0.05°C, along with a lidar point cloud density of 200 points per square meter. This system successfully constructed a high-precision digital twin model with a geometric accuracy better than 2cm and a temperature mapping accuracy better than 0.5°C.

[0135] Step 2: Design a cross-modal attention mechanism and establish a dynamic correlation model of conductor temperature, sag, and current carrying capacity through a graph neural network to achieve real-time evaluation of conductor thermal stability.

[0136] In this step, each transmission line component is first represented as a node in a graph structure, with node features containing multimodal data information. A three-layer graph convolutional network is then used to extract node features and high-order topological relationships. A cross-modal attention mechanism is then applied to adaptively assign weights to data from different modalities. A dynamic correlation model between conductor temperature, sag, and current carrying capacity is then established based on the fused features. Finally, a spatiotemporal fusion mechanism is used to capture how parameters change over time and in the environment, enabling real-time assessment of conductor thermal stability. For example, the system can model a dynamic relationship where a 1°C increase in conductor temperature results in an approximately 1.5-2cm increase in sag. This model also accounts for environmental factors such as wind speed and sunlight, providing more accurate assessment results than static models.

[0137] Step 3: Integrate edge computing nodes to complete partial discharge feature extraction within 10ms, and combine meteorological data to predict icing risks, with a classification accuracy of over 98.5%.

[0138] In this step, edge computing nodes are first deployed at key monitoring points. The hardware configuration is an industrial-grade ARM processor with 4GB of RAM and GPU acceleration support. A lightweight algorithm model is then used to optimize the computational process, enabling partial discharge feature extraction within 10ms. Meteorological data, including temperature, humidity, wind speed, and precipitation, is then acquired. Icing risk values ​​are then calculated using a FROST-like icing model. Finally, by combining preliminary analysis at the edge nodes with in-depth analysis in the cloud, a highly accurate icing risk prediction is achieved, with a classification accuracy exceeding 98.5%. In actual application on a power transmission line in a northern province, the system successfully issued 18 icing risk warnings during a full winter monitoring period, with an average lead time of 4.5 hours, allowing the operations and maintenance department ample time to take preventative measures. All warnings were proven effective, with no missed or false alarms.

[0139] Step 4: Through the self-evolution prediction and feedback correction mechanism, combined with the multi-level prediction framework, short-term, medium-term and long-term fault prediction and warning are achieved.

[0140] In this step, a three-level forecasting framework is first constructed: short-term, medium-term, and long-term, responsible for forecasting at different time scales. The deviation between the forecast results and the actual observed values ​​is then calculated in real time, and the error distribution characteristics are analyzed. Model parameters are then adjusted based on feedback to optimize forecast performance. Key historical experience is then retained through a knowledge accumulation unit to prevent "catastrophic forgetting." Finally, a closed-loop feedback mechanism is formed, enabling the system to continuously learn and evolve, continuously improving forecast accuracy. In a case study of a key power transmission channel, one year after the system was put into operation, the fault prediction accuracy increased from an initial 85% to over 95%, and the average warning lead time increased by 30%, demonstrating the long-term value of the self-evolutionary mechanism.

[0141] Step 5: Based on the abnormal status and prediction results, generate multi-level warning information and send it to the staff in the form of charts to achieve intelligent operation and maintenance decision support.

[0142] In this step, multi-level warning thresholds are first set based on different types of anomalies, such as excessive sag, excessive temperature, and frequent partial discharge. The predicted results are then compared with the thresholds to generate warning information at different levels. The warning information is then presented in intuitive graphical forms, including trend charts, heat maps, and 3D visualizations. Finally, action suggestions are provided to maintenance personnel, supporting intelligent O&M decision-making. For example, if the system detects increasing surface contamination on insulators on a certain section of line and predicts a potential flashover risk, an orange warning is generated, recommending that this section be prioritized during the next scheduled maintenance. A heat map of the contamination distribution and historical comparison data are also provided to help maintenance personnel fully understand the situation and make informed decisions.

[0143] Through the above steps, the method of the present invention can comprehensively monitor the status of transmission lines, timely discover potential hidden dangers, predict possible failure risks, provide a scientific basis for operation and maintenance decisions, and significantly improve the safety and reliability of power grid operation.

[0144] The above embodiments are merely preferred embodiments of the present invention, and are not exhaustive. Persons skilled in the art may, based on the insights gained from the description of the above embodiments, make various modifications and improvements to the above embodiments without departing from the basic principles of the present invention. Therefore, such modifications and improvements fall within the scope of protection of the present invention.

[0145] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. The digital twin real-time monitoring system for transmission lines based on multimodal data fusion is characterized by: Includes data acquisition module, digital twin module and online monitoring module; The data acquisition module is used to: Collect optical images, environmental feature data, current waveform data and point cloud data; transmitting the optical image, the environmental characteristic data, the current waveform data, and the point cloud data to the digital twin module; The digital twin module is electrically connected to the data acquisition module and is used to: receiving the optical image, the environmental characteristic data, the current waveform data, and the point cloud data; Build a three-dimensional digital twin of the transmission line; fusing the optical image, the environmental feature data, the current waveform data, and the point cloud data through a graph neural network and an attention mechanism to generate a multimodal fusion feature; Establishing a dynamic correlation model of conductor temperature, sag and current carrying capacity based on the multimodal fusion features; Transmitting the conductor temperature-sag-current carrying capacity dynamic correlation model to the online monitoring module; The online monitoring module is electrically connected to the digital twin module and is used to: receiving a dynamic correlation model of conductor temperature, sag and current carrying capacity; Integrate real-time collected data with historical data to analyze line status; Generate multi-level warning information; Output line status analysis results and fault prediction information.

2. The multimodal data fusion transmission line digital twin real-time monitoring system according to claim 1 is characterized in that: The digital twin module includes: The original image analysis module is used to establish the correlation between image features and partial discharge waveform features; The sag prediction module is used to calculate the conductor-tower distance using fused features; Model fusion module, used for point cloud registration and model update of point cloud data; Partial discharge analysis module, used to establish the correlation between the original image and the partial discharge waveform; The sag calculation module is used to calculate the current conductor sag information based on the real line model and the digital twin model.

3. The multimodal data fusion transmission line digital twin real-time monitoring system according to claim 1 is characterized in that: The online monitoring module includes: Data fusion mining module, used to calculate the correlation between line environment data and line current waveform and the change trend of historical data; The result analysis and evaluation module is used to analyze and evaluate the result data and send the results to the staff in the form of charts.

4. The multimodal data fusion transmission line digital twin real-time monitoring system according to claim 2 is characterized in that: The original image analysis module, the sag prediction module, the model fusion module, the partial discharge analysis module and the sag calculation module use a graph neural network combined with an attention mechanism to fuse multimodal data, wherein: The graph neural network includes three layers of graph convolutional layers, which are used to extract node features and capture high-order topological relationships between nodes; The attention mechanism includes intra-modality attention and inter-modality cross-attention, which are used to adaptively assign weights to data of different modalities; The graph neural network and the attention mechanism are combined to achieve deep fusion of multimodal data.

5. The multimodal data fusion transmission line digital twin real-time monitoring system according to claim 1 is characterized in that: The optical image is obtained by a linear array camera mounted on an unmanned patrol vehicle, the environmental feature data is obtained by a wireless sensor node, and the current waveform data and the point cloud data are obtained by a power transmission line inspection robot; wherein, The resolution of the linear array camera is not less than 2048×1536; The wireless sensor node collects environmental feature data every 5 minutes; The transmission line inspection robot is equipped with a laser radar, and the point cloud density is not less than 100 points per square meter.

6. The multimodal data fusion transmission line digital twin real-time monitoring system according to claim 1 is characterized in that: The digital twin module is based on the digital twin principle and uses laser radar technology and infrared thermal imaging technology to achieve three-dimensional digital twin imaging. Combined with the real-time current waveform data collected by the current sensor, the collected images, current data and point cloud data are digitally twinned to generate a digital twin circuit. The temperature information of the collection point, the current data collected by the current sensor and the image are associated to update the digital twin model and calculate the sag and sag prediction value.

7. The multimodal data fusion transmission line digital twin real-time monitoring system according to claim 1 is characterized in that: The input of the online monitoring module is the data detected by the sensor, and the output is the fault prediction result and fault identification; wherein, The fault prediction results include short-term prediction results, medium-term prediction results and long-term prediction results; The short-term prediction results predict the state changes within the next 10 minutes and are updated every 10 seconds; The medium-term forecast results predict the state changes of 10 minutes to 2 hours and are updated every 10 minutes; The long-term prediction results predict state changes of more than 2 hours and are updated every hour; The fault identification includes partial discharge fault, sag abnormality and icing risk.

8. The multimodal data fusion transmission line digital twin real-time monitoring system according to claim 1 is characterized in that: The conductor temperature-sag-current carrying capacity dynamic correlation model is implemented by a time-space fusion mechanism, which includes: Temporal information encoding is used to capture periodic changes and long-term trends in data; Spatial relationship modeling, used to assess the physical distance and functional similarity between nodes; Spatiotemporal fusion processing, used to integrate information in time and space dimensions; Multi-scale spatiotemporal analysis is used to analyze state changes at different spatiotemporal scales.

9. The multimodal data fusion transmission line digital twin real-time monitoring system according to claim 1 is characterized in that: The online monitoring module includes a self-evolution prediction and feedback correction mechanism, which includes: Error analysis and correction unit, used to calculate the deviation between the predicted results and the actual observed values, and adjust the prediction parameters based on the error distribution; Model self-evolution unit, used to update model structure and parameters based on performance evaluation and environmental changes; Knowledge accumulation unit, used to preserve key historical experiences and patterns and prevent new knowledge from overwriting useful old knowledge; The feedback correction unit triggers an update when the prediction accuracy is lower than a threshold or the environmental conditions change significantly.

10. A digital twin real-time monitoring method for transmission lines based on the system of claim 1, characterized in that: The specific steps are as follows: Step 1: Use laser point cloud, infrared thermal imaging, and current waveform data fusion technology to build a 3D digital twin of the transmission line; Step 2: Design a cross-modal attention mechanism and establish a dynamic correlation model between conductor temperature, sag, and current carrying capacity through a graph neural network to achieve real-time assessment of conductor thermal stability. Step 3: Integrate edge computing nodes to complete partial discharge feature extraction within 10ms and combine meteorological data to predict icing risk, with a classification accuracy of over 98.5%; Step 4: Through the self-evolution prediction and feedback correction mechanism, combined with a multi-level prediction framework, short-term, medium-term and long-term fault prediction and warning are achieved; Step 5: Based on the abnormal status and prediction results, generate multi-level warning information and send it to the staff in the form of charts to achieve intelligent operation and maintenance decision support.

Images