A power transmission line tower disaster monitoring data fusion method, device, medium and product
Patent Information
- Application Number
- CN202611062486.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-17
- Publication Date
- 2026-10-23
AI Technical Summary
[0004]本申请的目的是提供一种输电线路杆塔受灾监测数据的融合方法，可解决现有技术输电线路杆塔受灾监测数据的精度低、实时性和鲁棒性差的问题
本申请首先基于输电线路杆塔预设范围内设置的各类传感器，获取各类多源异构监测数据，然后设置全局时间基准实现各类传感器的统一时间基准，结合扩展卡尔曼滤波处理每一类多源异构监测数据，确定时间对齐的每一类多源异构监测数据，使各类多源异构监测数据在时间维度上的同步精度，从而显著提升了灾害事件定位和动态监测的准确性。进一步提取所述时间对齐的每一类多源异构监测数据中的激光雷达点云数据的激光雷达三维特征点和光学图像数据的二维特征点，并基于所述激光雷达三维特征点与二维特征点，建立匹配关系和确定空间校正的每一类融合监测数据，精确对齐点云数据的三维特征点与光学图像的二维特征点，实现杆塔及周边环境特征在空间维度上的高度一致性。有效克服了传统手工标定或单传感器标定带来的误差累积问题，保证了多源异构监测数据融合的可靠性和精度。最后根据自适应加权方法和所述空间校正的每一类融合监测数据，确定融合后的输电线路杆塔受灾监测数据。本申请实现了多源异构监测数据在时间维度和空间维度上的一致融合，提升了输电线路杆塔受灾监测数据的精度、实时性和鲁棒性。
Smart Images
Figure CN122962600A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of transmission line tower operation monitoring technology, and in particular to a method, equipment, medium and product for fusing disaster monitoring data of transmission line towers. Background Technology
[0002] In the field of existing transmission line tower operation monitoring, traditional monitoring methods mainly rely on single-type sensors, such as ground radar, optical cameras, or UAV aerial data. However, these single sensors have significant limitations in the data acquisition process: inconsistent temporal sampling, large differences in spatial coordinate references, and the impact of environmental conditions on data quality make it difficult to directly use the acquired monitoring information for accurate tower condition assessment and disaster risk analysis. For example, the time synchronization deviation between radar and optical images can easily lead to spatial mismatch of disaster location information, and the positioning accuracy of UAV aerial data is also limited in complex terrain. In addition, the data types collected by multi-source sensors are diverse, including point clouds, image sequences, strain, and tilt angles, with varying scales, resolutions, and noise characteristics, which poses challenges to subsequent data fusion and feature extraction. Existing methods usually rely on manual calibration or single-dimensional filtering algorithms for temporal or spatial correction, which is not only complex to operate but also significantly insufficient in terms of accuracy and real-time performance in integrating heterogeneous data, making it difficult to meet the requirements of rapid assessment and early warning in high-risk environments.
[0003] Meanwhile, intelligent preprocessing and noise suppression technologies for multi-source heterogeneous monitoring data are still in their early stages. Existing technologies mainly employ traditional filtering, interpolation, or feature matching algorithms to smooth and correct data, but these methods are difficult to adapt to application scenarios with diverse sensor types, irregular sampling periods, and frequent dynamic environmental changes. Particularly in monitoring power line towers in areas prone to geological disasters or affected by external mining, data noise and abnormal fluctuations interfere with disaster identification and risk calculation, resulting in unstable and delayed monitoring results. In such cases, the lack of a comprehensive solution that can simultaneously achieve time synchronization, spatial joint calibration, and intelligent noise reduction of multi-source information hinders the development of high-precision, rapid-response tower disaster monitoring technology. Summary of the Invention
[0004] The purpose of this application is to provide a method for fusing disaster monitoring data of transmission line towers, which can solve the problems of low accuracy, poor real-time performance, and poor robustness of existing disaster monitoring data of transmission line towers.
[0005] To achieve the above objectives, this application provides the following solution.
[0006] This application provides a method for fusing disaster monitoring data of transmission line towers, comprising: acquiring various types of multi-source heterogeneous monitoring data based on various sensors installed within a preset range of the transmission line towers; the multi-source heterogeneous monitoring data includes lidar point cloud data, optical camera image data, and tower operation status sensor data; setting a global time reference, processing each type of multi-source heterogeneous monitoring data using extended Kalman filtering, and determining time-aligned multi-source heterogeneous monitoring data for each type of time-aligned multi-source heterogeneous monitoring data; extracting lidar three-dimensional feature points from lidar point cloud data and two-dimensional feature points from optical image data from each type of time-aligned multi-source heterogeneous monitoring data; establishing a matching relationship based on the lidar three-dimensional feature points and two-dimensional feature points, and determining spatially corrected fused monitoring data for each type of monitoring data; and determining fused transmission line tower disaster monitoring data according to an adaptive weighting method and the spatially corrected fused monitoring data for each type of monitoring data, wherein the fused transmission line tower disaster monitoring data is used for tower status assessment and disaster risk early warning.
[0007] Furthermore, a global time base is set, and extended Kalman filtering is used to process each type of multi-source heterogeneous monitoring data to determine time-aligned multi-source heterogeneous monitoring data for each type. Specifically, this includes setting a global time base for each type of sensor. Based on the state update equation, an extended Kalman filter is used to predict and correct each type of multi-source heterogeneous monitoring data, thus determining the time-aligned multi-source heterogeneous monitoring data for each type. The state update equation is: ;in, For the first Sensor-like sensors Estimated values of tower operation status sensor data; For the first Kalman gain matrix of sensor-like sensors; The observation matrix; For the first The sensor-like data represents the actual tower operating status data sampled at the current moment. For the first The sensor-like data represents the tower's operational status as predicted at the previous moment.
[0008] In one embodiment, based on the three-dimensional and two-dimensional feature points of the lidar, a matching relationship is established and each type of fused monitoring data with spatial correction is determined. Specifically, this includes: projecting the three-dimensional feature points of the lidar onto the optical camera coordinate system to determine the image position of the lidar three-dimensional feature points projected onto the optical image; determining the spatial calibration relationship between the lidar and the optical camera with the goal of minimizing the error between the position of the two-dimensional feature points in the optical camera coordinate system and the image position; and using the spatial calibration relationship as the matching relationship to determine each type of fused monitoring data with spatial correction.
[0009] In one embodiment, the matching relationship for: ;in; Coordinate system for point cloud acquired by lidar The next Three-dimensional feature points; , , The first The coordinate values of a three-dimensional feature point in the three coordinate axes of the lidar coordinate system; The first optical image acquired by the optical camera Homogeneous pixel coordinates of a two-dimensional feature point; For the first The lateral pixel coordinates of each optical image feature point; For the first Vertical pixel coordinates of optical image feature points; For numbering; It is a rotation matrix; This is the camera intrinsic parameter matrix; This is the projection scale factor; . It is a 2-norm; It is a translation vector.
[0010] In one embodiment, before determining the fused transmission line tower disaster monitoring data based on each type of fused monitoring data according to the adaptive weighting method and the spatial correction, the method further includes: determining the adaptive weight of each type of multi-source heterogeneous monitoring data based on three indicators: signal-to-noise ratio, abnormal fluctuation degree, and consistency of multiple observations.
[0011] In one embodiment, the formula for the adaptive weights is: .
[0012] in, For the first Adaptive weights for sensor-like devices An index for sensor categories; For the first Quality evaluation indicators for multi-source heterogeneous monitoring data corresponding to sensor types; This refers to the total number of various sensors involved in data fusion. For the first Evaluation indicators for the quality of multi-source heterogeneous monitoring data of sensor-like devices. ; ; For the first Signal-to-noise ratio of multi-source heterogeneous monitoring data from sensor-like devices; The first one determined based on multiple observations Consistency index of multiple observations of multi-source heterogeneous monitoring data from sensor-like devices; For the first The degree of abnormal fluctuation in multi-source heterogeneous monitoring data of similar sensors; , and All are indicator adjustment coefficients.
[0013] In one embodiment, the fused transmission line tower disaster monitoring data is determined based on the adaptive weighting method and each type of fused monitoring data with spatial correction. Specifically, this includes: using the adaptive weighting method to perform noise suppression and outlier correction on each type of fused monitoring data with spatial correction to determine the fused transmission line tower disaster monitoring data; the adaptive weighting method is: ;in, This is the integrated disaster monitoring data for transmission line towers; It is the identity matrix; For the first Adaptive weights for sensor-like applications; For the first The actual observation status of fused monitoring data from various sensors; For the first Predictive observation status based on fusion monitoring data from sensor-like devices.
[0014] In a second aspect, this application provides a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the method for fusing disaster monitoring data of transmission line towers as described in the first aspect.
[0015] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the method for fusing disaster monitoring data of transmission line towers as described in the first aspect.
[0016] Fourthly, this application provides a computer program product, including a computer program, characterized in that, when the computer program is executed by a processor, it implements the method for fusing disaster monitoring data of transmission line towers as described in the first aspect.
[0017] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application first acquires various multi-source heterogeneous monitoring data based on different sensors installed within a preset range on transmission line towers. Then, a global time reference is established to achieve a unified time reference for all types of sensors. Combined with extended Kalman filtering, each type of multi-source heterogeneous monitoring data is processed to determine time-aligned data, improving the synchronization accuracy of the data in the time dimension and significantly enhancing the accuracy of disaster location and dynamic monitoring. Further, the application extracts 3D feature points from the lidar point cloud data and 2D feature points from the optical image data of each time-aligned multi-source heterogeneous monitoring data. Based on these lidar 3D and 2D feature points, a matching relationship is established, and spatial correction is determined for each type of fused monitoring data. This precisely aligns the 3D feature points of the point cloud data with the 2D feature points of the optical image, achieving a high degree of consistency in the spatial dimension of the tower and its surrounding environment features. This effectively overcomes the error accumulation problem caused by traditional manual calibration or single-sensor calibration, ensuring the reliability and accuracy of multi-source heterogeneous monitoring data fusion. Finally, based on the adaptive weighting method and the spatially corrected fused monitoring data, the fused disaster monitoring data for the transmission line towers is determined. This application achieves consistent fusion of multi-source heterogeneous monitoring data in both time and spatial dimensions, improving the accuracy, real-time performance, and robustness of disaster monitoring data for transmission line towers. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 This is a flowchart illustrating a method for fusing disaster monitoring data of transmission line towers according to an embodiment of this application. Detailed Implementation
[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0021] To make the objectives, features and advantages of this application more apparent and understandable, the application will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0022] Example 1 Existing technologies for monitoring disaster damage to transmission line towers have revealed problems in practical applications, including low accuracy of multi-source heterogeneous sensor data fusion, inconsistencies in temporal and spatial calibration, and insufficient intelligent preprocessing capabilities. These technical deficiencies lead to biases in monitoring data during disaster identification and risk assessment, making it difficult to meet the high-precision and rapid response requirements in complex geological environments. This application addresses these shortcomings by proposing key technical issues that urgently need to be resolved: First, data from different types of sensors differ in acquisition time and sampling frequency. Traditional time synchronization methods cannot guarantee accurate alignment of multi-source information in the temporal dimension, thus affecting the accurate location and dynamic monitoring of disaster events. Second, multi-source sensors exhibit deviations in spatial reference and coordinate systems. In particular, spatial fusion of data from lidar, optical cameras, and UAV aerial photography struggles to achieve high-precision calibration, resulting in mismatches in the spatial dimension of tower and surrounding environmental features, directly restricting the reliability of disaster feature extraction and risk analysis. Third, current data preprocessing methods largely rely on simple filtering or manual correction, which cannot adequately handle sensor noise, multi-scale anomalies, and dynamic interference in complex environments, leading to unstable results after information fusion and limiting the performance of intelligent disaster identification and rapid risk assessment systems.
[0023] Therefore, the core technical problem this application aims to solve is how to achieve precise synchronization in the time dimension and joint calibration in the spatial dimension while ensuring the accuracy and robustness of multi-source heterogeneous monitoring data, and to carry out intelligent preprocessing and noise suppression during data fusion, thereby providing high-precision, real-time, and reliable basic data for disaster monitoring of transmission line towers. Solving this technical problem will overcome the limitations of existing technologies in heterogeneous information fusion, dynamic disaster monitoring, and rapid risk assessment, providing a scientific and feasible technology for the safe operation of transmission lines and disaster prevention and control.
[0024] like Figure 1 As shown, this application proposes a method for fusing disaster monitoring data of transmission line towers, including: Step 101: Based on various sensors installed within a preset range of the transmission line tower, acquire various multi-source heterogeneous monitoring data; the multi-source heterogeneous monitoring data includes lidar point cloud data, optical camera image data, and tower operation status sensing data.
[0025] The types of sensors include structural condition sensors, lidar sensors, optical camera sensors, and strain / tilt sensors.
[0026] In practical applications, various types of sensors are installed on the transmission line tower itself, around the tower foundation, or on inspection vehicles to acquire raw, multi-source heterogeneous monitoring data of the tower and its surrounding environment. Specifically, lidar is used to collect 3D point cloud data of the tower and its surrounding area, optical cameras are used to collect texture image data of the tower structure and environment, and strain sensors and tilt sensors are used to collect data on the stress state and attitude changes of the tower structure, i.e., tower operating status sensing data. After this process, the multi-source heterogeneous monitoring data is output as input for subsequent time synchronization processing.
[0027] Step 102: Set a global time base, use extended Kalman filtering to process each type of multi-source heterogeneous monitoring data, and determine the time-aligned multi-source heterogeneous monitoring data for each type.
[0028] In some embodiments, step 102 specifically includes: setting a global time base for various sensors. Based on the state update equation, extended Kalman filtering is used to predict and correct each type of multi-source heterogeneous monitoring data, and time-aligned multi-source heterogeneous monitoring data for each type is determined.
[0029] The state update equation is: ;in, For the first Sensor-like sensors Estimated values of tower operation status sensor data; For the first The Kalman gain matrix of the sensor is used to balance prediction and observation errors; The observation matrix maps the predicted state to the observation space; For the first The sensor-like data represents the actual tower operating status data sampled at the current moment. For the first The system uses sensor data representing the tower's operational status at the previous moment. Through iterative calculations, the data from all sensors can be precisely aligned in the time dimension, laying the foundation for subsequent spatial fusion.
[0030] In practical applications, further consideration is given to sensor response delay, sampling error, and dynamic environmental changes. An extended Kalman filter (EDF) method is used to predict and correct the sensor's time state. By fusing historical state estimation information with current sensor observation data, state estimation results for each sensor are obtained under a unified reference time. After this step, time-aligned multi-source heterogeneous monitoring data are output.
[0031] Step 103: Extract the three-dimensional feature points of the lidar point cloud data and the two-dimensional feature points of the optical image data from each type of time-aligned multi-source heterogeneous monitoring data.
[0032] Step 104: Based on the three-dimensional and two-dimensional feature points of the lidar, establish a matching relationship and determine each type of fused monitoring data for spatial correction.
[0033] In some embodiments, step 104 specifically includes: projecting the three-dimensional feature points of the lidar onto the optical camera coordinate system to determine the image position of the lidar three-dimensional feature points projected onto the optical image; determining the spatial calibration relationship between the lidar and the optical camera with the goal of minimizing the error between the position of the two-dimensional feature points in the optical camera coordinate system and the image position; and using the spatial calibration relationship as the matching relationship to determine each type of fused monitoring data for spatial correction.
[0034] Specifically, for each type of time-aligned multi-source heterogeneous monitoring data, spatial joint calibration is performed after time consistency processing. Using the time-corrected multi-source heterogeneous monitoring data as input, spatial coordinate transformation is performed on the lidar point cloud data and optical image data. First, 3D structural feature points are extracted from the lidar point cloud and 2D visual feature points from the optical image, and a corresponding matching relationship is established between them. Then, based on the matched feature points, the spatial transformation parameters between the lidar coordinate system and the camera coordinate system are solved, mapping the data from different sensors to a unified spatial coordinate system. After this step, spatially aligned fused monitoring data is output, providing a foundation for subsequent intelligent preprocessing.
[0035] Step 105: Based on the adaptive weighting method and each type of fused monitoring data of the spatial correction, determine the fused transmission line tower disaster monitoring data. The fused transmission line tower disaster monitoring data is used for tower status assessment and early warning of disaster risks.
[0036] In some embodiments, the matching relationship for: ;in; Coordinate system for point cloud acquired by lidar The next Three-dimensional feature points; , , The first The coordinate values of a three-dimensional feature point in the three coordinate axes of the lidar coordinate system; The first optical image acquired by the optical camera Homogeneous pixel coordinates of a two-dimensional feature point; For the first The lateral pixel coordinates of each optical image feature point; For the first Vertical pixel coordinates of optical image feature points; For numbering; It is a rotation matrix; This is the camera intrinsic parameter matrix; This is the projection scale factor; . It is a 2-norm; It is a translation vector.
[0037] Specifically, in terms of joint spatial calibration, the coordinates of the lidar point cloud are set as follows: The optical camera pixel coordinates are This application solves for the rigid body transformation matrix, i.e., the matching relationship, through multi-feature matching and projection relationships. The rigid body transformation matrix between the lidar coordinate system and the camera coordinate system belongs to a special Euclidean group. .
[0038] in, This represents the rotation matrix, used to describe the attitude relationship between the LiDAR coordinate system and the camera coordinate system. Its function is to rotate 3D points in the LiDAR coordinate system to the camera coordinate system.
[0039] in, This represents the translation vector, used to describe the positional offset of the origin of the LiDAR coordinate system relative to the origin of the camera coordinate system. and Together they constitute the rigid body transformation relationship between the lidar and the camera.
[0040] This represents the camera intrinsic parameter matrix, which typically includes the focal length and principal point coordinates. The function of K is to map 3D points in the camera coordinate system to the image pixel coordinate system.
[0041] This represents the projection scale factor, which can also be understood as a depth-dependent scaling factor. Since camera imaging projects from three-dimensional space onto a two-dimensional image plane, the two sides of the homogeneous coordinates typically differ by one scale; therefore, it is necessary to introduce a scaling factor. .
[0042] Regarding the right-hand side of the formula: This means first transferring the three-dimensional points in the lidar coordinate system through... and Transform to the camera coordinate system, and then use the camera intrinsic parameter matrix. Projected onto the image plane.
[0043] left side of the formula , which means the expression of two-dimensional feature points actually detected in an optical image in homogeneous coordinate form.
[0044] The entire norm term is , representing the error between the actual image feature point positions and the image positions obtained by projecting 3D points from the LiDAR, is also known as the reprojection error. The optimal reprojection error can be determined by minimizing the reprojection error of all matching points. and Spatial calibration between the lidar and the camera is completed.
[0045] Among them, subscript and The meaning is: the first The three-dimensional feature points of the lidar and the first There are matching relationships between the two-dimensional feature points of the image. Therefore, the summation in the formula should be written as: 。
[0046] in, , This represents a pre-established point cloud-image feature matching set. Represents the rotation matrix. It is a translation vector. For the camera intrinsic parameter matrix, This represents the projection scale coefficient. By minimizing the projection error, the precise spatial calibration relationship between the lidar and the camera can be obtained, achieving alignment between the point cloud and the image in three-dimensional space, thereby ensuring the spatial consistency of the tower structure and surrounding geological features.
[0047] In some embodiments, before step 105, the method further includes: determining the adaptive weight of each type of multi-source heterogeneous monitoring data based on three indicators: signal-to-noise ratio, degree of abnormal fluctuation, and consistency of multiple observations.
[0048] In some embodiments, the formula for the adaptive weights is: .
[0049] in, For the first Adaptive weights for sensor-like devices An index for sensor categories; For the first Quality evaluation indicators for multi-source heterogeneous monitoring data corresponding to sensor types; This refers to the total number of various sensors involved in data fusion. For the first Evaluation indicators for the quality of multi-source heterogeneous monitoring data from sensor-like devices; ; ; For the first Signal-to-noise ratio of multi-source heterogeneous monitoring data from sensor-like devices; The first one determined based on multiple observations Consistency index of multiple observations of multi-source heterogeneous monitoring data from similar sensors; For the first The degree of abnormal fluctuation in multi-source heterogeneous monitoring data of similar sensors; , and All are indicator adjustment coefficients.
[0050] Assuming the transmission line tower monitoring system is equipped with The sensor categories are LiDAR sensors, optical camera sensors, and strain / tilt sensors. The sensor category index satisfies... .in, Indicates the first The evaluation index for the quality of multi-source heterogeneous monitoring data of similar sensors, therefore These can be represented as data quality evaluation indicators for LiDAR sensors. Data quality evaluation indicators for optical camera sensors and the data quality evaluation indicators corresponding to strain / tilt sensors ,Right now: .
[0051] When it is necessary to calculate the adaptive weights for a certain type of sensor, let the sensor category index be... For example, when calculating the adaptive weights for the second type of sensor, namely the optical camera sensor, let... The corresponding adaptive weights are: .
[0052] At this point, in the molecule This indicates the weight to be calculated at the current time. Data quality evaluation metrics for sensors, specifically data quality evaluation metrics for optical camera sensors. ; in the denominator This represents the data quality evaluation index of all sensors participating in the fusion, obtained by traversing... The data quality of all sensors is normalized.
[0053] In practical applications, Used for adjustment and The proportion in the fusion results. When the quality of multi-source heterogeneous monitoring data is high, the signal-to-noise ratio is large, and multiple observation results show high consistency, the proportion should be increased. The weighting is adjusted to ensure that actual observation information occupies a larger proportion in the fusion result; when the sensor is affected by noise interference, exhibits abnormal fluctuations, or has low consistency with other observation results, the weighting is reduced. The weighting is adjusted to reduce the impact of outlier data on the fusion results.
[0054] By calculating the adaptive weights described above, the fusion strategy can be adjusted according to the real-time data quality of different sensors in complex disaster environments, effectively suppressing the impact of low-reliability data and abnormal measurements on monitoring results, and improving the stability and reliability of multi-source heterogeneous sensor fusion data.
[0055] In some embodiments, step 105 specifically includes: using an adaptive weighting method to perform noise suppression and outlier correction on each type of fused monitoring data of the spatial correction, and determining the fused transmission line tower disaster monitoring data.
[0056] The adaptive weighting method is as follows: ;in, This is the integrated disaster monitoring data for transmission line towers; It is the identity matrix; For the first Adaptive weights for sensor-like applications; For the first The actual observation status of fused monitoring data from various sensors; For the first Predictive observation status based on fusion monitoring data from sensor-like devices.
[0057] This application performs noise suppression and outlier correction on fused monitoring data after time synchronization and spatial joint calibration. Unlike traditional fixed-parameter filtering methods, this application dynamically evaluates the reliability of each sensor's data based on the signal-to-noise ratio, consistency of multiple observations, and degree of abnormal fluctuations, and generates a corresponding adaptive weight matrix based on the evaluation results, thereby achieving dynamic adjustment of the contribution of different data sources.
[0058] Specifically, in practical applications, step 105 involves intelligent preprocessing and data fusion. Using each type of spatially corrected fused monitoring data as input, the quality of various sensor data is evaluated based on the signal-to-noise ratio, historical observation consistency, and degree of abnormal fluctuations. Adaptive weights are dynamically generated based on the evaluation results. These weights are then used to perform weighted fusion of multi-source data, suppressing noise interference and abnormal measurements, and reducing the impact of low-reliability data on the fusion results. After this step, a highly reliable fusion result (i.e., the fused transmission line tower disaster monitoring data) is output.
[0059] The highly reliable fusion results output combine tower structural status information, three-dimensional spatial structural information, and image environment information to analyze the disaster status of transmission line towers and identify disaster characteristics such as tower tilt, foundation settlement, and surrounding geological anomalies. After this step, the corresponding disaster monitoring and identification results can be output.
[0060] Furthermore, the highly reliable fusion results can also be used for risk assessment and output. Specifically, based on the disaster characteristic identification results obtained from the highly reliable fusion results, the current disaster status of the towers can be analyzed for risk level, and risk assessment results and operation and maintenance decision-making basis can be output, providing data support for disaster early warning, condition-based maintenance, and safe operation of transmission lines.
[0061] Through the above process, this application achieves a complete closed-loop processing procedure, from the acquisition of multi-source heterogeneous monitoring data, to time unification, time deviation correction, spatial coordinate alignment, and data quality optimization to obtain fused transmission line tower disaster monitoring data, and finally to the application of the fused transmission line tower disaster monitoring data to disaster feature identification and risk assessment. Compared with methods that only perform single time synchronization, spatial calibration, or data filtering, this application improves the accuracy, stability, and reliability of transmission line tower monitoring data fusion under complex disaster environments by establishing the correlation between multi-source data in time processing, spatial processing, and quality optimization.
[0062] Example 2 In practical applications, this application takes the disaster monitoring of transmission line towers as its application background. The following describes the data acquisition, time synchronization, spatial calibration, intelligent preprocessing and result output process of this application in conjunction with specific embodiments.
[0063] In this embodiment, multi-source heterogeneous monitoring data of the transmission line tower and its surrounding environment are acquired. This multi-source heterogeneous monitoring data includes three-dimensional point cloud data acquired by lidar, image data acquired by optical cameras, and state data such as strain and tilt angle acquired by structural state sensors. Specifically, the three-dimensional point cloud data is used to characterize the spatial structure information of the tower and its surrounding terrain; the image data is used to characterize the texture and target feature information of the tower and its surrounding environment; and the strain and tilt angle data are used to characterize the stress and attitude changes of the tower structure. The above data can be obtained by corresponding sensors installed on the tower body, around the tower foundation, or on the inspection carrier, but the specific installation method of the sensors does not constitute a limitation on the method steps of this application.
[0064] After acquiring multi-source heterogeneous monitoring data from various sensors, the multi-source heterogeneous data is time-synchronized based on a unified time reference. Specifically, a unified reference time, i.e., a global time reference, is used. As a time reference for aligning various multi-source heterogeneous monitoring data, the time deviation between the sampling time of different sensors and the unified reference time is determined, and time correction is performed on data with inconsistent sampling times. For sensors At sampling time Collected data The extended Kalman filter is used to predict and correct the time series state of the sensor, thus obtaining the sensor's state at a unified reference time. The estimated state is expressed as follows: . in, For the first Sensor-like sensors Estimated values of tower operation status sensor data; For the first The Kalman gain matrix of the sensor is used to balance the error between the predicted state and the actual observation; This is the observation matrix, used to map the predicted state to the observation space; For the first The sensor-like data represents the actual tower operating status data sampled at the current moment. For the first The sensor data is the tower operation status of the previous moment. Through the above processing, the multi-source heterogeneous data obtained at different sampling frequencies and sampling times can be unified under the same time reference, so as to obtain time-aligned multi-source heterogeneous monitoring data.
[0065] After time synchronization is completed, spatial joint calibration is performed on the time-aligned LiDAR point cloud data and optical image data. Specifically, three-dimensional feature points are extracted from the LiDAR point cloud, and two-dimensional feature points are extracted from the optical image, establishing a matching relationship between the three-dimensional point cloud feature points and the two-dimensional image feature points. Let the i-th feature point in the LiDAR coordinate system be... The three-dimensional feature points are: .
[0066] Among them, superscript This indicates that the point belongs to the lidar coordinate system, and the subscript... Indicates the first Let the nth 3D feature point be the one in the image that matches this 3D feature point. The two-dimensional feature points are: .
[0067] in, and These represent the horizontal and vertical pixel coordinates of feature points in the image, respectively, and the subscripts are... Indicates the first Identify image feature points. Based on the projection relationship between the matched feature points, solve for the rigid body transformation matrix (i.e., the matching relationship) from the LiDAR coordinate system to the camera coordinate system. The optimization objective is: .in, , This represents the matching set of 3D feature points of established point cloud data and 2D feature points of optical images; This represents the rotation matrix from the lidar coordinate system to the camera coordinate system; This represents the translation vector from the lidar coordinate system to the camera coordinate system; Represents the camera intrinsic parameter matrix; This represents the projection scale coefficient. The optimal scaling factor is obtained by minimizing the reprojection error between matching feature points. and Based on this rigid body transformation relationship (i.e. matching relationship), the lidar point cloud data and optical image data are mapped to a unified spatial coordinate system to obtain spatially aligned multi-source data, i.e., each type of fused monitoring data with spatial correction.
[0068] After obtaining time-aligned and spatially aligned multi-source data, intelligent preprocessing is performed on the multi-source data. Specifically, based on the signal-to-noise ratio, abnormal fluctuation degree, and consistency of multiple observations of various sensor data, corresponding adaptive weights are determined, and adaptive weighted filtering is used to suppress noise and correct outliers in the multi-source heterogeneous data. Let the preprocessed output (i.e., the fused transmission line tower disaster monitoring data) be... The predicted state after time and space correction is , No. The actual observed value of sensor data is The corresponding adaptive weight matrix is The preprocessing process can then be expressed as: . in, Represents the identity matrix; According to the The signal-to-noise ratio and consistency across multiple observations of sensor data are dynamically determined. When a sensor's data exhibits significant noise, abrupt changes, or low consistency with other observations, its corresponding weight is reduced; conversely, when a sensor's data is of high quality and consistent with multiple observations, its corresponding weight is increased. This process effectively reduces the impact of noisy data and outlier measurements on the fusion results, improving the stability and reliability of multi-source heterogeneous data.
[0069] Finally, the disaster monitoring data of transmission line towers, after time synchronization, spatial joint calibration, and intelligent preprocessing, is output. This processed data can be used for subsequent tower tilt identification, foundation settlement analysis, identification of surrounding geological anomalies, and risk level assessment. Therefore, this application achieves consistent fusion of multi-source heterogeneous data in both time and spatial dimensions through time correction under a unified time reference, spatial joint calibration between point clouds and images, and adaptive weighted preprocessing based on data quality. This improves the accuracy, real-time performance, and robustness of the disaster monitoring data for transmission line towers.
[0070] The fusion method for monitoring disaster-affected transmission line towers provided in this application effectively solves the problems of temporal and spatial deviations, data noise interference, and difficulties in fusion of heterogeneous information in existing technologies. It utilizes a GPS hardware triggering mechanism to achieve a unified time reference for various sensors, and combines extended Kalman filtering to predict and correct time-series data, enabling sub-millisecond synchronization accuracy in the time dimension, thereby significantly improving the accuracy of disaster event location and dynamic monitoring.
[0071] Furthermore, by employing multi-feature point matching and spatial joint calibration techniques using lidar and optical cameras, it is possible to accurately align 3D point cloud data with 2D optical images, achieving a high degree of spatial consistency in the features of the tower and its surrounding environment. This method effectively overcomes the error accumulation problem caused by traditional manual calibration or single-sensor calibration, ensuring the reliability and accuracy of multi-source information fusion.
[0072] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.
[0073] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.
[0074] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.
[0075] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.
[0076] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).
[0077] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, etc., and are not limited to these.
[0078] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0079] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for fusing disaster monitoring data of transmission line towers, characterized in that, The method includes: Based on various sensors installed within a preset range of transmission line towers, various multi-source heterogeneous monitoring data are acquired; the multi-source heterogeneous monitoring data includes lidar point cloud data, optical camera image data, and tower operation status sensing data. Set a global time base, use extended Kalman filtering to process each type of multi-source heterogeneous monitoring data, and determine time-aligned multi-source heterogeneous monitoring data for each type of multi-source heterogeneous monitoring data. Extract the three-dimensional feature points of the lidar point cloud data and the two-dimensional feature points of the optical image data from each type of time-aligned multi-source heterogeneous monitoring data; Based on the three-dimensional and two-dimensional feature points of the lidar, a matching relationship is established and each type of fused monitoring data with spatial correction is determined; Based on each type of fused monitoring data using the adaptive weighting method and the spatial correction, fused disaster monitoring data for transmission line towers is determined. This fused disaster monitoring data for transmission line towers is used for tower condition assessment and early warning of disaster risks.
2. The method for fusing disaster monitoring data of transmission line towers according to claim 1, characterized in that, A global time base is set, and extended Kalman filtering is used to process each type of multi-source heterogeneous monitoring data to determine the time-aligned multi-source heterogeneous monitoring data for each type, specifically including: Set global time base for various sensors ; Based on the state update equation, the extended Kalman filter is used to predict and correct each type of multi-source heterogeneous monitoring data, and to determine the time-aligned multi-source heterogeneous monitoring data for each type. The state update equation is: ； in, For the first Sensor-like sensors in global time base Estimated values of tower operation status sensor data; For the first The Kalman gain matrix of the sensor-like unit is used to balance the error between the predicted state and the actual observed state. This is the observation matrix, used to map the predicted state to the observation space; For the first The current moment is the actual sampled tower operating status sensing data of the structural state sensor. For the first The tower's operational status is predicted by a sensor at the previous moment. This operational status data is collected by strain sensors and tilt sensors installed on the tower structure and is used to characterize the stress state and attitude change of the tower structure.
3. The method for fusing disaster monitoring data of transmission line towers according to claim 1, characterized in that, Based on the three-dimensional and two-dimensional feature points of the lidar, a matching relationship is established and each type of fused monitoring data for spatial correction is determined, specifically including: The three-dimensional feature points of the lidar are projected onto the coordinate system of the optical camera to determine the image position of the three-dimensional feature points of the lidar projected onto the optical image. The spatial calibration relationship between the lidar and the optical camera is determined with the goal of minimizing the error between the position of the two-dimensional feature points in the optical camera coordinate system and the position of the image. Using the spatial calibration relationship as the matching relationship, each type of fused monitoring data for spatial correction is determined.
4. The method for fusing disaster monitoring data of transmission line towers according to claim 3, characterized in that, The matching relationship for: ； in; Coordinate system for point cloud acquired by lidar The next Three-dimensional feature points; , , The first The coordinate values of a three-dimensional feature point in the three coordinate axes of the lidar coordinate system; The first optical image acquired by the optical camera Homogeneous pixel coordinates of a two-dimensional feature point; For the first The lateral pixel coordinates of each optical image feature point; For the first Vertical pixel coordinates of optical image feature points; For numbering; It is a rotation matrix; This is the camera intrinsic parameter matrix; This is the projection scale factor; . It is a 2-norm; It is a translation vector.
5. The method for fusing disaster monitoring data of transmission line towers according to claim 1, characterized in that, Before determining the fused transmission line tower disaster monitoring data based on each type of fused monitoring data using the adaptive weighting method and the spatial correction, the process also includes: Based on three indicators—signal-to-noise ratio, degree of abnormal fluctuation, and consistency of multiple observations—adaptive weights are determined for each type of multi-source heterogeneous monitoring data.
6. The method for fusing disaster monitoring data of transmission line towers according to claim 5, characterized in that, The formula for the adaptive weight is: ； in, For the first Adaptive weights for sensor-like devices An index for sensor categories; For the first Evaluation indicators for the quality of multi-source heterogeneous monitoring data corresponding to sensor types; This refers to the total number of various sensors involved in data fusion. For the first Evaluation indicators for the quality of multi-source heterogeneous monitoring data of sensor-like devices. ; ; For the first Signal-to-noise ratio of multi-source heterogeneous monitoring data from sensor-like devices; The first one determined based on multiple observations Consistency index of multiple observations of multi-source heterogeneous monitoring data from sensor-like devices; For the first The degree of abnormal fluctuation in multi-source heterogeneous monitoring data of similar sensors; , and All are indicator adjustment coefficients.
7. The method for fusing disaster monitoring data of transmission line towers according to claim 6, characterized in that, Based on the adaptive weighting method and the spatial correction of each type of fused monitoring data, the fused disaster monitoring data of transmission line towers is determined, specifically including: An adaptive weighting method is used to perform noise suppression and outlier correction on each type of fused monitoring data of the spatial correction to determine the fused disaster monitoring data of the transmission line towers. The adaptive weighting method is as follows: ; in, This is the integrated disaster monitoring data for transmission line towers; It is the identity matrix; For the first Adaptive weights for sensor-like applications; For the first The actual observation status of fused monitoring data from various sensors; For the first Predictive observation status based on fusion monitoring data from sensor-like devices.
8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the method for fusing disaster monitoring data of transmission line towers according to any one of claims 1-7.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method for fusing disaster monitoring data of transmission line towers as described in any one of claims 1-7.
10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the method for fusing disaster monitoring data of transmission line towers as described in any one of claims 1-7.