Power transmission line operation state monitoring method based on laser point cloud multi-modal fusion

Abstract

This invention provides a method for monitoring the operational status of transmission lines based on multimodal fusion of laser point clouds, belonging to the technical field of monitoring and early warning. The method includes: extracting inertial data using laser point cloud radar and simultaneously acquiring spatial point cloud data; obtaining high-precision processed data through registration fusion, joint filtering, and multi-scale error correction; establishing a mapping relationship between point cloud contours and inertial response by combining a tower structural characteristic model; cross-validating and constraining the three-dimensional changes and attitude changes of the point cloud, converting them into tilt changes relative to a reference coordinate system; further constructing a multi-time-period state estimation model, fusing historical attitude and deformation data to identify deformation features, and setting differentiated thresholds for different types of towers; identifying structural deformation through deviation monitoring and rate over-limit judgment, and outputting the results. This invention can comprehensively identify changes in towers, thereby timely detecting structural anomalies to achieve transmission line operational status monitoring.

Description

Technical Field

[0001] This invention relates to the technical field of monitoring and early warning, and specifically to a method for monitoring the operating status of power transmission lines based on multimodal fusion of laser point clouds. Background Technology

[0002] Existing power transmission line operation status monitoring technologies mainly rely on observation methods using single sensors or limited sensor combinations. For example, they employ tilt sensors, accelerometers, or strain gauges for single-point monitoring of tower attitude, or use drone aerial photography and photogrammetry for visual inspection of tower geometry. These methods typically use discrete points or local component physical quantities as criteria, inferring whether the tower has tilted or deformed based on time-series trends. Some studies have introduced laser point cloud mapping technology to reconstruct the tower's shape, or combined it with inertial measurement units for attitude sensing, aiming to improve monitoring accuracy. However, due to differences in sampling frequencies among various sensor data, the influence of environmental noise, and inconsistencies in coordinate systems, high-precision fusion of multi-source data is difficult, thus limiting the refined assessment of the overall structural status of the tower.

[0003] The main drawback of existing technologies lies in the lack of unified fusion and modeling capabilities for multimodal data. There is a lack of effective registration and cross-validation mechanisms between inertial data and laser point cloud data, making it impossible to accurately reflect the correspondence between the tower's three-dimensional geometry and attitude changes in a reference coordinate system. This results in tilt determination results being susceptible to sensor bias errors, environmental disturbances, and random noise. Furthermore, traditional monitoring methods typically employ fixed threshold judgments, failing to establish differentiated criteria for the structural characteristics of different tower types, making it difficult to accurately distinguish between structural anomalies and minor fluctuations under normal operating conditions. These shortcomings make it difficult for existing technologies to comprehensively identify the amount of tilt change, dynamic rate change, and local deformation characteristics of towers, thus hindering the timely detection of structural anomalies or stability degradation.

[0004] Therefore, a method is needed to comprehensively identify changes in towers, thereby promptly detecting structural anomalies and enabling the monitoring of transmission line operation status. Summary of the Invention

[0005] This invention provides a method for monitoring the operation status of transmission lines based on multimodal fusion of laser point clouds, which can comprehensively identify changes in towers and promptly detect structural anomalies to achieve the monitoring of the operation status of transmission lines.

[0006] In a first aspect, the present invention provides a method for monitoring the operating status of transmission lines based on multimodal fusion of laser point clouds, the method comprising: Establish the initial installation vertical state parameters of the target towers of the target transmission line, and construct a reference coordinate system based on the initial installation vertical state parameters; Inertial data is extracted from the target tower by laser point cloud radar data processing, and spatial laser point cloud data of the target tower is collected simultaneously. The inertial data and the spatial laser point cloud data are sequentially registered and fused, jointly filtered, and multi-scale error corrected to obtain processed inertial data and processed point cloud data. A mapping relationship between point cloud contour and inertial response is established by combining the tower structure characteristic model. The three-dimensional change of the processed point cloud data and the attitude change of the processed inertial data are cross-validated and constrained for optimization. The results of the cross-validation and constraint optimization are converted into the tilt change relative to the reference coordinate system. A multi-period state estimation model is constructed, which integrates historical attitude data and historical deformation data in a time series manner to identify the deformation characteristics of the target tower. Different tilt criterion thresholds are established for different types of towers. The structural deformation of the target tower is determined by the deviation monitoring of the deformation characteristics and the rate threshold over-limit judgment method, and the results are output.

[0007] In a second aspect, the present invention provides a transmission line operation status monitoring device based on multimodal fusion of laser point clouds. The device is used to execute a transmission line operation status monitoring method based on multimodal fusion of laser point clouds as described above. The device includes an acquisition module, a processing module, and an output module, wherein: The acquisition module is used to establish the initial installation vertical state parameters of the target tower of the target transmission line, and to construct a reference coordinate system based on the initial installation vertical state parameters. The processing module is used to extract inertial data from the target tower through laser point cloud radar data processing, and simultaneously collect spatial laser point cloud data of the target tower. The processing module is used to sequentially perform registration and fusion, joint filtering and multi-scale error correction on the inertial data and the spatial laser point cloud data to obtain processed inertial data and processed point cloud data. The processing module is used to establish a mapping relationship between point cloud contour and inertial response by combining the tower structure characteristic model, cross-validate and constrain the three-dimensional change of the processed point cloud data and the attitude change in the processed inertial data, and convert the result of the cross-validation and constraint optimization into the tilt change relative to the reference coordinate system. The processing module is used to construct a multi-time period state estimation model, fuse historical attitude data and historical deformation data in a time series manner, and identify the deformation characteristics of the target tower. The output module is used to establish differentiated tilt criterion thresholds for different types of towers, determine whether the target tower has structural deformation through deviation monitoring of deformation features and rate threshold over-limit judgment, and output the results.

[0008] In a third aspect of the invention, an electronic device is provided, including a processor, a memory, a user interface, and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any of the preceding embodiments.

[0009] In a fourth aspect, the present invention provides a computer-readable storage medium storing instructions that, when executed, perform the method as described in any of the preceding claims.

[0010] In summary, one or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages: 1. This invention obtains high-precision and time-consistent processed data by registering and fusing inertial data and spatial laser point cloud data in a reference coordinate system, performing joint filtering and multi-scale error correction. It also establishes a mapping relationship between the point cloud profile and the inertial response by combining a tower structural characteristic model, achieving cross-validation and constraint optimization of three-dimensional changes and attitude changes, thereby obtaining accurate tilt changes. Based on this, a multi-time-period state estimation model is constructed, fusing historical attitude data and historical deformation data to extract long-term tilt trends, dynamic rate changes, and local deformation features. Simultaneously, differentiated tilt criterion thresholds are set for different types of towers, and deviation monitoring and rate over-limit judgment are performed. This enables comprehensive identification of overall tilt, hierarchical deformation, and cross-sectional torsion, allowing for timely detection of structural anomalies and ensuring the accuracy and reliability of transmission line operation status monitoring.

[0011] The first improvement lies in establishing a unified spatial reference system. Specifically, the geometric center of the tower foundation is used as the origin, and the line connecting the foundation to the top is used as the vertical reference axis. This is then calibrated and corrected using the gravity vector. Simultaneously, two horizontal reference axes are determined through the installation point in the middle of the tower, forming a strictly orthogonal coordinate system. This improvement ensures that all inertial data and point cloud data are mapped within the same coordinate frame, eliminating reference differences caused by different sensor installation locations and external disturbances, thus providing a unified geometric reference for subsequent multi-source data fusion.

[0012] 2. The consistency and accuracy of multimodal data were improved through registration fusion, joint filtering, and multi-scale error correction. Specifically, timestamp alignment, extrinsic parameter calibration, and coarse-to-fine registration were performed on inertial data and spatial laser point cloud data. Point cloud geometric residual constraints were introduced within the Kalman filtering framework to eliminate bias drift in inertial measurements. Multi-scale error correction then achieved smoothing and residual compensation in both the time and spatial domains. This overcame the noise and drift issues associated with single sensors, enabling synchronous high-precision output of both inertial and point cloud data.

[0013] 3. By establishing a mapping relationship between point cloud contours and inertial response through a tower structure characteristic model, joint estimation of attitude change and flexible micro-deformation was achieved. Specifically, the axis pose sequence and cross-sectional normal sequence of the processed point cloud data were cross-validated with the attitude change in the inertial data. The joint solution of attitude and micro-deformation was obtained through residual calculation and constraint optimization, and then converted into tilt change in the reference coordinate system. 7. This method overcomes the limitation of independent judgment of point cloud and inertial data, enabling the identification of tilt change to simultaneously possess geometric consistency and mechanical constraints, thus improving the reliability of the judgment.

[0014] 4. The key lies in achieving comprehensive identification of deformation characteristics through multi-time-period state estimation and differentiated threshold setting. Specifically, historical attitude data and historical deformation data are integrated into the state estimation model to extract long-term tilt trends, dynamic rate changes, and local deformation characteristics. Differentiated statistical and dynamic thresholds are set for towers of different types, including tension, straight-line, and angle towers. Combined with joint scoring, multi-level judgment and spatial positioning are achieved. This enables monitoring to cover long-term evolution and sudden anomalies, adapt to different structural characteristics, and effectively improve the accuracy and real-time performance of transmission line tower operation status monitoring. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating a method for monitoring the operation status of transmission lines based on multimodal fusion of laser point clouds, as disclosed in an embodiment of the present invention. Figure 2 This is a schematic diagram of a transmission line operation status monitoring device based on multimodal fusion of laser point clouds disclosed in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention.

[0016] Explanation of reference numerals in the attached drawings: 201, acquisition module; 202, processing module; 203, output module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. Detailed Implementation

[0017] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0018] In the description of the embodiments of the present invention, words such as "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0019] In the description of the embodiments of the present invention, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0020] Existing transmission line operation status monitoring technologies mostly rely on single sensors or limited sensor combinations, making it difficult to achieve unified fusion of multimodal data. The lack of effective registration and cross-validation between inertial data and laser point cloud data makes tilt determination results susceptible to errors and disturbances. Furthermore, fixed threshold determination cannot establish differentiated standards for different types of towers, making it difficult to accurately distinguish between structural anomalies and normal operating condition fluctuations. This limits the comprehensive identification of tilt changes, dynamic rate changes, and local deformation characteristics. There is an urgent need for a method that can achieve accurate fusion and differentiated criteria to improve the reliability and timeliness of transmission line operation status monitoring.

[0021] This embodiment discloses a method for monitoring the operating status of transmission lines based on multimodal fusion of laser point clouds, referring to... Figure 1 This includes the following steps S110-S160: S110, establish the initial installation vertical state parameters of the target tower of the target transmission line, and construct a reference coordinate system based on the initial installation vertical state parameters.

[0022] This invention discloses a method for monitoring the operational status of transmission lines based on multimodal fusion of laser point clouds, which is applied to a server. The server includes, but is not limited to, electronic devices such as mobile phones, tablets, wearable devices, and PCs (Personal Computers), and can also be a backend server running the method for monitoring the operational status of transmission lines based on multimodal fusion of laser point clouds. The server can be implemented using a standalone server or a server cluster composed of multiple servers.

[0023] In one possible implementation, a reference coordinate system is constructed based on the initial installation vertical state parameters. Specifically, this includes: acquiring the initial installation vertical state parameters of the target tower in its installed state, including the spatial three-dimensional coordinate information of the tower top, tower middle, and tower foundation locations, and the vector information corresponding to the gravity direction; using the geometric center point of the tower foundation as the origin, and defining the direction of the line connecting the tower foundation to the tower top as the vertical reference axis of the reference coordinate system, and calibrating and correcting the vertical reference axis based on the vector information corresponding to the gravity direction; selecting two directions orthogonal to the vertical reference axis as horizontal reference axes, determined by the line connecting the geometric center point of the tower foundation and the installation point in the middle of the tower, and forming a coordinate system that is orthogonal to each other with the vertical reference axis, thus constructing the reference coordinate system.

[0024] Specifically, when obtaining the initial vertical state parameters of the target tower in its installed state, the top, middle, and foundation positions of the tower are first measured using a total station, 3D laser scanner, or high-precision GPS to obtain the spatial 3D coordinate information of each position. This spatial 3D coordinate information refers to the precise positional description of key points on the tower in a 3D Cartesian coordinate system, reflecting the overall geometric characteristics of the tower. Simultaneously, a gravity sensor or high-precision inertial measurement unit is used to obtain the vector information corresponding to the direction of gravity. This vector information indicates the true direction of gravitational acceleration, providing a reference for subsequent calibration and correction of the vertical reference axis.

[0025] When using the geometric center point of the tower foundation as the origin, it is necessary to model the geometry of the tower foundation. The geometric center point of the tower foundation refers to the geometric center position of all boundary points within the plane boundary of the tower foundation. This point, as the origin of the reference coordinate system, possesses stability and uniqueness. Based on this, the direction of the line connecting the geometric center point of the tower foundation to the top of the tower is defined as the vertical reference axis of the reference coordinate system. This line direction can accurately reflect the spatial extension direction of the tower in its installed state. By comparing and registering this line direction with the vector information corresponding to the gravity direction, minor deviations that may exist during the initial installation of the tower can be identified. The vertical reference axis can then be calibrated and corrected based on vector projection, thereby ensuring that the vertical reference axis is consistent with the actual gravity direction.

[0026] After determining the vertical reference axis, a horizontal reference axis needs to be constructed in the reference coordinate system. Specifically, two directions orthogonal to the vertical reference axis are selected as the horizontal reference axes to satisfy the requirement of three-dimensional orthogonality of the coordinate system. The two horizontal reference axes are determined by connecting the geometric center point of the tower foundation to the installation point at the middle of the tower. The installation point at the middle of the tower refers to a pre-defined identifiable component node in the middle of the tower structure, used to enhance the physical constraint significance of the horizontal reference axes on the structure. By calculating the spatial three-dimensional coordinate information of the geometric center point of the tower foundation and the installation point at the middle of the tower, the direction of the connecting line is obtained. Then, using a vector cross product, two directions orthogonal to the vertical reference axis and to each other are generated, thus forming a complete three-axis coordinate system. The reference coordinate system established in this way has geometric orthogonality and physical constraints, and can be used as a unified spatial reference system in subsequent data processing.

[0027] The S120 extracts inertial data from the target tower through laser point cloud radar data processing and simultaneously collects spatial laser point cloud data of the target tower.

[0028] After the reference coordinate system is determined, the sensor configuration and installation are completed. The laser point cloud radar and inertial measurement unit are fixed in a maintainable position on the target tower, and the relative pose stability between the sensors is ensured by a rigid mounting bracket. After installation, the sensor serial number, calibration date and firmware version are recorded to form equipment metadata, which is used for traceable management and parameter reuse in subsequent data processing links.

[0029] After completing the sensor configuration, a time synchronization mechanism is established. A precise time protocol or hardware pulse synchronization is used to align the scanning timestamp of the laser point cloud radar with the sampling timestamp of the inertial measurement unit. After alignment, all observation frames are described with a unified time grid, and the time-aligned raw data of the inertial measurement unit and the time-aligned primary spatial laser point cloud data are output for subsequent external parameter calibration and motion estimation.

[0030] After time synchronization, sensor extrinsic parameter calibration is performed. Using the geometric center point of the tower foundation, the installation point in the middle of the tower, and the top component of the tower as structural references, multi-view static point clouds are acquired in the reference coordinate system, and short-term static inertial measurement unit sampling is triggered. Based on the commonly visible corner points and edge features, the pose transformation matrix and time delay parameters of the laser point cloud radar to the inertial measurement unit are solved. After calibration, the original observations are uniformly mapped to the reference coordinate system to obtain preliminary spatial laser point cloud data and preliminary inertial measurement unit data in the reference coordinate system, which are used as inputs for motion estimation.

[0031] After completing the extrinsic parameter calibration, motion estimation is performed on the spatial laser point cloud data. Feature-based point cloud odometry is used to perform inter-frame matching and frame map matching on continuous scans. Short-term pose sequences are estimated through the collaborative optimization of corner points and surface points. Fixed height layering and component normal constraints are used to suppress drift caused by occlusion and sparsity, thereby generating point cloud pose sequences in the reference coordinate system. After the point cloud pose sequences are aligned with the time-aligned inertial measurement unit raw data in the same time grid, geometric priors are provided for solving the inertial data.

[0032] After obtaining the point cloud pose sequence, inertial measurement unit (IMU) pre-integration fusion is performed. The attitude prior and gravity direction prior provided by the point cloud pose sequence are used to estimate the gyroscope zero bias and accelerometer zero bias of the IMU online. The bias, scale factor and extrinsic parameter perturbation are jointly updated through sliding window optimization. After optimization convergence, the original measurements of the IMU are debiased and de-drifted to obtain clean IMU data after joint correction, which serves as the basic input for solving the inertial data.

[0033] After joint calibration, inertial data is calculated based on a unified time grid. The angular velocity time series is obtained by using the consistency constraint between the gyroscope output after joint calibration and the attitude change of the point cloud pose sequence. The linear acceleration time series is obtained by using the accelerometer output after joint calibration combined with gravity component separation and low-frequency drift suppression. The pitch and roll angles are read from the point cloud pose sequence to form the attitude angle time series. The three types of quantities together constitute the inertial data and are output together with the spatial laser point cloud data under the same time grid to ensure the consistency of the input for subsequent registration fusion and joint filtering.

[0034] While generating inertial data, spatial laser point cloud data acquisition quality control is performed. For each scanning cycle, echo intensity statistics, point density statistics, and component coverage assessment are conducted. If the coverage is insufficient, an adaptive scanning strategy is triggered to extend the single-cycle accumulation time and increase the point cloud sampling density of key components. After the quality control is passed, quality-marked spatial laser point cloud data is output and bound to the aforementioned inertial data by frame index to form a data pair that can be directly entered into the registration and fusion stage.

[0035] After the above link is running stably, an online consistency check is performed. The point cloud pose sequence and attitude angle time sequence are checked for co-directionality using the axis reference in the reference coordinate system. If a short-term inconsistency is found, the process is rolled back to the most recent external bias snapshot, the point cloud odometry is locally re-estimated and the inertial measurement unit is recalibrated until consistency is restored. After the check is completed, a synchronous data stream of inertial data and spatial laser point cloud data is continuously generated as the direct input for subsequent registration and fusion, joint filtering and multi-scale error correction.

[0036] S130 performs registration and fusion, joint filtering and multi-scale error correction on inertial data and spatial laser point cloud data in sequence to obtain processed inertial data and processed point cloud data.

[0037] In one possible implementation, inertial data and spatial laser point cloud data are sequentially registered and fused, jointly filtered, and subjected to multi-scale error correction to obtain processed inertial data and processed point cloud data. Specifically, this includes: timestamping and resampling the inertial data and spatial laser point cloud data based on a time synchronization mechanism to obtain a time-aligned inertial data sequence and a time-aligned first point cloud data sequence; performing sensor extrinsic parameter calibration using the structural reference components of the target tower to establish the pose transformation relationship between the inertial measurement unit and the laser point cloud radar, and mapping the first point cloud data sequence to a reference coordinate system to obtain a preliminarily registered second point cloud data sequence; performing coarse-to-fine registration and fusion based on the second point cloud data sequence, and optimizing the registration result using the attitude change of the inertial data as a priori constraint to obtain a registered and optimized third point cloud data sequence. The system generates aligned first inertial data. A joint state estimation model is established on the third point cloud data sequence and the first inertial data. Extended Kalman filtering or unscented Kalman filtering is used to jointly filter the first inertial data, and bias drift is corrected using point cloud geometric residual constraints. The jointly filtered second inertial data is then output. Synchronous joint filtering is performed on the third point cloud data sequence based on the second inertial data. Combined with statistical outlier removal and surface reconstruction consistency constraints, the jointly filtered result is output. Multi-scale error correction is performed based on the joint filtering result. Through temporal-scale smoothing and backtracking correction, and spatial-scale resolution reconstruction and residual compensation, temporally consistent processed inertial data and shape-consistent processed point cloud data are output respectively. The processed inertial data and processed point cloud data are output under the condition that the cross-source consistency verification meets a preset threshold.

[0038] Specifically, when performing timestamp alignment and resampling on inertial data and spatial laser point cloud data under the time synchronization mechanism, a target time axis is first constructed using a unified time grid, and the timestamps of the original inertial data and the original spatial laser point cloud data are mapped to this target time axis. The timestamp alignment process uses hardware pulse synchronization or a precise time protocol to ensure consistency of the initial reference, and resampling is completed using linear interpolation or spline interpolation, so that data with different sampling frequencies obtain corresponding observation values ​​on the same time grid, forming a time-aligned inertial data sequence and a time-aligned first point cloud data sequence. Here, timestamp alignment refers to unifying the recording time of each observation frame to the same reference time scale, and resampling refers to interpolating or extracting irregular or asynchronous sampled data according to the target time grid to obtain a regular time sequence. The time-aligned inertial data sequence and the time-aligned first point cloud data sequence are used to ensure that subsequent geometric registration and state estimation avoid systematic errors introduced by time phase differences.

[0039] When performing sensor extrinsic calibration using the structural reference components of the target tower, the geometric center point of the tower foundation, the installation point in the middle of the tower, and the top component of the tower are first selected as stable and repeatable structural reference components. Multi-view static scans are acquired and short-time static data of the inertial measurement unit are recorded in the reference coordinate system. The pose transformation relationship between the laser point cloud radar and the inertial measurement unit is established through corresponding geometric features such as corners, edges, and planes. The pose transformation relationship refers to the rigid body transformation matrix describing the three-dimensional rotation and three-dimensional translation between the two sensor coordinate systems, as well as the necessary time delay parameters. The solved pose transformation relationship is applied to the time-aligned first point cloud data sequence to complete the coordinate mapping to the reference coordinate system, resulting in a preliminarily registered second point cloud data sequence, thus eliminating the influence of differences in the installation poses of different sensors on subsequent data fusion.

[0040] When performing coarse-to-fine registration fusion based on the second point cloud data sequence after initial registration, voxel downsampling and normal vector estimation are first performed to obtain a uniformly distributed point set with local geometric information. Initial alignment is obtained by global feature matching or coarse registration based on tower axis constraints. Then, iterative nearest-point registration with robust kernel function constraints is used to refine the alignment within a local window. During the registration process, the attitude change of the inertial data is used as a priori constraint. The prior constraint is added to the objective function to limit the search space of the solution, thereby suppressing local optima caused by occlusion and sparsity. Finally, the third point cloud data sequence after registration optimization and the first inertial data after alignment are output. Here, coarse-to-fine registration refers to first determining the approximate pose at the global or low-resolution level, and then refining the estimation at the high-resolution local level. The prior constraint refers to the attitude change range and direction information provided by the inertial data used to guide the matching to converge to a physically reliable solution.

[0041] When establishing a joint state estimation model on the third point cloud data sequence and the first inertial data, attitude, angular velocity bias, acceleration bias, and extrinsic parameter perturbation are used as joint state variables, and extended Kalman filtering or unscented Kalman filtering is used for recursive estimation. Extended Kalman filtering refers to achieving an approximate Gaussian optimal estimation by performing first-order linearization on the nonlinear model, while unscented Kalman filtering refers to using unscented transformation to propagate the mean and covariance to avoid first-order linearization errors. The point cloud geometric residual is used as an observation constraint and is updated together with the inertial observation. The point cloud geometric residual refers to the distance or normal consistency error between the third point cloud data sequence and the geometry predicted from the current state, which is used to correct bias drift and attitude drift. The second inertial data after joint filtering is output to obtain low-noise and zero-bias stable inertial observations.

[0042] When performing synchronous joint filtering on the third point cloud data sequence based on the second inertial data, the attitude prior and gravity direction prior provided by the second inertial data are used as dynamic constraints for filter updates. The point cloud observations are robustly processed by combining statistical outlier removal and surface reconstruction consistency constraints. Statistical outlier removal refers to identifying and removing high residual or isolated points based on the point-to-surface residual distribution or local density statistics. Surface reconstruction consistency constraints require that the normal and curvature changes of adjacent fragments in local triangular meshes or ordered surfaces remain within a reasonable range, thereby maintaining structural continuity. After the above synchronous joint filtering, the joint filtering result is output, realizing the consistency of point cloud pose and geometry in dynamics and geometry.

[0043] When performing multi-scale error correction based on the joint filtering results, firstly, fixed-interval smoothing and backward smoothing are applied to the second inertial data at the time scale to perform temporal smoothing and zero-bias backtracking correction, eliminating low-frequency drift and generating time-consistent processed inertial data. Then, at the spatial scale, resolution reconstruction and residual compensation are performed on the joint filtering results. Resolution reconstruction refers to establishing a pyramid-shaped multi-resolution representation to re-evaluate geometric consistency at coarse, medium, and fine scales. Residual compensation refers to feeding the fitting errors at different scales back to the high-resolution model to repair edges and details, and maintaining geometric regularity with the axis reference and cross-sectional normal constraints of the tower components. Under the condition that the cross-source consistency verification meets the preset threshold, that is, when the attitude change provided by the processed inertial data and the axis change of the processed point cloud data are in the same direction in amplitude and direction and the residual is lower than the threshold, the processed inertial data and processed point cloud data are output as stable inputs for subsequent mapping relationship construction and tilt change calculation.

[0044] S140, combined with the tower structure characteristic model, establishes the mapping relationship between point cloud contour and inertial response, cross-validates and constrains the three-dimensional changes in the processed point cloud data and the attitude changes in the processed inertial data, and converts the results of cross-validation and constraint optimization into the tilt changes relative to the reference coordinate system.

[0045] In one possible implementation, a mapping relationship between point cloud contours and inertial response is established by combining the tower structural characteristic model. The three-dimensional changes in the processed point cloud data and the attitude changes in the processed inertial data are cross-validated and constrained for optimization. The results of the cross-validation and constraint optimization are then converted into tilt changes relative to the reference coordinate system. Specifically, this includes: constructing a tower structural characteristic model based on the reference coordinate system and the design parameters, material parameters, and connecting component parameters of the target tower; performing structural semantic segmentation and geometric element fitting on the processed point cloud data to extract a point cloud contour parameter set, which includes an axis pose sequence, a node three-dimensional coordinate sequence, and a section normal sequence; and establishing a mapping relationship between rigid body pose terms, flexible micro-deformation terms, and connection constraint terms under the constraints of the tower structural characteristic model. The system projects the attitude change from the processed inertial data onto the point cloud contour parameter set to form a point cloud contour change prediction sequence. Through cross-validation, the axis pose residual and section normal residual between the point cloud contour change prediction sequence and the point cloud contour parameter set are calculated to generate a weighted composite residual. A constrained optimization problem is constructed using the attitude change and flexible micro-deformation compensation as optimization variables, and solved under the stiffness boundary, connection constraint, and time smoothing constraint conditions of the tower structure characteristic model. This yields a joint estimation result of the constrained attitude change and flexible micro-deformation compensation. Based on the joint estimation result, a pose update sequence and section update sequence consistent with the point cloud contour parameter set are generated in the reference coordinate system, and these sequences are converted into tilt changes relative to the reference coordinate system.

[0046] Specifically, when constructing the structural characteristic model of the tower based on the reference coordinate system and the design parameters, material parameters, and connecting component parameters of the target tower, the main columns, diagonal web members, crossarms, and foundation constraints are first expressed as component-level parameterizations. The stiffness characteristics are described using the equivalent Young's modulus, shear modulus, and moment of inertia of the members, and the rotational and displacement constraints of the connecting components are described using the nodal constraint matrix. This forms a unified computational framework for mapping rigid body posture and flexible micro-deformation. Within this framework, a continuous representation of the global stiffness matrix and mass distribution is obtained according to the finite element assembly concept. The axis reference and nodal positions are used as geometric anchor points to unify geometric and mechanical information in the same reference coordinate system, ensuring physical consistency between subsequent geometric predictions driven by posture changes and detailed changes corrected by flexible micro-deformation compensation. To ensure consistent mapping between stiffness and geometry, a component-level stiffness integral expression is introduced to verify the contribution of discrete parameters to the overall constraints.

[0047]

[0048] in, Let B represent the stiffness matrix of the component element, B represent the strain-displacement matrix composed of the derivatives of the component's shape functions, and E represent the constitutive matrix determined by the material parameters. This represents the integral region of the component element in the reference coordinate system; this expression constrains the material and geometric parameters into the tower structure characteristic model by constraining the energy consistency of the component's volume domain.

[0049] When performing structural semantic segmentation and geometric feature fitting on point cloud data, the point sets of main columns, diagonal braces, and crossarms are first distinguished in the reference coordinate system using a region growing strategy based on normal consistency and curvature threshold. Then, linear or planar least squares fitting is performed on each point set to obtain the axial reference and cross-sectional geometry. Based on this, hierarchical nodes are extracted at fixed intervals along the height direction to form a sequence of 3D coordinates of the nodes. At the same time, the cross-sectional normal is calculated through principal component analysis in the neighborhood of each hierarchical node to form a sequence of cross-sectional normals, thus jointly obtaining the point cloud contour parameter set. To improve the stability of the normal estimation, total least squares solution is used for the plane of each local neighborhood.

[0050] Where n represents the normal to the cross section, and d represents the directed distance from the cross section plane to the origin. Represents the three-dimensional coordinates of local neighborhood points. This indicates the number of points involved in the estimation; the above solution equivalences the impact of noise on the three axes, thereby obtaining a robust sequence of cross-sectional normals for subsequent mapping and verification.

[0051] When establishing a mapping relationship consisting of rigid body pose terms, flexible micro-deformation terms, and connection constraint terms under the constraints of the tower structure characteristic model, the attitude change amount in the inertial data is used as the driving variable, and the point cloud contour parameter set is predicted forward in the reference coordinate system. The rigid body pose term approximates the effect of attitude change amount on the axis pose sequence using a small-angle exponential mapping, the flexible micro-deformation term describes local deflection and torsion using nodal shape function interpolation, and the connection constraint term maintains physical feasibility by setting boundary conditions for the foundation and crossarm ends, forming a point cloud contour change prediction sequence:

[0052] in, This represents a comprehensive representation of the axis and cross-sectional geometry corresponding to the predicted sequence of point cloud contour changes. Represents the change in attitude The generated rigid body rotation matrix, where P represents the geometric representation of the point cloud contour parameter set at the previous time step, and t represents the rigid body translation compensation related to the installation zero bias. The expression represents the shape function matrix constructed from the tower structure characteristic model, and q represents the flexible micro-deformation compensation amount. This expression couples the attitude change amount and the flexible micro-deformation compensation amount into the geometric prediction within a linearly separable framework, making the prediction interpretable and constrainable.

[0053] When calculating the axial pose residual and cross-sectional normal residual between the predicted sequence of point cloud contour changes and the point cloud contour parameter set through cross-validation, first, layer-by-layer alignment is performed at the hierarchical nodes, and then cross-sectional alignment is performed at the cross-sections of each layer to form two types of residuals, which are further synthesized into a weighted composite residual to measure the degree of consistency between the prediction and the measurement:

[0054]

[0055] in, and Representing levels The predicted and measured axis directions, and Representing levels The predicted and measured cross-sectional normals are located at the cross-section. This represents the orthogonal basis used for this level in the reference coordinate system. This indicates that the vector sequence is formed by stacking the elements in a hierarchical manner. and These represent weight matrices set according to hierarchical importance and point cloud coverage, respectively. This represents the weighted composite residual; this metric is constrained simultaneously on both the axis and cross-section channels to ensure that rigid body consistency and cross-section consistency are achieved at the same time.

[0056] When constructing a constrained optimization problem with attitude change and flexible micro-deformation compensation as optimization variables, the weighted composite residual is used as the objective function. The problem is solved jointly under the stiffness boundary, connection constraint, and time smoothing constraint provided by the tower structure characteristic model to obtain the joint estimation results of attitude change and flexible micro-deformation compensation.

[0057]

[0058] in, and Here, K represents the coefficient of the tradeoff term, K represents the equivalent stiffness matrix determined by material and geometric parameters to penalize excessive flexibility, and D represents the time difference operator used to suppress non-smooth fluctuations in attitude changes between adjacent time steps. This represents the connection constraint matrix between the foundation and the crossarm end. and The inequality constraint represents the range of allowable rotation and displacement at the node; this optimization minimizes the geometric residual under the joint constraints of physical boundary and time continuity, thereby obtaining a joint estimation result that is consistent with the structural prior and is time-stable.

[0059] When generating pose and cross-section update sequences consistent with the point cloud contour parameter set in the reference coordinate system based on the joint estimation results, the axis pose sequence is first updated with the jointly estimated attitude change, and then the cross-section normal sequence is updated with the jointly estimated flexible micro-deformation compensation, while maintaining the connection constraint identity. The updated pose and normal are then decomposed on the vertical reference axis and two horizontal reference axes to obtain the tilt change relative to the reference coordinate system. The tilt change is represented by the sum of two orthogonal components: pitch and roll, in terms of amplitude and direction.

[0060] in, and They represent the times respectively. Regarding the pitch and roll components of the two horizontal reference axes of the reference coordinate system, This indicates the magnitude of the change in tilt. The tilt change is represented by a unit vector in the horizontal reference plane. This construction combines two orthogonal attitude components of the pose update sequence into a geometrically clear tilt amplitude and tilt direction, thereby outputting a tilt change time series that is consistent with the point cloud contour parameter set and can be directly used in subsequent multi-time state estimation and threshold determination.

[0061] S150: Construct a multi-time period state estimation model, which integrates historical attitude data and historical deformation data in a time series manner to identify the deformation characteristics of the target tower.

[0062] In one possible implementation, a multi-time-period state estimation model is constructed, which integrates historical attitude data and historical deformation data in a time-series manner to identify the deformation characteristics of the target tower. Specifically, this includes: unifying the tilt change sequence of the reference coordinate system with the historical attitude data and historical deformation data aligned with the corresponding time into a discrete time series, and generating an observation sequence using a unified time grid; establishing state variables including pitch angle, roll angle, angular velocity offset, acceleration offset, and flexible micro-deformation control quantities based on the tower structural characteristic model, and constructing state transition equations and observation equations. The state transition equations characterize unmodeled disturbances with process noise, and the observation equations map the observation sequence to the state variables with geometric consistency constraints; employing a joint strategy of improved Kalman filtering and fixed interval smoothing to perform forward recursion and backward smoothing on the state variables to generate a smooth state sequence spanning multiple time periods; calculating the long-term tilt trend, dynamic rate change, and local deformation characteristics on the smooth state sequence to obtain the deformation characteristics.

[0063] Specifically, when performing unified temporal processing on the tilt change sequence, historical attitude data, and historical deformation data within the reference coordinate system, a unified time grid is first established. Different sampling frequencies and irregular timestamps are aligned into discrete time series through time synchronization and interpolation resampling. Each channel quantity is then constructed into an observation sequence to ensure consistent input for subsequent state estimation. To this end, an observation vector is constructed at each moment on the unified time grid. The tilt change in the reference coordinate system is stacked with the corresponding attitude and deformation variables to form a temporally computable observation structure. The observation vector is defined as follows:

[0064] in, Indicates at time The observation vector; This represents the magnitude of the tilt change, which is the result of the pitch and roll components, in the reference coordinate system. and These represent the pitch and roll components in the two horizontal reference axes of the reference coordinate system, respectively. The multidimensional quantities representing historical deformation data (including geometric representation vectors of axis pose components and cross-sectional normal components) have all been aligned to a unified time grid through time synchronization and interpolation resampling.

[0065] When establishing state variables and constructing state transition equations and observation equations based on the tower structure characteristic model, the rigid body attitude and flexible response are uniformly described in the reference coordinate system. The state variables include pitch angle, roll angle, bias terms of gyroscope and accelerometer, and flexible micro-deformation control variables. Unmodeled disturbances are characterized by process noise, and the observation sequence is mapped to the state variables by geometric consistency constraints. The discrete state-space model is expressed as:

[0066]

[0067] in, Indicates at time The state variables, and For pitch angle and roll angle, and For angular velocity bias and acceleration bias, This is a flexible micro-deformation control quantity; The state transition matrix is ​​obtained by linearizing the tower structure characteristic model. The external stimulus matrix, For equivalent inputs to the operating conditions (including normalized quantities such as wind load, temperature gradient, and conductor tension); The process noise has a covariance of It is used to characterize the uncertainty of unmodeled disturbances and time-varying parameters; For the observation matrix, combined with geometric consistency constraints, a linear or piecewise linear correspondence is established between the observation vector and the state variables in the reference coordinate system; To observe the noise, the covariance is... It is used to describe the random fluctuations in sensor measurements and transient environmental disturbances.

[0068] When employing a joint strategy of improved Kalman filtering and fixed interval smoothing for forward and backward smoothing of state variables, the posterior estimate and covariance at each time step are first obtained through forward recursion. Then, fixed interval smoothing is used to backtrack and optimize the state estimates near the boundary and in the sparse observation interval throughout the entire time period, ensuring that the attitude and flexible response remain consistent in dynamics and geometry over the time series. To enhance time-varying and nonlinear adaptability, extended or unscented mechanisms are used for time-varying linearization and covariance propagation. In the backward stage, the Rauch–Tung–Striebel smoother is used to achieve full-segment uniformity. The smoothing recursion is expressed as follows:

[0069] in, This indicates the time interval under the condition of the entire data segment. Smooth state estimation, Indicates time The posterior filtering estimate, Indicates time Predict to time Prior estimates, To smooth the gain, and These are the corresponding covariances; this joint strategy is based on improving the robust update of Kalman filtering, and reduces the cumulative effect of noise and drift in the global time by smoothing the fixed interval, thereby generating a smooth state sequence across time periods.

[0070] When calculating the long-term tilt trend, dynamic rate change, and local deformation characteristics on a smooth state sequence, the smooth state sequence is first decomposed into a slowly changing trend term and a fast disturbance term. Then, the rate and local flexible response indices are extracted under the trend-disturbance framework to obtain a multi-granular characterization suitable for subsequent threshold determination. The long-term tilt trend is obtained by performing polynomial regular regression on the pitch and roll angles on a unified time grid. The dynamic rate change is obtained through robust statistics of first-order and second-order discrete differences within a sliding window. The local deformation characteristics are obtained by projecting the flexible micro-deformation control quantity onto the structural feature mode. The relevant calculations are defined as follows:

[0071]

[0072] in, Let them be polynomial basis function vectors. and The trend regression coefficients for pitch and roll angles are given. This is the trend regularization coefficient; and At time respectively The vector representation of the dynamic rate change and the dynamic second-order change. and These are first-order and second-order discrete-time difference operators, respectively; For local deformation feature vectors, To smooth the obtained flexible micro-deformation control quantity, M is the characteristic mode matrix constructed from the tower structure characteristic model, which is used to map the flexible control quantity into physically interpretable indices at the component level and node level in the reference coordinate system; the above results together constitute the multi-component output of deformation characteristics, and maintain temporal and coordinate consistency with the previously generated observation sequence and smoothed state sequence.

[0073] S160 establishes differentiated tilt criterion thresholds for different types of towers, determines whether the target tower has structural deformation by monitoring deviations in deformation characteristics and judging rate threshold exceedances, and outputs the results.

[0074] In one possible implementation, differentiated tilt criterion thresholds are established for different types of towers. Specifically, this includes: extracting a set of type parameters based on the tower structural characteristic model to distinguish the stress characteristics and boundary constraints of tension towers, straight towers, and angle towers; performing load condition normalization on the overall tilt change, layer tilt change, and cross-sectional torsional change under the constraints of the type parameter set to form a multi-dimensional observation sequence; constructing a robust baseline distribution based on the multi-dimensional observation sequence, obtaining the mean vector and covariance matrix using weighted estimation, and then calculating the Mahalanobis distance and setting a first-level statistical threshold based on the chi-square distribution quantile; and then... Based on the threshold calculation, robust scaling statistics are calculated for the first and second differences of the overall tilt change and the hierarchical tilt change, respectively. Rate threshold and acceleration threshold are set as secondary dynamic thresholds. Under the joint constraints of the primary statistical threshold and the secondary dynamic threshold, a multi-index normalized score is constructed and a reliability weight is introduced to generate a joint score. Based on the joint score, a graded threshold of early warning threshold, alarm threshold and critical threshold is set to form a set of differentiated tilt judgment thresholds. The mean vector, covariance matrix and robust scaling statistics are dynamically updated through an adaptive update mechanism to output a typed differentiated tilt judgment result.

[0075] Specifically, the design parameters, material parameters, and connecting component parameters of the target tower are read within the reference coordinate system to form a set of type parameters, including type identifier, axis reference, node position, connection constraint, foundation boundary, crossarm layout, and conductor tension line direction. The type identifier is used to distinguish the force characteristics and boundary constraints of tension towers, straight towers, and corner towers. The axis reference and node position are used to define the geometric skeleton. The connection constraint and foundation boundary are used to define the displacement and rotation degrees of freedom. The crossarm layout and conductor tension line direction are used to characterize the path of external working conditions. Based on the differences in type identifier and connection constraint, a grouping mapping rule is established to encode the high end constraint and high conductor tension coupling characteristics of tension towers, the neutral force transmission characteristics and symmetrical boundary of straight towers, and the deflection angle constraint and asymmetrical force characteristics of corner towers into the type parameter set, so that the subsequent threshold modeling is consistent in terms of structural priors.

[0076] Linear compensation and synchronous resampling are performed on the overall tilt change, hierarchical tilt change, and cross-sectional torsional change on a unified time grid, along with equivalent inputs such as wind speed, temperature gradient, and conductor tension, to eliminate consistency offsets in external operating conditions. An observation vector is constructed and geometrically consistent projection is performed using connection constraints within the type parameter set and the axis reference, resulting in a normalized multidimensional observation sequence. The linear compensation for operating condition normalization can be written as:

[0077] in, Indicated in type Next moment The original observation vector contains the combined components of the overall tilt change, the hierarchical tilt change, and the cross-sectional torsional change. Represents the equivalent input vector of the working condition; This represents the sensitivity matrix obtained from the set of type parameters and historical regression. This represents the observation vector after the operating condition is normalized, which is used as a consistent input for threshold modeling.

[0078] Within a sliding window, a weighted estimate with a forgetting factor is used to obtain the mean vector and covariance matrix, and this statistic is used to define the Mahalanobis distance and the first-order statistical threshold; the weighted estimate is written as:

[0079] in, and Types The mean vector and covariance matrix, Forgetting factor, For sliding window index, For the current time; Mahalanobis distance and first-order statistical threshold are defined as follows:

[0080] in, Indicates the intensity of the joint deviation. As an observation dimension, For type The level of false alarm control, For the chi-square distribution quantiles, This is the first-level statistical threshold.

[0081] First-order and second-order differences are calculated for the overall tilt change and hierarchical tilt change on a unified time grid, respectively. Robust scaling estimates are established using median absolute deviation, forming rate and acceleration thresholds. The robust scaling estimates and thresholds are defined as follows:

[0082]

[0083] in, and These are first-order and second-order time difference operators, respectively. This represents the median absolute deviation. Indicates the tilt correlation component, and For rate robustness and acceleration robustness, and These are the scale mapping coefficients. and This is a secondary dynamic threshold.

[0084] The normalized ratio of Mahalanobis distance to differential channels is weighted according to channel reliability to obtain a joint score. The reliability weights are derived from observation confidence and sensor health, and after normalization, they satisfy the constraint that the weight sum is one. The joint score is defined as follows:

[0085] in, and , The L2 norm is used; this joint score simultaneously measures the intensity of anomalies across three channels: statistical deviation, rate change, and acceleration transition, and the reliability weights reflect the differences in morphology and channel reliability.

[0086] Based on historical acceptable interval quantiles and safety redundancy, warning thresholds, alarm thresholds, and critical thresholds are set, and hysteresis bands and minimum dwell times are configured to suppress frequent jitter. Simultaneously, the mean vector, covariance matrix, and robust scaling statistics are updated online, employing a gated exponential weighting rule and limiting the maximum relative change of thresholds within a day. The tiered thresholds and online updates are written as follows:

[0087]

[0088] in, These are the warning threshold, alarm threshold, and critical threshold. To update the step size, a snapshot of the statistical quantity of the new batch of data that has passed quality control is added with the crossed-out amount; when the observation reliability or sensor health decreases, the threshold is frozen and the reliability weight is reduced; finally, a standardized and differentiated tilt judgment result is output, which includes the level label, the over-limit channel, the tilt direction and tilt amplitude in the reference coordinate system, and the trigger window index, for use in operation and maintenance decision-making.

[0089] In one possible implementation, the presence of structural deformation in the target tower is determined by monitoring deviations in deformation features and judging rate threshold exceedances, and the results are output. Specifically, this includes: unifying the time series of overall tilt change, the time series of layer tilt change, and the time series of cross-sectional torsional change into a unified input sequence in a reference coordinate system, and calculating a joint deviation statistic based on morphological distribution parameters; performing first-order and second-order difference operations on the overall tilt change and the layer tilt change respectively to obtain rate and acceleration statistics, and setting rate and acceleration thresholds using robust scalar estimation; constructing a joint score based on the joint deviation, rate, and acceleration statistics, and comparing it with the graded thresholds of warning, alarm, and critical thresholds, and outputting a graded judgment result under the condition of satisfying hysteresis and dwell time constraints; simultaneously with the graded judgment result, performing local backtracking on the axis pose sequence and cross-sectional normal sequence of the point cloud contour parameter set to determine the residual contribution rate and outputting the spatial positioning result, thereby generating a structural deformation judgment result containing grade labels, tilt direction, tilt amplitude, and component positioning identifiers.

[0090] Specifically, within the reference coordinate system, the time series of overall tilt change, the time series of hierarchical tilt change, and the time series of cross-sectional torsion change are stacked into a simultaneous input sequence using a unified time grid. Each component is then centered and its correlation is normalized according to the typological distribution parameters. A joint deviation statistic is calculated to measure the overall anomaly intensity across multiple channels. The joint deviation statistic is expressed as Mahalanobis distance, and a forgetting factor is used to suppress the weight of old samples within a sliding window. Its calculation is as follows:

[0091] in, Indicates at time The simultaneous input vectors include components of overall tilt change, hierarchical tilt change, and cross-sectional torsional change. and Indicated in type The mean vector and covariance matrix obtained by weighted estimation are used to provide a formalized statistical center and related structure. This represents the joint deviation statistic, used in conjunction with the dynamic threshold to drive subsequent hierarchical determination. The joint deviation statistic reflects the overall deviation magnitude of multiple components under relevant structural constraints, making different observation channels comparable within the same scale. After the joint deviation statistic is calculated, first-order and second-order differences are performed on the overall tilt change and hierarchical tilt change, respectively, to construct rate and acceleration statistics. Robust scaling estimates are then used to set the corresponding dynamic thresholds. The normalized expressions for the rate and acceleration statistics are:

[0092]

[0093] in, This represents a two-dimensional vector composed of the overall tilt change and the hierarchical tilt change. and These represent the first-order and second-order time difference operators, respectively. Represents the L2 norm, Indicates a sliding window Robust scaling estimation of median absolute deviation within the range. and To map robustness metrics to the detection threshold using a proportionality factor; Used to identify continuous and rapid changes To identify sudden transitions, these two metrics, along with the joint deviation statistic, form a complementary detection channel, thus taking into account both slow degradation and short-term shocks. A joint score is constructed based on the joint deviation statistic, rate statistic, and acceleration statistic, and compared with tiered thresholds for warning, alarm, and criticality thresholds. Hysteresis and dwell time constraints are applied to avoid critical jitter. The joint score is defined as:

[0094]

[0095] in, The reliability weights are derived from the observation reliability and sensor health assessment, and are non-negative and sum to one. Indicated in type The statistical threshold is set according to the chi-square quantile. and These represent the rate threshold and acceleration threshold set according to the robustness scale, respectively; when Upgrades are confirmed when the dwell time is not less than a preset threshold and the dwell time is not less than the minimum dwell time. A hysteresis band with high and low thresholds is used to ensure level stability in the fallback judgment. The graded threshold set is jointly set by historical qualified interval quantiles and safety redundancy, used to map the joint score to the warning level. While outputting the graded judgment results, local backtracking is performed on the axis pose sequence and section normal sequence of the point cloud contour parameter set to calculate the residual contribution rate of components and nodes within the trigger window and provide spatial positioning results, thus forming a structural deformation judgment result containing grade labels, tilt direction, tilt amplitude, and component positioning identifiers. The residual contribution rate measures the anomaly source of each component in the form of energy decomposition, and its calculation is as follows:

[0096]

[0097] in, and They represent the times respectively. For components or nodes The axis pose residual and the cross section normal residual, and This is a diagonal weight matrix set according to hierarchical weights and coverage. The residual contribution rate is represented; the location of the maximum residual contribution rate is used as the spatial positioning result, and the tilt direction and tilt amplitude are calculated by combining the vertical and horizontal reference axes of the reference coordinate system. The final output is the structural deformation judgment result that can be directly used for maintenance priority ranking and operation and maintenance scheduling instructions.

[0098] This embodiment also discloses a transmission line operation status monitoring device based on multimodal fusion of laser point clouds, referring to... Figure 2 The device includes an acquisition module 201, a processing module 202, and an output module 203. It is used to execute any of the above-described multimodal fusion-based transmission line operation status monitoring methods, wherein: The acquisition module 201 is used to establish the initial installation vertical state parameters of the target tower of the target transmission line and to construct a reference coordinate system based on the initial installation vertical state parameters.

[0099] The processing module 202 is used to extract inertial data from the target tower through laser point cloud radar data processing, and simultaneously collect spatial laser point cloud data of the target tower.

[0100] The processing module 202 is used to sequentially perform registration and fusion, joint filtering and multi-scale error correction on inertial data and spatial laser point cloud data to obtain processed inertial data and processed point cloud data.

[0101] The processing module 202 is used to establish a mapping relationship between the point cloud profile and the inertial response by combining the tower structure characteristic model, cross-validate and constrain the three-dimensional changes in the processed point cloud data and the attitude changes in the processed inertial data, and convert the results of cross-validation and constraint optimization into the tilt changes relative to the reference coordinate system.

[0102] The processing module 202 is used to construct a multi-time period state estimation model, which integrates historical attitude data and historical deformation data in a time series manner to identify the deformation characteristics of the target tower.

[0103] Output module 203 is used to establish differentiated tilt criterion thresholds for different types of towers. It determines whether the target tower has structural deformation by monitoring the deviation of deformation characteristics and judging the rate threshold exceeding the limit, and outputs the results.

[0104] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0105] This embodiment also discloses an electronic device, as shown in the reference. Figure 3 The electronic device may include: at least one processor 301, at least one communication bus 302, user interface 303, network interface 304, and at least one memory 305.

[0106] The communication bus 302 is used to enable communication between these components.

[0107] The user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0108] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0109] The processor 301 may include one or more processing cores. The processor 301 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 305, and by calling data stored in memory 305. Optionally, the processor 301 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 301 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications. The GPU is responsible for rendering and drawing the content required for display. The modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 301 and may be implemented as a separate chip.

[0110] The memory 305 may include random access memory (RAM) or read-only memory. Optionally, the memory may include a non-transitory computer-readable storage medium. The memory 305 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described above, etc. The data storage area may store data involved in the various method embodiments described above. Optionally, the memory 305 may also be at least one storage device located remotely from the aforementioned processor 301. As a computer storage medium, the memory 305 may include an operating system, a network communication module, a user interface 303 module, and an application program for a multimodal fusion method for monitoring the operating status of transmission lines based on laser point clouds.

[0111] exist Figure 3In the illustrated electronic device, the user interface 303 is mainly used to provide an input interface for the user and to acquire user input data. The processor 301 can be used to call an application program stored in the memory 305, which is a method for monitoring the operation status of transmission lines based on multimodal fusion of laser point clouds. When executed by one or more processors 301, the electronic device performs one or more methods as described in the above embodiments.

[0112] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, as some steps can be performed in other orders or simultaneously according to the present invention. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0113] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0114] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0115] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0116] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0117] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 305 and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned memory 305 includes various media capable of storing program code, such as a USB flash drive, external hard drive, magnetic disk, or optical disk.

[0118] The present invention also discloses a computer-readable storage medium storing instructions. When executed by one or more processors 301, these instructions cause an electronic device to perform one or more methods as described in the above embodiments.

[0119] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of other embodiments of this disclosure upon considering the specification and the disclosure of practical truths. This invention is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for monitoring the operating status of transmission lines based on multimodal fusion of laser point clouds, characterized in that, The method includes: Establish the initial installation vertical state parameters of the target towers of the target transmission line, and construct a reference coordinate system based on the initial installation vertical state parameters; Inertial data is extracted from the target tower using an inertial measurement unit, and spatial laser point cloud data of the target tower is collected simultaneously. The inertial data and the spatial laser point cloud data are sequentially registered and fused, jointly filtered, and multi-scale error corrected to obtain processed inertial data and processed point cloud data. A mapping relationship between point cloud contour and inertial response is established by combining the tower structure characteristic model. The three-dimensional change of the processed point cloud data and the attitude change of the processed inertial data are cross-validated and constrained for optimization. The results of the cross-validation and constraint optimization are converted into the tilt change relative to the reference coordinate system. A multi-period state estimation model is constructed, which integrates historical attitude data and historical deformation data in a time series manner to identify the deformation characteristics of the target tower. Different tilt criterion thresholds are established for different types of towers. The structural deformation of the target tower is determined by the deviation monitoring of the deformation characteristics and the rate threshold over-limit judgment method, and the results are output.

2. The method for monitoring the operating status of transmission lines based on multimodal fusion of laser point clouds according to claim 1, characterized in that, The process of sequentially registering and fusing the inertial data and the spatial laser point cloud data, performing joint filtering and multi-scale error correction to obtain processed inertial data and processed point cloud data specifically includes: Based on the time synchronization mechanism, the inertial data and the spatial laser point cloud data are timestamped and resampled to obtain a time-aligned inertial data sequence and a time-aligned first point cloud data sequence. Using the structural reference components of the target tower, sensor extrinsic parameter calibration is performed to establish the pose transformation relationship between the inertial measurement unit and the laser point cloud radar, and the first point cloud data sequence is mapped to the reference coordinate system to obtain a preliminarily registered second point cloud data sequence; Based on the second point cloud data sequence, coarse-to-fine registration and fusion are performed, and the attitude change of the inertial data is used as a priori constraint to optimize the registration result, so as to obtain the third point cloud data sequence after registration optimization and the first inertial data after alignment. A joint state estimation model is established on the third point cloud data sequence and the first inertial data. The first inertial data is jointly filtered using extended Kalman filtering or unscented Kalman filtering. The bias drift is corrected using point cloud geometric residual constraints, and the second inertial data after joint filtering is output. Based on the second inertial data, synchronous joint filtering is performed on the third point cloud data sequence, and the joint filtering result is output by combining statistical outlier removal and surface reconstruction consistency constraints. Based on the joint filtering results, multi-scale error correction is performed. Through time-scale smoothing and backtracking correction, and spatial-scale resolution reconstruction and residual compensation, time-consistent processed inertial data and shape-consistent processed point cloud data are output respectively. Under the condition that the cross-source consistency verification meets the preset threshold, the processed inertial data and the processed point cloud data are output.

3. The method for monitoring the operating status of transmission lines based on multimodal fusion of laser point clouds according to claim 2, characterized in that, The process involves establishing a mapping relationship between the point cloud profile and the inertial response using a tower structure characteristic model, cross-validating and optimizing the 3D changes in the processed point cloud data with the attitude changes in the processed inertial data, and converting the results of the cross-validation and optimization into tilt changes relative to the reference coordinate system. Specifically, this includes: The structural characteristic model of the tower is constructed based on the reference coordinate system and the design parameters, material parameters and connecting component parameters of the target tower. The processed point cloud data is subjected to structural semantic segmentation and geometric element fitting to extract a point cloud contour parameter set, which includes an axis pose sequence, a node three-dimensional coordinate sequence, and a cross-sectional normal sequence. Under the constraints of the tower structure characteristic model, a mapping relationship consisting of rigid body pose terms, flexible micro-deformation terms and connection constraint terms is established. The attitude change amount in the processed inertial data is projected onto the point cloud contour parameter set to form a point cloud contour change prediction sequence. The weighted composite residual is generated by calculating the axis pose residual and cross-sectional normal residual between the point cloud contour change prediction sequence and the point cloud contour parameter set through cross-validation. A constrained optimization problem is constructed using attitude change and flexible micro-deformation compensation as optimization variables, and solved under the stiffness boundary, connection constraint and time smoothing constraint of the tower structure characteristic model to obtain the joint estimation results of attitude change and flexible micro-deformation compensation under constraint optimization. Based on the joint estimation results, a pose update sequence and a cross-section update sequence consistent with the point cloud contour parameter set are generated in the reference coordinate system, and the pose update sequence and the cross-section update sequence are converted into tilt changes relative to the reference coordinate system.

4. The method for monitoring the operating status of transmission lines based on multimodal fusion of laser point clouds according to claim 3, characterized in that, The construction of a multi-time-period state estimation model, which fuses historical attitude data and historical deformation data in a time-series manner to identify the deformation characteristics of the target tower, specifically includes: The tilt change sequence of the reference coordinate system is unified with the historical attitude data and historical deformation data aligned with the corresponding time into a discrete time series, and an observation sequence is generated using a unified time grid. Based on the tower structure characteristic model, state variables including pitch angle, roll angle, angular velocity offset, acceleration offset and flexible micro-deformation control are established, and state transition equations and observation equations are constructed. The state transition equations characterize unmodeled disturbances with process noise, and the observation equations map the observation sequence to state variables with geometric consistency constraints. A combined strategy of improved Kalman filtering and fixed interval smoothing is used to perform forward recursion and backward smoothing on the state variables to generate a smooth state sequence across time periods. The deformation characteristics are obtained by calculating the long-term tilt trend, dynamic rate change and local deformation characteristics on the smooth state sequence.

5. The method for monitoring the operating status of transmission lines based on multimodal fusion of laser point clouds according to claim 4, characterized in that, The establishment of differentiated tilt criterion thresholds for different types of towers specifically includes: Based on the tower structure characteristic model, the set of type parameters is extracted to distinguish the stress characteristics and boundary constraints of tension towers, straight towers and angle towers; Under the constraints of the type parameter set, the overall tilt change, the hierarchical tilt change, and the cross-sectional torsion change are normalized according to the working conditions to form a multi-dimensional observation sequence. A robust baseline distribution is constructed based on the multidimensional observation sequence. The mean vector and covariance matrix are obtained by weighted estimation. Then, the Mahalanobis distance is calculated and the first-level statistical threshold is set by combining the chi-square distribution quantile. Based on the first-level statistical threshold, robust scale statistics are calculated for the first-order and second-order differences of the overall tilt change and the hierarchical tilt change, respectively, and rate threshold and acceleration threshold are set as second-level dynamic thresholds. Under the joint constraints of the first-level statistical threshold and the second-level dynamic threshold, a multi-index normalized score is constructed and a reliability weight is introduced to generate a joint score; Based on the joint scoring, a set of graded thresholds for early warning, alarm, and criticality is set to form a differentiated tilt criterion threshold set. The mean vector, covariance matrix, and robust scaling statistics are dynamically updated through an adaptive update mechanism to output a standardized differentiated tilt judgment result.

6. The method for monitoring the operating status of transmission lines based on multimodal fusion of laser point clouds according to claim 5, characterized in that, The method of determining whether the target tower has structural deformation through deviation monitoring of the deformation characteristics and rate threshold exceedance judgment, and outputting the result, specifically includes: Under the reference coordinate system, the time series of overall tilt change, the time series of hierarchical tilt change, and the time series of cross-sectional torsion change are unified into a joint input sequence, and the joint deviation statistics are calculated in combination with the set of type parameters. Based on the joint deviation statistics, first-order and second-order difference operations are performed on the overall tilt change and the hierarchical tilt change respectively to obtain the velocity statistics and acceleration statistics, and the velocity threshold and acceleration threshold are set using robust scalar estimation. A joint score is constructed based on the joint deviation statistic, the rate statistic, and the acceleration statistic, and compared with the classification thresholds of the warning threshold, alarm threshold, and critical threshold. Under the condition of satisfying the hysteresis and dwell time constraints, the classification judgment result is output. Simultaneously with the graded determination results, the axis pose sequence and cross-sectional normal sequence of the point cloud contour parameter set are locally backtracked to determine the residual contribution rate and output the spatial positioning results, thereby generating structural deformation determination results containing grade labels, tilt direction, tilt amplitude and component positioning identifiers.

7. The method for monitoring the operating status of transmission lines based on multimodal fusion of laser point clouds according to claim 6, characterized in that, The construction of the reference coordinate system based on the initial installation vertical state parameters specifically includes: The initial installation vertical state parameters of the target tower in the installation state are obtained. The initial installation vertical state parameters include the spatial three-dimensional coordinate information of the top of the tower, the middle of the tower, and the position of the tower foundation, as well as the vector information corresponding to the gravity direction. The origin is taken as the geometric center point of the tower foundation, and the direction of the line connecting the tower foundation to the top of the tower is defined as the vertical reference axis of the reference coordinate system. The vertical reference axis is calibrated and corrected based on the vector information corresponding to the direction of gravity. Based on the vertical reference axis, two directions orthogonal to the vertical reference axis are selected as horizontal reference axes. The two horizontal reference axes are determined by the line connecting the geometric center point of the tower foundation and the installation point in the middle of the tower, and are orthogonal to the vertical reference axis to form a coordinate system, thus constructing the reference coordinate system.

8. A transmission line operation status monitoring device based on multimodal fusion of laser point clouds, characterized in that, The device is used to execute a transmission line operation status monitoring method based on multimodal fusion of laser point clouds as described in any one of claims 1-7. The device includes an acquisition module, a processing module, and an output module, wherein: The acquisition module is used to establish the initial installation vertical state parameters of the target tower of the target transmission line, and to construct a reference coordinate system based on the initial installation vertical state parameters. The processing module is used to extract inertial data from the target tower through an inertial measurement unit and simultaneously acquire spatial laser point cloud data of the target tower. The processing module is used to sequentially perform registration and fusion, joint filtering and multi-scale error correction on the inertial data and the spatial laser point cloud data to obtain processed inertial data and processed point cloud data. The processing module is used to establish a mapping relationship between point cloud contour and inertial response by combining the tower structure characteristic model, cross-validate and constrain the three-dimensional change of the processed point cloud data and the attitude change in the processed inertial data, and convert the result of the cross-validation and constraint optimization into the tilt change relative to the reference coordinate system. The processing module is used to construct a multi-time period state estimation model, fuse historical attitude data and historical deformation data in a time series manner, and identify the deformation characteristics of the target tower. The output module is used to establish differentiated tilt criterion thresholds for different types of towers, determine whether the target tower has structural deformation through deviation monitoring of deformation features and rate threshold over-limit judgment, and output the results.

9. An electronic device, characterized in that, The device includes a processor, a communication bus, a user interface, a network interface, and a memory. The memory is used to store instructions. The user interface and the network interface are both used to communicate with other devices. The communication bus is used to enable communication between the components within the electronic device. The processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1-7.

Images