Method, device and equipment for damage prediction of power transmission tower and storage medium

Abstract

This application discloses a method, apparatus, device, and storage medium for damage prediction of transmission towers, relating to the field of power technology. The method includes: extracting features from comprehensive heterogeneous data of the transmission tower to obtain comprehensive heterogeneous features; mapping and fusing the comprehensive heterogeneous features based on the structural topology diagram of the transmission tower to obtain fused topology features; fusing and classifying the comprehensive heterogeneous features and the fused topology features to obtain damage detection results and damage level classification results for each component of the transmission tower; and performing damage time-series prediction based on the damage detection results and the damage level classification results to obtain damage prediction results for each component of the transmission tower. This application improves the comprehensiveness of transmission tower detection and the accuracy of prediction by extracting comprehensive heterogeneous features and combining them with fused topology features for weighted fusion, thereby performing damage detection and prediction.

Description

Technical Field

[0001] This application relates to the field of power technology, and in particular to methods, devices, equipment and storage media for predicting damage to transmission towers. Background Technology

[0002] To ensure the safe operation of transmission towers, regular and efficient damage detection is essential. Currently, transmission tower damage detection mainly relies on manual inspections and single-sensor methods. Manual inspections are labor-intensive, inefficient, and limited by objective conditions such as inclement weather and the risks of working at heights. While single-sensor-based detection methods, such as using only visible light images from UAVs for visual defect identification, have seen some application in recent years, their detection information is limited in scope and cannot comprehensively capture the complex characteristics of different types of damage, such as corrosion of members, loose bolts, and structural deformation. The most significant shortcoming of existing methods lies in the limited availability of data sources. They typically rely on data from only one modality (such as visible light images) for analysis. This limitation prevents the detection system from acquiring global and multi-dimensional feature information of the damage, severely restricting the comprehensiveness and accuracy of the detection and easily leading to missed detections or misjudgments.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, equipment, and storage medium for predicting damage to transmission towers, aiming to solve the technical problem of insufficient comprehensiveness in damage detection and prediction for transmission towers.

[0005] To achieve the above objectives, this application proposes a damage prediction method for transmission towers, the method comprising: Feature extraction is performed on the comprehensive heterogeneous data of transmission towers to obtain comprehensive heterogeneous features; Based on the structural topology diagram of the transmission tower, the integrated heterogeneous features are mapped and fused to obtain fused topology features; The integrated heterogeneous features and the fused topological features are fused and classified to obtain the damage detection results and damage level classification results of each component of the transmission tower; Based on the damage detection results and the damage level classification results, damage time sequence prediction is performed to obtain the damage prediction results for each component of the transmission tower.

[0006] In one embodiment, the integrated heterogeneous data includes visible light images, infrared thermal imaging data, point cloud time-series data, and vibration monitoring data. The integrated heterogeneous features include visible range features, infrared temperature field features, point cloud time-series features, and vibration time-series features. The step of extracting features from the integrated heterogeneous data of the transmission tower to obtain integrated heterogeneous features includes: Image features are extracted from the visible light image to obtain visible range features, which include rust texture features, rust area ratio features, and image spectrum features. Temperature field features are extracted from the infrared thermal imaging data to obtain infrared temperature field features; Point cloud features are extracted from the point cloud time series data to obtain point cloud time series features; Vibration features are extracted from the vibration monitoring data to obtain vibration time-series features.

[0007] In one embodiment, the step of mapping and fusing the integrated heterogeneous features based on the structural topology map of the transmission tower to obtain the fused topology features includes: A structural topology diagram is constructed based on the actual structure of the transmission tower, wherein the graph nodes of the structural topology diagram are the poles of the transmission tower; The integrated heterogeneous features are double-aligned to obtain the target aligned features; The target alignment features are fused based on the membership relationships of the members in the structural topology diagram to obtain fused topology features.

[0008] In one embodiment, the step of performing dual alignment on the synthesized heterogeneous features to obtain the target aligned features includes: The point cloud temporal features and the vibration temporal features are coarsely aligned using a dynamic time warping algorithm to obtain the initial aligned features after coarse alignment. The initial alignment features are input into the residual fine alignment network for fine-grained correction to obtain the target alignment features.

[0009] In one embodiment, the damage detection results include rod corrosion detection results and bolt condition detection results. The step of fusing and classifying the integrated heterogeneous features and the fused topological features to obtain the damage detection results and damage level classification results of each component of the transmission tower includes: The contribution weights are obtained by cross-modal attention weighting of the integrated heterogeneous features and the fused topological features through a multi-head self-attention mechanism, and feature fusion is performed based on the contribution weights to obtain the integrated damage features. The comprehensive damage features are input into the rod corrosion detection head, the bolt condition detection head, and the damage level classification head, respectively, to obtain the rod corrosion detection results, the bolt condition detection results, and the damage level classification results.

[0010] In one embodiment, the step of performing damage time-series prediction based on the damage detection results and the damage level classification results to obtain the damage prediction results for each component of the transmission tower includes: Based on the inspection time sequence, the damage detection results and damage level classification results of each component of the same transmission tower are sequenced to obtain the component-level damage time series. The component-level damage time series is input into the time series prediction network, and the damage level change prediction curve of each component in the future time window is output. Based on the damage level change prediction curve, the estimated remaining time for each component to upgrade from the current damage level to the next damage level is calculated, and the damage prediction result is obtained.

[0011] In one embodiment, the step of extracting image features from the visible light image to obtain visible range features includes: The contrast features, correlation features, energy features, and homogeneity features of the rusted region in the visible light image are extracted using the gray-level co-occurrence matrix to obtain the first texture sub-feature. The visible light image is input into a convolutional neural network to extract deep texture features, resulting in a second texture sub-feature. The first texture sub-feature is concatenated with the second texture sub-feature to obtain the rust texture feature; The corroded areas in the visible light image are segmented at the pixel level using a semantic segmentation network. The ratio of the corroded area to the total surface area of ​​the rod is calculated, and the shape features of the corroded areas are extracted to obtain the corroded area proportion feature. A two-dimensional Fourier transform is performed on the visible light image to divide the frequency domain image into low-frequency, mid-frequency, and high-frequency regions. The energy proportion and peak frequency of the low-frequency, mid-frequency, and high-frequency regions are calculated respectively to obtain the image spectral features.

[0012] Furthermore, to achieve the above objectives, this application also proposes a damage prediction device for transmission towers, the damage prediction device for transmission towers comprising: The feature extraction module is used to extract features from the comprehensive heterogeneous data of the transmission towers to obtain comprehensive heterogeneous features; The topology fusion module is used to map and fuse the integrated heterogeneous features based on the structural topology map of the transmission tower to obtain fused topology features. The damage detection module is used to fuse and classify the integrated heterogeneous features and the fused topological features to obtain the damage detection results and damage level classification results of each component of the transmission tower. The damage prediction module is used to perform damage time-series prediction based on the damage detection results and the damage level classification results, and obtain the damage prediction results of each component of the transmission tower.

[0013] In addition, to achieve the above objectives, this application also proposes a damage prediction device for transmission towers, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the damage prediction method for transmission towers as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the transmission tower damage prediction method described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the transmission tower damage prediction method described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: This application proposes a method, apparatus, device, and storage medium for damage prediction of transmission towers. The method involves extracting features from the comprehensive heterogeneous data of the transmission tower to obtain comprehensive heterogeneous features; mapping and fusing these features based on the structural topology diagram of the transmission tower to obtain fused topology features; classifying the comprehensive heterogeneous features and the fused topology features to obtain damage detection results and damage level classification results for each component of the transmission tower; and performing damage time-series prediction based on the damage detection results and damage level classification results to obtain damage prediction results for each component of the transmission tower. This application improves the comprehensiveness of transmission tower detection and the accuracy of prediction by extracting comprehensive heterogeneous features and combining them with fused topology features for weighted fusion, thereby performing damage detection and prediction. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an embodiment of the damage prediction method for transmission towers provided in this application. Figure 2This is a schematic diagram of the module structure of the damage prediction device for transmission towers according to an embodiment of this application; Figure 3 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the damage prediction method for transmission towers in this application embodiment.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] The main solution of this application embodiment is as follows: feature extraction is performed on the comprehensive heterogeneous data of the transmission tower to obtain comprehensive heterogeneous features; the comprehensive heterogeneous features are mapped and fused based on the structural topology map of the transmission tower to obtain fused topology features; the comprehensive heterogeneous features and the fused topology features are fused and classified to obtain damage detection results and damage level classification results of each component of the transmission tower; damage time series prediction is performed based on the damage detection results and the damage level classification results to obtain damage prediction results of each component of the transmission tower.

[0024] In this embodiment, for ease of description, the damage prediction device for transmission towers will be used as the main execution subject in the following description.

[0025] This application provides a solution that improves the comprehensiveness of transmission tower detection and the accuracy of prediction by extracting comprehensive heterogeneous features and combining them with fused topological features for weighted fusion.

[0026] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, such as a damage prediction device for transmission towers. The following description uses a damage prediction device for transmission towers as an example to illustrate this embodiment and the subsequent embodiments.

[0027] Based on this, embodiments of this application provide a method for predicting damage to transmission towers, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the damage prediction method for transmission towers according to this application.

[0028] In this embodiment, the damage prediction method for the transmission tower includes steps S11 to S14: Step S11: Extract features from the comprehensive heterogeneous data of the transmission tower to obtain comprehensive heterogeneous features.

[0029] It should be noted that integrated heterogeneous data refers to multimodal data about transmission towers simultaneously collected by UAVs equipped with visible light cameras, infrared thermal imagers, lidar, and pre-deployed vibration sensors. This includes visible light images reflecting surface conditions, infrared thermal imaging data reflecting temperature distribution, point cloud time-series data reflecting three-dimensional geometry and dynamics, and vibration monitoring data that directly measures structural response. Feature extraction refers to the process of calculating or learning feature vectors from this raw heterogeneous data using specialized algorithms or models to effectively characterize the essential attributes of various types of damage (such as corrosion, bolt loosening, and structural deformation).

[0030] Understandably, damage to transmission towers manifests in diverse ways, making comprehensive coverage difficult with data from a single source. While corrosion is readily apparent under visible light, internal defects or poor connections may initially present as localized overheating (infrared) or abnormal vibration. By systematically collecting and extracting features from multi-source heterogeneous data, this step constructs a comprehensive feature set incorporating visual, thermal radiation, geometric, and dynamic information. This multi-dimensional feature fusion strategy enables the detection model to comprehensively utilize information from surface appearance, internal thermal state, structural deformation, and dynamic response, significantly enhancing its ability to perceive complex, hidden, or composite damage and overcoming the limitations and blind spots of single data sources.

[0031] Specifically, firstly, the UAV flight platform and the ground sensor network are coordinated to synchronously trigger and receive visible light images, infrared images, laser point cloud sequences, and vibration acceleration signals. Then, multiple feature extraction sub-modules are activated: the visible light image processing module segments the images into rods / bolts and runs texture analysis, semantic segmentation, and frequency domain analysis algorithms respectively, outputting texture, area proportion, and spectral feature vectors. The infrared image processing module performs temperature matrix analysis on the segmented rod regions, calculates gradients and anomaly region statistics, and outputs temperature field feature vectors. The point cloud processing module registers and tracks continuous point cloud frames, extracting displacement time-series or overall shape change time-series features of key points on specific rods. The vibration signal processing module filters and transforms the acceleration signal, extracting frequency domain features (such as modal frequencies and amplitudes) or time domain statistical features. All extracted feature vectors are organized by rod ID, forming a comprehensive heterogeneous feature set for each rod.

[0032] For example, during an inspection of a transmission tower, for the main member A-12, texture feature vectors indicating coarse texture and high contrast, area feature vectors indicating a 18% rust area, and spectral feature vectors indicating high-frequency energy anomalies were extracted from its visible light image; temperature field feature vectors indicating a local high-temperature gradient at the connection point were extracted from its corresponding infrared image; temporal features indicating a lateral vibration peak at 1.5 Hz were extracted from the continuous point cloud; and temporal features indicating a slight increase in the damping ratio at 1.48 Hz were extracted from the vibration data. These features collectively constitute the comprehensive heterogeneous characteristics of member A-12.

[0033] Step S12: Based on the structural topology diagram of the transmission tower, the integrated heterogeneous features are mapped and fused to obtain fused topology features.

[0034] It should be noted that the structural topology graph is a graph model constructed based on the actual physical connection relationships of the transmission tower. Each node in the graph corresponds to a specific member (such as a main member or diagonal member). If two members are directly connected in the physical structure via node plates, bolts, etc., an edge is established between the two graph nodes representing them. Mapping and fusion is a two-stage process: First, the obtained comprehensive heterogeneous feature vectors belonging to each member are assigned as the initial features of the corresponding graph nodes, realizing the mapping from data features to structural nodes. Then, using a graph neural network, the node features are transmitted and aggregated along the edges of the topology graph, so that the final features of each node not only contain its own sensor information but also incorporate the contextual information of its directly and indirectly connected neighboring nodes, thereby generating fused topology features containing structural semantics.

[0035] It is understandable that the mechanical states of transmission tower members influence each other. Damage to one member (such as corrosion weakening) alters its internal forces, which in turn affects the stress state of adjacent members through connection points. This influence may be reflected in sensor data (such as vibration and strain) of connected members. By constructing a topology graph and performing feature fusion, the model can explicitly model the spatial dependencies arising from these physical connections. This allows the model to assess damage not only based on data from individual members but also by considering the state of their neighboring nodes. For example, when a member exhibits slightly abnormal vibration characteristics, while multiple connected members show stress or thermal anomalies, the model can more confidently determine that damage exists in the connection area, thereby improving detection accuracy and providing a structural explanation of the damage propagation path.

[0036] Specifically, firstly, based on the transmission tower design drawings or by automatically identifying the connection relationships of the poles through a point cloud model, an undirected graph G=(V, E) is constructed, where V is the set of graph nodes and E is the set of graph edges. Then, all extracted pole feature vectors are used as the initial feature vectors for the corresponding nodes in graph G, according to their pole ID indices. Next, this graph is input into a multi-layer graph attention network. In each layer, for each node v, the network calculates the attention coefficient between it and all its neighboring nodes u, which represents the importance of neighbor u's features to node v. Then, the new features of node v are obtained by weighted summation of the features of all its neighbors according to the attention coefficients, followed by a nonlinear transformation. After multiple layers of such propagation, the final feature output by each node is a fused topological feature that deeply integrates its own structural information and that of its multi-hop neighbors.

[0037] For example, for the aforementioned member A-12, it is connected to four diagonal members (B-7, C-9, D-3, E-5) in the topology graph. In the first layer of the GAT, node A-12 calculates the attention weights with these four neighboring nodes; for example, the weight with B-7 is 0.4, and the weight with C-9 is 0.3. Then, it aggregates the features of these four neighbors (each containing information such as corrosion, infrared, and vibration) into its own features. In the second layer, node A-12 aggregates again, this time the aggregated neighbor features already contain information about the neighbors' neighbors. Ultimately, the fused topology features of node A-12 not only contain all its own sensor features but also implicitly include structural context information such as the slight corrosion of member B-7 and the slightly higher temperature of member C-9.

[0038] Step S13: The integrated heterogeneous features and the fused topological features are fused and classified to obtain the damage detection results and damage level classification results of each component of the transmission tower.

[0039] It should be noted that fusion classification is an end-to-end process integrating feature fusion and multi-task classification. The comprehensive heterogeneous features represent multimodal features from the original data, while the fused topological features represent contextual features enhanced with structural relationships. First, an adaptive weighted fusion of these two sets of features is performed through a fusion layer (such as an attention-based fusion module) to generate a unified comprehensive damage feature. Subsequently, this feature is fed into multiple parallel, task-specific detection heads, simultaneously generating rod corrosion detection results (such as corrosion area location and severity grading), bolt status detection results (such as normal / loose / missing classification and loosening angle quantification), and damage level classification results (a rating of the overall health status of the component based on all information).

[0040] Understandably, by designing a multi-task learning framework and sharing the underlying feature extraction and fusion network, the model can efficiently infer multiple highly related tasks such as corrosion, bolt status, and overall damage level in a single forward propagation, improving computational efficiency and promoting knowledge transfer between tasks. Furthermore, the attention mechanism introduced during the fusion process automatically learns the contribution weights of different modal features and topological features to the final decision and outputs them as contribution weights. This provides maintenance personnel with interpretable evidence for the model's decisions. For example, it can be clearly stated that a bolt is judged to be loose primarily based on infrared features (hot spots) and geometric features (deflection angle), while visible light features contribute less, thus enhancing the credibility and practicality of the detection results.

[0041] Specifically, firstly, the feature fusion layer receives the comprehensive heterogeneous feature vector and the fused topological feature vector for each member. This layer typically contains a multi-head self-attention mechanism or gated fusion unit, which calculates the importance score (i.e., attention weight) of each feature vector and performs a weighted summation of all features based on the weights, outputting a more discriminative comprehensive damage feature vector. Simultaneously, the weights of each input feature are recorded as contribution weights. Then, this comprehensive damage feature vector is simultaneously fed into three independent detection heads: a member corrosion detection head, typically a fully connected network or a small convolutional network, outputting the confidence level of corrosion presence, bounding box (if needed), and severity level (e.g., none, light, medium, heavy); a bolt status detection head, which, in addition to receiving the comprehensive damage features, can also receive bolt geometric features specifically extracted from the point cloud (e.g., nut deflection angle), outputting the classification probability of the bolt status and the quantified angle of loosening; and a damage level classification head, which integrates all information and outputs the final overall damage level of the component (which may be a member or a connecting node). During training, the losses for the three tasks are jointly optimized using an adaptive weighting method.

[0042] For example, for rod A-12 and its connecting bolts, the fusion layer calculates its comprehensive damage characteristics and outputs the contribution of each mode: visible light texture accounts for 25%, area accounts for 20%, infrared features account for 30%, and topological features account for 25%. After inputting these characteristics into three detection heads, the rod corrosion detection head outputs moderate corrosion with a confidence level of 92%; the bolt condition detection head outputs bolt loosening with a confidence level of 88% and estimates a deflection angle of 4 degrees; the damage level classification head integrates the corrosion and bolt loosening information and outputs moderate damage, suggesting close monitoring.

[0043] Step S14: Based on the damage detection results and the damage level classification results, damage time sequence prediction is performed to obtain the damage prediction results of each component of the transmission tower.

[0044] It should be noted that damage time-series prediction is a predictive function based on time series analysis methods. Damage detection results and damage level classification results are snapshots of the component's state at each inspection point. Arranging these snapshots of the same component in chronological order constitutes the component's damage time series. A time-series prediction network is a neural network model specifically designed to process sequential data, such as a Long Short-Term Memory network or a Transformer encoder, capable of capturing long-term dependencies and trends in the sequence. This network takes historical damage time series as input, and its output is a damage level change prediction curve (a predicted sequence of damage levels over a future period) and estimated remaining time (the predicted time required for the damage to deteriorate from the current level to the next more severe level).

[0045] Understandably, this step aims to address the problem in the background technology of failing to utilize the temporal evolution information of historical inspection data, thus hindering the prediction of damage development trends. Damage to transmission towers (such as corrosion development and continuous bolt loosening) is a gradual process over time. A single inspection only reflects the current state and cannot predict future risks. By constructing component-level time series data and using deep learning models to learn its evolutionary patterns, this step achieves a leap from state detection to trend warning. Predicting future damage level curves and remaining safe time provides forward-looking guidance for operation and maintenance decisions, enabling predictive maintenance, thereby optimizing maintenance plans, reducing the risk of sudden failures, and improving the intelligence and economic efficiency of power grid asset management.

[0046] Specifically, a historical database is maintained to store the timestamps of each inspection and the corresponding detection results for each key component of each transmission tower (e.g., damage levels can be encoded as numbers, or multi-task detection results can be concatenated into a feature vector). When trend prediction is needed, the state sequence of a specified component over the past N time points (e.g., the last 8 quarters) is extracted. This sequence is then input into a pre-trained temporal prediction network, such as LSTM (Long Short-Term Memory). LSTM, through its internal gating units, learns the pattern of state evolution over time, such as linear deterioration, accelerated deterioration, or entering a stable period. The network output can be a sequence of length M, representing the predicted damage state (which can be a level or a feature vector) at the next M time points, i.e., a change prediction curve. Simultaneously, the network can also output a scalar representing the estimated time required for the component to transition from its current state to the next worse state based on the learned degradation rate.

[0047] For example, for pole A-12, its overall damage level sequence over the past two years (8 quarters) is: [1 (None), 1, 2 (Slight), 2, 2, 3 (Moderate), 3, 3]. This sequence is fed into an LSTM network. The network learns that the pole transitioned steadily from slightly to moderate damage over the past year and has maintained moderate damage for three cycles. Based on this pattern, the network predicts the damage level for the next four quarters to be [3, 3, 3, 4 (Severe)], and outputs an estimated remaining time of approximately 2.5 inspection cycles (i.e., it may develop into severe damage in about 7-8 months). This prediction report provides a clear basis for planning maintenance during the next inspection cycle.

[0048] Furthermore, a damage distribution visualization report is generated. The output results of member corrosion detection, bolt status detection, damage level classification, and damage trend prediction are correlated with the constructed transmission tower structural topology map. Each node (member) and its connection relationship are assigned corresponding attributes (such as damage type, level, and predicted trend direction). The report can be displayed in two main forms: (a) 3D model form: On the 3D point cloud or CAD model of the transmission tower, each member is rendered with different colors (e.g., green represents no damage, yellow represents slight damage, orange represents moderate damage, and red represents severe damage), bolt status is marked with special icons, and dynamic arrows indicate the possible future development direction of the damage. (b) 2D unfolded diagram form: The complex spatial tower structure is unfolded into a 2D schematic diagram with multiple planes, also labeled with colors, icons, and arrows. This visualization report allows maintenance personnel to have a clear understanding of the overall health status and high-risk areas of the entire tower.

[0049] Further, a decision-making basis report is generated, which includes a structural critical path heatmap, modal contribution weights, damage trend prediction curves, and remaining time estimates. The attention weight distribution between nodes, recorded and output during the attention topology fusion process in step S12, generates a structural critical path heatmap. This heatmap is overlaid on the structural topology map, using line thickness or color intensity to identify the member connections that have the greatest impact on the current overall damage state—that is, the critical force transmission paths. The report specifically points out members located on these critical paths that have been detected as damaged, indicating the severity of their risk. In step S13, based on the contribution of each modal feature output from the feature fusion layer in feature fusion and classification to the final decision, the contribution weights of each modality are determined. The report lists, in tabular or bar chart form, the contribution ratio of each visible light texture feature, area feature, infrared feature, point cloud feature, vibration feature, and topological feature to the judgment result for each member judged as damaged. In step S14, based on the output of the damage time-series prediction, the damage trend prediction curve and the remaining time estimate are determined. The report will plot the historical damage level change curve and the future prediction curve for each member with a deterioration trend, and clearly give the textual conclusion that "it is expected that the current moderate damage may develop into severe damage in about X months." By generating a visual report that integrates spatial damage distribution and multi-level decision-making basis, the output of cutting-edge artificial intelligence algorithms is transformed into intuitive and deeply reliable decision support information for engineering sites.

[0050] This embodiment, through the aforementioned scheme, firstly overcomes the limitations of a single data source by comprehensively extracting features from multi-source heterogeneous data, providing a multi-dimensional information foundation for comprehensive damage perception. Secondly, it introduces structural topology maps for feature fusion, enabling the model to possess structural mechanics-level reasoning capabilities, significantly improving the accuracy and robustness of damage localization and assessment in complex spatial structures. Then, through an end-to-end multi-task fusion classification framework, it simultaneously achieves refined identification of corrosion, bolt condition, and overall damage level, and leverages an attention mechanism to endow the model results with interpretability. Finally, by mining the temporal evolution patterns of historical data, it achieves a leap from static state assessment to dynamic trend prediction, greatly improving the automation level and accuracy of inspection and detection.

[0051] Based on the above implementation scheme, in one feasible implementation, the integrated heterogeneous data includes visible light images, infrared thermal imaging data, point cloud time-series data, and vibration monitoring data; the integrated heterogeneous features include visible range features, infrared temperature field features, point cloud time-series features, and vibration time-series features; and the step of extracting features from the integrated heterogeneous data of the transmission tower to obtain integrated heterogeneous features includes S21~S24: Step S21: Extract image features from the visible light image to obtain visible range features, which include rust texture features, rust area ratio features, and image spectrum features.

[0052] It should be noted that visible light images are two-dimensional image data acquired by a high-resolution RGB camera mounted on a drone, directly reflecting visual information such as color, texture, and shape of the transmission tower components. Image feature extraction specifically refers to the process of quantifying visual indicators directly related to corrosion damage from these images. Corrosion texture features are used to describe the pattern regularity, roughness, and contrast of the corroded area at the microscale, such as contrast and correlation statistics calculated through the gray-level co-occurrence matrix. Corrosion area proportion features are used to quantify the degree of corrosion coverage on the component surface at the macroscale, i.e., the ratio of the pixel area of ​​the corroded area to the total pixel area of ​​the component surface. Image spectral features are features obtained by transforming the image from the spatial domain to the frequency domain through Fourier transform, reflecting the periodicity, edge sharpness, and frequency domain distribution of the overall texture in the image.

[0053] Understandably, the purpose of this step is to overcome the shortcomings of traditional methods that rely solely on single visual features (such as color or simple texture) by constructing a more robust visual descriptor for corrosion through multi-angle and multi-level feature extraction. Using area proportion alone may lead to misjudgments due to shadows or stains; using texture alone may fail to distinguish the degree of corrosion; frequency domain features are sensitive to periodically occurring defects (such as uniform rust spots or regular cracks) and edge information in the image. Combining these three features allows for a more comprehensive and accurate characterization of the appearance, severity, and potential microstructural changes of corrosion, significantly improving the accuracy and reliability of visible light images in corrosion detection, and providing a solid and rich visual foundation for subsequent fusion with heterogeneous features such as infrared and geometric features.

[0054] Specifically, the rust texture feature extraction involves cropping or segmenting the target rod region from the image. First, the gray-level co-occurrence matrix of this region is calculated, and statistics such as contrast, correlation, energy, and homogeneity are extracted in four directions: 0°, 45°, 90°, and 135°, constituting the handcrafted texture features. Simultaneously, this region's image is input into a pre-trained convolutional neural network, extracting high-dimensional feature vectors from its deeper layers (such as the last convolutional layer) as deep learning texture features. The handcrafted features and deep learning features are then concatenated to form the final rust texture feature vector.

[0055] Rust area proportion feature extraction: A trained semantic segmentation network is used to perform pixel-level classification of the image, outputting a probability map of each pixel belonging to rust or background. A binary mask of the rust region is obtained through thresholding. The ratio of the total number of white pixels (rust) in the mask to the total number of pixels in the rod region is calculated to obtain the area proportion. Furthermore, shape descriptors such as the aspect ratio of the minimum bounding rectangle, contour roundness, and area-to-perimeter ratio are calculated based on the rust mask to form a shape feature vector. The area proportion value and the shape feature vector together constitute the area proportion feature.

[0056] Image spectral feature extraction: A two-dimensional fast Fourier transform is performed on the image of the rod region to obtain its amplitude spectrum. The frequency domain coordinates are converted to polar coordinates, and the spectrum is divided into multiple concentric annular regions (e.g., low frequency, mid frequency, high frequency) according to radial distance (frequency magnitude). The energy in each annular region is calculated as the ratio of the total energy in the entire spectrum, serving as the energy proportion feature of each frequency band. Simultaneously, the frequency component with the largest amplitude within each ring is identified as the representative frequency of that frequency band. The energy proportion and the representative frequency together constitute the image spectral feature vector.

[0057] Step S22: Extract temperature field features from the infrared thermal imaging data to obtain infrared temperature field features.

[0058] It's important to note that infrared thermal imaging data is acquired by an infrared thermal imager and records the surface temperature distribution of an object in matrix form, with each pixel value representing the temperature at that point. Temperature field feature extraction involves calculating statistical or structural features that characterize abnormal heat distribution patterns from this temperature matrix, rather than simply reading the highest or average temperature. Specifically, temperature gradient distribution features describe the intensity and directionality of heat transfer by calculating the rate of change (gradient magnitude) and direction of change (gradient direction) of the temperature field in space; anomalous hotspot area proportion features identify anomalous areas with significantly higher temperatures than the surrounding background by setting a statistical threshold (such as the average temperature plus a certain standard deviation) and calculating their relative area; and regional temperature standard deviation features reflect the uniformity or local abrupt changes in the temperature field by dividing the target area into sub-blocks and calculating the dispersion of temperature within each sub-block. These features collectively constitute the infrared temperature field features describing thermal anomalies.

[0059] Understandably, many types of damage to transmission towers, such as increased contact resistance due to loose electrical connections and uneven heat conduction caused by internal corrosion or cracks in the members, can lead to localized overheating. However, environmental factors such as temperature, sunlight, and wind speed can cause significant fluctuations in absolute temperature values. By extracting structural features such as temperature gradients, anomaly statistics, and local uniformity, it is possible to capture damage-related thermal anomaly patterns more stably and fundamentally, effectively suppressing environmental interference. For example, a well-maintained connection point exhibits uniform temperature and a gentle gradient; while a loose connection point, even with a low average temperature, may still have sharp temperature gradients and small localized hot spots at the contact surface. This specialized feature extraction transforms the raw infrared image into a feature vector that is sensitive to damage and robust to interference, providing an irreplaceable information dimension for diagnosing connection-related and internal defects.

[0060] Specifically, the following calculations are performed on the preprocessed infrared temperature matrix (corresponding to a specific rod region): First, the temperature gradient distribution characteristics are calculated: using the Sobel operator or directly calculating the difference, the gradient components Gx and Gy of the temperature field in the x and y directions are obtained. The gradient magnitude map is calculated by sqrt(Gx^2 + Gy^2), and the gradient direction map is calculated by arctan(Gy / Gx). Then, the mean, variance, maximum value, and other statistics of the gradient magnitude of the entire rod region are calculated, as well as the entropy of the gradient direction map (reflecting the degree of disorder in the directional distribution). Then, the abnormal hot spot area proportion characteristics are calculated: first, the average value μ and standard deviation σ of the temperature of the rod region are calculated. An abnormal temperature threshold T_th = μ + n*σ (n is a constant used to control the strictness of judging abnormal temperatures, usually taken as 2 or 3). All pixels in the temperature matrix greater than T_th are marked as abnormal hot spots. The ratio of the total area Area_hot of these hot spot pixels to the total area Area_total of the rod region is calculated, i.e., the abnormal hot spot area proportion. Finally, the standard deviation of the temperature in the region is calculated as follows: The rod region is divided into multiple non-overlapping sub-blocks (e.g., a 4x4 grid) on the image. The standard deviation of the temperature of all pixels within each sub-block is calculated. Then, the maximum, minimum, mean, and variance of the standard deviations of all sub-blocks are taken as features. These calculated statistics together form a multi-dimensional infrared temperature field feature vector.

[0061] Step S23: Extract point cloud features from the point cloud time series data to obtain point cloud time series features.

[0062] It should be noted that point cloud time-series data refers to a series of three-dimensional point cloud frames acquired by continuous scanning with a lidar system. Each frame contains the three-dimensional coordinates (x, y, z) of millions of spatial points, and sometimes also includes reflection intensity information. These frames are arranged in chronological order and can record the geometric shape changes and motion of the measured object over time. The goal of point cloud feature extraction is to extract features from these time-series three-dimensional data that characterize the dynamic response (such as vibration, deflection) or slow deformation of the transmission tower structure. This typically involves tracking the three-dimensional positional changes of specific points or regions from continuous point cloud frames to form a displacement time series, or calculating the temporal changes of the overall geometric properties of the point cloud (such as normal vector distribution, curvature, and moment of inertia). The extracted features can be the original displacement sequence or statistical features (such as mean, variance, and spectral components) further calculated from it.

[0063] Understandably, this step aims to leverage the high-precision, non-contact 3D measurement capabilities of lidar to acquire the structural dynamic response characteristics of transmission towers under environmental excitations (such as wind and light wind vibrations). The vibration characteristics (frequency, damping, mode shape) of a structure are directly related to its physical parameters such as mass, stiffness, and damping. When bolts loosen or members are damaged, local stiffness changes, thus affecting its vibration modes. By extracting point cloud temporal features (such as the vibration displacement spectrum of specific points) from point cloud temporal data, the vibration fingerprint of the structure can be obtained non-contactly. This complements vibration sensor data: point clouds provide high-resolution displacement information across the entire spatial field, but may be affected by weather; vibration sensors provide high-precision, stable time-series signals, but have limited coverage. Combining the two allows for a more comprehensive characterization of the structural dynamic state, providing crucial geometric kinematic evidence for damage detection (especially overall damage and connection status assessment).

[0064] Specifically, firstly, the continuous multi-frame point cloud data is preprocessed, including denoising, filtering, and inter-frame registration (aligning point clouds at different times to the same coordinate system). Then, specific members of interest are identified in the point cloud using a segmentation algorithm. Next, features can be extracted. In one embodiment of this application, a point-tracking-based approach is adopted: stable feature points (such as corner points or planar intersection points) are selected on the target member, and the three-dimensional coordinates of these points are tracked in continuous frames using an iterative nearest-point algorithm or feature matching algorithm, thereby obtaining a displacement time series of these points in the three-axis directions. These displacement sequences are subjected to a Fast Fourier Transform to extract their dominant frequency components, amplitudes, etc., as features. In another embodiment of this application, a global attribute-based approach is adopted: certain global geometric attributes of the target member point cloud in each frame are calculated, such as the centroid coordinates, offset relative to the tower base, average curvature of the point cloud surface, or normal vector distribution entropy, forming a sequence of these attributes changing over time, from which trend or periodic features are extracted. Finally, these spectrum or statistical quantities extracted from the time series constitute the point cloud temporal features.

[0065] Step S24: Extract vibration features from the vibration monitoring data to obtain vibration time sequence features.

[0066] It should be noted that vibration monitoring data is typically collected by accelerometers pre-installed at key locations on transmission towers (such as the tower head and crossarms). This data is the most direct physical quantity reflecting the structural dynamics, usually a waveform signal showing acceleration, velocity, or displacement changing over time. Vibration feature extraction involves calculating parameters characterizing the inherent properties of the structure from these waveform signals. Common features include: Frequency domain features: obtaining the spectrum by performing a Fourier transform on the signal, identifying the structure's first few natural frequencies, the amplitude (or mode shape participation factor) at the corresponding frequencies, and the damping ratio estimated by the half-power bandwidth method. Time domain statistical features: the signal's effective value, peak value, peak-to-peak value, skewness, kurtosis, waveform factor, etc. Time-frequency domain features: time-frequency diagrams obtained through wavelet transforms, or energy features extracted from wavelet coefficients. These parametric features extracted from vibration signals are collectively referred to as vibration time-series features.

[0067] Understandably, the purpose of this step is to obtain structural health monitoring indicators—modal parameters. A structure's natural frequencies, damping ratios, and mode shapes are a concentrated reflection of its physical properties (mass, stiffness, damping). When a structure suffers damage (such as a decrease in stiffness), its natural frequencies typically decrease, and its damping ratio may change. By extracting vibration time-series characteristics (especially modal parameters) and comparing them with baseline characteristics under healthy structural conditions, changes in the overall or local stiffness of the structure can be sensitively detected, thereby determining the presence, location, and even extent of damage. Vibration monitoring is a contact measurement method, providing stable and reliable data, and is the gold standard for evaluating the dynamic characteristics of large structures. Combining it with non-contact point cloud time-series characteristics allows for multi-dimensional verification and supplementation, including contact and non-contact methods, point and full-field methods, and direct measurement and indirect inversion, greatly enhancing the reliability and accuracy of damage detection results.

[0068] Specifically, the raw acceleration signals acquired are processed as follows: First, data preprocessing is performed, including trend removal, bandpass filtering (to retain the main vibration frequency bands of the structure), and downsampling. Then, feature extraction methods are selected based on the analysis objectives: for modal parameter identification, frequency domain decomposition and stochastic subspace identification methods can be used. For example, power spectral density estimation is performed on multi-point acceleration signals, and the natural frequencies of each order are obtained through peak picking; or the SSI (Stochastic Subspace Identification) algorithm is used to directly identify the state-space model of the system, thereby obtaining the frequencies, damping, and mode shapes. For time-domain statistical features, the preprocessed acceleration time series is directly calculated, such as root mean square value, peak value, peak-to-peak value, impulse factor, and margin factor. For more refined analysis, continuous wavelet transform can be performed to obtain the time-frequency distribution of the signal, and the characteristics of energy changes in specific frequency bands over time can be extracted. Finally, the selected natural frequencies, damping ratios, amplitudes, or time-domain statistics constitute the vibration time-series feature vector for this monitoring.

[0069] This embodiment, through the aforementioned scheme, achieves a fine characterization of surface corrosion from microscopic to macroscopic levels and from the spatial to frequency domains by extracting texture, area, and spectral features from visible light images; effectively diagnoses thermal anomalies caused by electrical or mechanical connection problems by extracting specialized features such as temperature field gradients and anomalous regions from infrared thermal imaging; realizes non-contact structural vibration monitoring by extracting structural dynamic response features from point cloud time-series data; and obtains the most direct structural dynamic health indicators by extracting features such as modal parameters from vibration monitoring data. These four feature extraction paths are optimized for different types of damage (surface corrosion, connection failure, structural deformation, and stiffness changes). Complementary evidence of damage is mined from different perspectives.

[0070] Based on the above implementation scheme, in one feasible implementation, the step of mapping and fusing the integrated heterogeneous features based on the structural topology map of the transmission tower to obtain the fused topology features includes S31~S33: Step S31: Construct a structural topology diagram based on the actual structure of the transmission tower, wherein the graph nodes of the structural topology diagram are the poles of the transmission tower.

[0071] It's important to note that a structural topology diagram is a graph data structure used to abstractly represent the physical connections of transmission towers. The basic elements of the diagram are nodes and edges. Nodes represent the members of the transmission tower, i.e., each independent main member, diagonal member, or auxiliary member such as an angle steel or steel pipe. The actual structure refers to the actual geometric and connection relationships of the transmission tower, determined by design drawings and on-site installation. Constructing this diagram involves mapping the physical system of members connected by bolts and gusset plates into a computer-processable network model composed of points and connecting lines. The attributes of nodes (later filled with feature vectors) represent the state of the member itself, while the existence of edges indicates a direct force transmission path between two members.

[0072] Understandably, the mechanical performance of transmission towers is highly dependent on their topological configuration. The effects of damage are often not limited to a single member but are propagated to adjacent members through connection nodes. By constructing a structural topology graph, the model obtains an explicit encoding of the spatial connectivity relationships within the transmission tower. This allows the algorithm to understand the neighbor relationships between members, thereby simulating the propagation effects of damage or anomalies in the structure in subsequent steps. Treating members as graph nodes captures the basic unit of structural stress, ensuring the correctness of the topology graph in terms of mechanical semantics, and providing an accurate blueprint for the model to learn structural constraints and spatial dependencies.

[0073] Specifically, in one embodiment of this application, a structural topology diagram is constructed based on design drawings. The CAD design drawings or standard drawings of the transmission tower are input, and the list of members and connection relationship tables are automatically or semi-automatically parsed. Each unique member (defined by model, length, and location) is created as a graph node. If the drawings explicitly indicate that two members are connected to the same node plate or bolt group, an undirected edge is established between the corresponding two graph nodes. In another embodiment of this application, a structural topology diagram is constructed based on 3D point cloud data. High-precision lidar point clouds are automatically processed, and each independent member point cloud cluster is identified through a point cloud segmentation algorithm. Each cluster represents a member and serves as a graph node. Then, by analyzing the spatial proximity of different member point cloud clusters (e.g., calculating the distance between the nearest points between point cloud clusters; if it is less than a threshold, it is considered a connection) or by detecting the node plates at the connection points through plane or cylinder fitting, the connection relationship between members is inferred, and graph edges are established. The constructed graph G can be represented as an adjacency matrix A, where A[i][j]=1 indicates that the members corresponding to nodes i and j are connected.

[0074] Step S32: Perform double alignment on the integrated heterogeneous features to obtain the target aligned features.

[0075] It should be noted that dual alignment specifically refers to a two-stage spatiotemporal alignment strategy, first coarse and then fine, specifically designed to handle comprehensive heterogeneous feature sequences from different sensors with varying sampling rates and timestamps, particularly time-sensitive features such as extracted point cloud temporal features and vibration temporal features. Coarse alignment typically uses traditional sequence matching algorithms, such as DTW (Dynamic Time Warping), to quickly eliminate major temporal offsets and scaling differences between data. Fine-grained correction utilizes learnable neural networks (such as residual fine alignment networks) to further learn and compensate for complex, nonlinear residual spatiotemporal biases between the two types of features, based on the coarse alignment, to obtain the target aligned features.

[0076] Understandably, this step aims to address the lack of a precise spatiotemporal alignment mechanism between multi-source data in the background technology. It addresses the issues of hard alignment using existing traditional algorithms such as dynamic time warping (DTW), which are sensitive to noise and cannot be jointly optimized end-to-end with the detection network. While both point cloud data and vibration data reflect structural dynamics, their acquisition principles, sampling frequencies, start times, and temporal jitter differ. Direct fusion can lead to misattribution, severely reducing the accuracy of damage localization and identification. Single DTW alignment is sensitive to noise and local deformation and is an independent preprocessing step, disconnected from subsequent detection targets. The dual alignment mechanism first uses DTW to achieve fast and robust global coarse alignment, then introduces an end-to-end trainable neural network for local fine-tuning. This network can learn more complex mapping relationships between the two modalities and continuously adjust as the detection task is optimized, thereby synergistically improving alignment accuracy and final detection performance, ensuring the effectiveness of cross-modal information fusion.

[0077] Specifically, coarse alignment is first performed by extracting the point cloud temporal feature sequence P and the vibration temporal feature sequence V from the integrated heterogeneous features. Since the two sequences may have different lengths, a dynamic time warping algorithm is used to calculate the optimal bending path between them. DTW constructs a cumulative cost matrix and backtracks to find the alignment method that minimizes the overall distance between the two sequences, thereby stretching or compressing sequence V to match the time scale of sequence P, resulting in coarsely aligned feature sequences P' and V'. Then, fine-grained correction is performed by inputting the coarsely aligned sequences P' and V' into a residual fine alignment network. This network typically employs an attention-based architecture; for example, using P' as the query vector and V' as the key and value vectors, a cross-attention layer is used to calculate the fine correlation between them. The network outputs a residual adjustment Δ (which can be a time offset or a feature correction), which is applied to P' and V' to obtain the final target aligned feature sequences P and V that are precisely matched in both time and feature dimensions. The parameters of this network are trained together with the graph fusion network and the detection network through backpropagation.

[0078] Step S33: The target alignment features are fused based on the membership relationships of the members in the structural topology diagram to obtain fused topology features.

[0079] It's important to note that attribution refers to which specific node each feature vector belongs to in the constructed structural topology graph. The task of this step is to use the target alignment features belonging to each node as initial features, assign them to the corresponding nodes in the topology graph, and then use a graph neural network algorithm to allow information to be transmitted, interacted with, and aggregated along the edges of the topology graph between nodes. The final output feature of each node not only includes its own aligned sensor information but also incorporates the feature information of its neighboring nodes (i.e., directly connected nodes), resulting in fused topology features.

[0080] Understandably, damage assessment of transmission towers cannot be approached in isolation, considering each individual member. An anomaly in one member (such as a decrease in stiffness) alters its stress, subsequently affecting connected members. This impact may be reflected in sensor data from neighboring members (e.g., stress redistribution leading to micro-strain or temperature changes in adjacent members). By using graph neural networks for feature fusion, the model can automatically learn this message-passing mechanism based on physical connections. Each node, when updating its own features, references the state of its neighbors. This allows the model to identify collaborative anomaly patterns across multiple members; for example, the overall risk of minor anomalies in multiple connected members simultaneously may be higher than that of a significant anomaly in a single member. This structural awareness significantly improves the model's accuracy in damage localization and its ability to assess the overall structural safety status.

[0081] Specifically, first, feature mapping is performed. This is done for each node in the structural topology graph G. (Representing rod i), find all feature vectors corresponding to rod i from the target alignment feature set (such as aligned point cloud features, vibration features, and visible light and infrared features), and concatenate these vectors or project them through a linear layer to form nodes. initial features Then, perform multi-round graph attention propagation. In each round (layer)... In the middle, for each node Calculate its relationship with all neighboring nodes. Attention coefficient : ; Among them, attention coefficient This indicates that in this round of transmission, neighbors Features of nodes The importance of the coefficients. The calculation of the coefficients involves the nodes. and The features are subjected to a learnable linear transformation and similarity is calculated. It refers to the first In the layer, nodes Its neighboring nodes The original attention score between nodes is a scalar representing the node's attention score. Features of nodes The importance of.

[0082] After that, the node The new features are obtained by summing all neighbor features weighted by attention coefficients and then passing the sum through a non-linear activation function: ; in, Pointer node In the New features of the layer; For the first A learnable weight matrix in a layered graph attention network; The neighboring node is Features of the layer; It is a non-linear activation function.

[0083] After propagation through L layers, the features output by each node It is a fusion topology feature that deeply integrates the structural information of itself and its 1-hop, 2-hop and even L-hop neighbor nodes.

[0084] This embodiment, through the aforementioned scheme, firstly ensures the accuracy of the graph model in mechanical semantics by explicitly defining the members as basic nodes of the topological graph, providing a correct structural skeleton for subsequent fusion. Next, it innovatively proposes a dual alignment mechanism: first, it uses traditional algorithms to quickly correct the main spatiotemporal deviations of cross-modal data; then, it uses a learnable neural network for refined, task-oriented fine-tuning. This design cleverly combines the robustness of classical methods with the flexibility of deep learning, effectively solving the most critical spatiotemporal synchronization problem in multi-source heterogeneous data fusion, providing high-quality, highly consistent input for subsequent feature fusion. Finally, by mapping the precisely aligned features to topological graph nodes according to the member attribution relationships, and utilizing a graph attention network for feature propagation and aggregation, a deep and organic fusion of sensor data and structural topological knowledge is achieved.

[0085] Based on the above implementation scheme, in one feasible implementation, the step of performing dual alignment on the integrated heterogeneous features to obtain the target aligned features includes S41~S42: Step S41: The point cloud temporal features and the vibration temporal features are coarsely aligned using a dynamic time warping algorithm to obtain the initial aligned features after coarse alignment.

[0086] It's important to note that Dynamic Time Warping (DTW) is a classic algorithm for measuring the similarity between two time series of potentially different lengths. It finds the optimal matching path by allowing the sequences to bend non-linearly along the time axis, thus calculating the minimum cumulative distance. Point cloud temporal features and vibration temporal features are time series data features from different sensors. Coarse alignment refers to using the DTW algorithm to quickly find a globally optimal matching relationship between these two feature sequences in the time dimension, and then resampling or interpolating them to make them roughly correspond at specific time points, thereby eliminating the main misalignment phenomena caused by different sampling start times, sampling frequency differences, or local time scaling. These two aligned feature sequences are the initial aligned features.

[0087] Understandably, this step, as the first stage of double alignment, aims to provide a robust, globally approximate correct starting point for alignment. Although point cloud and vibration data describe the same physical phenomenon (structural dynamics), their timestamps are difficult to synchronize perfectly due to the independent sensor principles and acquisition systems, and their sequence lengths often differ. Direct comparison or fusion is ineffective. The DTW algorithm can flexibly match two sequences, tolerating local time scaling and offsets, making it well-suited for handling such problems. Coarse alignment using DTW can resolve most time asynchrony issues caused by system differences, laying a good foundation for subsequent finer corrections. Its advantages lie in its mature algorithm, relatively high computational efficiency, and independence from model training, making it a reliable preprocessing step.

[0088] Specifically, assume the point cloud temporal feature sequence is P=[p1, p2, ..., p_M], and the vibration temporal feature sequence is V=[v1, v2, ..., v_N]. First, construct an M×N cost matrix D, where D(i, j)=d(p_i, v_j), and d is a distance metric function (such as Euclidean distance). Then, calculate the cumulative cost matrix C, where C(i, j)=D(i, j)+min{C(i-1, j), C(i, j-1), C(i-1, j-1)}, which represents the minimum cumulative path cost from (1, 1) to (i, j). Finally, backtrack from C(M, N) to find the path W=(w1, w2, ..., w_K) that minimizes the cumulative cost, where each w_k=(i_k, j_k) represents the alignment of the i_k-th point in P with the j_k-th point in V. Based on this path, shorter or longer sequences can be interpolated to generate a pair of new sequences P' and V' with the same length and corresponding time points. These are the initial alignment features.

[0089] Step S42: Input the initial alignment features into the residual fine alignment network for fine-grained correction to obtain the target alignment features.

[0090] It's important to note that the residual fine alignment network is a trainable neural network module whose input is the initial alignment feature sequences P' and V' after DTW coarse alignment. Fine-grained correction refers to the network learning the complex alignment errors remaining between the two coarse alignment sequences. These errors may originate from nonlinear phase differences, waveform distortion caused by different sensor frequency responses, or complex local deformations that DTW cannot handle. The residual concept is reflected in the fact that the network does not directly learn the complete alignment transformation, but rather learns a small correction amount (residual) relative to the coarse alignment result. By superimposing this correction amount onto the coarse alignment result, the target alignment features P and V, which are more accurately aligned in time and / or feature space, can be obtained.

[0091] Understandably, this step, as the second stage of dual alignment, aims to address the limitations of DTW coarse alignment and achieve end-to-end joint optimization with the detection task. DTW is a global optimization algorithm, sensitive to local noise, and assumes an overall monotonic mapping relationship between sequences, making it difficult to handle complex local nonlinear distortions. Furthermore, DTW is an independent preprocessing step, and its alignment quality is not optimized for the needs of the detection task. The introduced trainable residual fine alignment network can learn a more refined alignment mapping between the two modalities in a data-driven manner. This network is trained end-to-end along with the subsequent graph fusion network and detection network. During training, gradients propagate back from the loss function of the detection task, forcing the alignment network to learn the alignment method most conducive to improving the final damage detection accuracy. This task-driven alignment is superior to alignment based solely on signal similarity (such as DTW).

[0092] Specifically, residual fine alignment networks typically employ an encoder-attention mechanism. First, P' and V' are encoded using two encoders (such as a 1D convolutional neural network or a fully connected layer) to obtain high-dimensional representations H_p and H_v. Then, the core is a cross-attention module: using H_p as the query vector and H_v as the key and value vectors, an attention weight matrix A is calculated. The attention weights A reflect which parts of H_v are most important for correcting H_p at each time step. Using these attention weights, a context vector C = A * H_v can be calculated. Next, the network passes through one or more fully connected layers to regress the residual correction Δ from [H_p, C] (concatenation operation). This Δ can be a feature offset added to P' (feature space correction) or an offset field used to adjust the temporal index of P' (temporal correction). Finally, the target alignment feature P* = P' + Δ (feature correction) or obtained by temporally warping P' using Δ. V* can directly use V' or undergo similar processing. The parameters of the entire network are optimized using the total loss of the downstream detection tasks.

[0093] This embodiment, through the aforementioned scheme, explicitly designates the first-level alignment as the classic Dynamic Time Warping (DTW) algorithm. This method, with its robustness and effectiveness in sequence matching, provides a stable and reliable global initial solution for the entire alignment process. Furthermore, a second-level residual fine alignment network is introduced. This is a differentiable and trainable neural network module specifically designed to overcome complex local nonlinear alignment problems that the DTW algorithm struggles with. End-to-end joint optimization of spatiotemporal alignment and the final detection task is achieved, making the alignment process no longer an isolated preprocessing step, but directly serving and optimizing the final damage detection target. This ensures that cross-modal data reaches the best possible synchronization state before fusion, significantly improving the effectiveness of fusion of temporally heterogeneous features such as point clouds and vibrations. It lays a solid foundation for data consistency in subsequent topology-based feature fusion and multi-task detection, addressing the problem of information silos in multi-source data.

[0094] Based on the above implementation scheme, in one feasible implementation, the damage detection results include rod corrosion detection results and bolt condition detection results. The step of fusing and classifying the integrated heterogeneous features and the fused topological features to obtain the damage detection results and damage level classification results of each component of the transmission tower includes S51~S52: Step S51: The comprehensive heterogeneous features and the fused topological features are subjected to cross-modal attention weighting through a multi-head self-attention mechanism to obtain contribution weights, and feature fusion is performed based on the contribution weights to obtain comprehensive damage features.

[0095] It's important to note that the multi-head self-attention mechanism is a core component of the Transformer architecture and is used here for feature fusion. Its multi-head design allows the model to focus on different parts of the integrated heterogeneous features and fused topological features in parallel across different representation subspaces. Cross-modal attention weighting treats all these feature vectors (modalities) to be fused as a sequence, and the self-attention mechanism calculates the correlation weight between each feature vector in the sequence and all other feature vectors (including itself). The contribution weights are these calculated attention weights, quantifying the importance of each input feature vector (e.g., visible light texture features, infrared temperature field features, topological features, etc.) in forming the final decision. Feature fusion involves weighted summation of all input feature vectors based on these contribution weights, generating a comprehensive damage feature vector that integrates all modal information and highlights key information.

[0096] Understandably, simple feature concatenation or averaging cannot distinguish the differences in importance of different modalities and features to a specific damage assessment task. For example, when assessing corrosion, visible light features may be more important; when assessing bolt loosening, infrared temperature and bolt geometry may be more critical. Multi-head self-attention mechanisms can dynamically learn this importance relationship based on context (i.e., the overall situation of all current features). By calculating cross-modal attention and outputting contribution weights, the model not only achieves more intelligent feature fusion and improves the representational ability of comprehensive damage features, but also obtains the contribution of each modality to the final decision. This greatly enhances the transparency of the model's decision-making, allowing operations personnel to understand the main basis for decisions and improving the system's credibility in practical applications.

[0097] Specifically, firstly, all feature vectors to be fused (assuming there are K vectors, including multiple sub-features of visible light, infrared features, point cloud vibration alignment features, fused topological features, etc.) are concatenated into a sequence. The sequence is then fed into a multi-head self-attention layer, and a learnable positional encoding is added to each feature to distinguish its modality. For each attention head, the query vector Q, key vector K, and value vector V matrix are computed. , , ,in To query the weight matrix, The key weight matrix is... This is the weight matrix. Next, the attention score is calculated using the following formula: ; in, Let be the dimension of the key vector. A score is given for attention.

[0098] Furthermore, the attention scores from multiple head outputs are concatenated and linearly projected. The fused comprehensive impairment feature can be taken as the specialized output at position [CLS] (where [CLS] is the classification label), or obtained by pooling the outputs at all positions. Simultaneously, the weights corresponding to each input feature can be extracted from the attention weight matrix A (e.g., by averaging the columns of matrix A) as the contribution weight output for that feature.

[0099] Step S52: Input the comprehensive damage features into the rod corrosion detection head, bolt condition detection head, and damage level classification head respectively to obtain the rod corrosion detection result, the bolt condition detection result, and the damage level classification result.

[0100] It should be noted that the member corrosion detection head, bolt condition detection head, and damage level classification head are three parallel, task-specific neural network output layers (usually composed of several fully connected layers). The shared feature of the comprehensive damage characteristics is input into each of the three independent detection heads, and each head is responsible for decoding it into a task-specific output format. Member corrosion detection results typically include the presence and location of corrosion (e.g., bounding box), and severity level (e.g., none, light, medium, severe). Bolt condition detection results include bolt condition classification (normal, loose, missing) and possible state quantification values ​​(e.g., loosening angle). The damage level classification result is a comprehensive evaluation of the overall health status of the component (or area) after integrating all damage information.

[0101] Understandably, this step constructs a typical multi-task learning framework, aiming to simultaneously address the problem in the background technology where single-task detection is often insufficient to simultaneously perform multiple detection tasks such as corrosion detection, bolt condition detection, and damage grading. These three tasks are highly correlated and share underlying features (e.g., bolt loosening and corrosion can both affect the overall damage level). By sharing most of the network (feature extraction, alignment, topology fusion, cross-modal fusion) and then branching out three lightweight dedicated detection heads, the model can achieve efficient multi-task parallel inference. This approach not only reduces the overall computational cost but also promotes knowledge transfer and regularization between tasks through shared representations, helping to improve the generalization performance of each task.

[0102] Specifically, the three detection heads use the obtained comprehensive damage feature vector z as a common input. The rod corrosion detection head is a small fully connected network, including fully connected layers and a ReLU activation function. For classification tasks, the output is the probability distribution of corrosion severity. For more complex detection (e.g., with localization), an architecture similar to Faster R-CNN (Faster Region-Based Convolutional Neural Network) can be used to generate region proposals and classify based on shared features. The bolt condition detection head, in addition to the comprehensive damage feature vector z, typically receives a bolt geometric feature vector g (e.g., nut deflection angle, contour integrity) specifically extracted from the original point cloud. After concatenating [z, g], it is input into a fully connected network, outputting two parts: the classification probability of the bolt condition (normal / loose / missing) and the regressed nut deflection angle value (a continuous scalar). The damage level classification head, similar in structure to the corrosion detection head, is a fully connected network that outputs the probability distribution of the overall damage level of the component (e.g., level 1-5).

[0103] During training, the model employs an adaptive uncertainty loss weighting mechanism. This method introduces a learnable uncertainty parameter for each detection task, dynamically adjusting the weight of that task's loss in the total loss. The joint loss function L is calculated as follows: ; in, For joint losses. The loss for the rod corrosion detection task is typically represented by the cross-entropy loss function, which measures the difference between the corrosion level predicted by the model and the true label. The loss function for bolt condition detection is typically the cross-entropy loss function (used for condition classification) combined with mean squared error loss (used for deflection angle regression). The loss function for the damage level classification task is represented by the cross-entropy loss function, which measures the accuracy of the overall damage level prediction. express The corresponding learnable uncertainty parameters. express The corresponding learnable uncertainty parameters. express The corresponding learnable uncertainty parameters. , , Initialized to 1.0, it is automatically updated during training via backpropagation. When a task becomes difficult to learn (resulting in significant loss), its σ value increases, and the corresponding weight 1 / (2σ²) decreases, causing the optimization focus to automatically shift to a more easily optimized task, thereby achieving adaptive balance among multiple tasks.

[0104] This embodiment, through the aforementioned scheme, firstly, introduces a multi-head self-attention mechanism for cross-modal feature fusion. This mechanism dynamically evaluates and weights the importance of features from different sources, thereby generating a highly discriminative comprehensive damage feature. This process not only improves the efficiency of feature fusion, but its accompanying contribution weight output provides an intuitive and interpretable basis for model decision-making, directly addressing the interpretability requirements in the background technology. Subsequently, by setting up three parallel dedicated detection heads that share this comprehensive feature, efficient and synchronous detection of rod corrosion, bolt condition, and overall damage level is achieved. This multi-task learning framework avoids the redundancy of training models separately for each task, and through parameter sharing and knowledge transfer, it potentially improves the performance of each task, especially the few-sample task.

[0105] Based on the above implementation scheme, in one feasible implementation, the step of performing damage time-series prediction based on the damage detection results and the damage level classification results to obtain the damage prediction results of each component of the transmission tower includes S61~S63: Step S61: Based on the inspection time sequence, the damage detection results and damage level classification results of each component of the same transmission tower are sequenced to obtain the component-level damage time series.

[0106] It should be noted that the inspection time sequence refers to the timestamps of each inspection of the same transmission tower, arranged in ascending order. For each component of the same transmission tower, the sequence construction object is a specific, traceable rod or bolt. Organizing the damage detection results and damage level classification results according to the inspection time points constitutes the sequence construction. For each component, its state at different time points t1, t2, ..., tN (which can be the final damage level or the encoding vector of each sub-item detection result) constitutes a time series, i.e., a component-level damage time series.

[0107] Understandably, the purpose of this step is to organize discrete, periodic inspection snapshot data into a continuous historical trajectory that can be analyzed by time series models. Damage and degradation of transmission towers is a continuous and gradual process. A single inspection can only reflect the current state and cannot reveal its changing patterns and development trends. By constructing a damage history sequence for each component in chronological order, we transform static inspection data into dynamic evolutionary data. This is the foundation for predictive maintenance. Only with such time-series data can the model learn the evolution pattern, rate, and potential acceleration points of damage from non-existence to severity, thus providing the possibility of predicting future states. Component-level sequence construction ensures targeted prediction, allowing for focused tracking of high-risk components.

[0108] Specifically, the database maintains the inspection history records for each component of each transmission tower. When trend prediction is needed for a specific component, a query is executed: based on the component ID, all inspection task records containing that component are retrieved and sorted in ascending order by inspection time. For each record, a standardized state representation of the component at that time is extracted. This state representation can be a scalar (e.g., comprehensive damage level: 1, 2, 3, 4) or a vector (e.g., [corrosion level, bolt status code, infrared anomaly score, ...]). These states are arranged chronologically to form a sequence of length L (number of historical inspections) S = [s{t1}, s{t2}, ..., s_{tL}], which is the damage time series of that component. Missing time points can be handled through interpolation or by marking missing values.

[0109] Step S62: Input the component-level damage time series into the time series prediction network and output the damage level change prediction curve of each component within the future time window.

[0110] It's important to note that temporal prediction networks are neural network models specifically designed to process sequential data and predict future values. Examples include RNNs (Recurrent Neural Networks), LSTM (Long Short-Term Memory) networks, GRUs (Gated Recurrent Units), and Transformers. The future time window refers to the range of future time we wish to predict, such as the four inspection cycles shown below. The damage level change prediction curve is the direct output of the temporal prediction network. It is a sequence of length M (the size of the future time window), where each value represents the predicted damage state (either a damage level or a state vector) for that component at the corresponding future time point. This curve visually illustrates the possible future development path of the damage.

[0111] Understandably, this step leverages the powerful sequence modeling capabilities of deep learning models to automatically learn the intrinsic patterns of damage evolution from historical damage time series and extrapolate them to the future. Networks such as LSTM, through their internal gating states, can memorize long-term dependencies and capture complex nonlinear patterns. By inputting historical sequences into a trained network, the network can generate the most probable state sequence for a future period based on the learned patterns. This makes predictions no longer dependent on simple assumptions, but rather on data-driven induction and deduction of historical patterns, thus significantly improving prediction accuracy and achieving a leap from describing the present to predicting the future.

[0112] Specifically, a pre-trained time-series prediction model (e.g., a two-layer LSTM network) is loaded. The constructed component history damage time series S (possibly normalized) is used as the model input. The model processes the data according to its forward propagation logic: for the LSTM, each state s_t in the sequence is read sequentially, and its internal hidden state h_t and cell state c_t are updated, thereby encoding the information of the entire history sequence. In the prediction phase, the hidden state h_L of the last time step can be input into a fully connected layer, directly outputting the predicted sequence for the next M time points. {L+1}, {L+2},..., Alternatively, an autoregressive approach can be used, using the previous prediction output as the input for the next, iteratively generating M-step predictions. This output sequence constitutes the damage level change prediction curve.

[0113] Step S63: Based on the damage level change prediction curve, calculate the estimated remaining time for each component to upgrade from the current damage level to the next damage level, and obtain the damage prediction result.

[0114] It's important to note that the damage severity prediction curve provides a forecast of the damage's condition at various future points in time. Based on this curve, the timing of potential condition transitions (from the current severity level to the next, more severe level) can be analyzed. The estimated remaining time, calculated from the current moment (the time of the last inspection) to the first predicted escalation of the damage severity, is based on the prediction curve. This time can be measured in days, months, or inspection cycles. Combining the prediction curve and this crucial remaining time information as the damage prediction output provides the most direct time-based guidance for maintenance decisions.

[0115] Understandably, knowing how a component will change in the future is important, but knowing roughly when it will fail is even more urgent and instructive for developing maintenance plans, scheduling outage windows, and allocating resources. By calculating the estimated remaining time, the results of trend forecasting are condensed into a clear time node or time range. This allows maintenance personnel to prioritize high-risk components, enabling true predictive maintenance—intervention before failure occurs and before significant deterioration occurs—thereby maximizing the efficiency of maintenance funds and significantly improving the reliability of the power grid.

[0116] Specifically, let the damage level of the current component be C_current, and the prediction curve be [P_1, P_2, ..., P_M]. The algorithm iterates through this prediction sequence and finds the first index k that satisfies P_k > C_current (i.e., the first time the predicted level is higher than the current level). The estimated remaining time T_remaining can then be calculated as: T_remaining = (k - 0.5) * ΔT, where ΔT is the length of the inspection cycle (e.g., 90 days). Here, k - 0.5 is used to account for the uncertainty of the prediction, taking the median of the escalation time range. If no escalation occurs after iterating through the entire prediction window, the algorithm can output that no escalation is expected within the next M cycles or provide an estimate based on curve extrapolation. Finally, the damage prediction result is a structure containing the prediction curve and the estimated remaining time.

[0117] This embodiment, through the aforementioned scheme, firstly, constructs a component-level damage time series, connecting discrete inspection points into a continuous trajectory reflecting the individual degradation history, providing a data foundation for prediction. Then, it utilizes advanced time-series prediction networks (such as LSTM) to learn complex degradation patterns from historical trajectories and outputs a prediction curve for future damage level changes, achieving data-driven extrapolation of damage development paths. Finally, through intelligent analysis of the prediction curve, it calculates the crucial estimated remaining time, transforming the prediction results into action recommendations with a clear time dimension that can directly guide maintenance decisions. This series of steps systematically addresses the shortcoming of the prior art—the inability to predict damage development trends—and elevates transmission tower health management from passive, reactive post-event maintenance to proactive, planned pre-event early warning.

[0118] Based on the above implementation scheme, in one feasible implementation, the step of extracting image features from the visible light image to obtain visible range features includes S71~S75: Step S71: Extract the contrast features, correlation features, energy features, and homogeneity features of the rusted area in the visible light image using the gray-level co-occurrence matrix to obtain the first texture sub-feature.

[0119] It's important to note that the gray-level co-occurrence matrix (GLCM) is a classic method for describing texture by studying the spatial correlation of image gray levels. It defines a matrix P(i, j|d, θ) representing the probability of a pair of pixels with gray levels i and j occurring given a spatial distance d and a direction θ. Several statistics can be calculated from this matrix: contrast features measure image sharpness and the depth of texture grooves; correlation features measure the similarity of image textures along rows or columns; energy features (also called angular second moments) are the sum of squares of the matrix elements, reflecting the uniformity of image gray-level distribution and the coarseness of texture; and homogeneity features measure the uniformity of local variations in image texture. Calculating these GLCM features from an image of a rusted region yields a feature vector called the first texture sub-feature, which statistically quantifies the macroscopic texture pattern of the rusted surface.

[0120] Understandably, corrosion causes metal surfaces to lose their luster, become rough and uneven, and alter their color distribution. These changes are reflected in the gray-level co-occurrence matrix features: for example, severe corrosion may exhibit high contrast (large differences in brightness), low correlation (chaotic texture orientation), low energy (uneven distribution), and low homogeneity. These features are computationally fast, highly interpretable, and require little training data. They can provide robust, prior-knowledge-based texture cues to the model, supplementing subsequent deep learning features and enhancing the robustness and generalization ability of the feature set.

[0121] Specifically, the following operations are performed on the rod region (obtained through image segmentation) in the input visible light image: First, the image is converted to grayscale. Then, for each pixel, in its four principal directions (0°, 45°, 90°, 135°), pixel pairs at a fixed distance d (e.g., d=1) are counted, constructing grayscale co-occurrence matrices for the four directions. Next, for each direction's grayscale co-occurrence matrix, four features—contrast, correlation, energy, and homogeneity—are calculated.

[0122] Among them, contrast features The calculation formula is: ; The gray-level co-occurrence matrix P is an L*L matrix. For a given spatial distance d and direction θ, the elements of the gray-level co-occurrence matrix are... This represents the probability that a pixel with gray value i and a pixel with gray value j located d pixels away along the θ direction will appear simultaneously in the entire image.

[0123] Among them, correlation features The calculation formula is: ; in, Let $\mathbf{P}$ be the row mean of matrix $P$, and $\mathbf{P}$ be the gray value of the first pixel. The expectation. The column mean of matrix P is the gray value of the second pixel. Expectations; Let be the row standard deviation of matrix P; Let be the column standard deviation of matrix P.

[0124] Among them, energy characteristics The calculation formula is: ; Among them, homogeneity features The calculation formula is: ; Finally, take the mean and standard deviation of the four directional feature values, or directly concatenate all 16 values ​​(4 directions × 4 features) to form a 16-dimensional first texture sub-feature vector.

[0125] Step S72: Input the visible light image into a convolutional neural network to extract deep texture features and obtain the second texture sub-feature.

[0126] It's important to note that a convolutional neural network (CNN) is a deep neural network with structures such as convolutional layers and pooling layers, capable of automatically learning hierarchical feature representations from images. Deep texture features refer to feature vectors extracted from the deeper layers of a CNN (typically fully connected layers near the classification layer or the last convolutional layer). CNN features are obtained through training on large amounts of data and can capture more abstract, high-level semantic information, such as complex texture combinations and pattern structures, which may be difficult to describe using simple statistics. This feature vector extracted from the CNN is called the second texture sub-feature.

[0127] Understandably, the purpose of this step is to leverage the powerful representation learning capabilities of convolutional neural networks (CNNs) to automatically discover and extract latent, complex texture patterns relevant to corrosion discrimination. The morphology of corrosion is highly variable, greatly influenced by lighting, angle, and the type of corrosion products. Hand-designed features may not cover all situations. Through end-to-end training, CNNs can adaptively learn the most effective feature representations for corrosion classification; these features may correspond to patterns that are difficult for the human eye to define but that the model considers valid. Combining CNN features with hand-crafted gray-level co-occurrence matrix (GLCM) features offers complementary advantages: the GLCM provides a stable and interpretable statistical basis, while the CNN provides powerful and adaptive high-level semantic information, together forming a more comprehensive and robust description of corrosion texture.

[0128] Specifically, the execution entity (such as the deep learning feature extraction submodule) operates as follows: A pre-trained CNN (Convolutional Neural Network) model (such as ResNet, VGG, MobileNet) on a large image dataset is used as the feature extractor. The cropped image of the pole region (which may need to be scaled to the model's required input size, such as 224x224) is input into the CNN. The image is forward-propagated to a specified layer of the network (e.g., the output of a convolutional layer before a global average pooling layer, or the layer before the last fully connected layer). The output feature map of this layer is then subjected to global average pooling, or directly flattened, to obtain a high-dimensional feature vector (e.g., ResNet-50 might output a 2048-dimensional vector). This vector is the second texture sub-feature. To save computation, the pre-trained model is usually fine-tuned to better suit the characteristics of the transmission tower rust texture.

[0129] Step S73: The first texture sub-feature and the second texture sub-feature are spliced ​​together to obtain the rust texture feature.

[0130] It's important to note that concatenation is a feature fusion method, which involves linking the first and second texture sub-feature vectors end-to-end in terms of dimension to create a longer feature vector. For example, if the first sub-feature has a dimension of 16 and the second sub-feature has a dimension of 2048, then the concatenated feature vector will have a dimension of 2064. This concatenated vector is the final rust texture feature.

[0131] Understandably, the goal of this step is to integrate the advantages of handcrafted features and deep learning features to create a more informative texture descriptor. Simple concatenation preserves all the original information from both types of features, allowing subsequent models to decide how to utilize this information. This approach avoids the risk of early information loss. By combining interpretable statistical features with powerful semantic features, the rust texture features obtained through the gray-level co-occurrence matrix incorporate both human expert knowledge and data-driven insights gained through convolutional neural networks, giving them greater discriminative power and robustness when facing diverse and complex real-world rust scenes.

[0132] Specifically, we receive the first texture sub-feature vector and the second texture sub-feature vector. We then perform a concatenation operation to obtain a new vector, which is the rust texture feature.

[0133] Step S74: The corroded area in the visible light image is segmented at the pixel level using a semantic segmentation network. The ratio of the corroded area area to the total surface area of ​​the rod is calculated, and the shape features of the corroded area are extracted to obtain the corroded area ratio feature.

[0134] It should be noted that this step is used to extract the rust area proportion feature. Semantic segmentation networks are pixel-level classification CNNs, such as U-Net and DeepLabV3+. Their output is a mask image of the same size as the input image, where each pixel is classified into a specific category (e.g., rust, background). Pixel-level segmentation obtains the category label for each pixel. The rust region area is the total number of pixels classified as rust in the mask image. The total surface area of ​​the rod is the total number of pixels in the pre-segmented rod region. The ratio of the rust region area to the total surface area of ​​the rod directly quantifies the macroscopic severity of rust. The shape features of the rust region are geometric descriptors calculated from the binary rust mask, such as roundness and aspect ratio, used to describe the distribution pattern of rust (whether it is dot-like, sheet-like, or strip-like).

[0135] Understandably, area percentage is the most direct indicator of severity and a commonly used quantitative basis in operation and maintenance standards. However, area percentage alone may not be sufficient to distinguish risks. For example, large-area, concentrated corrosion and scattered, point-like corrosion, even if the area is the same, may have different impacts on the structure. Introducing shape features can help determine the type of corrosion and potential risks: point-like corrosion may only be in its initial stage, while elongated or edge-like corrosion may indicate the beginning of stress concentration or coating peeling. By combining area percentage and shape features, the features output in this step can more comprehensively reflect the macroscopic manifestation of corrosion, providing a more refined basis for damage level assessment.

[0136] Specifically, first, the image of the rod region is input into a trained rust semantic segmentation network to obtain a probability map. This map is then thresholded (e.g., to 0.5) to obtain a binary rust mask M. Next, the area ratio is calculated, representing the total pixel area of ​​the rusted region, i.e., the number of pixels in the binary mask M with a value of 1 (white, representing rust). Simultaneously, the aspect ratio of the minimum bounding rectangle can be selected according to actual needs. Roundness is the area of ​​the rusted region. The area ratio of the rusted region to its minimum bounding rectangle is also considered. The Hu moments of the contour (which are invariant to translation, rotation, and scale) are also used. The area ratio and several selected shape feature values ​​are combined into a feature vector, i.e., the rust area ratio feature.

[0137] Step S75: Perform a two-dimensional Fourier transform on the visible light image to divide the frequency domain image into a low-frequency region, a mid-frequency region, and a high-frequency region. Calculate the energy proportion and peak frequency of the low-frequency region, the mid-frequency region, and the high-frequency region respectively to obtain the image spectral features.

[0138] It's important to note that the two-dimensional Fourier transform is a transformation that converts an image from the spatial domain to the frequency domain. The resulting frequency domain image reflects the intensity and phase information of different frequency components in the original image. The low-frequency, mid-frequency, and high-frequency regions are areas on the frequency domain plane divided from low to high frequency, typically using concentric rings centered at the origin (zero frequency). Low frequencies correspond to gently changing areas and the overall outline of the image, mid frequencies to texture and edge information, and high frequencies to details, noise, and sharp edges. Energy percentage refers to the ratio of the sum of the squares of the amplitudes of all pixels within a certain frequency region to the total energy of the entire spectrum, reflecting the importance of that frequency component in the image. Peak frequency refers to the frequency value corresponding to the frequency component with the largest amplitude within that frequency region.

[0139] It is understandable that damages such as rust and cracks will change the periodic patterns and edge characteristics of the image surface. For example, uniform rust may increase the mid- and high-frequency energy; regular grid-like rust or periodically occurring pitting will generate peaks at specific frequencies; the blurring or sharpening of edges will also affect the high-frequency components. By analyzing the energy distribution and peak frequencies in different frequency bands, it is possible to detect uniformity changes, periodic defects, and subtle texture changes that are difficult to directly observe with the human eye. The spectral features provide another orthogonal dimension of information for rust detection, effectively complementing the texture and area features in the spatial domain, and helping to discover early or special damage types.

[0140] Specifically, perform a two-dimensional fast Fourier transform on the grayscale image I(x, y) of the rod region to obtain its spectrum F(u, v). Move the origin of the spectrum to the center and calculate the magnitude spectrum |F(u, v)|. On the frequency domain plane, with the origin as the center, set two radii r1 and r2 (r1 < r2), and divide the frequency region into: low-frequency region (frequency < r1), mid-frequency region (r1 ≤ frequency < r2), and high-frequency region (frequency ≥ r2). Then, calculate the energy of each region and the total energy, and further calculate the energy ratio: ; ; ; ; ; ; ; Among them, is the low-frequency energy; is the mid-frequency energy; is the high-frequency energy. is the low-frequency region; is the mid-frequency region; is the high-frequency region. is the total energy.

[0141] Next, within each region, find the point that maximizes the amplitude |F(u, v)|, and its corresponding frequency value , and calculate the peak frequency : ; Finally, combine the three energy ratios and the three peak frequencies (a total of 6 values) to form an image spectral feature vector.

[0142] This embodiment, through the above-described scheme, combines the pixel-level precision of semantic segmentation with the analysis of the geometry of the rusted area for the area proportion feature. This allows the feature to reflect not only the quantity of rust but also its shape, thus enabling a more accurate risk assessment. For the spectral features, Fourier transform and frequency band energy analysis are used to capture information on surface uniformity and periodic variations from the unique perspective of the frequency domain, compensating for the shortcomings of spatial domain analysis.

[0143] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the damage prediction method for transmission towers in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0144] This application also provides a damage prediction device for transmission towers, please refer to... Figure 2 The damage prediction device for the transmission tower includes: The feature extraction module 201 is used to extract features from the comprehensive heterogeneous data of the transmission tower to obtain comprehensive heterogeneous features; Topology fusion module 202 is used to map and fuse the integrated heterogeneous features based on the structural topology map of the transmission tower to obtain fused topology features; The damage detection module 203 is used to fuse and classify the integrated heterogeneous features and the fused topological features to obtain the damage detection results and damage level classification results of each component of the transmission tower. The damage prediction module 204 is used to perform damage time-series prediction based on the damage detection results and the damage level classification results, so as to obtain the damage prediction results of each component of the transmission tower.

[0145] The damage prediction device for transmission towers provided in this application, employing the damage prediction method for transmission towers described in the above embodiments, can solve the technical problem of insufficient comprehensiveness in damage detection and prediction for transmission towers. Compared with the prior art, the beneficial effects of the damage prediction device for transmission towers provided in this application are the same as those of the damage prediction method for transmission towers provided in the above embodiments, and other technical features in the damage prediction device for transmission towers are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0146] This application provides a damage prediction device for transmission towers. The damage prediction device for transmission towers includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the damage prediction method for transmission towers in the above embodiment 1.

[0147] The following is for reference. Figure 3The diagram illustrates a structural schematic of a damage prediction device for transmission towers suitable for implementing embodiments of this application. The damage prediction device for transmission towers in embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), vehicle terminals (e.g., vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 3 The damage prediction device for the transmission tower shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0148] like Figure 3 As shown, the damage prediction device for transmission towers may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the transmission tower damage prediction device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the damage prediction device for the transmission tower to communicate wirelessly or wiredly with other devices to exchange data. Although the figures show damage prediction devices for transmission towers with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0149] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0150] The damage prediction device for transmission towers provided in this application, employing the damage prediction method for transmission towers described in the above embodiments, can solve the technical problem of insufficient comprehensiveness in damage detection and prediction for transmission towers. Compared with the prior art, the beneficial effects of the damage prediction device for transmission towers provided in this application are the same as those of the damage prediction method for transmission towers provided in the above embodiments, and other technical features in this damage prediction device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0151] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0152] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0153] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the transmission tower damage prediction method in the above embodiments.

[0154] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0155] The aforementioned computer-readable storage medium may be included in the damage prediction device of the transmission tower; or it may exist independently and not be assembled into the damage prediction device of the transmission tower.

[0156] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the damage prediction device for the transmission tower, the damage prediction device for the transmission tower causes the following: it extracts features from the comprehensive heterogeneous data of the transmission tower to obtain comprehensive heterogeneous features; it maps and fuses the comprehensive heterogeneous features based on the structural topology map of the transmission tower to obtain fused topology features; it fuses and classifies the comprehensive heterogeneous features and the fused topology features to obtain damage detection results and damage level classification results for each component of the transmission tower; and it performs damage time-series prediction based on the damage detection results and the damage level classification results to obtain damage prediction results for each component of the transmission tower.

[0157] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0158] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0159] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0160] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described damage prediction method for transmission towers, thereby solving the technical problem of insufficient comprehensiveness in damage detection and prediction for transmission towers. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the damage prediction method for transmission towers provided in the above embodiments, and will not be repeated here.

[0161] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method for predicting damage to transmission towers.

[0162] The computer program product provided in this application can solve the technical problem of insufficient comprehensiveness in damage detection and prediction of transmission towers. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the damage prediction method for transmission towers provided in the above embodiments, and will not be repeated here.

[0163] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for predicting damage to transmission towers, characterized in that, The damage prediction method for the transmission tower includes: Feature extraction is performed on the comprehensive heterogeneous data of transmission towers to obtain comprehensive heterogeneous features; Based on the structural topology diagram of the transmission tower, the integrated heterogeneous features are mapped and fused to obtain fused topology features; The integrated heterogeneous features and the fused topological features are fused and classified to obtain the damage detection results and damage level classification results of each component of the transmission tower; Based on the damage detection results and the damage level classification results, damage time sequence prediction is performed to obtain the damage prediction results for each component of the transmission tower.

2. The damage prediction method for transmission towers as described in claim 1, characterized in that, The integrated heterogeneous data includes visible light images, infrared thermal imaging data, point cloud time-series data, and vibration monitoring data. The integrated heterogeneous features include visible range features, infrared temperature field features, point cloud time-series features, and vibration time-series features. The step of extracting features from the integrated heterogeneous data of the transmission tower to obtain integrated heterogeneous features includes: Image features are extracted from the visible light image to obtain visible range features, which include rust texture features, rust area ratio features, and image spectrum features. Temperature field features are extracted from the infrared thermal imaging data to obtain infrared temperature field features; Point cloud features are extracted from the point cloud time series data to obtain point cloud time series features; Vibration features are extracted from the vibration monitoring data to obtain vibration time-series features.

3. The damage prediction method for transmission towers as described in claim 2, characterized in that, The step of mapping and fusing the integrated heterogeneous features based on the structural topology map of the transmission tower to obtain the fused topology features includes: A structural topology diagram is constructed based on the actual structure of the transmission tower, wherein the graph nodes of the structural topology diagram are the poles of the transmission tower; The integrated heterogeneous features are double-aligned to obtain the target aligned features; The target alignment features are fused based on the membership relationships of the members in the structural topology diagram to obtain fused topology features.

4. The damage prediction method for transmission towers as described in claim 3, characterized in that, The step of performing dual alignment on the integrated heterogeneous features to obtain the target aligned features includes: The point cloud temporal features and the vibration temporal features are coarsely aligned using a dynamic time warping algorithm to obtain the initial aligned features after coarse alignment. The initial alignment features are input into the residual fine alignment network for fine-grained correction to obtain the target alignment features.

5. The damage prediction method for transmission towers as described in claim 1, characterized in that, The damage detection results include the corrosion detection results of the members and the condition detection results of the bolts. The step of fusing and classifying the integrated heterogeneous features and the fused topological features to obtain the damage detection results and damage level classification results of each component of the transmission tower includes: The contribution weights are obtained by cross-modal attention weighting of the integrated heterogeneous features and the fused topological features through a multi-head self-attention mechanism, and feature fusion is performed based on the contribution weights to obtain the integrated damage features. The comprehensive damage features are input into the rod corrosion detection head, the bolt condition detection head, and the damage level classification head, respectively, to obtain the rod corrosion detection results, the bolt condition detection results, and the damage level classification results.

6. The damage prediction method for transmission towers as described in claim 1, characterized in that, The step of performing damage time-series prediction based on the damage detection results and the damage level classification results to obtain the damage prediction results of each component of the transmission tower includes: Based on the inspection time sequence, the damage detection results and damage level classification results of each component of the same transmission tower are sequenced to obtain the component-level damage time series. The component-level damage time series is input into the time series prediction network, and the damage level change prediction curve of each component in the future time window is output. Based on the damage level change prediction curve, the estimated remaining time for each component to upgrade from the current damage level to the next damage level is calculated, and the damage prediction result is obtained.

7. The damage prediction method for transmission towers as described in claim 2, characterized in that, The step of extracting image features from the visible light image to obtain visible range features includes: The contrast features, correlation features, energy features, and homogeneity features of the rusted region in the visible light image are extracted using the gray-level co-occurrence matrix to obtain the first texture sub-feature. The visible light image is input into a convolutional neural network to extract deep texture features, resulting in a second texture sub-feature. The first texture sub-feature is concatenated with the second texture sub-feature to obtain the rust texture feature; The corroded areas in the visible light image are segmented at the pixel level using a semantic segmentation network. The ratio of the corroded area to the total surface area of ​​the rod is calculated, and the shape features of the corroded areas are extracted to obtain the corroded area proportion feature. A two-dimensional Fourier transform is performed on the visible light image to divide the frequency domain image into low-frequency, mid-frequency, and high-frequency regions. The energy proportion and peak frequency of the low-frequency, mid-frequency, and high-frequency regions are calculated respectively to obtain the image spectral features.

8. A damage prediction device for transmission towers, characterized in that, The damage prediction device for the transmission tower includes: The feature extraction module is used to extract features from the comprehensive heterogeneous data of the transmission towers to obtain comprehensive heterogeneous features; The topology fusion module is used to map and fuse the integrated heterogeneous features based on the structural topology map of the transmission tower to obtain fused topology features. The damage detection module is used to fuse and classify the integrated heterogeneous features and the fused topological features to obtain the damage detection results and damage level classification results of each component of the transmission tower. The damage prediction module is used to perform damage time-series prediction based on the damage detection results and the damage level classification results, and obtain the damage prediction results of each component of the transmission tower.

9. A damage prediction device for transmission towers, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the damage prediction method for transmission towers as claimed in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the damage prediction method for transmission towers as described in any one of claims 1 to 7.

Images